Digital Red Queen - Adversarial Program Evolution in Core War with LLMs

2601.03335v1

• Website
• GitHub

Abstract

Digital Red Queen: Adversarial Program Evolution in Core War with LLMs

Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most LLM-evolution frameworks are formulated as static optimization problems, overlooking the open-ended adversarial dynamics that characterize real-world evolutionary processes. Here, we study Digital Red Queen (DRQ), a simple self-play algorithm that embraces these so-called “Red Queen” dynamics via continual adaptation to a changing objective. DRQ uses an LLM to evolve assembly-like programs, called warriors, which compete against each other for control of a virtual machine in the game of Core War, a Turing-complete environment studied in artificial life and connected to cybersecurity. In each round of DRQ, the model evolves a new warrior to defeat all previous ones, producing a sequence of adapted warriors. Over many rounds, we observe that warriors become increasingly general (relative to a set of held-out human warriors). Interestingly, warriors also become less behaviorally diverse across independent runs, indicating a convergence pressure toward a general-purpose behavioral strategy, much like convergent evolution in nature. This result highlights a potential value of shifting from static objectives to dynamic Red Queen objectives. Our work positions Core War as a rich, controllable sandbox for studying adversarial adaptation in artificial systems and for evaluating LLM-based evolution methods. More broadly, the simplicity and effectiveness of DRQ suggest that similarly minimal self-play approaches could prove useful in other more practical multi-agent adversarial domains, like real-wor…

Authors

SakanaAI

• Akarsh Kumar
• Ryan Bahlous-Boldi
• Prafull Sharma
• Phillip Isola
• Sebastian Risi
• Yujin Tang
• David Ha

Summary

The paper introduces Digital Red Queen (DRQ), a self-play evolutionary framework that uses LLMs as mutation operators to navigate the strategic complexity of Core War. By replacing static fitness functions with a dynamic adversarial lineage, DRQ induces “Red Queen” dynamics where agents must continually innovate to maintain relative fitness. The study demonstrates that iterative adaptation against a historical pool of adversaries leads to the emergence of robust, generalist combat strategies and reveals a phenomenon of phenotypic convergence in the space of Turing-complete programs.

Mechanism

Adversarial Self-Play Loop: Let $\mathcal{W}$ be the space of Redcode programs. DRQ constructs a sequence of warriors $(w_0,w_1,\dots ,w_T)$ where $w_t$ is optimised against the history pool ${\mathcal{P}}_{t-1}=\{w_0,\dots ,w_{t-1}\}$. In round $t$, the objective is to find:

$$w_t=\arg \underset{w\in \mathcal{W}}{\max }{\mathbb{E}}_{w_i\sim {\mathcal{P}}_{t-1}}[B(w,w_i)]$$

where $B(w,w_i)$ is the battle fitness. History length $K$ controls the look-back window; $K=T$ (full DRQ) mitigates cyclic Rock-Paper-Scissors dynamics.

Fitness Function: For $N$ warriors in a battle of $T$ cycles, fitness is the cumulative time-distributed reward:
$f(w_i)={\sum }_{\tau =1}^T\frac{N}{T}\frac{A_i^{\tau }}{{\sum }_j{A}_j^{\tau }}$

where $A_i^{\tau }\in \{0,1\}$ denotes if warrior $i$ is active at time $\tau$. This incentivises both longevity and the rapid termination of competitors.

Intra-round Optimisation (MAP-Elites): Search is conducted via MAP-Elites to preserve strategic diversity. The behavioural descriptor is $BD(w)=(\log (\text{threads}),\log (\text{coverage}))$. The LLM (GPT-4.1 mini) performs “informed” mutations and crossovers on elite solutions, utilising a Redcode manual provided in the system prompt to remain within the functional program subspace.

Findings

Generality Emergence: DRQ warriors exhibit superior zero-shot performance against unseen human experts $\mathcal{H}$ compared to static optimisation baselines. Generality $G(w)=\frac{1}{|\mathcal{H}|}{\sum }_{h\in \mathcal{H}}\mathbb{I}(B(w,h)\ge B(h,w))$ increases monotonically with rounds $T$.

Convergent Evolution: As $T\to \infty$, independent DRQ runs exhibit phenotypic convergence. Variance in the behavioural vector ${\mathbf{v}}_w=[B(w,h_1),\dots ,B(w,h_k)]$ decreases, while genotypic variance (distance in embedding space of source code) remains high. This mirrors biological convergence where similar functional traits emerge from distinct genetic lineages.

Generality Prediction: A linear probe $\hat{G}(w)={\omega }^{\top }\varphi (w)$ trained on text embeddings $\varphi (w)$ of Redcode source code achieves $R^2=0.461$. This suggests that the strategic utility of a program is partially encoded in its static structural features, allowing for potential surrogate-based search acceleration.