Digital Red Queen - Adversarial Program Evolution in Core War with LLMs

2601.03335v1

• Website
• GitHub

Abstract

Digital Red Queen: Adversarial Program Evolution in Core War with LLMs

Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most LLM-evolution frameworks are formulated as static optimization problems, overlooking the open-ended adversarial dynamics that characterize real-world evolutionary processes. Here, we study Digital Red Queen (DRQ), a simple self-play algorithm that embraces these so-called “Red Queen” dynamics via continual adaptation to a changing objective. DRQ uses an LLM to evolve assembly-like programs, called warriors, which compete against each other for control of a virtual machine in the game of Core War, a Turing Completeness environment studied in artificial life and connected to cybersecurity. In each round of DRQ, the model evolves a new warrior to defeat all previous ones, producing a sequence of adapted warriors. Over many rounds, we observe that warriors become increasingly general (relative to a set of held-out human warriors). Interestingly, warriors also become less behaviorally diverse across independent runs, indicating a convergence pressure toward a general-purpose behavioral strategy, much like convergent evolution in nature. This result highlights a potential value of shifting from static objectives to dynamic Red Queen objectives. Our work positions Core War as a rich, controllable sandbox for studying adversarial adaptation in artificial systems and for evaluating LLM-based evolution methods. More broadly, the simplicity and effectiveness of DRQ suggest that similarly minimal self-play approaches could prove useful in other more practical multi-agent adversarial domains, like real-wor…

Authors

SakanaAI

• Akarsh Kumar
• Ryan Bahlous-Boldi
• Prafull Sharma
• Phillip Isola
• Sebastian Risi
• Yujin Tang
• David Ha

Summary

The paper introduces Digital Red Queen (DRQ), a self-play evolutionary framework that uses LLMs as mutation operators to navigate the strategic complexity of Core War. By replacing static fitness functions with a dynamic adversarial lineage, DRQ induces “Red Queen” dynamics where agents must continually innovate to maintain relative fitness. The study demonstrates that iterative adaptation against a historical pool of adversaries leads to the emergence of robust, generalist combat strategies and reveals a phenomenon of phenotypic convergence in the space of Turing Completeness programs.

Mechanism

Adversarial Self-Play Loop: Let be the space of Redcode programs. DRQ constructs a sequence of warriors where is optimised against the history pool . In round , the objective is to find:

where is the battle fitness. History length controls the look-back window; (full DRQ) mitigates cyclic Rock-Paper-Scissors dynamics.

Fitness Function: For warriors in a battle of cycles, fitness is the cumulative time-distributed reward:

where denotes if warrior is active at time . This incentivises both longevity and the rapid termination of competitors.

Intra-round Optimisation (MAP-Elites): Search is conducted via MAP-Elites to preserve strategic diversity. The behavioural descriptor is . The LLM (GPT-4.1 mini) performs “informed” mutations and crossovers on elite solutions, utilising a Redcode manual provided in the system prompt to remain within the functional program subspace.

Findings

Generality Emergence: DRQ warriors exhibit superior zero-shot performance against unseen human experts compared to static optimisation baselines. Generality increases monotonically with rounds .

Convergent Evolution: As , independent DRQ runs exhibit phenotypic convergence. Variance in the behavioural vector decreases, while genotypic variance (distance in embedding space of source code) remains high. This mirrors biological convergence where similar functional traits emerge from distinct genetic lineages.

Generality Prediction: A linear probe trained on text embeddings of Redcode source code achieves . This suggests that the strategic utility of a program is partially encoded in its static structural features, allowing for potential surrogate-based search acceleration.