Self-Evolving Agents: Model, Harness, and Artifact Evolution

Introduction

Three routes to agent self-evolution

Terms like "self-evolving," "self-improving," "autonomous adaptation," and "meta-learning" appear interchangeably across papers doing fundamentally different things to fundamentally different parts of the system. One paper updates weights through self-generated training signal. Another rewrites system prompts based on trajectory reflection. A third iteratively mutates a population of algorithms scored by an external evaluator. All three call themselves self-improving—but the label obscures more than it reveals.

This review focuses on eight representative papers from the past year or so, selected to cover three routes to self-evolution: changing the artifact, changing the harness, and changing the model itself. They range from a zero-data reinforcement learning paradigm (Absolute Zero) to a fully automated research system deployed across 166 papers and 67 topics (FARS), from prompt evolution that outperforms gradient-based optimization (GEPA) to an evolutionary coding agent that discovered improvements to Strassen's algorithm after 56 years (AlphaEvolve).

This survey introduces three anchoring constructs—Model, Harness, and Artifact—and uses them to partition self-evolution strategies into three mutually exclusive categories, each defined by one question: which layer of the agent is changing?

Conceptual Foundations: Model, Harness, Artifact, and the Agent Identity

Model denotes the parametric language model whose weights encode compressed knowledge. Those weights may be frozen or trainable; when the Model changes, gradient updates flow into it and weights shift.

Harness denotes everything surrounding the Model at inference time without requiring weight updates: prompts, system instructions, routing logic, memory stores, tool dispatchers, workflow scaffolds, MCP libraries. The Harness is software, not learned parameters; it can be rewritten between calls with no gradient involved.

Alita workflow

Artifact denotes any persistent output the Agent produces as the residue of task execution: code files, algorithm implementations, research manuscripts, experience templates, MCP definitions. Artifacts live outside the agent loop and can be stored, versioned, evaluated, and fed back in.

The key architectural claim is: Agent = Model + Harness. Observable behavior is a joint function of what the Model knows and how the Harness directs it. Neither layer alone constitutes the agent, and conflating the two is a primary source of terminology confusion in this literature.

Artifacts are the connective tissue of all self-evolution. A code solution scored by an evaluator feeds the Artifact-layer loop. A trajectory trace reflected upon to update a prompt feeds the Harness-layer loop. A set of self-proposed tasks with verified solutions feeds the Model-layer loop.

GEPA results

Three papers concisely illustrate the distinctions. Alita generates MCP tools on demand and accumulates them in the Harness inventory—Harness-layer evolution. GEPA reflects on trajectories and rewrites system prompts—Harness-layer evolution by a different mechanism. AZR proposes code-reasoning tasks, solves them, and updates weights via verifiable execution rewards—Model-layer evolution. All three call their systems "self-evolving"; the taxonomy makes clear they operate on three different layers.

Artifact-Layer Evolution: Iterative Refinement of What the Agent Produces

In Artifact-layer evolution, neither Model weights nor Harness configuration are permanently altered. What evolves is the population of Artifacts—code files, algorithms, research manuscripts—through iterative generation, evaluation, and selection. The Model and Harness serve as fixed generators; an automated evaluator provides scores; highest-scoring Artifacts seed the next generation. The loop closes at the Artifact store.

AlphaEvolve Original paper ↗

AlphaEvolve results

AlphaEvolve was developed by Alexander Novikov, Matej Balog and colleagues, with participating institutions including Google DeepMind, Google. AlphaEvolve, from Google DeepMind, is the cleanest instantiation of this category. The system orchestrates a pipeline of LLMs—Gemini Flash for rapid generation, Gemini Pro for higher-quality proposals—to evolve entire code files. Each candidate is scored by automated evaluators (correctness verifiers, benchmark metrics, resource monitors), and a program database maintains a diverse evolutionary population, sampling from past candidates weighted by recency and quality to prevent premature convergence.

AlphaEvolve discovered the first improvement over Strassen's algorithm for multiplying two 4×4 complex-valued matrices in 56 years, reducing required scalar multiplications from 49 to 48. It also optimized data-center scheduling heuristics at Google and accelerated training of the LLM that powers AlphaEvolve itself. Automated executable evaluation is a structural prerequisite: without reliable selection pressure, the evolutionary population degenerates. The program database's diversity management—retaining candidates across performance levels rather than only the global best—enables discovery of structurally novel algorithms.

FARS Original paper ↗

FARS architecture

FARS (Fully Automated Research System) extends Artifact-layer evolution to the full scientific research pipeline. Stage-specific agents handle Ideation, Planning, Experimentation, and Writing, coordinated through a shared workspace that functions as both project memory and auditable artifact store. In its first public deployment, FARS produced 166 complete papers spanning 67 AI/ML topics. Volunteer reviewers provided 282 structured reviews covering 140 of the 166 papers, including overall ratings, sub-scores for soundness and contribution, integrity checks, and disclosure of LLM-assisted review. Recurring weaknesses include narrow experimental scope, overclaiming, and insufficient experimental validation.

The human review process is an external audit of the Artifact population, not a self-evolution feedback loop—the system does not automatically update from reviewer feedback. FARS demonstrates that Artifact-layer evolution at scale surfaces failure modes that curated demonstrations systematically suppress. AlphaEvolve achieves high-fidelity evolution by restricting scope to problems with executable automated evaluators; FARS achieves broad scope by expanding to full research pipelines, compensating with structured human review. The tension between scope and evaluability is a structural constraint.

Harness-Layer Evolution: Self-Improvement Without Touching Model Weights

Harness-layer evolution requires no gradient computation and no weight update, making it applicable to any Model including black-box APIs. What evolves is the Harness configuration: prompt text, routing assignments, memory structures, and tool inventories. Feedback comes from task outcomes consumed by a meta-level optimizer operating entirely on Harness components. The loop closes inside the Harness. Five papers instantiate this category through distinct mechanisms.

GEPA Original paper ↗

GEPA was developed by Lakshya A Agrawal, Omar Khattab and colleagues, with participating institutions including Stanford University, MIT. GEPA (Genetic-Pareto) argues directly against gradient-based RL as the right tool for adapting LLM agents. A trajectory contains nothing but language—instructions, reasoning chains, tool calls, compiler messages, reward signals—which is precisely what LLMs are best at understanding. Rather than compress that richness into a scalar reward and run policy gradient ascent, GEPA reflects on trajectories in natural language to diagnose problems and propose prompt mutations, then maintains a Pareto frontier of top-performing prompts to avoid greedy convergence.

On HotpotQA, HoVer, IFBench, and PUPA, GEPA outperforms GRPO (with 24,000 rollouts and LoRA fine-tuning) by an average of 10% and up to 20%, while requiring up to 35× fewer rollouts. It also outperforms MIPROv2 by over 10% across two LLMs. GEPA never updates Model weights. The information-efficiency asymmetry is the key insight: prompt optimization operates in language space, where each rollout carries a full natural-language explanation of what went wrong, whereas RL operates in weight space via scalar rewards.

EEVEE Original paper ↗

EEVEE architecture

GEPA assumes a single-benchmark setting. Real-world deployment breaks this: queries arrive from heterogeneous domains, and optimizing a prompt for one domain degrades performance on another. EEVEE extends test-time prompt learning to heterogeneous multi-dataset streams by adding a router that partitions incoming inputs into task clusters and assigns each cluster a suitable prompt configuration. The router and prompt learner are mutually dependent; EEVEE resolves this coupling through interleaved router and prompt learning phases.

EEVEE improves average multi-benchmark scores by 10.38 points over Qwen3-4B-Instruct and 24.32 points over DeepSeek-V3.2, surpassing GEPA and ACE by up to 37.2% and 48.2% respectively. In the incremental setting—benchmarks introduced one at a time—EEVEE ends at a cumulative +41.53 retention gain while GEPA and ACE end at −15.36 and −18.58. The negative numbers reveal the cross-dataset interference problem: optimizing for a new task destroys the Harness configuration that worked for prior tasks.

UI-Mem Original paper ↗

UI-Mem workflow

UI-Mem addresses how a mobile GUI agent should accumulate experience across multiple tasks and applications where online RL must contend with sparse rewards and long-horizon credit assignment. The solution is a hierarchical experience memory with three levels: high-level workflows for planning, subtask skills for execution, and failure patterns to prevent repetitive errors. Experiences are stored as parameterized templates that abstract away task-specific details, enabling reuse across applications sharing structural similarities.

UI-Mem's Stratified Group Sampling injects varying levels of memory guidance across trajectories within each rollout group. Without unguided trajectories the agent would follow memory rather than internalize it; the mix drives the unguided policy toward reproducing guided behaviors—imitation-through-contrast that makes the RL signal more informative. Model weights are updated via online RL (GRPO), but the memory component evolves independently of and in addition to the weight updates; the memory itself is a Harness-layer artifact.

Alita Original paper ↗

Alita was developed by Jiahao Qiu, Mengdi Wang and colleagues, with participating institutions including Princeton University, Tsinghua University. Alita generates new Harness components on demand rather than optimizing existing ones. When Alita encounters a task requiring a capability it lacks, it generates an MCP implementing that capability, tests it, refines it, and adds it to its reusable library. The design philosophy is minimalist: Alita ships with only a single core capability (a web agent) and a small set of general-purpose modules; everything else is generated and accumulated on demand.

On the GAIA benchmark validation set, Alita achieves 75.15% pass@1 and 87.27% pass@3, outperforming OpenAI Deep Research (67.36% pass@1). A system with fewer prebuilt components outperforms systems with more, because dynamic generation constructs precisely the right tool for each task. MCPs are generated as Artifacts—persistent, versioned outputs of task execution—but stored as Harness components that alter future agent behavior without any Model weight update.

BoundaryRouter Original paper ↗

Learning Agent Routing From Early Experience^[1] was developed by Yimin Wang, Mengdi Wang and colleagues, with participating institutions including Princeton University, Tsinghua University. BoundaryRouter addresses a routing problem: should the system route a query to lightweight direct LLM inference or escalate to full agent execution? The cold-start constraint is central—in real deployment there is no labeled training set of queries with known correct routing decisions. BoundaryRouter builds an early experience memory from a small seed set of queries executed by both the LLM and the full agent. Comparing their outputs creates a behavioral reference capturing systematic differences between the two systems. At inference time, BoundaryRouter retrieves similar past experiences and uses rubric-guided reasoning to make routing decisions.

BoundaryRouter reduces average inference time by 60.6% compared to always using the agent, while improving performance by 28.6% over always using direct LLM inference. The rubric-guided reasoning encodes structural knowledge about task difficulty rather than surface-level similarity matching, enabling generalization to out-of-domain scenarios in the RouteBench evaluation.

Model-Layer Evolution: Learning Without Ground-Truth Answer Labels

Absolute Zero results

Model-layer evolution is the most fundamental form: weights change through signal generated by the agent itself, with no external dataset of (question, answer) pairs. The defining constraint is verifiability—there must be a reliable automated mechanism to determine whether a proposed solution is correct, because the loop must close thousands of times during training with no human in the critical path. This restricts the category to domains where correctness has an executable definition: code that runs or doesn't, proofs that check out or don't.

Absolute Zero Reasoner (AZR) Original paper ↗

AZR is the paradigmatic instance. A single language model both proposes coding and reasoning tasks and solves them, using a code executor as the sole source of verifiable reward, with no external data whatsoever.

The technical mechanism separates proposer and solver roles within the same model. In the proposer role, the model generates tasks parameterized by three types: deduction (predict output from code and input), abduction (infer input from code and output), and induction (infer code from input-output pairs). In the solver role, the model receives a proposed task, generates a solution, and receives a binary reward based on execution verification. Policy gradient (RLVR) updates the weights, and the updated model then proposes harder tasks—a curriculum co-evolving with capability. AZR achieves state-of-the-art performance on coding and mathematical reasoning benchmarks, outperforming models trained on tens of thousands of in-domain human-curated examples.

The separation of proposer and solver roles within a single model prevents reward collapse. Tasks the model already solves trivially produce no gradient signal, pushing the proposer toward tasks at the boundary of current capability. The three task types provide structural diversity preventing collapse to a narrow class of easily-generated tasks.

The scope limitation is equally important: AZR is bounded by the verifiable domain. Open-ended dialogue, subjective judgment, and aesthetic evaluation cannot be automatically verified. This is an honest boundary condition on the entire Model-layer self-evolution paradigm as it currently exists.

Boundaries, Open Problems, and Real-World Deployment Constraints

Verifiability is the master constraint. Both Artifact-layer and Model-layer evolution require automated correctness signals: AlphaEvolve's code executor, AZR's verification environment. When such signals are unavailable, neither category can function without human evaluation in the loop. FARS demonstrates what happens at the boundary: automated metrics for research quality are insufficient, so evaluation relies on 282 volunteer reviews—an honest acknowledgment that automated verifiability for open-ended scientific writing does not yet exist at acceptable quality. Harness-layer evolution remains the most broadly applicable category for domains where verification is soft or subjective.

Learning Agent Routing workflow

Cold-start scarcity affects all categories differently. BoundaryRouter confronts routing cold-start—no prior routing labels exist for new deployments—and builds a behavioral reference from a seed set. EEVEE confronts prompt cold-start under domain shift—as new benchmark domains enter the stream, existing configurations have no experience with them. FARS confronts quality cold-start at scale—in early deployment, there is no evidence about which experimental configurations produce strong papers. Each paper develops a specific mitigation, but none eliminates the cold-start problem; they manage it.

Proxy optimization and faithfulness failures are structurally analogous. AZR faces reward hacking risk: a sufficiently powerful proposer could generate tasks that are technically executable-verifiable but solved by surface pattern matching rather than actual reasoning. FARS faces faithfulness failures: the writing agent occasionally overclaims or inadequately represents experimental evidence, optimizing for plausible-sounding language rather than accurate scientific description. Both represent the same structural issue—optimizing a measurable proxy at the expense of the intended objective.

The Model ceiling limits Harness evolution. Every Harness-layer approach in this survey operates on a frozen Model with a capability ceiling: there are tasks it cannot solve regardless of prompt quality or memory richness. Once Harness optimization saturates, further gains require Model-layer updates. This motivates co-evolution architectures in which Harness improvements generate training data for Model updates, which in turn enable further Harness optimization. No paper in this survey implements full three-layer co-evolution, but the taxonomy points toward it as the natural next step.

Evaluation methodology is itself an open problem. FARS's structured volunteer reviews, AlphaEvolve's formal correctness proofs, and AZR's out-of-distribution benchmark transfer are not interchangeable. Formal proofs are available only for well-specified mathematical problems. Volunteer reviews require significant human effort and carry inter-reviewer variance. Benchmark transfer measures a specific proxy for generalization but says nothing about real-world utility. FARS's decision to publish all 166 papers with all 282 reviews—including failures—is a methodological commitment that the field would benefit from adopting more broadly.

Multi-domain heterogeneity requires explicit architectural accommodation. EEVEE prevents prompt specialization for one domain from destroying performance on others; BoundaryRouter prevents routing experience from failing to transfer to out-of-domain queries. In both cases the solution involves explicit structure in the Harness—a router partitioning the input space, rubric-guided reasoning encoding structural knowledge—rather than hoping a single configuration generalizes.

Conclusion

What evolves, where feedback comes from, and where the loop closes each have a clean answer per category.

In Artifact-layer evolution (AlphaEvolve, FARS), the Artifact population evolves while the agent stays fixed. Feedback comes from automated evaluators—code executors, benchmark scores—or structured human review when automation is insufficient. The loop closes at the Artifact store: surviving Artifacts seed the next generation, and neither Model weights nor Harness configuration are modified. The power of this category—genuine discovery at scale—is inseparable from its prerequisite: reliable evaluation.

In Harness-layer evolution (GEPA, EEVEE, UI-Mem, Alita, BoundaryRouter), the Harness configuration evolves while Model weights remain frozen. Feedback comes from task outcomes consumed by meta-level optimizers operating in language space (GEPA, EEVEE), abstracted into structured memory (UI-Mem, Alita), or encoded as behavioral reference (BoundaryRouter). The loop closes inside the Harness: updated prompts, memory entries, routing policies, and tool libraries alter future behavior without any gradient flowing into the Model. This is the most broadly applicable category because it requires no weight access.

In Model-layer evolution (AZR), parametric weights evolve through a fully internal self-play circuit. Feedback comes exclusively from verifiable execution outcomes—compiler results, test-case pass rates—that substitute for human annotation. The loop closes inside the training process: the Model proposes tasks, solves them, receives execution-verified rewards, updates its own weights, and proposes harder tasks aligned with its new capability level. The result matches curated supervised training in well-verifiable domains with zero external data, bounded precisely by what can be automatically verified.

Original Papers

1. Learning Agent Routing From Early Experience. arXiv:2605.07180. 2026. View on arXiv.
2. FARS: A Fully Automated Research System Deployed at Scale. arXiv:2606.31651v1. 2026. View on arXiv.
3. EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents. arXiv:2606.11182v1. 2026. View on arXiv.
4. UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents. arXiv:2602.05832v1. 2026. View on arXiv.
5. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457v1. 2025. View on arXiv.
6. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv:2506.13131v1. 2025. View on arXiv.
7. Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution. arXiv:2505.20286v1. 2025. View on arXiv.
8. Absolute Zero: Reinforced Self-play Reasoning with Zero Data. arXiv:2505.03335v2. 2025. View on arXiv.