Self-Evolving Agents: Survey, Taxonomy, and How Self-Improving AI Agents Work

What Are Self-Evolving Agents?

Self-evolving agents are AI systems that persistently improve part of themselves using feedback from their own prior execution. Depending on the architecture, the change can land in model weights, in the surrounding harness of prompts, memory, routing, and tools, or in external artifacts such as code and research outputs. Unlike temporary in-context adaptation, self evolution in AI agents produces changes that persist across tasks, sessions, or iterations, so the next run starts from a genuinely different system.

This self-evolving agents survey reviews eight representative systems and sorts them by a single question: which layer of the agent actually changes? The answer separates papers that share the "self-improving" label but do fundamentally different things. The comparison and taxonomy below make that split explicit before the paper-by-paper review.

Three nearby labels are easy to mix up. An autonomous agent can act without a human in the loop, but it need not change itself. An adaptive agent may change behavior within a session—retrieving extra context, switching tools, or rewriting a plan—without leaving a durable update. A self-improving AI agent is the overlapping commercial and research label for systems that get better over time. This review treats self-evolving agents as the stricter case: some component of the agent is updated from feedback, and that update is still there on the next run.

Self-Evolving Agents vs Self-Improving AI Agents

In papers and product copy the two phrases are often treated as synonyms. They overlap, but they are not identical. Self-improving AI agents is the broader search and marketing term: any agent that claims to get better from experience. Self-evolving agents is the more specific research term: the system runs a closed loop that writes a persistent change into memory, tools, policy, workflow, model weights, or the environment.

The practical test is persistence. If the next session starts from the same persistent state—including memory, prompts, tools, policies, workflows, and weights—the system adapted once; it did not evolve.

A Taxonomy of Self-Evolving Agents

Readers looking for a taxonomy of self-evolving agents usually want the evolution target, not a paper-by-paper recap. Gao et al. organize the field by what, when, and how to evolve. This review keeps a complementary cut: which layer changes (Model, Harness, Artifact), then names the write target inside that layer.

A system can occupy more than one row. UI-Mem updates memory and weights. MemGen trains memory modules without a full backbone fine-tune. AgentEvolver generates tasks, reuses experience, and trains a policy. The taxonomy locates the write; it does not claim that each paper occupies a single cell.

Survey of Self-Evolving Agents

Two recent surveys map the broader literature. Gao, Geng, Hua, Wang and colleagues organize self-evolving agents by what, when, how, and where to evolve (arXiv:2507.21046). Fang, Peng, Zhang and colleagues review self-evolving AI agents as a bridge from static foundation models to lifelong agentic systems, with a feedback loop over system inputs, the agent, the environment, and optimisers (arXiv:2508.07407).

This page is a narrower self-evolving agents survey: eight systems scored by a single question—which layer changes?—then three additional representative systems that engineers now search for by name. The eight-paper table below is the reading cut; the surveys above are the map of the field.

8 Papers at a Glance

The table below compares all eight systems on the layer they evolve, the feedback signal that drives evolution, whether model weights are updated, and the headline result reported by the authors.

UI-Mem is the one hybrid case: its memory component is a harness artifact, but it is trained jointly with online reinforcement learning, so weights move as well.

Conceptual Foundations: Model, Harness, Artifact, and the Agent Identity

The eight-paper cut covers three primary loci of evolution—changing the artifact, the harness, or the model. They range from a zero-data reinforcement learning paradigm (Absolute Zero) to a fully automated research system deployed across 166 papers and 67 topics (FARS), from prompt evolution that outperforms gradient-based optimization (GEPA) to an evolutionary coding agent that discovered improvements to Strassen's algorithm after 56 years (AlphaEvolve). Model, Harness, and Artifact are analytical lenses for locating the write, not mutually exclusive buckets. Hybrid systems such as UI-Mem can change more than one layer.

Model denotes the parametric language model whose weights encode compressed knowledge. Those weights may be frozen or trainable; when the Model changes, gradient updates flow into it and weights shift.

Harness denotes everything surrounding the Model at inference time without requiring weight updates: prompts, system instructions, routing logic, memory stores, tool dispatchers, workflow scaffolds, MCP libraries. The Harness is software, not learned parameters; it can be rewritten between calls with no gradient involved.

Alita workflow

Artifact denotes any persistent output the Agent produces as the residue of task execution: code files, algorithm implementations, research manuscripts, experience templates, MCP definitions. Artifacts live outside the agent loop and can be stored, versioned, evaluated, and fed back in.

The key architectural claim is: Agent = Model + Harness. Observable behavior is a joint function of what the Model knows and how the Harness directs it. Neither layer alone constitutes the agent, and conflating the two is a primary source of terminology confusion in this literature.

Artifacts are the connective tissue of all self-evolution. A code solution scored by an evaluator feeds the Artifact-layer loop. A trajectory trace reflected upon to update a prompt feeds the Harness-layer loop. A set of self-proposed tasks with verified solutions feeds the Model-layer loop.

GEPA results

Three papers concisely illustrate the distinctions. Alita generates MCP tools on demand and accumulates them in the Harness inventory—Harness-layer evolution. GEPA reflects on trajectories and rewrites system prompts—Harness-layer evolution by a different mechanism. AZR proposes code-reasoning tasks, solves them, and updates weights via verifiable execution rewards—Model-layer evolution. All three call their systems "self-evolving"; the taxonomy makes clear they operate on three different layers.

Artifact-Layer Evolution: Iterative Refinement of What the Agent Produces

In Artifact-layer evolution, neither Model weights nor Harness configuration are permanently altered. What evolves is the population of Artifacts—code files, algorithms, research manuscripts—through iterative generation, evaluation, and selection. The Model and Harness serve as fixed generators; an automated evaluator provides scores; highest-scoring Artifacts seed the next generation. The loop closes at the Artifact store.

AlphaEvolve Original paper ↗

AlphaEvolve results

AlphaEvolve was developed by Alexander Novikov, Matej Balog and colleagues, with participating institutions including Google DeepMind, Google. AlphaEvolve, from Google DeepMind, is the cleanest instantiation of this category. The system orchestrates a pipeline of LLMs—Gemini Flash for rapid generation, Gemini Pro for higher-quality proposals—to evolve entire code files. Each candidate is scored by automated evaluators (correctness verifiers, benchmark metrics, resource monitors), and a program database maintains a diverse evolutionary population, sampling from past candidates weighted by recency and quality to prevent premature convergence.

AlphaEvolve discovered the first improvement over Strassen's algorithm for multiplying two 4×4 complex-valued matrices in 56 years, reducing required scalar multiplications from 49 to 48. It also optimized data-center scheduling heuristics at Google and accelerated training of the LLM that powers AlphaEvolve itself. Automated executable evaluation is a structural prerequisite: without reliable selection pressure, the evolutionary population degenerates. The program database's diversity management—retaining candidates across performance levels rather than only the global best—enables discovery of structurally novel algorithms.

FARS Original paper ↗

FARS architecture

FARS (Fully Automated Research System) extends Artifact-layer evolution to the full scientific research pipeline. Stage-specific agents handle Ideation, Planning, Experimentation, and Writing, coordinated through a shared workspace that functions as both project memory and auditable artifact store. In its first public deployment, FARS produced 166 complete papers spanning 67 AI/ML topics. Volunteer reviewers provided 282 structured reviews covering 140 of the 166 papers, including overall ratings, sub-scores for soundness and contribution, integrity checks, and disclosure of LLM-assisted review. Recurring weaknesses include narrow experimental scope, overclaiming, and insufficient experimental validation.

The human review process is an external audit of the Artifact population, not a self-evolution feedback loop—the system does not automatically update from reviewer feedback. FARS demonstrates that Artifact-layer evolution at scale surfaces failure modes that curated demonstrations systematically suppress. AlphaEvolve achieves high-fidelity evolution by restricting scope to problems with executable automated evaluators; FARS achieves broad scope by expanding to full research pipelines, compensating with structured human review. The tension between scope and evaluability is a structural constraint.

Harness-Layer Evolution: Self-Improvement Without Touching Model Weights

Most self-improving AI agents that are practical to deploy today operate at this layer. Harness-layer evolution requires no gradient computation and no weight update, making it applicable to any Model including black-box APIs. What evolves is the Harness configuration: prompt text, routing assignments, memory structures, and tool inventories. Feedback comes from task outcomes consumed by a meta-level optimizer operating entirely on Harness components. The loop closes inside the Harness. Five papers instantiate this category through distinct mechanisms.

Those papers mainly evolve harness configuration—prompts, routing, memory, and tools. HarnessDev instead asks whether a model can write and later revise the harness code itself, then checks whether those revisions help on held-out tasks.

GEPA Original paper ↗

GEPA was developed by Lakshya A Agrawal, Omar Khattab and colleagues, with participating institutions including Stanford University, MIT. GEPA (Genetic-Pareto) argues directly against gradient-based RL as the right tool for adapting LLM agents. A trajectory contains nothing but language—instructions, reasoning chains, tool calls, compiler messages, reward signals—which is precisely what LLMs are best at understanding. Rather than compress that richness into a scalar reward and run policy gradient ascent, GEPA reflects on trajectories in natural language to diagnose problems and propose prompt mutations, then maintains a Pareto frontier of top-performing prompts to avoid greedy convergence.

On HotpotQA, HoVer, IFBench, and PUPA, GEPA outperforms GRPO (with 24,000 rollouts and LoRA fine-tuning) by an average of 10% and up to 20%, while requiring up to 35× fewer rollouts. It also outperforms MIPROv2 by over 10% across two LLMs. GEPA never updates Model weights. The information-efficiency asymmetry is the key insight: prompt optimization operates in language space, where each rollout carries a full natural-language explanation of what went wrong, whereas RL operates in weight space via scalar rewards.

EEVEE Original paper ↗

EEVEE architecture

GEPA assumes a single-benchmark setting. Real-world deployment breaks this: queries arrive from heterogeneous domains, and optimizing a prompt for one domain degrades performance on another. EEVEE extends test-time prompt learning to heterogeneous multi-dataset streams by adding a router that partitions incoming inputs into task clusters and assigns each cluster a suitable prompt configuration. The router and prompt learner are mutually dependent; EEVEE resolves this coupling through interleaved router and prompt learning phases.

EEVEE improves average multi-benchmark scores by 10.38 points over Qwen3-4B-Instruct and 24.32 points over DeepSeek-V3.2, surpassing GEPA and ACE by up to 37.2% and 48.2% respectively. In the incremental setting—benchmarks introduced one at a time—EEVEE ends at a cumulative +41.53 retention gain while GEPA and ACE end at −15.36 and −18.58. The negative numbers reveal the cross-dataset interference problem: optimizing for a new task destroys the Harness configuration that worked for prior tasks.

UI-Mem Original paper ↗

UI-Mem workflow

UI-Mem addresses how a mobile GUI agent should accumulate experience across multiple tasks and applications where online RL must contend with sparse rewards and long-horizon credit assignment. The solution is a hierarchical experience memory with three levels: high-level workflows for planning, subtask skills for execution, and failure patterns to prevent repetitive errors. Experiences are stored as parameterized templates that abstract away task-specific details, enabling reuse across applications sharing structural similarities.

UI-Mem's Stratified Group Sampling injects varying levels of memory guidance across trajectories within each rollout group. Without unguided trajectories the agent would follow memory rather than internalize it; the mix drives the unguided policy toward reproducing guided behaviors—imitation-through-contrast that makes the RL signal more informative. Model weights are updated via online RL (GRPO), but the memory component evolves independently of and in addition to the weight updates; the memory itself is a Harness-layer artifact.

Alita Original paper ↗

Alita was developed by Jiahao Qiu, Mengdi Wang and colleagues, with participating institutions including Princeton University, Tsinghua University. Alita generates new Harness components on demand rather than optimizing existing ones. When Alita encounters a task requiring a capability it lacks, it generates an MCP implementing that capability, tests it, refines it, and adds it to its reusable library. The design philosophy is minimalist: Alita ships with only a single core capability (a web agent) and a small set of general-purpose modules; everything else is generated and accumulated on demand.

On the GAIA benchmark validation set, Alita achieves 75.15% pass@1 and 87.27% pass@3, outperforming OpenAI Deep Research (67.36% pass@1). A system with fewer prebuilt components outperforms systems with more, because dynamic generation constructs precisely the right tool for each task. MCPs are generated as Artifacts—persistent, versioned outputs of task execution—but stored as Harness components that alter future agent behavior without any Model weight update.

BoundaryRouter Original paper ↗

Learning Agent Routing From Early Experience^[1] was developed by Yimin Wang, Mengdi Wang and colleagues, with participating institutions including Princeton University, Tsinghua University. BoundaryRouter addresses a routing problem: should the system route a query to lightweight direct LLM inference or escalate to full agent execution? The cold-start constraint is central—in real deployment there is no labeled training set of queries with known correct routing decisions. BoundaryRouter builds an early experience memory from a small seed set of queries executed by both the LLM and the full agent. Comparing their outputs creates a behavioral reference capturing systematic differences between the two systems. At inference time, BoundaryRouter retrieves similar past experiences and uses rubric-guided reasoning to make routing decisions.

BoundaryRouter reduces average inference time by 60.6% compared to always using the agent, while improving performance by 28.6% over always using direct LLM inference. The rubric-guided reasoning encodes structural knowledge about task difficulty rather than surface-level similarity matching, enabling generalization to out-of-domain scenarios in the RouteBench evaluation.

Model-Layer Evolution: Learning Without Ground-Truth Answer Labels

Absolute Zero results

Model-layer evolution is the most fundamental form: weights change through signal generated by the agent itself, with no external dataset of (question, answer) pairs. The defining constraint is verifiability—there must be a reliable automated mechanism to determine whether a proposed solution is correct, because the loop must close thousands of times during training with no human in the critical path. This restricts the category to domains where correctness has an executable definition: code that runs or doesn't, proofs that check out or don't.

Absolute Zero Reasoner (AZR) Original paper ↗

AZR is the paradigmatic instance. A single language model both proposes coding and reasoning tasks and solves them, using a code executor as the sole source of verifiable reward, with no external data whatsoever.

The technical mechanism separates proposer and solver roles within the same model. In the proposer role, the model generates tasks parameterized by three types: deduction (predict output from code and input), abduction (infer input from code and output), and induction (infer code from input-output pairs). In the solver role, the model receives a proposed task, generates a solution, and receives a binary reward based on execution verification. Policy gradient (RLVR) updates the weights, and the updated model then proposes harder tasks—a curriculum co-evolving with capability. AZR achieves state-of-the-art performance on coding and mathematical reasoning benchmarks, outperforming models trained on tens of thousands of in-domain human-curated examples.

The separation of proposer and solver roles within a single model prevents reward collapse. Tasks the model already solves trivially produce no gradient signal, pushing the proposer toward tasks at the boundary of current capability. The three task types provide structural diversity preventing collapse to a narrow class of easily-generated tasks.

The scope limitation is equally important: AZR is bounded by the verifiable domain. Open-ended dialogue, subjective judgment, and aesthetic evaluation cannot be automatically verified. This is an honest boundary condition on the entire Model-layer self-evolution paradigm as it currently exists.

Representative Systems: ReasoningBank, MemGen, and AgentEvolver

The eight-paper cut above is organized by layer. Three later systems extend the taxonomy with memory, latent recall, and an end-to-end training stack. They do not replace AlphaEvolve, GEPA, or AZR.

ReasoningBank Original paper ↗

ReasoningBank, from Ouyang, Yan, Hsu and colleagues at UIUC, Google Cloud AI Research, and Yale, distills generalizable reasoning strategies from an agent's self-judged successes and failures. At test time the agent retrieves relevant memories, acts, then writes new learnings back. Memory-aware test-time scaling (MaTTS) spends extra compute on each task to produce diverse traces, which in turn yield higher-quality memory. On web browsing and software engineering benchmarks the authors report better success and fewer steps than memory that stores raw trajectories or only successful routines. Code: google-research/reasoning-bank.

In this taxonomy ReasoningBank is harness-layer memory evolution. Weights stay frozen; what persists is a structured strategy bank, not a transcript dump.

MemGen Original paper ↗

MemGen, from Zhang, Fu, and Yan at the National University of Singapore, argues that neither full weight updates nor an external retrieval database captures how memory and reasoning interleave. A memory trigger decides when to invoke memory; a memory weaver turns the current state into a latent token sequence that is woven into ongoing reasoning. The backbone is not fully fine-tuned; the trigger and weaver are trained modules, typically as LoRA adapters, so some parameters do move. Across eight benchmarks the authors report gains of up to 38.22% over external memory systems such as ExpeL and AWM, and up to 13.44% over GRPO, with unplanned emergence of planning, procedural, and working-memory behaviors.

MemGen is hybrid. The write lands in trained memory modules rather than a document store or a full backbone update, which is why it appears in both the memory and model-weights rows.

AgentEvolver Original paper ↗

AgentEvolver, from Zhai, Tao, Chen and colleagues, is a self-evolving agent system aimed at the cost of RL for tool-using agents. It combines three mechanisms: self-questioning (curiosity-driven task generation, so novel environments do not require a hand-built dataset), self-navigating (experience reuse and hybrid policy guidance), and self-attributing (credit that distinguishes which states and actions actually contributed). The authors report more efficient exploration and better sample use than standard RL baselines. Code: modelscope/AgentEvolver.

AgentEvolver is the closest of the three to a full training stack: it evolves the task distribution, the exploration policy, and the learned policy together. That is why it appears under environment, policy, and model weights in the taxonomy table.

How to Build a Self-Evolving Agent Framework

A self-evolving agent framework is not "add memory." It is a write path with an evaluation gate. The papers above disagree on where the write lands; they converge on five engineering pieces.

collect trace
  -> evaluate against a frozen holdout
  -> propose one update to a named layer
  -> regression-test the candidate
  -> promote a versioned snapshot
  -> monitor live metrics
  -> roll back if retention or safety drops

Keep a versioned record for every promoted change, for example:

UpdateRecord
  id: mem-0142
  layer: memory | tools | policy | workflow | weights | environment
  parent: mem-0141
  evidence: eval-suite-v3 score, trace ids
  gate: pass | reject
  rollback_to: mem-0141

1. Feedback collection

Log the trajectory, the outcome, and the evaluator that scored it. Prefer executable checks (unit tests, sandbox traces, structured graders) over a single LLM-as-judge scalar. GEPA shows that natural-language reflection is useful in addition to a score, not instead of one. If you cannot say what counted as success, you cannot say the agent evolved.

2. Memory and update loop

Decide the write target before you store anything: strategy memory (ReasoningBank), tool inventory (Alita), prompt population (GEPA), or weights (AZR, AgentEvolver). Retrieve before acting, write after judging, and keep failed traces—success-only memory repeats the same blind spots. Version every write. A memory store with no schema is a log, not a loop.

3. Evaluation harness

Hold out tasks the updater cannot see. Measure retention across domains, not only the latest benchmark; EEVEE exists because prompt evolution on one stream destroys another. Separate automation (the agent finished) from durable improvement (the next run starts better). If the only eval is the training distribution, you are overfitting a loop.

4. Safety gates

Do not let the agent promote a change that failed evaluation, changed permissions, or cannot be explained. Put a gate in front of tool installation, prompt replacement, and any weight update. Self-play and self-questioning can invent tasks that are easy to reward-hack; treat proposer output as untrusted data.

5. Rollback and versioning

Every promoted prompt, memory item, tool, and checkpoint needs an ID and a rollback command. Artifact-layer systems already do this with program databases; harness-layer systems often skip it and then cannot undo a bad prompt. Without rollback, a bad write is hard to undo; treat versioning as part of the loop, not an afterthought.

A minimal production skeleton: collect traces → judge with a frozen eval set → write to one named layer → promote only behind a gate → keep the previous version. That is the implementation counterpart of the taxonomy.

Boundaries, Open Problems, and Real-World Deployment Constraints

Verifiability is the master constraint. Both Artifact-layer and Model-layer evolution require automated correctness signals: AlphaEvolve's code executor, AZR's verification environment. When such signals are unavailable, neither category can function without human evaluation in the loop. FARS demonstrates what happens at the boundary: automated metrics for research quality are insufficient, so evaluation relies on 282 volunteer reviews—an honest acknowledgment that automated verifiability for open-ended scientific writing does not yet exist at acceptable quality. Harness-layer evolution remains the most broadly applicable category for domains where verification is soft or subjective.

Learning Agent Routing workflow

Cold-start scarcity affects all categories differently. BoundaryRouter confronts routing cold-start—no prior routing labels exist for new deployments—and builds a behavioral reference from a seed set. EEVEE confronts prompt cold-start under domain shift—as new benchmark domains enter the stream, existing configurations have no experience with them. FARS confronts quality cold-start at scale—in early deployment, there is no evidence about which experimental configurations produce strong papers. Each paper develops a specific mitigation, but none eliminates the cold-start problem; they manage it.

Proxy optimization and faithfulness failures are structurally analogous. AZR faces reward hacking risk: a sufficiently powerful proposer could generate tasks that are technically executable-verifiable but solved by surface pattern matching rather than actual reasoning. FARS faces faithfulness failures: the writing agent occasionally overclaims or inadequately represents experimental evidence, optimizing for plausible-sounding language rather than accurate scientific description. Both represent the same structural issue—optimizing a measurable proxy at the expense of the intended objective.

The Model ceiling limits Harness evolution. Most Harness-layer approaches in this survey operate on a frozen Model with a capability ceiling: there are tasks it cannot solve regardless of prompt quality or memory richness. UI-Mem is the hybrid exception, because it also updates weights. Once Harness optimization saturates, further gains require Model-layer updates. This motivates co-evolution architectures in which Harness improvements generate training data for Model updates, which in turn enable further Harness optimization. No paper in this survey implements full three-layer co-evolution, but the taxonomy points toward it as the natural next step.

Evaluation methodology is itself an open problem. A central difficulty for self evolution in AI agents is showing that a reported gain is durable improvement rather than automation, one-off adaptation, or benchmark overfitting. FARS's structured volunteer reviews, AlphaEvolve's formal correctness proofs, and AZR's out-of-distribution benchmark transfer are not interchangeable. Formal proofs are available only for well-specified mathematical problems. Volunteer reviews require significant human effort and carry inter-reviewer variance. Benchmark transfer measures a specific proxy for generalization but says nothing about real-world utility. FARS's decision to publish all 166 papers with all 282 reviews—including failures—is a methodological commitment that the field would benefit from adopting more broadly.

Multi-domain heterogeneity requires explicit architectural accommodation. EEVEE prevents prompt specialization for one domain from destroying performance on others; BoundaryRouter prevents routing experience from failing to transfer to out-of-domain queries. In both cases the solution involves explicit structure in the Harness—a router partitioning the input space, rubric-guided reasoning encoding structural knowledge—rather than hoping a single configuration generalizes.

Conclusion

What evolves, where feedback comes from, and where the loop closes each have a clean answer per category.

In Artifact-layer evolution (AlphaEvolve, FARS), the Artifact population evolves while the agent stays fixed. Feedback comes from automated evaluators—code executors, benchmark scores—or structured human review when automation is insufficient. The loop closes at the Artifact store: surviving Artifacts seed the next generation, and neither Model weights nor Harness configuration are modified. The power of this category—genuine discovery at scale—is inseparable from its prerequisite: reliable evaluation.

In Harness-layer evolution (GEPA, EEVEE, Alita, BoundaryRouter), the Harness configuration evolves while Model weights remain frozen. Feedback comes from task outcomes consumed by meta-level optimizers operating in language space (GEPA, EEVEE), converted into reusable tools (Alita), or encoded as behavioral reference (BoundaryRouter). The loop closes inside the Harness: updated prompts, memory entries, routing policies, and tool libraries alter future behavior without any gradient flowing into the Model. This is the most broadly applicable category because it requires no weight access. UI-Mem is hybrid: its memory store is a harness artifact, but online RL also moves weights.

In Model-layer evolution (AZR), parametric weights evolve through a fully internal self-play circuit. Feedback comes exclusively from verifiable execution outcomes—compiler results, test-case pass rates—that substitute for human annotation. The loop closes inside the training process: the Model proposes tasks, solves them, receives execution-verified rewards, updates its own weights, and proposes harder tasks aligned with its new capability level. The result matches curated supervised training in well-verifiable domains with zero external data, bounded precisely by what can be automatically verified.

Awesome Self-Evolving Agents: Papers, Repos, and Tools

A short reading and tooling list, not a second bibliography. Use it when you want the survey papers, the named systems, or a repo to clone.

Surveys and taxonomies

• A Survey of Self-Evolving Agents (Gao et al., arXiv:2507.21046) — what / when / how / where to evolve. Companion list: CharlesQ9/Self-Evolving-Agents.
• A Comprehensive Survey of Self-Evolving AI Agents (Fang et al., arXiv:2508.07407) — lifelong agentic systems and the input–agent–environment–optimiser loop. Companion list: EvoAgentX/Awesome-Self-Evolving-Agents.
• A Taxonomy of Self-evolving Agents — a short public taxonomy organized by what the agent changes.

Named systems

• ReasoningBank — paper, code
• MemGen — paper, code
• AgentEvolver — paper, code
• The eight systems in the table above — AlphaEvolve, FARS, GEPA, EEVEE, UI-Mem, Alita, BoundaryRouter, Absolute Zero

Implementation guides

• OpenAI Cookbook: Autonomous Agent Retraining — evals, a retraining loop, and LLM-as-a-judge in an executable notebook.
• This page's build checklist — feedback, memory writes, eval harness, safety gates, rollback.

Data and Citation

The comparison table above and the full reference list are available as downloadable files for reuse in your own reading notes, slides, or literature reviews.

• Comparison table (CSV, 8 papers)
• All references (BibTeX, 13 entries)

Suggested citation

AgentsPulse Editorial Team. "Self-Evolving Agents: Survey, Taxonomy, and How Self-Improving AI Agents Work." AgentsPulse, September 3, 2026. https://agentspulse.github.io/tutorials/self-evolving-agents-review-en/

BibTeX for this survey

@misc{agentspulse2026selfevolving,
  title        = {Self-Evolving Agents: Survey, Taxonomy, and How Self-Improving AI Agents Work},
  author       = {{AgentsPulse Editorial Team}},
  year         = {2026},
  howpublished = {AgentsPulse},
  note         = {Survey, taxonomy, and implementation guide for self-evolving agents},
  url          = {https://agentspulse.github.io/tutorials/self-evolving-agents-review-en/}
}

Original Papers

1. Learning Agent Routing From Early Experience. arXiv:2605.07180. 2026. View on arXiv.
2. FARS: A Fully Automated Research System Deployed at Scale. arXiv:2606.31651v1. 2026. View on arXiv.
3. EEVEE: Towards Test-time Prompt Learning in the Real World for Self-Improving Agents. arXiv:2606.11182v1. 2026. View on arXiv.
4. UI-Mem: Self-Evolving Experience Memory for Online Reinforcement Learning in Mobile GUI Agents. arXiv:2602.05832v1. 2026. View on arXiv.
5. GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457v1. 2025. View on arXiv.
6. AlphaEvolve: A coding agent for scientific and algorithmic discovery. arXiv:2506.13131v1. 2025. View on arXiv.
7. Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution. arXiv:2505.20286v1. 2025. View on arXiv.
8. Absolute Zero: Reinforced Self-play Reasoning with Zero Data. arXiv:2505.03335v2. 2025. View on arXiv.
9. A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence. arXiv:2507.21046. 2025. View on arXiv.
10. A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. arXiv:2508.07407. 2025. View on arXiv.
11. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. arXiv:2509.25140. 2025. View on arXiv.
12. MemGen: Weaving Generative Latent Memory for Self-Evolving Agents. arXiv:2509.24704. 2025. View on arXiv.
13. AgentEvolver: Towards Efficient Self-Evolving Agent System. arXiv:2511.10395. 2025. View on arXiv.