One-sentence Summary

Researchers at Jilin University propose EVISKILL, an evidence-driven framework for continual LLM skill evolution that stores execution observations as Replayable Evidence Cards, synthesizes edits explicitly linked to supporting contexts, verifies those edits through targeted replay, and provisionally retains supported edits for further refinement before global validation incorporates them into final skills, with effectiveness demonstrated on three interactive benchmarks across six LLM backbones.

Key Contributions

• EVISKILL is an evidence-driven skill evolution framework that organizes execution observations into Replayable Evidence Cards and links skill edits to their supporting behavioral contexts.
• Replay-guided edit verification tests whether edits produce their intended behavioral effects through re-execution, while cross-epoch evidence propagation retains and refines supported edits even when the overall revision is rejected.
• Experiments on three interactive benchmarks across six LLM backbones show consistent improvements over strong skill evolution baselines, with analyses indicating that evidence-grounded replay filters unsupported revisions and cross-epoch propagation enables prior evidence to support continued refinement.

Introduction

LLM agents increasingly handle interactive tasks such as tool use, web navigation, and long-horizon decision making, where reusable external skills provide procedural guidance without changing model parameters. Prior experience-driven skill evolution methods extract updates directly from execution trajectories, but execution experience is local and context-dependent, so edits can overfit observed cases or capture incidental behavior. Rejected revisions may also contain useful edits that are discarded. The authors introduce EVISKILL, an evidence-driven framework that organizes continual skill evolution into evidence construction, behavioral verification, and cross-epoch refinement. EVISKILL grounds candidate edits in replayable execution evidence, re-executes referenced behaviors to verify intended effects, and preserves uncertain or unresolved evidence for reconsideration across future epochs. Experiments on three interactive benchmarks across multiple LLM backbones show consistent improvements over strong skill evolution baselines.

Method

The authors introduce EVISKILL, a framework that organizes skill evolution around Replayable Evidence Cards. These cards record proposed skill corrections alongside their supporting execution evidence, enabling a structured pipeline for continuous improvement. The overall architecture consists of three interconnected phases: Evidence-Grounded Edit Synthesis, Replay-Guided Edit Verification, and Cross-Epoch Evidence Propagation.

As shown in the figure below:

In the first phase, Evidence-Grounded Edit Synthesis, the system distinguishes between the Validated Skill, which is updated only upon successful global validation, and the Working Skill used for ongoing interactions. At epoch $e$e, the Working Skill $S_e^W$SeW​ combines the latest Validated Skill $S_e^V$SeV​ with a Provisional Edit Ledger $P_e$Pe​, which contains replay-supported edits retained for further refinement:

where $\oplus$⊕ applies the edits sequentially. The Action Agent collects trajectories using $S_e^W$SeW​, providing the foundational evidence for skill revision. An LLM evidence extractor analyzes the Working Skill and its trajectories to propose corrections, recording each proposal as a Replayable Evidence Card. A card $C_i$Ci​ is formally defined as:

where $i$i is a persistent identifier, $\mathcal{G}_i$Gi​ represents supporting trigger ranges from trajectories, and $d_i$di​ is the proposed correction. The framework extracts Trajectory Evidence Cards from current interactions and Contrastive Evidence Cards from adjacent-epoch trajectory comparisons to identify persistent deficiencies. These cards are combined into an Evidence Pool, grouped into Evidence Windows based on compatible contexts, and processed by an LLM editor to synthesize a candidate edit set $\mathcal{U}_e$Ue​.

The second phase, Replay-Guided Edit Verification, ensures that synthesized edits produce their intended behavioral effects. For each candidate edit $u \in \mathcal{U}_e$u∈Ue​, the system retrieves its supporting cards and replays the associated trigger ranges. By reproducing the prefix of the source trajectory, the framework reconstructs the execution state and re-executes the segment under the modified skill $S_e^W \oplus u$SeW​⊕u. An LLM evaluator assesses the source and replayed segments, returning a decision to accept, reflect, or reject the edit. If the decision is to reflect, the editor generates a revised edit $u^\prime$u′ based on the feedback, which then undergoes another round of replay and evaluation. Successfully verified edits are consolidated into an ordered collection, yielding a candidate revision $\widetilde{S}_e$Se​ for global validation.

The final phase, Cross-Epoch Evidence Propagation, governs how skills and evidence are updated across epochs. Global validation compares the candidate revision $\widetilde{S}_e$Se​ against the current Validated Skill $S_e^V$SeV​ on a validation set. If $\widetilde{S}_e$Se​ demonstrates superior performance, it replaces $S_e^V$SeV​, the Provisional Edit Ledger is cleared, and the incorporated edits become the new Tracked Edits for future cross-epoch comparisons.

If the candidate revision is globally rejected, the framework employs post-rejection replay to salvage effective edits. Each consolidated edit is replayed independently under the original Validated Skill $S_e^V \oplus v$SeV​⊕v to determine if it remains effective without the provisional edits. Edits that pass this isolated evaluation are retained in the updated Provisional Edit Ledger for the next epoch, ensuring that locally beneficial corrections are not discarded due to global incompatibilities. Furthermore, the lifecycle of the Evidence Cards is dynamically managed. Cards supporting accepted edits are archived, while those supporting rejected edits may be marked as stale or protected for future cross-epoch comparisons. After $E$E evolution epochs, the final Validated Skill is frozen for inference, ensuring that only thoroughly validated and replay-supported skills are deployed.

Experiment

The preliminary study identifies two weaknesses in existing skill evolution: edits based on experience often fail to produce intended execution effects, and globally rejected revisions may still contain useful constituent edits. The main experiments evaluate EVISKILL across App-World, ScienceWorld, and ALFWorld with multiple LLM backbones, showing that grounding skill updates in execution evidence yields broad gains and that evolving a skill does not guarantee improvement over its initial version. Ablation and mechanism analyses confirm that replay-guided edit verification improves behavioral corrections, while cross-epoch evidence propagation preserves supported edits and unresolved evidence, with many edits from rejected revisions later incorporated into accepted skills and evidence cards reused across epochs.

Execution-grounded skill evolution yields broad task-success gains across interactive benchmarks, improving over the no-skill baseline in every setting and ranking among the top methods in most comparisons. Several skill-evolution baselines regress on ScienceWorld, falling below both the no-skill and initial-skill accuracy, while the execution-grounded approach avoids this pattern. The results indicate that verifying updates through observed behavior helps preserve useful skills while incorporating new experience. Execution-grounded skill updates achieve the best accuracy in most model-dataset settings and improve over no skill in all reported settings. On ScienceWorld, several evolution baselines underperform the initial skill, with some dropping below the no-skill baseline, whereas execution-grounded updates improve over the initial skill in nearly all settings.

The experiments evaluate execution-grounded skill evolution on interactive benchmarks, testing whether updates grounded in observed behavior improve task success. The approach consistently improves over the no-skill baseline across all settings and ranks among the top methods, while several skill-evolution baselines regress on ScienceWorld and fall below both no-skill and initial-skill accuracy. Execution-grounded updates also achieve the best accuracy in most model-dataset settings and improve over the initial skill in nearly all ScienceWorld settings. These results suggest that grounding skill updates in execution helps retain useful skills while integrating new experience.