One-sentence Summary

Microsoft and Peking University researchers present WizardLM, which applies Evol-Instruct to iteratively refine an initial instruction set into increasingly complex open-domain instructions for fine-tuning LLaMA, and both automatic and human evaluations demonstrate that WizardLM outperforms Alpaca and Vicuna, confirming that instruction data evolved via Evol-Instruct significantly boosts instruction-following performance.

Key Contributions

• Evol-Instruct is a method that automatically generates diverse open-domain instruction data at scale by iteratively rewriting seed instructions into more complex and diverse versions.
• WizardLM, a model fine-tuned on Evol-Instruct data, outperforms open-source baselines Alpaca and Vicuna, with notable gains in code, math, GPT-4 evaluations, and human evaluations.
• An initial investigation shows that instruction complexity is important for improving supervised fine-tuning performance of large pre-trained language models.

Introduction

Large language models are widely used for natural language tasks, but they often struggle to follow real user instructions. Prior instruction-tuning efforts either relied on closed-domain datasets with limited diversity and single-task instructions, or on expensive human-created open-domain data that tends to skew toward easier examples. The authors introduce Evol-Instruct, an automatic method that uses a large language model to iteratively generate more complex and diverse open-domain instructions, and they show that WizardLM, a model fine-tuned on this evolved data, substantially outperforms open-source baselines such as Alpaca and Vicuna.

Dataset

The authors construct the dataset through an iterative evolution process:

• Base dataset (seed): The Alpaca instruction-tuning dataset of 52k instruction-response pairs, serving as the starting point $D^{(0)}$D(0).
• Evolution rounds: They perform M = 4 evolution passes. In each round, every instruction is evolved by applying a randomly chosen prompt from a set of six evolution prompts (five “in-depth” and one “in-breadth”). The new response is then generated by ChatGPT (temperature=1, top‑p=0.9, max tokens=2048).
• Full evolved dataset: After four rounds, the collection grows to 250k instruction-response pairs.
• Training subset: For training WizardLM, they randomly sample 70k pairs from the 250k set, matching the amount of data used by Vicuna. No additional filtering is applied.
• Data characteristics: The instructions are open-domain, with tasks and inputs often blended together without a strict separation between the instruction part and the input. No cropping or special metadata construction is employed beyond the evolution prompts.

Method

The authors propose Evol-Instruct, a method designed to automatically evolve instruction data to improve the complexity, diversity, and difficulty of open-domain instruction datasets. The pipeline primarily consists of two core components: the Instruction Evolver and the Instruction Eliminator.

As shown in the figure below:

The instruction data evolution process begins with an initial instruction dataset $D^{(0)} = \{(I_k^{(0)}, R_k^{(0)})\}_{1 \leq k \leq N}$D(0)={(Ik(0)​,Rk(0)​)}1≤k≤N​, where $I_k^{(0)}$Ik(0)​ represents the $k$k-th instruction, $R_k^{(0)}$Rk(0)​ is the corresponding response, and $N$N is the total number of samples. In each evolution step, the authors upgrade the instructions $I^{(t)}$I(t) in the current dataset $D^{(t)}$D(t) to $I^{(t+1)}$I(t+1) by prompting a large language model (LLM) with specific Evol-Instruct prompts. The LLM then generates corresponding responses $\mathring{R}^{(t+1)}$R˚(t+1) for these newly evolved instructions, yielding an evolved dataset $D^{(t+1)}$D(t+1). By iteratively performing $M$M evolutions, the system sequentially generates $M$M evolved datasets.

The Instruction Evolver utilizes two distinct types of prompts to enhance the instruction pool: In-Depth Evolving and In-Breadth Evolving. In-Depth Evolving aims to make instructions more complex and difficult through five specific strategies: adding constraints, deepening, concretizing, increasing reasoning steps, and complicating input. The core objective is to rewrite prompts into more complex versions that remain reasonable and understandable for humans, while being slightly harder for AI systems. To prevent a sudden spike in difficulty that could harm model generalization, the authors restrict the difficulty increase to be gradual, limiting additions to a maximum of 10 to 20 words per evolution. In-Breadth Evolving focuses on expanding topic and skill coverage to enhance overall dataset diversity. It generates completely new, long-tailed instructions based on the given ones, addressing the typical lack of diversity in small-scale open-domain datasets.

Following instruction evolution, the pipeline proceeds to Response Generation and Elimination Evolving. The same LLM used for evolving generates responses for the new instructions. Subsequently, the Instruction Eliminator filters out instructions that fail to evolve properly. The authors classify four situations as evolution failures: the evolved instruction provides no information gain compared to the original; the LLM struggles to generate a response, which is often indicated by short responses containing the word "sorry"; the generated response consists solely of punctuation and stop words; or the evolved instruction explicitly copies phrases from the evolving prompt. Failed instructions are either discarded or placed back into the pool for potential successful upgrading in the next epoch.

Once all evolution epochs are complete, the authors merge the initial instruction dataset with the evolved data from all epochs and randomly shuffle the samples to create the final fine-tuning dataset. This ensures an even distribution of instructions across varying difficulty levels, maximizing the smoothness of model fine-tuning. To validate that performance gains stem from the Evol-Instruct methodology rather than merely increased data volume, they randomly sample an equal amount of data from this merged pool to match the training baselines. The foundation LLM is then fine-tuned on this curated dataset using a specific chat prompt format.

Experiment

WizardLM is evaluated against Alpaca, Vicuna, ChatGPT, and other open-source models using a combination of automatic benchmarks (covering knowledge, reasoning, code, math, and GPT-4-based assessments) and human pairwise comparisons on a diverse, manually crafted test set. The model consistently outperforms its same-sized peers, with notable gains in math, code, and overall quality, and human evaluations confirm these advantages with high inter-annotator agreement. Ablation experiments further demonstrate that the Evol-Instruct method benefits from richer seed data and larger instruction sets, is not tied to a specific base model or evolution LLM, and generalizes well across different pre-trained architectures.

WizardLM-13b achieves the highest average score among open-source 13b models, outperforming all baselines on ARC, HellaSwag, HumanEval, GSM8k, AlpacaEval, and WizardEval. While ChatGPT-3.5 remains the strongest overall, WizardLM-13b surpasses it on TruthfulQA and nearly matches it on HellaSwag, demonstrating competitive commonsense and truthfulness capabilities. The model shows particular strength in code generation and math reasoning, where it roughly doubles the performance of the next best open-source model. WizardLM-13b exceeds ChatGPT-3.5 on TruthfulQA (50.55 vs. 47.0), indicating a lower propensity to reproduce falsehoods. On HumanEval, WizardLM-13b achieves a pass@1 of 24.0, roughly double Vicuna-13b's 12.5 and more than double Alpaca-13b's 9.2. WizardLM-13b leads all open-source models on AlpacaEval with a score of 75.31, substantially ahead of Vicuna-13b's 70.43. WizardLM-13b's GSM8k math reasoning score of 37.15 is over 50% higher than Vicuna-13b's 24.34, the next best open-source result. Open-source baselines like Alpaca-13b and Baize-13b severely underperform on code generation and math, scoring below 10 on HumanEval and GSM8k.

Ablation experiments reveal that using ShareGPT as seed data improves overall performance but reduces GSM8k accuracy due to fewer math examples. Scaling the evolved dataset to 250k yields consistent gains, and the evol-instruct method works effectively with alternative evolution models like Llama-2-70B-Chat and various base architectures such as Mistral-7B, consistently outperforming training on Supernatural Instructions. WizardLM-13b with ShareGPT seed achieves a higher average score than the original Alpaca-seeded version, but its GSM8k score drops because ShareGPT contains a smaller proportion of math instructions. Increasing the evolved data from 70k to 250k boosts performance on knowledge-intensive benchmarks like MMLU and GSM8k. Substituting ChatGPT with Llama-2-70B-Chat as the evolution model still yields competitive results, though slightly below the ChatGPT-based baseline. Training on Supernatural Instructions leads to the lowest average across all metrics, confirming the advantage of evol-instruct generated data. Applying the method to the Mistral-7B base produces the highest overall average (65.81), surpassing the Llama-13b-based WizardLM and demonstrating cross-architecture generalization.

The experiments demonstrate that the Evol-Instruct method produces a 13B model that leads open-source alternatives across diverse reasoning benchmarks, particularly excelling in code generation and math reasoning, and even surpassing ChatGPT-3.5 on truthfulness. Ablation studies confirm that scaling the evolved dataset and using ShareGPT seed data improve overall performance, though math accuracy may decline if seed data lacks math examples, while the method generalizes effectively across different evolution models and base architectures, with Mistral-7B achieving the highest overall results.