LLM WORLD
SC

The open instruction ecosystem

Stanford CRFM · · Research

Within two weeks, Stanford released Alpaca (LLaMA 7B fine-tuned on 52K synthetic instructions for $600) and LMSYS released Vicuna (LLaMA 13B fine-tuned on 70K ShareGPT conversations for $300). Both demonstrated that instruction-tuning a base model was trivially cheap and replicable.

Why it matters

Proved that instruction-tuning was trivially cheap. The "base model + synthetic instructions" recipe became the template for every open instruction-tuned model. Also introduced GPT-4-as-judge evaluation.

This is LLM World's historical reading, not a claim by Stanford CRFM. The facts above and the sources below are what it rests on.

Sources

Record status: partly verified · confidence 70/100

A composite milestone: Alpaca (13 March 2023) and Vicuna (30 March 2023) are seventeen days apart and the record is dated to the first. Neither Stanford's post nor LMSYS's states Alpaca's date in its text — the 13 March came from the Stanford URL path, which is not a claim the page makes. Wikipedia states it, so it is now cited. The Vicuna source states its own date, 30 March, which is the later half of the same transition.