How We Built Zeta2: Training an Edit Prediction Model in Production — Ben Kunkle, Zed

Ben Kunkel, edit predictions lead at Zed, presents a detailed account of how the Zeta 2 (Zed 2) model was trained — a small, specialized fine-tuned model that predicts the next code edit a developer will make in real time, running on every keystroke. Because the model must respond at keystroke speed, a large frontier model is impractical; instead, Zed trains a compact student model via knowledge distillation from a frontier teacher.

The core training pipeline begins with opt-in production edit traces collected from Zed users. Each trace is a snapshot that captures the surrounding code context at prediction time: recent edits, cursor position, nearby type and variable definitions, and compiler diagnostics. This snapshot-based approach is a significant design advantage because it preserves all contextual signals exactly as they existed when the prediction was requested. These snapshots are fed to a frontier teacher model, which is prompted to predict the next edit the user will make.

Generating high-quality teacher labels at scale is non-trivial. Prompting a frontier model 100,000 times yields inconsistent outputs, so the team invested heavily in prompt engineering. They also introduced a repair step: a set of static heuristics identifies bad teacher predictions — for example, predictions that simply undo what the user just typed, or that violate the declared editable region boundary — and a second frontier model call attempts to fix them before the label is used. All pipeline stages are structured as JSONL files where each line is a complete JSON object, with successive stages adding or rearranging fields. This design makes the pre-distillation stages fully cacheable and reusable across experiments, decoupling data preparation from prompt-formatting choices like whether to include diagnostics or how much edit history to provide.

Training runs typically use 100,000 examples, with smaller exploratory experiments using 10,000–50,000. To evaluate model quality, the team uses delta char-F, an n-gram variant of Levenshtein distance, and a metric called reversal ratio — measuring how often the model predicts exactly the inverse of what the user just typed — as the key diagnostic for detecting regressions before shipping. Offline evals run against three distinct frontier teacher predictions per example, since many coding edits have multiple valid continuations. Production A/B experiments are managed through an internal dashboard tracking acceptance rate, latency, and kept rate, with traffic split controls (e.g., 15% to a new experiment, the remainder to the current live model).

The most technically interesting section concerns settled data: the idea of waiting until a user stops editing a region (defined as a 10-second pause) to capture the final state as ground truth. This is potentially very valuable for curriculum-style training, as it provides the "real" answer rather than a teacher-model approximation. However, it is noisy — agents may rewrite code, users may change their minds, and the settled state may bear no relationship to the prediction input. To filter noise, Zed originally generated 10 frontier teacher predictions per example and used Levenshtein distance to check whether any were close to the settled state; proximity indicated the example was both predictable and non-noisy. At 100,000 training examples, this required 1 million frontier model calls — prohibitively expensive. The solution: because Zeta 2 now approaches frontier teacher quality, they can instead run the student model checkpoint 50 times per example at negligible cost and apply the same proximity filter. The target training examples are those in the middle of the distance distribution: far-from-settled examples are discarded as noise, near-settled examples are too trivial, and the interesting middle region represents genuinely hard, non-obvious predictions — including post-training-cutoff code the student has never encountered.