Fast Models Need Slow Developers — Sarah Chieng, Cerebras

Sarah Chieng, Head of Developer Experience at Cerebras, argues that the 20x jump in AI inference speed — from the 40–60 tokens/second typical of the Sonnet/Opus families to the 1,200 tokens/second achieved by Codex Spark (a model Cerebras built jointly with OpenAI) — does not automatically make developers better. Instead, it amplifies whatever habits they already have: good practices become compounding advantages, while sloppy ones produce technical debt at a previously unimaginable rate. The core thesis is that fast models demand slow, deliberate developers.

Chieng opens with a brief history of inference speed, noting that across the major model families — Gemini, Claude, GPT, Sonnet — speeds have stayed within 50–150 tokens/second for roughly two years, keeping the Y-axis of any speed chart relatively static. Codex Spark forced a rescaling of that axis. She then explains the hardware and algorithmic reasons behind the acceleration: the memory wall (memory movement accounts for 50–80% of inference latency on NVIDIA GPUs with off-chip HBM); disaggregated inference (splitting compute-bound prefill from memory-bound decode onto separate, purpose-optimized hardware — the strategic logic behind NVIDIA's $20 billion acquisition of Groq and the Cerebras/AWS partnership with the Cerebras wafer and AWS Trainium); Mixture of Experts architectures that activate only a subset of experts per token, achieving large-model intelligence at small-model compute cost; pruning techniques like REAP (Router-Weighted Expert Activation Pruning) that eliminate unused experts entirely; and KV cache reuse at the inference software layer.

The practical playbook Chieng prescribes has six pillars. First, model orchestration: use a large, high-intelligence model such as GPT-5.4 for long-horizon planning, then delegate execution to Codex Spark sub-agents. Successful trajectories can be captured as reusable skills, verified once by the slow model and then replayed cheaply by the fast model. Second, continuous validation: at 1,200 tokens/second, running test suites, linters, pre-commit hooks, diff reviews, and browser-based QA automations at every step is effectively free — there is no longer any reason to defer these checks to the end of a session. Third, cherrypicking through variety: instead of generating one navbar component, generate 15 variants in the same wall-clock time a slower model would take for one; for higher variety needs, spawn five sub-agents each producing 15 variants to yield 75 candidates and select the best. Chieng frames this as artificially inducing taste into model output — a workaround for models' intrinsic lack of aesthetic judgment that is less expensive than writing exhaustive prompts or curating examples manually. Fourth, real-time collaboration: context that used to take 10 minutes to fill now reaches compaction in 30 seconds (10 minutes ÷ 20x speedup). Developers must sit alongside the model, make active steering decisions, and treat the interaction as pair programming rather than as a job dispatched and reviewed later. Fifth, anti-slob practices: developers should give explicit guardrails — banning the model from deleting files, capping diff size, restricting the model to read/write only, issuing targeted steering directives like "only change this, don't touch types yet." The goal is maintaining comprehension of what the code actually does, not just trusting the output. Sixth, context management via a four-file external memory system: agents.md (sub-agent definitions), plan.md (a step-by-step checklist generated at session start), progress.md (completed vs. remaining tasks so each new session resumes rather than restarts), and verify.md (a per-step quality gate). This system externalizes memory so that 30-second context windows do not reset all accumulated state.

Chieng closes by emphasizing that the real prize is not raw speed but a fundamentally improved developer experience — one where verification, refactoring, and quality checks are no longer painful interruptions but continuous, automatic, and nearly costless steps in the workflow.