← back
Fast Models Need Slow Developers — Sarah Chieng, Cerebras
Takeaway
Faster coding models make deliberate development and verification more valuable because unchecked errors can accumulate faster too.
Summary
- The talk presents Codex Spark at a reported 1,200 tokens per second and argues that faster generation can amplify poor development habits.
- Inference acceleration spans on-chip memory, separating compute-heavy prefill from memory-heavy decode, model architecture, and KV-cache reuse.
- Large one-shot prompts, huge commits, and many unattended agents can create more code than developers can verify.
- Model selection increasingly needs to account for speed alongside intelligence and cost.
cerebrasfast-inferencecoding-workflows
Original description
Codex Spark, a model Cerebras built with OpenAI, generates code at 1,200 tokens per second. The Sonnet and Opus families run at 40 to 60. At that 20x difference, a context window that used to take ten minutes to fill now takes 30 seconds, and every habit built around slow generation starts producing technical debt at a scale nobody has dealt with before. Sarah Chieng from Cerebras covers what the playbook looks like in this regime. Validation and linting at every step is now instant, so there is no excuse not to run it continuously. Generating 75 component variations across five sub-agents and cherrypicking the best one becomes practical where it was not before. And when context burns in 30 seconds, a four file external memory system (agents, plan, progress, verify) is what keeps each new session from starting over instead of from scratch. Speaker info: https://x.com/sarahchieng / sarah-chieng-888595139 Timestamps: 0:00 - Introduction to the impact of fast AI code generation 2:29 - Historical context of model speeds 3:10 - Why AI inference speeds are increasing (Hardware/Stack optimization) 7:05 - The current developer landscape and risks of "slob" 8:27 - Playbook: Orchestrating models and sub-agents 9:56 - Playbook: Validation and automated testing 10:47 - Playbook: Cherrypicking and variety in output 12:07 - Playbook: Adopting a real-time collaborative mental model 12:53 - Playbook: Avoiding "slob" and active steering 13:54 - Playbook: Continuous refactoring 14:30 - Playbook: Context management and external memory systems