← back
Your Coding Agent Should Do AI System Engineering — Ben Burtenshaw, Hugging Face
Takeaway
Pair coding agents with maintained examples, hardware benchmarks, and reusable distribution formats to make AI systems optimization practical.
Summary
- Coding agents can tackle GPU kernels, model training, and automated research when repositories, tools, and evaluation interfaces are standardized.
- Kernel optimization often targets memory movement and overhead, increasing useful computation per memory access rather than focusing only on arithmetic throughput.
- Hugging Face's kernels library and Hub repositories package compatibility metadata and distribution for hardware-specific implementations.
- Maintained skills provide examples, benchmarking scripts, and tests; a Qwen 3 8B kernel experiment on H100 reportedly achieved a 94% speedup.
- The open-source upskill tool generates and evaluates skills to compare model performance and token use on the same task.
cuda-kernelshugging-facecoding-agents
Original description
An agent written RMSNorm kernel hit 1.88x speedups on H100s. A finetuned Qwen3 0.6B hit 35% on LiveCodeBench. Neither result required a systems engineer. Just coding agents with the right skills loaded. Ben Burtenshaw from Hugging Face walks through three levels: using Claude Code interactively to write and benchmark CUDA kernels distributed as versioned repos on the Hub, a zero-shot task where an agent finetunes a model end to end from a single prompt, and a multi agent research lab running parallel experiments overnight on Hub compute while a reporter agent pushes results to a live Trackio dashboard. The through line is skills: file based context that turns a zero shot failure into a few shot workflow. CUDA programming and ML training pipelines were deep specializations that took years. Skills compress that timeline to hours. Speaker info: https://x.com/ben_burtenshaw / ben-burtenshaw https://github.com/burtenshaw Timestamps: 0:00 Introduction to AI Systems Engineering 1:59 Boss 1: Writing and Distributing CUDA Kernels 3:48 Efficiency in Deep Learning 6:08 Using Skills for Agentic Workflows 8:37 Benchmarking and Evaluating Skills with Upskill 9:26 Boss 2: End-to-End Fine-tuning of LLMs 10:16 Boss 3: Multi-Agent Auto Research Labs 12:09 Architecture of the Multi-Agent Research System 13:40 Implementing the Research Agent in OpenCode 15:28 Monitoring Experiments with Trackio 16:45 Final Takeaways and Conclusion