← back
Special Topics in Kernels, RL, Reward Hacking in Agents — Daniel Han, Unsloth
Takeaway
Assess agent reliability using success thresholds and checks for reward hacking, rather than headline benchmark scores alone.
Summary
- Unsloth distributes dynamically quantized language and diffusion models and fixes bugs across model implementations and training stacks.
- METR task horizons depend strongly on the required success rate: the talk contrasts roughly 16-hour tasks at 50% success with roughly three-hour tasks at 80%.
- Repeated independent attempts can raise aggregate success probability, but benchmark cheating can drastically inflate apparent capability.
- Improving scores across coding, mathematics, and knowledge benchmarks raises the question of whether benchmark saturation actually establishes general intelligence.
benchmarksreward-hackingunsloth
Original description
An advanced seminar (good prerequisites: Daniel's 2024 and 2025 hit AIE workshops, but all are welcome!) PLS WATCH: / @aidotengineer Timestamps: 0:00 Introduction to Unsloth and model distribution 2:32 The State of AI: Meter plots and performance trends 20:26 Open Source vs. Closed Source models 38:51 Throughput maxing and accuracy minimizing 1:03:00 Benchmarking and cheating in AI 1:37:49 Kernels and algorithmic improvements 2:04:16 Reinforcement learning primer 2:05:19 Reward hacking and AI agents Viral Quotes & Pull Quotes: "If you make the model 86% smaller, it does not get 86% dumber... it only gets 14% less dumb." (29:52) "Reinforcement learning is terrible, but everything else is even worse." (20:43) "The model becomes not important anymore; it's the harness or the tool that is actually the most important thing." (38:23) "If a model can finish a task that takes a human 16 hours, can a model finish that task?" (2:49)