← back
Training Frontier Models to Out-Think Hackers — Uri Rolls, Arithmetic & Thom Wolf, Hugging Face
Takeaway
Improving AI cyber defense requires realistic reasoning benchmarks that capture system-level access-control failures.
Summary
- Arithmetic and Hugging Face frame cybersecurity as a test of dynamic world modeling and reasoning rather than code-pattern recognition alone.
- Cheaper, faster AI-assisted attacks challenge defensive systems designed around attackers having limited time and resources.
- The proposed response is stronger defensive models supported by high-quality data, evaluations, and open-source collaboration.
- Arithmetic introduces an access-control benchmark focused on finding logic failures across interconnected systems and reasoning about unauthorized privilege.
cybersecuritybenchmarksopen-models
Original description
NOTE: see further context from Thom: https://x.com/Thom_Wolf/status/207995... Give a frontier model a real chain of Keycloak, Vault, and a broker, start it as a low privileged user, and ask it to reach production code. There is a genuine zero day in there: one check validates the admin by name while another checks by ID, so a user can simply rename themselves to the admin and inherit the privilege. GPT 5.5 and Opus probe everything, even reach the check, and never make that logical leap. That gap is the point of Uri Rolls and Hugging Face cofounder Thom Wolf's talk: today's models can do the reconnaissance but not the reasoning jump a skilled hacker makes. Their argument is optimistic, which is rare in AI and cyber right now. Just as high quality data transformed coding, Arithmetic builds cyber training data by having human vulnerability researchers find their own zero days in open source software, then wrapping them in blackbox environments where every step of discovery and exploitation is deterministically graded. The benchmark, focused on access control, the top vulnerability class, is brutal: exactly one solve at K1. The bet is that if open source models get fast and good enough at these logic leaps, defenders finally get a lasting edge over attackers, instead of leaving it to two labs. Speaker info: Uri Rolls, Arithmetic: https://x.com/uri_rolls / urirolls Thom Wolf, Hugging Face: https://x.com/Thom_Wolf / thom-wolf https://thomwolf.io Timestamps: 0:00 - Why cyber is a wide new field for AI 1:34 - The ARC-AGI-3 parallel: models can't model the world 2:24 - Open source models as part of the defense 3:45 - The shifting economics of cyber 5:52 - The optimistic thesis: models are the solution 7:06 - The first benchmark: access control 8:20 - Data quality: finding your own zero days 10:01 - A real solve: the Keycloak name versus ID exploit 11:57 - Live demo: one solve at K1 14:22 - Only models can replace the old stack 15:00 - The speed challenge and specialized defenders