Your LLM Deception Monitor Is Broken. The Fix Is in the Training Data - Sachin Kumar, LexisNexis

744 views · Jul 08, 2026 · 13:58 min · Watch on YouTube ↗

Takeaway

Activation differences can expose fine-tuning-induced backdoors in controlled experiments, but high precision alone does not guarantee broad trigger detection.

Summary

The study investigates sleeper backdoors that behave normally until a trigger appears, which ordinary behavioral evaluations may miss.
A difference sparse autoencoder learns features from fine-tuned activations minus base-model activations, targeting changes introduced during training.
Experiments fine-tune SmolLM2-360M on 5,000 examples where a year cue controls safe versus vulnerable SQL generation, then evaluate on 2,500 examples.
The reported backdoor isolation score is about 0.4 for difference features versus about 0.01 for joint cross-model features, with consistent results across middle layers and fine-tuning regimes.
The strongest individual feature achieved perfect observed precision and no false positives but detected only about a quarter of triggers, limiting single-feature coverage.

backdoor-detectionsparse-autoencodersinterpretability

Original description

You fine-tune LLMs and ship them. Your evals are green, your behavioral monitors are green — and a sleeper-agent backdoor can still flip the model to harmful output on a trigger you never tested. Behavioral testing can't reach it, and the interpretability tool people reach for — joint cross-model features (crosscoders) — dilutes the signal until it sits at the noise floor.

The fix is in what the training data changed. A backdoor is a directional shift that fine-tuning writes into the model's activations, so you isolate it by watching the difference between the base and fine-tuned model. In a controlled SQL-injection backdoor, a sparse autoencoder trained on that difference flags it with 40× the signal of joint features, perfect precision, and zero false positives — from a single cheap layer. You'll leave knowing how to wire a "delta monitor" into your fine-tuning pipeline as a quiet CI gate. Based on my peer-reviewed paper accepted at IJCNN.

Speakers:
Sachin Kumar (LexisNexis): Sachin Kumar is a Senior Data Scientist III and Tech Lead at LexisNexis, building agentic AI for the legal domain. His independent AI-safety and interpretability research has been accepted at top-tier venues including ACL, AAAI, and IJCNN.
  LinkedIn:   / techsachinkumar  
  GitHub: https://github.com/techsachinkr