← back
Your LLM Stack Is a 2008 Database With Better Marketing — Lovina Dmello, NVIDIA
Takeaway
Production AI security depends on correctly configured infrastructure and practical defense costs as much as model-specific protections.
Summary
- Exposed Ray clusters illustrate how disabled authentication and open job APIs can dominate production ML risk without any novel model attack.
- The security model layers infrastructure isolation, access controls, runtime monitoring, and operational practices such as incident response.
- Overprivileged accounts, unsegmented systems, and accessible secrets or model weights are recurring misconfiguration failures.
- Security controls have latency and throughput costs, so basic authentication and validation should be standard while heavier defenses are engineered and targeted to relevant workloads.
ml-securityinfrastructuremisconfiguration
Original description
In 2023, researchers found thousands of Ray clusters sitting wide open on the public internet, dashboards and job APIs exposed to anyone, because authentication ships off by default and nobody turned it on before going to production. The data at risk was worth more than a billion dollars. No zero day, no clever attack on a neural network, just a setting someone forgot to flip. In production ML security that is not the exception, it is the rule. Lovina Dmello read 139 peer reviewed papers on production ML security and kept hitting the same pattern: misconfiguration, not missing features, is the dominant failure mode. One audit of 50 real systems found a critical mistake in 78% of them, almost always the same three offenders: overprivileged accounts that can touch anything, flat networks where one foothold reaches everything, and secrets and model weights left in storage anyone can read. The heavier defenses researchers love tend to die in production because they add 15% to 30% inference overhead, so the real question is never whether to run a control but how to run it cheaply enough to keep. The fix is not a new attack and defense pair. It is to stop securing ML like a model and start securing it like the infrastructure it actually is, because that is where the breaches keep landing. Speaker info: / lovina25 https://developer.nvidia.com/blog/aut... https://scholar.google.com/citations?...