← back
Deterministic Infra for Non-Deterministic AI Agents - Nishant Gupta, Meta Superintelligence Labs
Takeaway
Keep probabilistic decision-making inside deterministic infrastructure that bounds failures, cost, and authority.
Summary
- Long-running, stateful agents violate assumptions behind conventional short-lived cloud requests and introduce retry amplification, deadlocks, and memory-consistency failures.
- Models should propose actions while infrastructure validates them, policy engines approve them, and execution gateways enforce them.
- An agent control plane coordinates scheduling, memory, policy, routing, evaluation, and monitoring; traces should capture decisions, tool calls, and state transitions.
- Circuit breakers, bounded recovery, quotas, layered permissions, and human exception handling adapt established distributed-systems practices to agents.
agent-infrastructuredistributed-systemsreliability
Original description
AI agents are rapidly evolving from copilots into autonomous systems capable of reasoning, invoking tools, coordinating workflows, and interacting with production infrastructure. But most platforms today were designed for deterministic microservices — not long-running, non-deterministic systems powered by LLMs. This creates a massive infrastructure gap. In this talk, I’ll share lessons from building large-scale agentic and elastic compute infrastructure powering production AI workloads. We’ll explore the emerging “control plane” required for reliable AI agents: orchestration, observability, retries, evaluation, safety guardrails, workload isolation, memory coordination, and operational control loops. Topics include: Why most AI agents fail outside demos Building deterministic systems around stochastic models Observability for autonomous AI workflows Failure handling and retry storms in agent systems Human oversight and safety guardrails Elastic GPU infrastructure for agentic workloads Reliability patterns for production AI systems The shift from “prompt engineering” to “systems engineering” Attendees will leave with practical architectural patterns for building resilient AI infrastructure capable of supporting autonomous systems safely and efficiently in production. Speakers: Nishant Gupta (Meta Superintelligence Labs): Nishant Gupta is a Software Engineering Tech Lead at Meta Superintelligence Labs building the training and inference AI infrastructure. LinkedIn: / nishantgupta-ai GitHub: https://github.com/nishantgpt-lab