🛡️ Safety & Alignment
Prompt injection defenses, jailbreak resistance, hallucination mitigation, PII handling, red-teaming, responsible scaling.
The workflow
flowchart LR
A[Threat model] --> B{Risk type}
B -->|Prompt injection| C[Input sanitization<br/>+ instruction hierarchy]
B -->|Jailbreak| D[Red-team set<br/>refusal training]
B -->|Hallucination| E[Citation +<br/>retrieval grounding]
B -->|PII leak| F[Output filter<br/>+ redaction]
C --> G[Continuous eval<br/>+ monitoring]
D --> G
E --> G
F --> G
Prompt injection is the #1 unsolved AI security issue. Every shipping team needs a layered defense.
Key takeaways
Videos (45)
AI’s Jurassic Park Period — Aaron Stanley, dbt Labs
Agent safety requires respect for the intent behind constraints as well as enforceable technical boundaries.
Prompt Engineering and AI Red Teaming — Sander Schulhoff, HackAPrompt/LearnPrompting
Prompt injection is a fundamentally unsolved AI-security problem and red-teaming competitions like HackAPrompt are the empirical baseline for measuring it.
How to Secure Agents using OAuth — Jared Hanson (Keycard, Passport.js)
Stop pasting long-lived API keys into MCP configs — treat MCP servers as OAuth resource servers and let a real authorization server mediate agent access.
What if the network was the sandbox? — Remy Guercio, Tailscale
Identity-aware network gateways can enforce agent access while keeping valuable provider credentials outside agent-controlled environments.
Your company brain will leak secrets: how we stopped it for big banks — Tanmai Gopal, PromptQL
A useful company brain must combine continuously maintained shared knowledge with explicit controls over what each user and agent may access.
$1 AI Guardrails: The Unreasonable Effectiveness of Finetuned ModernBERTs – Diego Carpentero
A fine-tuned ModernBERT classifier sidecar provides low-latency, sub-dollar guardrails against today's broad LLM attack surface across prompt, context, RAG, and MCP vectors.
"I've never seen anything scarier than an LLM with tool calls." — Erik Meijer aka @HeadinTheBox
Agent safety needs precise controls over effects rather than relying solely on the apparent safety of generated text.
Tethered: Our Agents Are Us — Shu Fang, Two Sigma
Agents can reuse user permissions when attribution and controlled external access make their actions traceable.
Trust, but Verify: Shreya Rajpal
Treat every LLM call as untrusted and wrap it in a verification suite with reask/fix/refrain policies so correctness becomes a programmable property, not a hope.
Build AI Systems for Discernment, Not Approval - Angel Ortmann Lee, Duolingo
Human review works better when the workflow explicitly demands independent evidence rather than approval of AI output.
Your Agent Just Authorized What?! — Jay Mok & Ben Coumes, Paypal
Match agent authority and evidence requirements to transaction stakes and counterparty trust.
Why you should care about AI interpretability - Mark Bissell, Goodfire AI
Interpretability tools like Goodfire's Ember are ready to use today — they let AI engineers debug and steer models at the neuron level instead of fighting prompts.
The Unofficial Guide to Apple's Private Cloud Compute - Jmo, CONFSEC
PCC's stack — OHTTP, blind signatures, secure enclaves, code attestation — is a reusable blueprint for making remote AI compute provably private.
How to Build Trustworthy AI — Allie Howe
Trustworthy AI requires shifting right with runtime guardrails alongside build-time scanning and red-teaming because non-determinism means CI/CD alone can't catch failures.
OpenAI on Securing Code-Executing AI Agents — Fouad Matin (Codex, Agent Robustness)
Secure code-executing agents require sandboxing, internet allowlists, and human approval surfaces — Codex CLI is OpenAI's open reference implementation.
Agentic Security: Permissions, Provenance, and the Agent Supply Chain — Steve Yegge, Gas Town
Integrate dedicated security checks into the agent’s coding loop and treat model self-review as insufficient evidence of security.
The AI bugpocalypse is here. Now what? - Jack Cable, Corridor
Apply AI to structural vulnerability prevention so defensive improvements keep pace with faster code generation and exploitation.
Security Firewall for Agents — Ryan Dahl, Deno
Enforce agent permissions outside the model at the actual network and protocol boundaries where actions occur.
How we hacked YC Spring 2025 batch's AI agents — Rene Brandel, Casco
Agents are users, not services — apply classic web-app authn/authz, IDOR mitigation and sandbox hardening downstream of the LLM, not at the prompt layer.
Give the Agent a Budget, Not a Token — Sachin Malhotra, Anthropic
Agent authority should reflect blast radius and recoverability, with infrastructure enforcing the limits.
How to defend your sites from AI bots — David Mytton, Arcjet
Layered defenses (robots.txt + UA + reverse-DNS verification + behavioral signals) are needed to distinguish good AI crawlers from training/abuse bots and agent-driven browsers.
CIAM for AI: Authn/Authz for Agents — Michael Grinich, CEO of WorkOS
Agents need a new identity class — neither human nor service-account — with patterns like shadow personas, delegation chains, and capability tokens to keep them safe in enterprise systems.
What's Next After RLHF? — Diogo Almeida, TypeSafe AI
Moving beyond assistance requires training objectives that reward dependable task outcomes rather than only human preference.
Full Workshop: Better Auth — Paola Estefania, Better Auth
Give each agent an auditable identity and narrowly delegated capabilities so access can be understood and revoked.
We Vetted 2000 AI Skills Before They Reached Developers — Lucas Palma, Nubank
Secure the instructions and tools shaping AI development workflows before distributing them to engineers.
Unlock Agent Autonomy: The Runtime for AI-Native Systems — Tushar Jain, Docker
Agent autonomy needs external runtime enforcement that grants narrowly scoped capabilities as tasks evolve.
AI + Security & Safety — Don Bosco Durai
Single-process agent frameworks violate zero-trust — credentials, prompts, and tool outputs need isolation boundaries to prevent injection-based privilege escalation.
LLM Safeguards: Security Privacy Compliance Anti Hallucination: Daniel Whitenack
Production LLM safety requires a layered checklist covering hallucination, supply chain, server resilience, PII leakage, and prompt injection — not a single guardrail.
Through the AI Fog: The Architectural Decision Agentic Security Depends On — Manoj Nair, Snyk
Autonomous software delivery needs independent validation across code, agent environments, and behavior.
Claws Out: Securing and Building with OpenClaw - Nick Taylor, Pomerium
Correctly configured identity-aware proxying can simplify OpenClaw access while explicit permissions remain essential for agent actions.
Securing Agents with Open Standards — Bobby Tiernay and Kam Sween, Auth0
Use open identity standards (OAuth 2.1, token exchange, CIBA, FGA at retrieval) to keep agent actions tied to real users with scoped short-lived tokens, not env-var keys.
Guardrails First: Engineering Member-Facing Health AI — Rashi Agrawal, Hinge Health
Member-facing healthcare AI needs enforced architectural boundaries and ongoing review of real interactions.
AI Frontiers in Trust and Safety Combatting Multifaceted Harm on Tinder at Scale: Vibhor Kumar
Fine-tuning open-source LLMs on hybrid LLM-mined + human-verified data is the practical playbook for trust-and-safety classification at consumer scale.
AI is the World’s largest Relationship Therapist — Clay Cockrell & Tony Fabrikant, CoupleWork AI
Relationship AI should help users return to human relationships with greater insight rather than maximize validation and time in chat.
IT Admin for the AI Workforce — Sarthak Aggarwal, Decawork
Enterprise agents need a governed identity and permission lifecycle with enforceable controls immediately before consequential actions.
You Didn't Ship a Bug. You Just Wrote It for a Human. - Ravi Madabhushi, Scalekit
Agent-facing APIs need explicit delegated identity and context-specific permissions rather than inherited human access patterns.
It's 10pm. Do You Know Where Your Agents Are? — Kim Maida, Keycard
Use identity-aware, per-action token exchange to replace broad API keys with governed delegated access.
AI Red Teaming Agent: Azure AI Foundry — Nagkumar Arkalgud & Keiji Kanazawa, Microsoft
Azure AI Foundry packages Microsoft's PyRIT red-teaming toolkit as a managed SDK + dashboard so AI engineers can red-team their own apps without standing up the framework themselves.
Your LLM Deception Monitor Is Broken. The Fix Is in the Training Data - Sachin Kumar, LexisNexis
Activation differences can expose fine-tuning-induced backdoors in controlled experiments, but high precision alone does not guarantee broad trigger detection.
Security Track Intro — Randall Degges, Snyk
Secure-by-default development and controlled agent execution are prerequisites for confidently scaling AI-assisted software.
Critical AI Inference your CIO can Trust — Sahil Yadav, Hariharan Ganesan, Telemetrak
For CIOs to trust AI in critical systems, MLOps needs to evolve into XTOps with explainability, traceability and guardrails plus business-facing metrics like MTRE and trust-adjusted risk.
Agentic Development Security — Ezra Tanzer, Snyk
Securing autonomous development requires enforceable scanning hooks and inspection of agent components and privileges alongside generated code.
Using LLMs to Secure Source Code — Eugene Yan, Anthropic
Effective LLM-assisted security depends on explicit system context and reproducible exploit verification so teams can prioritize and patch real vulnerabilities.
Building security around ML: Dr. Andrew Davis
ML security has moved from anti-malware-style ML to defending the ML pipeline itself — poisoning, theft, and adversarial examples remain open problems.
Cognitive Shield Real Time Real Smart - Rachna Srivastava
AI-powered fraud requires AI-powered defenses—layered detection (data hygiene + multi-modal real-time + human escalation) rebuilds trust against deepfakes and synthetic identities.