← back
Evals-Driven Development for a Mental Health AI Coach — Akele Reed & Dave Revere, SonderMind
Takeaway
Safety evaluation for mental-health AI must measure clinically appropriate responses in context, including the harm of unnecessary refusal.
Summary
- SonderMind's Sonder is a clinically grounded conversational coach intended to support reflection, coping exercises, and preparation for or support between therapy sessions.
- Independent input and output LLM-judge guardrails surround the core agent so safety controls can be evaluated and updated separately from personalization and memory.
- Guardrails distinguish active danger requiring resources and disengagement from past distress where support can continue, and ordinary difficulties needing no intervention.
- The team prioritizes correct intervention over frequent intervention, calibrating nuanced behavior with clinicians and emphasizing an ongoing evaluation learning loop.
mental-healthguardrailsllm-as-judge
Original description
In the world of AI mental health, vibes can be dangerous, with real consequences. Building SonderMind's Mental Health AI Coach required a new playbook for eval-driven development that balances effectiveness and safety. The team explains the clinical feedback loop that turns human therapist insights into machine-readable evaluations across thousands of conversations; the Ethics Engine of modular guardrails that can evolve with clinical guidelines; the move from single-prompt agents to a closed-loop Supervisor/Executor/Evaluator architecture; and the human oversight used to improve safety and quality. Speakers: Akele Reed — Principal AI Engineer, SonderMind Akele leads the team behind SonderMind's conversational AI mental health feature and helped architect its guardrails and evaluations framework. LinkedIn: / akele-reed Dave Revere — Staff AI Engineer, SonderMind Dave builds evaluation, guardrail, clinical-feedback, and regression-testing infrastructure for high-stakes mental health AI. X/Twitter: https://x.com/daverevere LinkedIn: / daverevere Doug Keller — Senior Staff AI Engineer, SonderMind Doug is the lead architect of SonderMind's agent platform and a core builder of its mental health coach. LinkedIn: / doug-keller