← back
The Missing Layer After Launch - Raphael Kalandadze, Wandero AI
Takeaway
Treat post-launch monitoring and repair as a product capability that must itself be calibrated and evaluated.
Summary
- Successful completion can conceal agent struggles, lucky recoveries, incorrect prices, or outcomes that do not help the user.
- Wandero links production trajectories and source code to an investigating agent that diagnoses failures and proposes PRs.
- A separate reviewer with fresh context challenges proposed fixes and runs focused checks rather than inheriting the investigator’s assumptions.
- Frequent log-monitoring runs support rapid repairs, complemented by a broader view of overall system health.
production-agentsfeedback-loopsobservability
Original description
We run a production system of agents for real customers. The team that keeps it healthy is also made of agents. Operating an agent product isn't like operating software. When our agent fails a customer — a dropped constraint, a stale price, a confident wrong answer — nothing crashes and no log lights up. The failure is in the conversation, not the stack trace. So we put agents on the operations: One monitors production conversations and judges where the agent actually let a customer down — across thousands of live sessions, not a sampled few. One watches logs and system health and traces real problems back into the code. One writes and runs tests, because "green CI" means nothing for a non-deterministic agent. One reviews every PR — human or agent-authored — against a single question: root cause, or just the symptom? Humans stay at the merge and approval boundaries. The agents do the watching, judging, testing, and drafting that no human team could keep up with at this volume. This talk is the honest version: what each operating agent actually checks, where we trust it and where we don't, what breaks, and why operating an agent system is becoming its own engineering discipline — done, increasingly, by agents. Speakers: Raphael Kalandadze (Wandero AI): Co-founder and CTO of Wandero AI, an agent-native operating system for travel and hospitality, and co-founder of Tbilisi AI Lab, where we build the first Georgian large language model. X/Twitter: @RaphaelKalan LinkedIn: / rapael-kalandadze GitHub: https://github.com/RRaphaellRaphaelKalan