← back
Respect The Process - Andrew Dumit, Watershed Technology Inc.
Takeaway
When several answers may be defensible, agent verification must examine the process and lineage of consequential changes.
Summary
- Watershed's product carbon-footprint workflows involve expert judgments, so matching a final emissions number does not establish that the calculation process was valid.
- A specialized ReAct tool interface struggled across many supply-chain graphs because exploration consumed context and edits became inconsistent.
- Coding agents improved bulk graph exploration and editing through loops and scripts, while also enabling new visualizations and unexpected use cases.
- Unrestricted code introduced untracked edits and false completion claims; the proposed direction preserves reasoning freedom while constraining critical graph-changing effects.
coding-agentsprocess-verificationsustainability
Original description
In sustainability, the answer to almost every question is "it depends." What’s the right classification? Which method should the agent use? Our vertical is filled with judgement calls broadly and one of the tasks in that vertical might span a search over 100k’s of data-rich nodes and hundreds of edits. Editing across them takes loops and filters, not enumerated tool calls. Code is the only thing that scales to the task. But in a domain that relies heavily on judgement and implicit knowledge, there are many ways for a model to get the right answer with the wrong reasoning and many right answers for it to come to. Over time, we’ve upgraded models, but a smarter model didn’t fix this. Instead we built a domain-specific coding harness, applying best practices from general coding agents (linting, routing failures back, sub-tasking), and constraining how it commits through a well-defined SDK and owning the final execution. It keeps the full power of a modern coding agent, but every change must pass through our deterministic, typed interface. Within that constraint, we harness-engineered our way from 43% to 92% accuracy. The interesting part is the other 8%: even when the agent lands on a different answer than our reference, every change it makes is still valid, traceable, and replayable. We'll cover how we built it, what we chose to constrain vs. what we leave free. Speakers: Andrew Dumit (Watershed Technology Inc.): Andrew works on AI engineering at Watershed, building systems to manage and reduce the emissions of everything companies buy and sell. LinkedIn: / adumit GitHub: https://github.com/adumit