← back
Your Agent Is Wasting Tokens and You Don't Know It - Erik Hanchett, AWS
Takeaway
Control agent costs by reducing repeated context, matching models to task difficulty, and bounding unnecessary work.
Summary
- Cache stable system prompts, tool definitions, and messages to reduce repeated input costs in agent loops.
- Route easy tasks to cheaper models and reserve more capable models for difficult work.
- Store large tool results externally and return summaries; cap tool-loop iterations and inspect traces for repeated calls.
- Use a sliding conversation window, such as the Strands conversation manager, and summarize older history to preserve essential context.
token-costsstrands-agentscontext-engineering
Original description
I deployed an agent to production and the bill was not good. Not because the model was bad, but because it was doing too much. I was using the most expensive models for simple inference calls. The context was filling up. And my tool loops ran longer than they needed to. The agent worked fine, it just cost way more than it should have. This talk covers three small changes I made that dropped my costs without hurting quality. Each one was a few lines of code, and none of them required changing my prompts or switching models. I'll cover things like prompt caching and model routing. I'll show code. Speakers: Erik Hanchett (Amazon Web Services): Erik Hanchett is a Developer Advocate at AWS who helps developers build with frontend, fullstack, and AI/agent technologies through hands-on tutorials, talks, and videos. X/Twitter: https://x.com/erikch LinkedIn: / erikhanchett GitHub: https://github.com/erikch