← back
Voice In, Visuals Out: The Agony and the Ecstasy - Allen Pike, Forestwalk Labs
Takeaway
Voice-in, visuals-out interfaces become practical when fast inference and incremental processing stay within the user's attention window.
Summary
- Forestwalk combines spoken input with visual output and tool actions, such as creating a Linear issue during a live team call.
- Visual responses can feel responsive within roughly a second, whereas fluid spoken turn-taking demands a much tighter latency budget around 200 milliseconds.
- A small model is insufficient unless its serving platform also delivers low tail latency; heavier reasoning can be delegated asynchronously to a larger model.
- Frequent inference while people speak avoids spending the entire response budget waiting for end-of-speech silence.
voice-interfaceslatencymultimodal-ux
Original description
The latest AI models have made it possible to go past text-based chats, and build what Andrej Karpathy argues is the pinnacle of AI UX: voice-in, visuals-out. In this talk, Forestwalk Labs co-founder Allen Pike shares why this approach for LLM-powered product development is so useful, what's necessary to make it actually delight users, and lessons his team has learned building products with highly responsive AI agents like these – with a key focus on techniques for achieving low latency. Speakers: Allen Pike (Forestwalk Labs): Allen is co-founder of Forestwalk Labs, runs the Infer AI engineering meetup, and hosts the It Shipped That Way podcast. X/Twitter: / apike LinkedIn: / allenpike GitHub: https://github.com/apike