← back
Tell the Robot What You Want — Sandhya Subramani, AWS
Takeaway
An agent layer can turn existing robot policies into flexible natural-language workflows while keeping action selection separate from motor execution.
Summary
- Scout demonstrates natural-language robot commands, visual observations, and physical actions coordinated through AWS Strands Agents.
- Three cooperating agents handle environmental reasoning, text communication, and voice interaction.
- The architecture separates the agent’s choice of action from robot policies, execution backends, and physical hardware, with observations flowing back upward.
- The presented approach combines cloud and edge execution so existing robot capabilities can be orchestrated through a common tool interface.
roboticsstrands-agentsedge-ai
Original description
Sandhya Subramani asks her rover how many people it can see, a question nobody ever trained it to answer. It thinks for a second, checks its front camera, and reports two, one near the speaker monitor and one further right. Scout is a small four legged robot running a Raspberry Pi, reaching the internet over a SIM card and a 4G connection, and its whole personality comes from an agent layer sitting above the movement policies it already had. The demo goes exactly as live robot demos go. It falls over, gets coaxed upright, announces a signature performance and then just turns its headlights off. The idea underneath is neat and portable. If you give an agent software tools, you can also hand it a hardware tool and let it choose which preset policy to fire, which turns a robot trained for a fixed task list into something you can address in plain language. She wires it up in about five lines using an open source AWS framework that already covers 40 or so robots. Scout actually runs three agents at once, one thinking about the environment, one talking to her over Telegram, and a voice agent she disabled so it would stop interrupting her on stage. Her framing is that the agent decides what to do while the trained policy decides how, and the robot doubles as a rig for collecting the next round of training data. Speaker info: / sandhyasubramani Timestamps: 0:00 - Meet Scout, running on a Raspberry Pi over 4G 1:27 - Answering a question nobody trained it for 3:46 - Giving an agent a hardware tool 5:02 - Five lines to hand a robot to an agent 6:09 - Three agents running at once 7:17 - The four layers, and how observations travel up 9:35 - Splitting work between cloud and edge 10:45 - Where policies go if they scale like language models 11:53 - Asking it to do something complex 14:12 - What is actually under the hood 15:21 - Mirror mirror, over Telegram