← back
OpenClaw in Your Hand: Building a Physical AI Terminal - Lech Kalinowski, Callstack
Takeaway
A useful physical AI terminal can keep the device simple by separating responsive local interaction from heavyweight backend intelligence.
Summary
- The physical OpenClaw terminal pairs a fast monochrome OLED for live typing with an e-paper display for persistent, low-power reading.
- An ESP32 microcontroller uses preallocated buffers and one-bit rendered pages, leaving agent orchestration and model inference to a backend connected to a DGX Spark.
- The prototype serves a 120-billion-parameter open model through TensorRT and an OpenAI-style proxy; hardware iteration exposed power regulation, wiring, GPIO, and encoder-noise problems.
- A text RPG mode adds generated worlds, characters, and memory, demonstrating a use case well suited to the dedicated terminal.
openclawembedded-aiesp32
Original description
What if an AI device felt calm — closer to peace than a glowing distraction? Vault is a text-first, dual-display handheld AI terminal built on a single ESP32-S3 and powered by one battery cell. It pairs a fast, emissive OLED "live" surface with a slow, bistable e-paper "content" surface, switches between four modes with a single slash command — shell, assist, control, and an LLM-native RPG — and drives autonomous OpenClaw agents, all against a fully local, self-hosted model (gpt-oss:120b served by NVIDIA TensorRT-LLM). This talk walks the system end to end: an AI-native architecture where the firmware talks to a dependency-free Python backend that dispatches to local models, agents, and a game engine; why inference stays on the backend instead of the microcontroller; and how the device surfaces an agent thinking and calling tools in real time on e-paper. Along the way: the engineering war stories that each cost a day, the LLM-native game design that tracks narrative state instead of HP and dice, and what it actually takes to put a local agent in your pocket. Speakers: Lech Kalinowski (Callstack): Dr. Lech Kalinowski is an AI and data science leader, PhD in Physical Sciences, and startup co-founder specializing in machine learning, AI strategy, printed electronics, and applied innovation. X/Twitter: https://x.com/LeSiOO LinkedIn: / lech-kalinowski GitHub: https://github.com/lech-kalinowski