← back
Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind
Takeaway
Choose local, hybrid, or cloud inference according to device capability, privacy needs, and workload demands.
Summary
- Android supports on-device, hybrid, and cloud inference, with local execution offering offline use and keeping prompts off servers.
- ML Kit GenAI APIs expose Gemini Nano through AICore, which shares an optimized model across apps while isolating requests.
- Task APIs cover summarization, rewriting, and proofreading; the prompt API accepts text and images and returns text.
- Firebase AI Logic provides a hybrid route to cloud inference when local support is unavailable, while battery and hardware constraints shape suitable usage.
gemini-nanoandroidon-device-ai
Original description
Gemini Nano on device weighs three to four gigabytes. Shipping that per app is not realistic, which is why AI core puts it in the system once and every app shares it. Foreground apps get top priority. Background batch jobs queue and run overnight on charge. The developer never manages any of that. The tradeoff is reach. The GenAI MLKit APIs require flagship devices from the last couple of years. Classic MLKit for vision and OCR runs on a billion plus devices without issue. Hybrid inference, launched a few weeks before this talk, falls back from Nano to Gemini Flash in the cloud when the on device model is not available. An embedding API is coming soon for RAG style solutions. For anything beyond that, LiteRT is the other path. Speaker info: https://x.com/FMuntenescu / florina-muntenescu-314b8921 https://github.com/florina-muntenescu / ogaymond