Gemini Nano on device — Florina Muntenescu & Oli Gaymond, Google DeepMind

Florina Muntenescu (Developer Relations Engineer, Google DeepMind) and Oli Gaymond (PM for Android AI, Google DeepMind) present an overview of on-device and hybrid AI inference for Android developers at AI Engineer Europe 2026. The session is structured as a presentation followed by an open Q&A.

The core of the talk covers three inference pathways available on Android: fully on-device, hybrid (on-device with cloud fallback), and fully cloud-based. On-device inference using Gemini Nano is the primary focus. Gemini Nano is Google's most efficient on-device model, sharing the same architecture as Gemma 4 (launched the week before the talk) but optimized specifically for Android hardware. The model weighs approximately 3–4 GB and is delivered to devices via AI Core, a system-level service. This centralized approach means the model is downloaded once and shared across all apps, eliminating the burden of individual apps shipping the model themselves — a deployment pattern the speakers explicitly call out as unrealistic for app developers to manage independently.

AI Core handles scheduling, hardware optimization at runtime (leveraging silicon partner integrations), and privacy isolation: each app's requests run in isolation and no input/output data is persisted on device. Foreground apps get top scheduling priority; background batch jobs are queued and typically processed overnight while the device charges. Developers interact purely through the ML Kit GenAI APIs and do not manage any of this infrastructure.

The ML Kit GenAI APIs provide task-specific endpoints (summarization, proofreading, rewrite) as well as the general-purpose Prompt API, which accepts text and image input and returns text output. The speakers highlight use cases including image understanding, content assistance, entity extraction, and RAG-style retrieval. An embedding API is described as coming soon, which will expose the Gemini embedding model to enable vector similarity search directly on device.

The reach tradeoff is addressed directly: ML Kit GenAI APIs and Gemini Nano currently require flagship-class devices from the last couple of years (e.g., Pixel 9, Pixel 10, and equivalent OEM hardware). Classic ML Kit — covering vision, OCR, and natural language tasks using smaller classical models — runs across 1 billion+ devices without this constraint. For developers who need broader reach with generative capabilities, hybrid inference via Firebase AI Logic (launched a few weeks before the talk) falls back from Gemini Nano to Gemini Flash in the cloud when on-device inference is unavailable. Firebase AI Logic also supports fully cloud-based inference against the Gemini API and Vertex AI.

For developers who need custom models beyond Gemini Nano, LiteRT (formerly TFLite) and specifically LiteRT LM is mentioned as an alternative path, with a follow-on talk by Cormac covering that topic. The speakers note that LiteRT LM requires developers to own their own testing and profiling across device variants, whereas AI Core abstracts that entirely.

In the Q&A, questions probe battery and RAM impact (acknowledged, but attributed-to-app battery reporting handles user transparency), multi-app concurrency (AI Core queuing manages contention), RAG solution support (confirmed possible today; embedding API will simplify it), the relationship between AI Edge Gallery and AI Core (Gallery is a showcase of frontier capabilities; AI Core is the production-ready path), and the device-capability matrix (classic ML Kit = broad; GenAI APIs = flagship only for now; LiteRT LM = custom with self-managed testing).