From smartphones to Raspberry Pi: Running Gemma 4 anywhere
Summary
In this Google Cloud Next '26 session, Omar Sanseviero, Lead AI Developer Experience at Google DeepMind, joins host Muhammad Farooq to discuss the record-breaking launch of Gemma 4, Google's latest open model family. Within just three weeks of its release, Gemma 4 surpassed 40 million downloads, making it Google's largest open model release to date.
Omar describes the Gemma 4 family as spanning from a 2 billion parameter model optimized for mobile devices all the way up to a 31 billion parameter model designed to run on consumer GPUs and gaming workstations. The key design philosophy is maximizing 'intelligence per parameter per watt,' making high-capability AI accessible without requiring enterprise-grade hardware. The smallest models (2B and 4B) support full multimodal input — audio, video, and images — while the larger models focus on advanced vision capabilities without audio input.
A major highlight of the launch is that Gemma 4 was released under the Apache 2.0 license, replacing the previous custom Google Gemma license. While the previous license was commercially permissive, community feedback over 18 months revealed skepticism and friction — particularly from enterprises, NGOs, and regulated institutions with strict internal license policies. The switch to Apache 2.0 has already generated significant community excitement.
Omar introduces the concept of the 'Gemmaverse' — a growing ecosystem of community-fine-tuned variants built on top of Gemma's open weights. He cites a notable example of developers fine-tuning Gemma for Quechua-to-Spanish translation, serving indigenous communities in Peru — illustrating the model's potential as a truly global tool. Gemma 4 was trained on over 140 languages.
On the topic of hybrid inference, Omar references Cactus Compute, a Y Combinator startup, as an example of the emerging pattern where a local router model like Gemma handles approximately 70–80% of daily tasks, while more complex queries are escalated to cloud-based models like Gemini. This 'Cactus Compute' model allows developers to balance cost, privacy, and intelligence dynamically.
Omar outlines the primary use cases for local and open-access models: privacy-first and highly regulated environments such as healthcare and government sovereign AI initiatives, fully offline deployments in regions without internet connectivity, and specialized fine-tuning for domain-specific applications such as finance. He also highlights Gemma's agentic capabilities, noting it supports function calling and tool use, and can act as an on-device agent via the Android AI Edge Gallery — controlling hardware like flashlights, drafting emails, and performing translation entirely locally. Omar cautions that while small agentic models excel at narrowly scoped tool-use tasks, they are not yet suited for complex tasks like full codebase refactoring.
Key Takeaways
- •Gemma 4 achieved 40 million downloads in its first three weeks, signaling massive developer demand for capable open-weight models.
- •The model family runs across the full hardware spectrum — from Android phones to Raspberry Pis to consumer GPUs — without sacrificing agentic reasoning capability.
- •Switching to Apache 2.0 removed a key friction point for enterprises and regulated institutions that had strict internal license requirements.
- •Hybrid inference (local Gemma for ~80% of tasks, cloud Gemini for complex queries) is an emerging architecture pattern that balances cost, privacy, and intelligence.
- •Small local models are now capable enough for agentic tasks involving 5–10 tools, but should not be expected to handle tasks requiring broad world knowledge or large codebase reasoning.
- •Fully offline and sovereign AI deployments — healthcare, government, underserved communities without internet — are a primary design target for Gemma 4.
- •The Gemmaverse community is already fine-tuning Gemma for niche languages and specialized domains, enabled by the open Apache 2.0 license.
- •On-device agents using Gemma can control device hardware, draft emails, and perform translation entirely locally via the Android AI Edge Gallery.
- •Developers should use larger cloud models (e.g., Gemini) when tasks require comprehensive world knowledge or factual recall; Gemma is best for reasoning and tool-use within constrained domains.
- •Gemma 4's multilingual training across 140+ languages positions it as a genuinely global model, not US-centric.
Transcript
Hey everyone. This is Mohammed and today I have Omar with me from Google DeepMind and we're going to be talking about all things Gemma. Omar, how are you doing today? Hi, very good. Thank you. All right. So, you guys had Gemma for lunch recently and it's been a pretty successful launch. How do you feel about it? Yeah, we are super excited. So, this has been our largest open model release ever. The community reaction has been amazing. The model was actually launched just 3 weeks ago and we've had...
