Imagine building a fully autonomous robot that can see, hear, and react to its environment in real time, completely offline on a single compact device like Raspberry Pi. Edge AI unlocks this exact autonomy. It enables developers to build highly secure and self-contained systems like intelligent robotics and local AI agents with zero cloud dependencies, ultra-low latency, and total data privacy.
We’ve made running edge AI on the Raspberry Pi a breeze with Google AI Edge’s LiteRT, high-performance production-proven on-device inference runtime. LiteRT allows you to seamlessly deploy everything from classical ML models to state-of-the-art LLMs right out of the box across platforms. By delivering optimized execution and hyper-efficient memory usage on both CPU and GPU, LiteRT maximizes your Raspberry Pi's full computing potential.
This hardware-software synergy shines brightest when paired with Gemma, Google’s family of lightweight open models. To show you what’s possible, we'll demonstrate how Gemma and LiteRT on a Raspberry Pi 5 can power the Reachy Mini robot to perceive and react to its environment entirely locally in real time. Read on to get started with your own deployment.
Gemma models are well suited for building autonomous agents, smart cameras, and social robotics that can reason and execute complex, multi-step workflows directly on your Raspberry Pi. To accommodate different hardware constraints, the Gemma family of models provides several highly efficient options:
Through LiteRT-LM, a specialized orchestration layer on top of LiteRT, developers can seamlessly deploy Gemma right out of the box. Under the hood, sophisticated CPU acceleration via LiteRT and XNNPACK ensures the Gemma family of models is highly optimized for resource efficiency and low-latency execution directly on the Raspberry Pi.
On a Raspberry Pi 5, LiteRT-LM delivers a robust performance for Gemma 4 E2B, achieving 99 tokens/sec for prefill and 9 tokens/sec for decode, all while maintaining a remarkably low peak memory footprint of just 1432 MB. This brings Gemma’s highly responsive, general-purpose intelligence to Raspberry Pi.
Thanks to Gemma 4 E2B's highly efficient tokenizer, which packs more text into fewer tokens (averaging ~4.2 characters per token), LiteRT-LM achieves an impressive end-to-end generation speed of ~27.3 characters per sec, roughly 300 words per minute (wpm), in the Reachy Mini voice demo. This throughput makes Gemma 4 E2B excellent for real-time speech and translation tasks, delivering text at twice the speed of normal human speech (~150 wpm).
Explore more ready-to-use open models to run on Raspberry Pi from the LiteRT Hugging Face Community.
On the Raspberry Pi 5, the quad-core ARM Cortex-A76 CPU is a raw computing powerhouse, delivering ~153.6 GFLOPS (FP32) and up to ~2.0 TOPS (INT8). In comparison, the integrated Broadcom VideoCore VII GPU is clocked at 800 MHz and offers a peak of ~76.8 GFLOPS (FP32) and ~0.24 TOPS (INT8).
While the CPU possesses a massive capacity advantage, the GPU introduces heterogeneous parallel execution, a paradigm critical for real-time edge applications. Rather than saturating the CPU, developers can delegate tasks across both processors to optimize overall system and thermal efficiency. For example, by offloading continuous vision or audio models to the VideoCore VII GPU, it preserves high-priority CPU cycles for overall system monitoring, pipeline orchestration, or computationally demanding LLM inference.
As such, we have enabled GPU inference on the Raspberry Pi with LiteRT’s WebGPU (Vulkan) backend via ML Drift. This integration allows you to run a wide range of computer vision, audio, and embedding models directly from the LiteRT Hugging Face Community, including seamless support for popular MediaPipe models, Ultralytics YOLO models, Moonshine, and much more.
The table below demonstrates the CPU and GPU latency of running classic computer vision and audio models via LiteRT:
The Reachy Mini pipeline is a powerful showcase of low-latency, real-time edge AI inference running entirely on the Raspberry Pi 5. By leveraging LiteRT, the system splits intensive vision and language workloads into a concurrent, dual-processing architecture across CPU and GPU.
Here is how the parallel architecture works under the hood to ensure seamless interactions:
See the full source code of the Reachy demo in the LiteRT Samples Github repo.
LiteRT provides a comprehensive suite of tools that covers the full development cycle: conversion, quantization, benchmark, and inference. For a fast, frictionless setup, the most straightforward approach is using the LiteRT CLI tool. Rather than requiring developers or coding agents to manually manage multiple independent libraries, the LiteRT CLI aggregates core edge workflows into a single, unified command set.
You can now supercharge your development cycle by adding the LiteRT CLI skill and other advanced LiteRT skills into your AI coding agent, such as Google Antigravity. This empowers agents to autonomously orchestrate and execute complex, multi-stage machine learning workflows on your behalf. For example, you can easily build your own voice translator completely offline on a Raspberry Pi, like the Gemma Translator shown below.
Link to Youtube Video (visible only when JS is disabled)
Explore the complete implementation details in the Gemma Translator GitHub repo.
For resource-constrained IoT devices, minimizing storage and memory overhead is critical. Without special optimization, generic AI runtimes often bundle heavy desktop or server dependencies. In contrast, LiteRT is engineered specifically for on-device deployment, maintaining an exceptionally lean and modular distribution.
The table below compares the download footprint required to run LLM inference on a Raspberry Pi (ARM64 Linux).
You can install the LiteRT CLI and run your first model on a Raspberry Pi 5 with just a few simple commands.
1.Install LiteRT CLI
To get started, install the LiteRT CLI via pip (ideally within a virtual environment):
pip install litert-cli2. Run the model
Download and run any compatible model directly from the LiteRT Hugging Face Community. The code snippet below demonstrates how to execute Gemma 4 E2B (e.g. gemma-4-E2B-it-litert-lm) on the Raspberry Pi 5.
Run the model by providing your Hugging Face authentication token:
export HUGGING_FACE_HUB_TOKEN=<your_hugging_face_token_here>
litert lm run \
--from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm \
gemma-4-E2B-it.litertlm \
--attachment=image.jpg \
--prompt="You are Reachy Mini. Identify the main object in front of you, "\
"state its location (Left/Right/Center), and suggest head action in "\
"10 words or less."We are excited to share that LiteRT integration and Gemma models are coming soon to Hailo AI accelerators! This update will allow you to seamlessly offload model inference to the Raspberry Pi AI HAT+ and AI HAT+ 2, delivering massive hardware acceleration benefits through the exact same, familiar LiteRT workflows you use today.
Explore our resources and start your journey with LiteRT:
We value your input. Please share your thoughts, feedback, or feature requests by opening an issue on our GitHub Issue Tracker. Share your cool Raspberry Pi + LiteRT + Gemma projects with @googlegemma. We can't wait to see what you build!
Google: Changming Sun, Chintan Parikh, Cormac Brick, Dillon Sharlet, Daisuke Majima, Erin Walsh, Frank Barchard, Glenn Cameron, Ian Ballantyne, Jingjiang Li, Jun Jiang, Kimish Patel, Lu Wang, Matthias Grundmann, Rodney Witcher, Sachin Kotwani, Sasha Denisov, Scott Loftin, Shuangfeng Li, Somdatta Banerjee, Terry (Woncheol) Heo, Volodymyr Kysenko, Weiyi Wang, Yi-Chun Kuo, Yu-hui Chen, and gtech team
Raspberry Pi & Hailo: Ashley Whittaker, Eldad Rubinstein, José María Casanova (Igalia), Naushir Patuck, and Sarah Cunningham
Ultralytics: Francesco Mattioli, Onuralp Sezer, Lakshantha Dissanayake
Moonshine AI: Pete Warden
