Since we first introduced LiteRT in 2024, we have focused on evolving our ML tech stack from its TensorFlow Lite (TFLite) foundation into a modern on-device AI framework. While TFLite set the standard for classical ML, our mission is to empower developers to deploy today’s cutting-edge AI on-device just as seamlessly as they integrated classical ML in the past.
At Google I/O ‘25, we shared a preview of this evolution: a high-performance runtime designed specifically for advanced hardware acceleration. Today, we are excited to announce that these advanced acceleration capabilities have fully graduated into the LiteRT production stack, available now for all developers.
This milestone solidifies LiteRT as the universal on-device inference framework for the AI era, representing a significant leap over TFLite for being:
All of this is delivered while maintaining the same reliable, cross-platform deployment you trust since TFLite.
Here is how LiteRT empowers you in building the next-generation of on-device AI.
Moving beyond the initial GPU acceleration on Android announced at I/O ‘25, we are excited to introduce the full, comprehensive GPU support across Android, iOS, macOS, Windows, Linux, and Web. This expansion provides developers with a reliable, high-performance acceleration option that scales significantly beyond classical CPU inference.
LiteRT maximizes the reach by introducing robust support for OpenCL, OpenGL, Metal, and WebGPU, via ML Drift, our next-generation GPU engine, allowing you to deploy models efficiently across mobile, desktop, and web. On Android, LiteRT optimizes this further by automatically prioritizing OpenCL when available for peak performance, while falling back to OpenGL for broader device coverage.
Empowered by ML Drift, LiteRT GPU has achieved a significant leap in efficiency, delivering substantial performance gains that average 1.4x faster over the legacy TFLite GPU delegate, significantly reducing latency across a broad range of models. See more benchmark results in our previous announcement.
To enable high-performance AI applications, we have also introduced key technical advancements to optimize end-to-end latency, specifically asynchronous execution and zero-copy buffer interoperability. These features significantly reduce unnecessary CPU overhead and boost overall performance, fulfilling the stringent requirements for real-time use cases like background segmentation and speech recognition (ASR). In practice, these optimizations can result in up to 2x faster performance, as demonstrated in our Segmentation sample app. For a closer look at the improvements, see our technical deep dive.
The following examples demonstrate how easily you can leverage GPU acceleration with the new CompiledModel API in C++:
// 1. Create a compiled model targeting GPU in C++.
auto compiled_model = CompiledModel::Create(env, "mymodel.tflite",
kLiteRtHwAcceleratorGpu);
// 2. Create an input TensorBuffer that wraps the OpenGL buffer (i.e. from
image pre-processing) with zero-copy.
auto input_buffer = TensorBuffer::CreateFromGlBuffer(env, tensor_type,
opengl_buffer);
std::vector<TensorBuffer> input_buffers{input_buffer};
auto output_buffers = compiled_model.CreateOutputBuffers();
// 3. Execute the model.
compiled_model.Run(inputs, outputs);
// 4. Access model output, i.e. AHardwareBuffer.
auto ahwb = output_buffer[0]->GetAhwb();See more instructions on LiteRT cross-platform development and GPU acceleration from LiteRT DevSite.
While CPU and GPU offer broad versatility for AI tasks, the NPU is the key to unlock the smooth, responsive, and high-speed AI experience that modern applications demand. However, fragmentation across hundreds of NPU SoC variants often forces developers to navigate a maze of disparate compilers and runtimes. Furthermore, because traditional ML infrastructure has historically lacked deep integration with specialized NPU SDKs, the result has been complex, ad-hoc deployment workflows that are difficult to manage in production.
LiteRT addresses these challenges by providing a unified, simplified NPU deployment workflow that abstracts away low-level, vendor-specific SDKs and handles fragmentation across numerous SoC variants. We have streamlined this into a simple, three-step process to get your models running with NPU acceleration easily:
For a full, detailed guide, including colab and sample apps, visit our LiteRT NPU documentation.
To provide flexible integration options that fit your specific deployment needs, LiteRT offers both ahead-of-time (AOT) and on-device (JIT) compilation. This allows you to choose the best strategy based on your application’s unique requirements:
We are collaborating closely with silicon leaders across the industry to bring high-performance NPU acceleration to developers. Our first production-ready integrations with MediaTek and Qualcomm are available now. Read our technical deep-dives to see how we achieved best-in-class NPU performance, reaching speeds up to 100x faster than CPU and 10x faster than GPU:
Building on this momentum, we are actively expanding LiteRT’s NPU support to additional hardware. Stay tuned for further announcements!
Open models offer unparalleled flexibility and customization, yet deploying them remains a high-friction process. Navigating the complexities of model lowering, inference, and benchmarking often demands significant engineering overhead. To bridge this gap and enable developers to build custom experiences efficiently, we provide the following integrated tech stack:
Together, these components offer a production-grade path for running popular open models with leading performance. To demonstrate this, we benchmarked Gemma 3 1B on Samsung Galaxy S25 Ultra, comparing LiteRT and Llama.cpp.
LiteRT demonstrates a clear performance advantage, outperforming llama.cpp by 3x on CPU, 7x on GPU for decode (memory-bound), and 19x on GPU for prefill (compute-bound). Furthermore, LiteRT’s NPU acceleration delivers an additional 2x performance gain over the GPU for prefill, maximizing the potential of compute hardware. For a detailed look at the engineering behind these benchmarks, read our deep dive into LiteRT’s optimizations under-the-hood.
LiteRT supports an extensive and growing collection of popular open-weight models, meticulously optimized and pre-converted for immediate deployment, including:
These models are available on the LiteRT Hugging Face Community and can be explored interactively via the Google AI Edge Gallery app on Android/Play and iOS.
For more development details, visit our LiteRT GenAI documentation.
Deployment shouldn't be dictated by your choice of training framework. LiteRT offers seamless model conversion from the industry's most popular ML frameworks: PyTorch, TensorFlow, and JAX.
By consolidating these paths, LiteRT enables high research-to-production velocity regardless of your development environment. You can author models in your preferred framework and rely on LiteRT to deliver performance across CPU, GPU, and NPU backends.
To get started, explore the LiteRT Torch Colab and try the conversion process yourself, or dive into the technical details of our PyTorch integration in this tech deep dive.
While the capabilities of LiteRT have significantly expanded, our commitment to long-term reliability and cross-platform consistency remains unchanged. LiteRT continues to build on the proven .tflite model format, the industry-standard, single-file format that ensures your existing models remain portable and compatible across Android, iOS, macOS, Linux, Windows, Web, and IOT.
To provide developers with a continuous experience, LiteRT offers robust support for both existing and next-generation execution paths:
Ready to build the future of on-device AI? Get started today with these essential resources:
Let us know your feedback and feature requests by opening an issue on our GitHub channel. We can’t wait to see what you build with LiteRT!
Thank you to the members of the team, and collaborators for their contributions in making the advancements in this release possible: Advait Jain, Andrew Zhang, Andrei Kulik, Akshat Sharma, Arian Arfaian, Byungchul Kim, Changming Sun, Chunlei Niu, Chun-nien Chan, Cormac Brick, David Massoud, Dillon Sharlet, Fengwu Yao, Gerardo Carranza, Jingjiang Li, Jing Jin, Grant Jensen, Jae Yoo, Juhyun Lee, Jun Jiang, Kris Tonthat, Lin Chen, Lu Wang, Luke Boyer, Marissa Ikonomidis, Matt Kreileder, Matthias Grundmann, Majid Dadashi, Marko Ristić, Matthew Soulanille, Na Li, Ping Yu, Quentin Khan, Raman Sarokin, Ram Iyengar, Rishika Sinha, Sachin Kotwani, Shuangfeng Li, Steven Toribio, Suleman Shahid, Teng-Hui Zhu, Terry (Woncheol) Heo, Vitalii Dziuba, Volodymyr Kysenko, Weiyi Wang, Yu-Hui Chen, Pradeep Kuppala and gTech team.
