On 18th October, 2024, Google hosted the very first Web AI Summit to bring together top minds from around the world working with machine learning models client-side in the web browser. This means after the initial page load, all of these solutions could work entirely offline on the client device, allowing the users to benefit from low latency inference, lower costs, and privacy.
Our lineup included presenters not only from Google’s teams such as Chrome and MediaPipe, but also active 3P representatives in the space such as Intel, Hugging Face, Microsoft, LangChain and beyond. From consumer packaged goods detection to healthcare solutions - talks covered a wide range of industries and subject areas showing just how far Web AI can reach.
Read on for more details or view the YouTube playlist to catch up right away and see the talks for yourself!
We had over 1,100 registrations from folk spanning 22 countries, 59 cities, and 179 different Google offices joining us for this historic event with a full house the whole day - it was great to see how engaged everyone was for the talks.
We had a mixture of software engineers, business decision makers, and executive leadership in the audience, creating a productive synergy between technical expertise and strategic planning.
Our expert speakers shared invaluable insights to equip Javascript developers with knowledge on sophisticated and complex AI-powered features that are becoming the industry standard to meet clients’ demands. Check all the talks below or view them yourself on a coffee break.
Jason Mayes - Web AI Lead, Google
Jim Bankoski - VP Engineering, Chrome, Google
An overview for the state of Web AI in 2024 and why the Web AI Summit was created. See what's possible with machine learning on-device, as well as where it is heading, to get the 101 before you watch the other talks in this series. This talk is suitable for everyone and covers subject areas such as generative AI, LLMs, diffusion models, WebGPU, WebAssembly, and emerging APIs like WebNN along with examples from industry that are already using Web AI today.
Joshua Lochner - ML Engineer (Transformers.js), Hugging Face
Learn about Transformers.js, an exciting new JavaScript library that empowers developers to build never-before-seen web applications. It is designed to be functionally equivalent to Hugging Face's Python transformers library and supports over 120 architectures across a diverse set of tasks and modalities. Users can choose from over 1,000 pretrained models or convert their own to run locally in the browser, offering privacy-preserving, low-latency, and scalable machine learning. The latest addition of WebGPU support enables highly-performant execution of models by utilizing modern GPU capabilities directly in the browser.
Rob Kochman - Group Product Manager (Chrome), Google
Rafael Cintron - Principle Software Design Engineer, Microsoft
Advanced web technologies like WebAssembly and WebGPU have recently brought real AI capabilities to the browser. The proposed Web Neural Network (WebNN) API aims to build on that momentum, enabling AI workloads to run faster and more efficiently on a variety of devices, including devices with AI accelerator hardware (NPUs), all based on web standards. This session will start with a brief overview of WebNN, then describe recent developments, including API shape, device support, framework support, and browser implementations. We’ll also describe the plan forward, as we work to get feedback from the community.
Moh Haghighat - Fellow, Intel
Intel showcased WebNN, an emerging unified W3C web standard API for on-device web ML acceleration across client AI execution engines: CPU, GPU, and NPU. Currently in Developer Preview on Chrome and Edge browsers and integrated in popular ML frameworks (e.g., ONNX Runtime Web), WebNN delivers “near-native” performance and power characteristics. We will show exciting WebNN demos and adoption previews that bring a new class of experiences to the web.
Yu Lee - Resident Researcher, ML5.js, NYU
Aidan Nelson - Visiting Faculty, ML5.js, NYU
This talk focused on ml5.js - an open source library built on top of TensorFlow.js with a goal of making machine learning approachable for a broad audience of artists, creative coders, and students. This project has been built as a collaborative effort at NYU’s ITP Program, drawing inspiration from Processing and the p5.js project’s focus on making coding accessible and inclusive. ml5.js aims to expand this mission to the domain of machine learning, bridging the gap between the technical complexity of machine learning and the creativity of beginners and artists.
Charlie Ruan - student researcher, CMU
This talk covered WebLLM, a high-performance in-browser LLM inference engine. WebLLM allows building AI-enabled web apps that are fast (native GPU acceleration via WebGPU), private (100% client-side computation), and convenient (zero environment setup). For developers, WebLLM features an OpenAI-API style interface for standardized integration, supports chat applications and efficient structured JSON generation, and offers built-in support for Web/Service Workers to separate backend executions from the UI flow. In this talk, we will explore WebLLM’s key features, overall architecture, and how developers can build AI-enabled web applications with it.
Jacob Lee - Founding Engineer, LangChain
Exciting new advances from projects like WebLLM, Transformers.js, and Chrome AI have brought local LLMs closer than ever to anyone with a browser. This has immense potential to expand the frontiers of web development, but these small models are more limited than state-of-the-art hosted models and require more careful considerations around design and prompting.
This talk focused on addressing these constraints by covering techniques for implementing practical apps that make the most of small models using the powerful toolkit provided by LangGraph.js, a new framework for orchestrating stateful LLM apps.
Ruofei Du - Interactive Perception & Graphics Lead, Google
Visual Blocks for ML is a visual programming platform that empowers rapid AI and multimedia prototyping. In this talk, we will showcase how to build interactive AI pipelines, perform interactive data augmentation, and test pipelines with live data using simple drag-and-drop actions. We will also highlight a range of community-contributed pipelines and custom nodes demonstrating diverse applications in interactive graphics, large language model chains, computer vision, and multi-modal solutions. Finally, we encourage all Web AI practitioners to contribute their own ML pipelines and custom nodes, further enriching the shared platform and inspiring innovative use cases.
Charlie Gerard - Senior Research Engineer, CrowdStrike
The latest advancements in AI have mainly focused on large language models and new ways of creating and consuming content. However, AI also offers the opportunity to rethink the way we interact with interfaces. Using JavaScript and models focused on body tracking or audio classification, web developers have a unique opportunity to experiment with alternative interactions to create more innovative web experiences.
Kenji Baheux - Product Manager, Chrome, Google
Sharing what we've been up to in Chrome for built-in AI, what we've learned, and what's next. We'll talk about how we see the Prompt API, our status for high-level task APIs such as summarization, write / rewrite, learnings from the early preview program, and where we are going from here.
Hugo Zanini - Technical Project Lead, Nubank
This talk showcased how one of the top 10 largest consumer packaged goods (CPG) companies in the world utilized Web AI to expand its in-store trade marketing strategy in Brazil and how it evolved into an open-source project that has been beneficial to other companies in the industry.
Thomas Steiner - Developer Relations Engineer, Chrome, Google
In this talk, Thomas summarized some of the things Developer Relations has learnt in their role as customer zero of Chrome's built-in APIs. Using an example of an AI-powered synonym finder app, he will show how to work with the Prompt API focusing on aspects from tweaking the prompt, to reliably parsing the output, to optimizing the app for maximum performance.
Yuriko Hirota - Partner Solutions Engineer, Google
Kazunari Hara - Developer Expert, CyberAgent
This lightning talk reveals the practical power of client-side AI not just for the sake of using AI, but for enhancing user experiences. The talk took a deep dive into a real-world case study featured in Google I/O 2024, showcasing how CyberAgent, the powerhouse behind one of Japan's top blog services, plans to leverage the magic of client-side AI to empower users with effortless blog title generation. Join us to learn how CyberAgent maximized the potential of client-side AI through innovative use case design and a user-centric approach.
David Li - Product Manager, Chrome, Google
In this talk we will showcase the potential of AI and Chrome Extensions. Chrome Extensions allow you to control the browser, observe web content, and add your own UI. When combined, AI and Chrome Extensions can make the browsing experience truly helpful and more productive. This talk will give an overview on how extensions on the WebStore are using AI today and where we see the biggest potential.
Evgeny Peshkov - CTO, GEENEE
Discover how Web AI is revolutionizing personalized paid media by introducing groundbreaking virtual try-on ads across every screen.
Tyler Mullen - Staff Software Engineer, Mediapipe, Google
Learn about MediaPipe's cross-platform approach to building AI pipelines and bringing them to the browser. We'll highlight some of the benefits of our method and talk about a few of the major products we help power (like Google Meet). Then we'll cover our latest technological advancements and developer APIs. These offerings include solutions for traditional machine learning tasks like image segmentation, as well as generative AI tasks like LLM inference. Finally, we will give a sneak peak into the future with some exciting demos!
Chris Slee - CTO, Include Health
IncludeHealth, a virtual physical therapy provider, harnesses the power of WebAI to break down logistical and economic barriers, allowing patients to receive personalized, measured care anywhere, any time, and on any device
Richard Stotz - Software Engineer, Core ML, Google
Learn how we built Simple ML for Sheets, a free Google Sheets add-on for ML and AI. Simple ML for Sheets uses on-device Machine Learning powered by WebAssembly, Javascript and Chrome’s new built-in AI to unlock advanced Machine Learning tasks for all users. This talk highlights the tools we used to successfully bring Simple ML for Sheets to market and how our team’s open source libraries help developers achieve their own ML successes on the web.
This event would not be possible without the numerous people involved in the creation and running of the event. We would like thank our 3 event creators, Jason Mayes, Jenna Zheng, and Marcus Chang for putting on the event and of course a huge thank you to all of our presenters listed above, along with our helpers and assistants on the day, and our AV teams who ensured the run of show was smoothly recorded for your viewing pleasure after the show.
If you missed the event this time around, catch up via the videos above, and be sure to subscribe to our public Web AI Newsletter to be informed when we next go live!
Federated Credential Management (FedCM) Migration for Google Identity Services
Sharing our latest differential privacy milestones and advancements
People of AI: Season 3 Takeaways and Season 4 Previews
Indie Games Fund: Google Play’s $2m fund in Latin America is back
Chrome on Android to support third-party autofill services natively