6 results
JULY 24, 2026 / AI
This second installment explores how Ray’s higher-level libraries—Serve, Data, and Train—abstract the complexities of running AI workloads on Google's TPU slices. Ray Serve uses a simple topology configuration to correctly gang-schedule large multi-host models, while Ray Data eliminates data-loading bottlenecks by feeding accelerators directly with native JAX batches. Finally, JaxTrainer streamlines distributed training across TPUs by automatically handling cross-slice coordination, checkpointing, and fault tolerance.
JULY 20, 2026 / AI
Ray 2.55 introduces official, first-class support for Google Cloud TPUs, enabling developers to run distributed Python workloads on Google's accelerators using the familiar Ray task-and-actor APIs. To handle the strict networking requirement of keeping multi-host TPU "slices" together over their Inter-Chip Interconnect (ICI), the KubeRay Operator on GKE automatically provisions and labels the underlying hardware layout. Ray Core utilizes these labels via its slice_placement_group() primitive to atomically reserve complete slices, allowing developers to deploy jobs through KubeRay, Ray Train, or Ray Serve simply by declaring a hardware topology (like "4x4") without writing custom placement code.
JULY 6, 2026 / AI
Distributed AI training is notoriously fragile because losing a single machine typically crashes the entire multi-node job, forcing a time-consuming, full-workload infrastructure restart. To address this, Google’s JAX ecosystem utilizes elastic training via Pathways, which converts a hardware failure into a catchable Python exception so the running process can survive. When an unplanned failure occurs, the system automatically replaces only the broken worker, restores the last viable checkpoint from Cloud Storage, and resumes training in place—minimizing total downtime to under two minutes without ever restarting the main controller process.
APRIL 29, 2025 / Cloud
Llama 4, Meta's advanced large language model, is now generally available as a fully managed API on Vertex AI, simplifying deployment and management. The Llama 3.3 70B managed API is also generally available, offering users greater flexibility.
SEPT. 26, 2024 / AI
Vertex AI Prompt Optimizer, a new managed automated prompt engineering service, can help save time and effort in prompt engineering while ensuring performing prompts ready for your generative AI applications.
JULY 9, 2024 / Gemma
Use Gemma Supervised tuning on Ray on Vertex AI to train and serve machine learning models efficiently and effectively.
