Posts by Chris Zuo

1 results

Clear filters
  • NOV. 19, 2026 / AI

    Distributed LLM Training Across Metros: Streaming DiLoCo in MaxText

    Streaming DiLoCo in MaxText enables developers to efficiently train large language models across geographically distributed TPU slices by streaming parameter fragments instead of synchronizing full models at every step. This approach slashes cross-slice communication volume by over 60× and eliminates network stalls, allowing seamless multi-datacenter scaling without degrading model convergence or quality.

    dilco header