Overview
- Google, which announced the eighth‑generation TPUs Wednesday at Cloud Next in Las Vegas, split its line into TPU 8t for training and TPU 8i for inference with Broadcom as co‑developer and new Axion Arm CPUs as hosts.
- TPU 8t targets massive training runs by scaling a superpod to 9,600 chips with 2 petabytes of shared high‑bandwidth memory and optical switching on a new Virgo network to keep jobs running at high utilization.
- TPU 8i is tuned for low‑latency serving with 384 MB of on‑chip SRAM to keep key model data close to the cores, a Boardfly topology that shortens chip‑to‑chip paths, and a Collectives Acceleration Engine that cuts synchronization waits by up to five times.
- Google claims TPU 8t delivers about 2.8x the training performance of Ironwood at the same price, and says TPU 8i improves inference performance per dollar by roughly 80% with better energy efficiency for both chips.
- Both TPUs arrive later this year in Google Cloud, where customers will also get access to Nvidia’s next‑generation Vera Rubin GPUs, as Google touts growing TPU use by Anthropic, Citadel Securities, and all 17 U.S. Department of Energy national labs.