Content
summary Summary

At its annual Next '24 developer conference, Google unveiled further developments to its AI Hypercomputer architecture. The focus is on new performance-optimized hardware components such as the Cloud TPU v5p and Nvidia's upcoming Blackwell GPUs.

Ad

Google Cloud is announcing a number of enhancements to its AI Hypercomputer architecture at Next '24, including general availability of the TPU v5p and integration with the latest Nvidia Blackwell platform. These innovations are designed to accelerate the training and deployment of sophisticated AI models.

The Cloud TPU v5p is now generally available and is the most powerful and scalable TPU generation to date, according to Google. A single TPU v5p pod contains 8,960 connected chips - more than twice as many as a TPU v4 pod. In addition, the TPU v5p offers more than twice as many chip-level FLOPS and three times as much high-speed memory as the previous generation.

This makes the TPU v5p ideal for training large AI models. To facilitate this, the Google Kubernetes Engine (GKE) now fully supports TPU v5p clusters and multi-host serving. According to Google, the latter makes it possible to manage and monitor a group of model servers distributed across multiple hosts as a single logical unit.

Ad
Ad

Google launches Blackwell instances in early 2025

Google Cloud is also expanding its GPU offerings. A new Nvidia H100 GPU-based A3 Mega instance will be generally available next month. It will offer twice the GPU-to-GPU network bandwidth of the A3.

Google also announced that the new Nvidia Blackwell platform will be integrated into its AI Hypercomputer architecture in two configurations. Google Cloud customers will have access to Nvidia's HGX B200 and GB200 NVL72 systems in the spring of 2025. The new HGX B200 systems are designed for today's most demanding AI, data analytics, and HPC workloads. Taking it a step further, the liquid-cooled GB200 NVL72 systems are designed for real-time language model inference and trillion-parameter model training.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Google announced at its Next '24 developer conference the general availability of the powerful TPU v5p and the integration of the upcoming Nvidia Blackwell platform to accelerate the training and deployment of sophisticated AI models.
  • A single TPU v5p pod contains 8,960 synchronous chips, twice as many as the previous generation TPU v4. In addition, the TPU v5p offers more than twice as many FLOPS and three times as much high-speed memory at the chip level as the TPU v4.
  • From spring 2025, Google Cloud customers will have access to Nvidia's HGX B200 and GB200 NVL72 systems from the new Blackwell platform. These systems are designed for today's most demanding AI, data analytics and HPC workloads, as well as real-time language model inference and training of trillion-parameter models.
Sources
Max is managing editor at THE DECODER. As a trained philosopher, he deals with consciousness, AI, and the question of whether machines can really think or just pretend to.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.