Meta is building the world's fastest AI supercomputer. It's designed to accelerate Meta's AI research and Metaverse development.
The supercomputer goes by the name AI Research SuperCluster, or RSC for short. It is already being used by Meta, and after the end of the second construction phase in mid-2022, RSC is set to become the world's fastest supercomputer specializing in artificial intelligence.
Construction began in early 2020, and the supercomputer's infrastructure has been redesigned from the ground up, from its cooling systems and power supply to its network and cabling.
The goal is to train AI models with more than a trillion parameters and data sets that are up to an exabyte in size. That's equivalent to 36,000 years of high-quality video, Meta writes.
RSC: Gigantic AI training capacity.
When the second phase of construction is complete, the supercomputer will have access to 16,000 GPUs and will be able to process data at the rate of 16 terabytes per second. By comparison, Microsoft's AI supercomputer used to train GPT-3 can access 10,000 GPUs.
According to initial benchmarks, computer vision workflows run twenty times faster than on Meta's previous GPU cluster, and large language models can be trained three times faster, the company writes.
Whereas it used to take nine weeks to train a language model with tens of billions of parameters, the task takes three weeks with RSC. The goal is to train language models with several trillion parameters. By comparison, GPT-3 comes up with 175 billion parameters.
One AI supercomputer, many applications
The supercomputer serves many purposes: to accelerate Meta's own AI research in areas such as embodied, egocentric, and multimodal AI, to improve content moderation on platforms such as Facebook and Instagram by evaluating images, videos, and text more quickly and accurately.
Furthermore, it is supposed to train more complex language models for translation and speech recognition and to enable advances in augmented reality features such as object recognition that will be important in the development of AR glasses.
"We hope RSC will help us develop entirely new AI systems that, for example, enable real-time language translation for large groups of people who all speak a different language, so they can seamlessly collaborate on a research project or play an AR game together," Meta writes.
"Ultimately, the work done with RSC will pave the way to developing technologies for the next big computing platform - the Metaverse, where AI-driven applications and products will play an important role."
