Google is reportedly gearing up for a massive expansion of its AI infrastructure. Internal documents show the company wants to boost performance by a factor of 1,000 over the next four to five years.
Google's AI infrastructure boss, Amin Vahdat, told employees that the company has to double its serving capacity every six months to meet demand for AI services. According to CNBC, he shared a slide in early November outlining how the company could reach a thousandfold performance increase without using much more energy or money.
Meeting that goal will require more efficient AI models, new AI chips, tighter hardware-software co-design, and support from Deepmind's research teams, which are helping Google anticipate future model capabilities and compute demands.
Vahdat said the company has to race to build out compute capacity in order to meet demand. He described the race to build AI infrastructure as "the most critical and also the most expensive part" of the AI race. Google doesn't have to outspend competitors, he said, but it will "spend a lot" to build an infrastructure that is "far more reliable, more performant and more scalable than what's available anywhere else."
Google's own hardware is a major part of that strategy. Last week, the company unveiled the seventh generation of its Tensor Processing Units, codenamed "Ironwood". Google says the new TPU is nearly 30 times more energy-efficient than the first cloud TPU introduced in 2018.
OpenAI CEO Sam Altman recently made a similar point, arguing that the AI race ultimately comes down to securing as much compute as possible. To keep pace with chip makers and cloud providers, OpenAI is taking on significant debt.
Even Google employees worry about a potential AI bubble
At the same meeting, Google employees raised concerns about the financial risks tied to these investments. CEO Sundar Pichai acknowledged those worries, noting that fears of an AI bubble are "definitely in the zeitgeist." Still, he argued, as before, that underinvesting would be riskier than spending too much.
Pichai pointed to strong demand in Google's cloud business, which just recorded 34% annual revenue growth to more than $15 billion in the quarter. He said the numbers could have been even higher if more compute capacity had been available.
As an example, Pichai cited Google's viral video model Veo. "If we could've given it to more people in the Gemini app, I think we would have gotten more users but we just couldn't because we are at a compute constraint," he said.
Another highly rated employee question read at the meeting criticized the widening gap between capital spending and operating income and asked how Google plans to maintain healthy free cash flow over the next 18 to 24 months. CFO Anat Ashkenazi pointed to existing opportunities, including moving more customers from on-premise data centers to Google Cloud.
