Update from July 22, 2024:
Musk announced that his new "Gigafactory of Compute" has begun training its first model. With 100,000 Nvidia H100 chips, it is currently the world's largest GPU cluster for AI training. According to Musk, it will train the "world's most powerful AI by every metric by December," likely referring to Grok 3 (see article below). Grok 2 is supposed to launch in August.
Original article from July 10, 2024:
Elon Musk delays Grok 2 AI model to August, promises Grok 3 by end of year
Elon Musk has delayed the release of Grok 2, originally scheduled for May, until August. Grok 3, a more significant upgrade, is planned for the end of the year.
Musk announced Grok 2 in March, claiming it would outperform OpenAI's GPT-4. At the time, he didn't provide specifics on the new model's capabilities.
xAI has leased 24,000 H100 GPUs from Oracle to train Grok 2. The model is currently being fine-tuned and debugged.
The delay to August, Musk suggests, is partly due to the time-consuming process of cleaning the training data to remove content from other language models. He claims that Grok 2 will be a "giant improvement" in this area.
Musk teased Grok 3 in early July, saying that it will be trained on "100k H100s" and promising that this amount of computing power will make Grok 3 "really something special."
For Grok 3, xAI is building its own hardware infrastructure, moving away from Oracle. xAI claims it's creating the world's most powerful training cluster, and according to Musk, AI training on this cluster will begin this month.
Musk explains the shift away from Oracle, which he calls a "great company," by saying that xAI needs to "have our own hands on the wheel, rather than be a backseat driver" to outpace other AI companies.
Grok's current model as X's chatbot hasn't yet reached the level of ChatGPT or Claude, despite (or perhaps because of) its access to X's real-time data. Its news implementation has led to some absurd fake news stories.
