OpenAI's dissatisfaction with Nvidia chips sparked Cerebras deal

The ChatGPT developer is reportedly unhappy with the speed of certain Nvidia chips and is negotiating with startups that offer alternatives.

OpenAI has been unhappy with some of Nvidia's latest AI chips and has been looking for alternatives since last year, according to Reuters, citing eight sources.

The criticism isn't aimed at chips used for training AI models, where Nvidia dominates. Instead, it's about inference chips, the hardware that lets trained models respond to user queries. Seven sources told Reuters that OpenAI is unhappy with how fast Nvidia's hardware generates responses. Applications like software development with Codex are said to be particularly problematic because speed matters most there. OpenAI reportedly is looking for new hardware for about ten percent of its future inference workload.

Why inference needs different chip designs

Employees have partly attributed these weaknesses to Nvidia hardware. Inference requires more memory access than training. Nvidia GPUs use external memory, which slows down processing. OpenAI is therefore looking for chips with SRAM embedded directly on the silicon, which offers speed advantages.

According to Reuters, OpenAI has been negotiating with startups like Cerebras and Groq for a few month. Cerebras turned down an acquisition offer from Nvidia and instead signed a deal with OpenAI. CEO Sam Altman confirmed in late January that the Cerebras deal is meant to meet speed requirements for coding models.

Things went differently with Groq: In December, Nvidia signed a $20 billion licensing agreement with the startup, which ended OpenAI's negotiations. Nvidia also hired Groq's chip designers. Meanwhile, Nvidia has introduced Rubin CPX, a specialized accelerator designed specifically for the prefill phase of AI inference.

$100 billion investment stalls

In September, Nvidia announced plans to invest up to $100 billion in OpenAI. The deal was expected to close within weeks, but instead negotiations have dragged on for months. OpenAI's shifting product roadmap has slowed the talks, according to one source.

Nvidia CEO Jensen Huang dismissed reports of tensions as "nonsense" on Saturday. The company still plans to invest tens of billions of dollars. An OpenAI spokesperson said the company continues to rely on Nvidia for the majority of its inference fleet.