OpenAI Seeks Chip Alternatives Amid Strategic Pivot, Testing Nvidia’s AI Dominance

OpenAI has grown dissatisfied with certain Nvidia artificial intelligence chips and has been actively exploring alternatives since last year, according to eight sources familiar with the matter—a move that could strain one of the most high-profile partnerships in the AI industry.

The strategic shift, details of which are reported here for the first time, stems from OpenAI’s increasing focus on chips specialized for AI inference—the stage in which trained models, such as the one behind ChatGPT, process and respond to user queries in real time. While Nvidia remains the dominant supplier of chips for training massive AI models, inference has emerged as a critical new battleground in the accelerating race for AI efficiency and scale.

This search for inference-optimized hardware marks a significant test of Nvidia’s grip on the AI chip market and comes as investment talks between the two companies have stretched for months. Last September, Nvidia signaled its intent to invest up to $100 billion in OpenAI—a deal that would give the chipmaker equity in the startup while funding OpenAI’s enormous appetite for advanced semiconductors. Although initially expected to close within weeks, negotiations have since slowed, partly due to OpenAI’s evolving hardware requirements and its parallel engagements with competitors.

During this period, OpenAI has secured GPU supplies from AMD and others, while also holding discussions with inference-focused chip startups such as Cerebras and Groq, according to two sources. However, Nvidia’s recent $20 billion licensing agreement with Groq effectively ended OpenAI’s talks with the startup, one source told Reuters—a move industry executives interpreted as an attempt by Nvidia to consolidate key inference technologies amid rapidly shifting competitive dynamics.

Nvidia, in a statement, said Groq’s intellectual property was “highly complementary to NVIDIA’s product roadmap.” The company also emphasized its continued strength in inference, stating, “Customers continue to choose NVIDIA for inference because we deliver the best performance and total cost of ownership at scale.”

The Inference Imperative

Nvidia’s graphics processing units (GPUs) have powered the global AI boom, excelling at the immense computational workloads required to train large language models. However, as AI adoption grows, the industry’s focus is expanding toward deploying and running these models at scale—a task that places different demands on hardware.

According to seven sources, OpenAI has encountered performance limitations with Nvidia’s hardware in specific inference workloads, particularly in domains like software development and AI-to-software communication, where response speed is critical. Internally, some at OpenAI attributed weaknesses in Codex, its code-generation product, to the constraints of GPU-based systems, one source noted.

Inference often requires greater memory bandwidth and faster data retrieval than training, leading OpenAI to evaluate chips built with large embedded SRAM (Static Random-Access Memory)—memory integrated directly into the processor. Such designs can reduce latency and accelerate responses for services like ChatGPT, which handles millions of simultaneous queries.

By contrast, Nvidia and AMD GPUs typically rely on external memory, which can introduce delays during data fetching. Competing products like Anthropic’s Claude and Google’s Gemini have benefited from deployments on custom in-house chips, such as Google’s Tensor Processing Units (TPUs), which are engineered for inference-specific calculations.

OpenAI’s Balancing Act

Despite its search for alternatives, OpenAI has been careful to publicly affirm its reliance on Nvidia. After the Reuters report was published, CEO Sam Altman posted on X that Nvidia makes “the best AI chips in the world” and that OpenAI hopes to remain a “gigantic customer for a very long time.” An OpenAI spokesperson separately noted that the vast majority of its inference fleet still runs on Nvidia hardware, which delivers “the best performance per dollar for inference.”

Yet during a January 30 call with reporters, Altman acknowledged that speed is a premium for certain applications, particularly AI-assisted coding—a demand he said would be addressed in part through OpenAI’s recently announced partnership with Cerebras. He contrasted this with casual ChatGPT use, where latency is less critical.

Behind the scenes, OpenAI’s goal is to secure new hardware that could eventually meet roughly 10% of its inference computing needs, one source said—a seemingly modest portion that nonetheless represents a strategic foothold in diversifying its supply chain and pushing innovation in inference-optimized architectures.

Nvidia’s Countermoves

As OpenAI’s interest in alternatives became apparent, Nvidia approached several companies developing SRAM-heavy chips, including Cerebras and Groq, regarding potential acquisitions, according to sources. Cerebras declined, opting instead to strike a commercial agreement with OpenAI last month. Groq, which had been in talks to provide computing power to OpenAI, instead entered into a non-exclusive licensing deal with Nvidia in December—a transaction that has since shifted Groq’s focus toward cloud-based software offerings, after Nvidia hired away key Groq chip designers.

Nvidia CEO Jensen Huang dismissed reports of tension with OpenAI as “nonsense” in remarks on Saturday, reiterating Nvidia’s planned investment in the AI leader. Still, the flurry of deal-making highlights how quickly the inference chip landscape is evolving—and how determined both tech giants and startups are to carve out roles in the next phase of AI infrastructure.

Looking Ahead

OpenAI’s dual path—continuing to depend on Nvidia while nurturing partnerships with competitors—reflects the delicate balance many AI companies must strike between performance, cost, and supply chain resilience. As inference workloads grow in volume and complexity, the market is likely to see more specialized chips emerge, challenging Nvidia’s end-to-end dominance.

Whether Nvidia’s upcoming architectures will address OpenAI’s specific inference concerns—or whether alternatives from Cerebras, AMD, or future startups will gain meaningful traction—remains an open question. What is clear is that the race to build faster, more efficient AI inference hardware is now a central front in the wider battle for AI supremacy.

 

 

Our Pashto-Dari Website

  Donate Here