Microsoft's New Maia 300 Chip Is Designed to Make Nvidia Optional

Microsoft wants to stop depending on someone else's chips to run its AI business. Today the company unveiled Maia 300, its third-generation in-house AI accelerator, and positioned it as nothing less than the future default hardware for running AI across Microsoft's products.

That's a notable shift in ambition. Microsoft has built custom silicon before, but it has always run alongside Nvidia and AMD GPUs rather than aiming to replace them. Maia 300 is explicitly designed to change that balance. According to Scott Guthrie, executive vice president of Microsoft's cloud group, the company plans to scale its Azure Maia chip family to multi-gigawatt deployments starting in 2027, which he says will make Maia "the primary AI inference system for Microsoft products and services."

Inference, in this context, means the actual work of running a trained AI model to generate answers, images, or actions, as opposed to the separate and even more computationally brutal process of training the model in the first place. Microsoft's own products, from Copilot to GitHub Copilot, lean heavily on inference at massive scale, which is exactly where a custom chip tuned for the company's own workloads can pay off.

What's actually new

Maia 300 is the successor to Maia 200, which Microsoft only announced in January and has already deployed across its fleet. That's a fast turnaround for a chip generation, and it signals how aggressively Microsoft is iterating on its silicon strategy. Guthrie says Maia 200 is already "the best performing AI infrastructure" in Microsoft's fleet, measured by real-world service metrics, and it currently powers production workloads including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft's own AI models.

Maia 300 is meant to be a bigger leap than a typical generational bump. Guthrie says Microsoft expects it to deliver 40 percent more tokens per megawatt than leading GPU-based systems — tokens being the chunks of text (or other data) that AI models process and generate, and tokens-per-megawatt being a way of measuring how much useful AI work a chip squeezes out of a given amount of power. In a full 96-accelerator rack configuration, Microsoft says Maia 300 delivers up to seven times the rack-scale token throughput of Maia 200.

Power efficiency has become the central currency of the AI infrastructure race. Data centers running large language models are power-constrained in a way that's increasingly expensive and, in some regions, genuinely hard to solve by just building more capacity. A chip that does meaningfully more work per megawatt isn't just a cost saver, it's a way to squeeze more AI capability out of power and data center space that's already scarce.

The engineering behind the gains

Microsoft credits much of that efficiency jump to a new chiplet architecture. Instead of cramming compute, memory control, and input/output functions onto a single piece of silicon, Maia 300 splits them apart, built on TSMC's N2P manufacturing process. Microsoft's Azure hardware and infrastructure team says this approach "increases the silicon area dedicated to compute performance and allows each function to use the technology best suited to its role."

In plain terms: rather than forcing every part of the chip to use the same manufacturing compromise, Microsoft can optimize memory handling, data movement, and raw computation separately, then stitch them together. That's a design philosophy increasingly common in advanced chips, and it's part of why Microsoft believes Maia 300 can outpace its predecessor by such a wide margin rather than improving incrementally.

Microsoft also frames Maia 300 as purpose-built for the kind of AI workloads that have emerged over just the past year or so: long-context reasoning, where a model has to track and work with huge amounts of information across a conversation or document; latency-sensitive conversational AI, where any lag is immediately noticeable to a user; and sustained, multi-step agentic tasks, where an AI system has to carry out a chain of actions over time rather than just answering a single prompt. These are heavier, more complex workloads than the simpler chatbot queries that defined the first wave of generative AI products, and Microsoft is betting that generic GPU infrastructure, however powerful, isn't the most efficient way to serve them at scale.

Why Microsoft wants off the Nvidia treadmill

The business logic here is straightforward. Nvidia's AI chips are in enormous demand and command steep prices, and Microsoft, like every major cloud provider, has spent years buying as many as it could get. Owning its own silicon stack gives Microsoft more control over cost, supply, and the pace of its own roadmap, rather than being at the mercy of Nvidia's production schedule and pricing.

There's also a strategic angle tied to Microsoft's AI model ambitions. Mustafa Suleyman, Microsoft's AI CEO, framed Maia 300 as inseparable from the company's push to build competitive AI models of its own, alongside the likes of Google DeepMind, OpenAI, and Anthropic. "We have to work across the full stack, all the way down to the silicon," Suleyman said. "At that scale we need a platform that we own, and that we co-design with our models." He called running Microsoft's own AI models on Maia "a critical focus" for his team.

That's a telling comment. Co-designing chips and models together, rather than building models for whatever hardware happens to be available, is the same approach Google has taken with its TPU chips for years, and it's one reason Google has been able to run its AI products at scale without being entirely dependent on Nvidia. Microsoft now appears to be chasing that same advantage.

None of this means Nvidia and AMD GPUs are disappearing from Microsoft's data centers overnight. Maia 300 doesn't start rolling out to production workloads until 2027, and "primary" inference system doesn't mean exclusive. But the direction is clear: Microsoft is building toward a future where its own silicon, tuned specifically for its own models and its own customers, carries the bulk of the AI workload running through Azure. If the efficiency numbers hold up outside of Microsoft's own marketing, that's a meaningful shift in how much leverage Nvidia has over one of its biggest customers.