Microsoft has sunk billions of dollars into AI software, building out its Azure virtual machine infrastructure and rolling ChatGPT into products like Bing and SwiftKey. In 2019, reports surfaced that Microsoft was moving into AI hardware, designing a chip purpose-built for AI. Now, The Information reports that engineering samples of the new chip, codename Athena, have gone out to some employees of Microsoft and OpenAI.
According to the report (paywalled), Athena will use TSMC’s 5-nanometer process, at least for its initial version. Currently, 5nm is the go-to node for much, if not most, high-performance silicon manufacturing. Both AMD and Apple build on TSMC 5nm. But The Information also claims that the Athena roadmap includes ‘multiple generations,’ which could include TSMC’s forthcoming 3nm process.
In-house silicon, à la Google’s tensor processing units (TPUs) or Amazon’s Graviton 3, could help Microsoft slash costs. Athena will reportedly cost Microsoft about $100M. Currently, Nvidia owns the lion’s share of the GPU market, and Microsoft already extensively uses Nvidia hardware in its Azure server centers. Dylan Patel of research firm SemiAnalysis told The Information, “Athena, if competitive, could reduce the cost per chip by a third when compared with Nvidia’s offerings.”
Credit: Microsoft
Microsoft allegedly bringing its AI chip design in-house is further evidence of a long-term shift in chip manufacturing. For decades, companies like Microsoft and Google took the hardware designs that AMD, Intel, and Nvidia built. This is still the case for most companies, but the biggest players like Apple, Microsoft, and Google now design at least some of their own hardware rather than solely depending on GPU or CPU companies to move the market forward.
Vertically integrated hardware/software design could also help Microsoft wring better performance out of its code. Even though the companies collaborate, Nvidia’s silicon design and Microsoft’s planned data pipeline don’t align perfectly. And running ChatGPT is expensive—to the tune of $700,000 daily. Patel estimated that each ChatGPT query costs Microsoft about $0.36. Purpose-built silicon could cut costs for Microsoft by some $84 million, which would knock out almost the entire cost of getting Athena off the ground.
While Microsoft may be opting to avoid the ‘Nvidia tax’ by building its own chips for AI processing, Athena won’t be the end of Microsoft’s work with Nvidia. It won’t even be the end of Microsoft using Nvidia hardware for AI projects. The two companies are collaborating on a next-gen AI supercomputer, which would pair Microsoft’s Azure virtual machines with ‘tens of thousands’ of Nvidia’s A100 and H100 GPUs.
