Microsoft unveiled two chips at its Ignite convention in Seattle on Wednesday.
The primary, its Maia 100 synthetic intelligence chip, might compete with Nvidia’s extremely sought-after AI graphics processing models. The second, a Cobalt 100 Arm chip, is aimed toward basic computing duties and will compete with Intel processors.
Money-rich expertise firms have begun giving their purchasers extra choices for cloud infrastructure they’ll use to run functions. Alibaba, Amazon and Google have achieved this for years. Microsoft, with about $144 billion in money on the finish of October, had 21.5% cloud market share in 2022, behind solely Amazon, in line with one estimate.
Digital-machine cases operating on the Cobalt chips will change into commercially out there via Microsoft’s Azure cloud in 2024, Rani Borkar, a company vice chairman, advised CNBC in an interview. She didn’t present a timeline for releasing the Maia 100.
Google introduced its unique tensor processing unit for AI in 2016. Amazon Internet Providers revealed its Graviton Arm-based chip and Inferentia AI processor in 2018, and it introduced Trainium, for coaching fashions, in 2020.
Particular AI chips from cloud suppliers may be capable of assist meet demand when there is a GPU scarcity. However Microsoft and its friends in cloud computing aren’t planning to let firms purchase servers containing their chips, not like Nvidia or AMD.
The corporate constructed its chip for AI computing based mostly on buyer suggestions, Borkar defined.
Microsoft is testing how Maia 100 stands as much as the wants of its Bing search engine’s AI chatbot (now known as Copilot as an alternative of Bing Chat), the GitHub Copilot coding assistant and GPT-3.5-Turbo, a big language mannequin from Microsoft-backed OpenAI, Borkar stated. OpenAI has fed its language fashions with giant portions of knowledge from the web, and so they can generate e-mail messages, summarize paperwork and reply questions with a number of phrases of human instruction.
The GPT-3.5-Turbo mannequin works in OpenAI’s ChatGPT assistant, which turned fashionable quickly after changing into out there final yr. Then firms moved rapidly so as to add comparable chat capabilities to their software program, growing demand for GPUs.
“We have been working throughout the board and [with] all of our completely different suppliers to assist enhance our provide place and assist a lot of our prospects and the demand that they’ve put in entrance of us,” Colette Kress, Nvidia’s finance chief, stated at an Evercore convention in New York in September.
OpenAI has beforehand educated fashions on Nvidia GPUs in Azure.
Along with designing the Maia chip, Microsoft has devised customized liquid-cooled {hardware} known as Sidekicks that slot in racks proper subsequent to racks containing Maia servers. The corporate can set up the server racks and the Sidekick racks with out the necessity for retrofitting, a spokesperson stated.
With GPUs, profiting from restricted knowledge heart area can pose challenges. Corporations generally put a number of servers containing GPUs on the backside of a rack like “orphans” to stop overheating, moderately than filling up the rack from prime to backside, stated Steve Tuck, co-founder and CEO of server startup Oxide Laptop. Corporations generally add cooling programs to cut back temperatures, Tuck stated.
Microsoft may see sooner adoption of Cobalt processors than the Maia AI chips if Amazon’s expertise is a information. Microsoft is testing its Groups app and Azure SQL Database service on Cobalt. Thus far, they’ve carried out 40% higher than on Azure’s current Arm-based chips, which come from startup Ampere, Microsoft stated.
Previously yr and a half, as costs and rates of interest have moved increased, many firms have sought out strategies of constructing their cloud spending extra environment friendly, and for AWS prospects, Graviton has been one in every of them. All of AWS’ prime 100 prospects at the moment are utilizing the Arm-based chips, which may yield a 40% price-performance enchancment, Vice President Dave Brown stated.
Shifting from GPUs to AWS Trainium AI chips will be extra difficult than migrating from Intel Xeons to Gravitons, although. Every AI mannequin has its personal quirks. Many individuals have labored to make quite a lot of instruments work on Arm due to their prevalence in cell units, and that is much less true in silicon for AI, Brown stated. However over time, he stated, he would anticipate organizations to see comparable price-performance good points with Trainium compared with GPUs.
“We’ve shared these specs with the ecosystem and with plenty of our companions within the ecosystem, which advantages all of our Azure prospects,” she stated.
Borkar stated she did not have particulars on Maia’s efficiency in contrast with options akin to Nvidia’s H100. On Monday, Nvidia stated its H200 will begin delivery within the second quarter of 2024.
WATCH: Nvidia notches tenth straight day of good points, pushed by new AI chip announcement
