The accelerator, called Trainium3, was recently installed in a few data centers and will be available for customers beginning on Tuesday, Dave Brown, a vice president with Amazon Web Services, said
By Matt Day
Amazon.com Inc.’s cloud unit raced to get the latest version of its artificial intelligence chip to market, renewing efforts to sell hardware capable of rivaling products from Nvidia Corp. and Google.
The accelerator, called Trainium3, was recently installed in a few data centers and will be available for customers beginning on Tuesday, Dave Brown, a vice president with Amazon Web Services, said in an interview.
“As we get into early next year we’ll start to scale out very, very quickly,” he said.
The chip push is a key element of Amazon’s strategy to stand out in AI. AWS is the largest seller of rented computing power and data storage. But it has struggled to replicate that dominance among leading developers of AI tools, as some companies opt instead to rely on Microsoft Corp., which has close ties to ChatGPT maker OpenAI, or Alphabet Inc.’s Google.
Amazon shares rose 1.6 per cent to $237.71 at 11:09 a.m. in New York. Nvidia shares pared gains, while AI chip rival Advanced Micro Devices Inc. dropped to a session low.
Amazon hopes to entice companies looking for a bargain. Trainium chips are capable of powering the intensive calculations behind AI models more cheaply and efficiently than Nvidia’s market-leading graphics processing units, according to the company. “We’ve been very pleased with our ability to get the right price performance with Trainium,” Brown said.
Amazon is releasing Trainium3 about a year after deploying its predecessor accelerator. That’s a sprint by chip industry standards. “The main thing we’re gonna be hoping for here is just that we don’t see any kind of smoke or fire,” an AWS engineer joked when the chip was first fired up in August.
The quick turnaround also matches the pace of Nvidia, which has pledged to release a new chip every year.
There’s a catch: Amazon’s chips lack the deep software libraries that help customers get Nvidia’s graphics processing units up and running fast. Bedrock Robotics, a company that uses artificial intelligence models to enable construction equipment to operate autonomously, runs its infrastructure on AWS servers. But when it comes to building models to help guide an excavator, Bedrock uses Nvidia chips, according to Chief Technology Officer Kevin Peterson.
“We need it to be performant and easy to use,” Peterson said. “That’s Nvidia.”
Many of the Trainium chips in use today are at the disposal of Anthropic, inside data centers in Indiana, Mississippi and Pennsylvania. AWS said earlier this year that it had strung more than 500,000 of them together to help the AI startup train its latest models and aims to dedicate 1 million of the chips to Anthropic by the end of the year.
Amazon is betting Anthropic’s success, along with its own AI services, can entice other companies. Amazon has announced few other major customers for the chip, leaving analysts struggling to assess Trainium’s effectiveness.
Anthropic is also using Google’s Tensor Processing Units and cut a deal earlier this year with the Alphabet unit that gives the startup access to tens of billions of dollars worth of computing power. In an interview with Bloomberg Television, AWS Chief Executive Officer Matt Garman said the company’s relationship with Anthropic was “incredibly strong.” He said the startup has an enormous demand for computing power, leading Anthropic to use a variety of providers.
Amazon made the chip announcement at re:Invent, its annual user conference, which in recent years has become a rolling advertisement for AI services where Amazon courts builders of cutting-edge tools and companies that might want to pay for access to them.
On Tuesday, Amazon also announced updates to its main line of AI models, called Nova. New Nova 2 products include a variant called Omni, which can receive text, image, speech or video inputs and respond with both text or images.
As with its chips, Amazon has tried to sell customers on the performance of its models for their price. Prior Nova models generally haven’t ranked among the industry leaders in benchmarks that track how AI models perform in answering standardized questions.
“The real benchmark is the real world,” Rohit Prasad, who leads much of Amazon’s model development and the company’s Artificial General Intelligence team, said in an interview, adding that he expected the new models to be competitive.
The company also plans to let customers bring more data to bear when customizing Amazon’s models. Nova Forge, a new product, is designed to let sophisticated users grab versions of Amazon’s Nova models before their training is complete and customize them with their own data.
Reddit Inc. is using Nova Forge to build a model capable of assessing whether a post on the digital message board violates the site’s safety policies. Chris Slowe, the company’s chief technology officer, says that some AI customers are tempted to use the most advanced model to solve every problem, rather than seeking one with a particular expertise.
“The fact that we can make it an expert in our specific area is where the value comes from,” he said in an interview.
