Amazon unveils super fast AI chips to compete with Nvidia, AMD and Microsoft

What just happened? At its re:Invent conference in Las Vegas, Amazon Web Services (AWS) made a slew of announcements, many of which revolve around generative AI and how it could be used by companies to modernize their services and increase efficiency. The company also unveiled its next-generation chips for a wide range of cloud-based workloads and AI training models with the promise of better performance and higher energy efficiency.

One of the new chips is Trainium2, which is meant for AI model training and is said to deliver up to 4x better performance and 2x energy efficiency when compared to its predecessor. It is also expected to offer 3x more memory capacity than the first-gen Trainium chips. In its press release, Amazon said that Trainium2 is purpose-built for high-performance training of foundation models (FMs) and large language models (LLMs) with up to trillions of parameters.

The company also claimed that Trainium2 will be super fast, allowing programmers to train models in a fraction of the time required by the first-gen Trainium chips. According to Amazon, Trainium2 will deliver up to 65 exaflops of compute power, offering “supercomputer-class performance” and enabling customers to train a 300-billion parameter LLM in weeks rather than months. An Amazon-backed AI firm called Anthropic has already announced plans to use Trainium2 to train its models.

Another new chip is the Arm-based Graviton4, which Amazon says is the “most powerful and energy-efficient AWS processor to date.” It is designed for a wide range of applications running on Amazon Elastic Compute Cloud (EC2) Ultraclusters, and is said to offer up to 30 percent better compute performance, 50 percent more cores, and 75 percent more memory bandwidth than Graviton3.

According to Amazon, the new chip will enable customers to improve the execution of their high-performance databases, in-memory caches, and big data analytics workloads. It can be used to process larger amounts of data faster than the third-gen Graviton chips, thereby reducing the time-to-results and lowering operating costs.

Amazon will start offering the Trainium2 chips next year, while Graviton4-powered R8g instances are now available in preview, with general availability expected in the coming months. The new chips are going to increase competition in the AI hardware sector, which is currently dominated by Nvidia. With Microsoft also raising the heat with the recent announcement of its Azure Maia 100 chip and Azure Cobalt CPU, it will be interesting to see how the battle ramps up in the months and years ahead.