In the AI chip part, NVIDIA is currently the undisputed leader in the world, not only the highest shipments, but also the strongest performance, almost all large AI models in the world use NVIDIA AI chips to provide computing power. Recently, AMD held an "Advancing AI" conference and officially launched the Instinct Mi300A Mi300X accelerator, a GPU product for AI and HPC, which directly competes with NVIDIA's previous strongest AI chip, H100.
In fact, AMD released the MI300A and MI300X on paper in June this year, and now the MI300A and MI300X have begun mass production, so AMD also released more performance data about the MI300A and MI300X at this conference. The AMD Mi300A features a chiplet design with up to 13 chiplets inside, based on TSMC's 5nm or 6nm process technology (5nm for CPU GPU compute cores, 6nm for HBM memory and IO, etc.), many of which are 3D stacked to create a single-chip package with a controllable area and a total of 146 billion transistors.
The Mi300A uses a next-generation CDN3 GPU architecture with 228 compute units (14,592 cores) and integrates 24 ZEN 4 CPU cores with 128GB of HBM3 memory. The Mi300A's compute core is surrounded by 8 HBM3 memory, and a single HBM3 has a bandwidth of 63GB S, eight 16GB stacks form 128GB unified memory with up to 53 tb/s。The MI300A provides up to 61 Tflops FP64 hashrate and up to 122 TflOps FP32 hashrate.
As for the Mi300X, it integrates 12 chiplets with a 5 6nm process (6nm for HMB and IO) with 153 billion transistors and a CDNA 3 GPU with more computing cores. Each GCD based on the cDNA 3 GPU architecture of the Mi300X has a total of 40 compute units, which is equivalent to 2560 cores. There are a total of eight computing chips (GCDs), so there are a total of 320 compute and 20,480 core units. However, for the current production version, there will be a slight shrinkage.
In terms of memory bandwidth, the Mi300X also comes with a larger 192GB HBM3 RAM, bringing up to 52TBS of bandwidth and 896GB of Infinity Fabric bandwidth. Large memory pools are very important in LLMs (large language models), which are mostly memory-bound, and AMD can improve AI capabilities by leading the way in HBM memory capacity.
Specifically, the Mi300A APU offers up to 4x more performance than the H100, and the system also delivers up to 2x more performance per watt compared to NVIDIA's Grace Hopper superchip. The Mi300X, which is more like a pure GPU architecture, provides up to 20% performance improvement in FlashAttention-2 and LLAMA 2 70B compared to the H100 in the specific AI large model acceleration performance comparison. On a variety of integer and floating-point performance, the Mi300X is 1 of the H1003 times, which is quite shocking indeed.
AMD internally estimated that the global data center AI accelerator market could reach approximately $30 billion in 2023 and grow at a compound annual growth rate of more than $50 in the future, reaching a value of more than $150 billion by 2027. Now, AMD has raised its forecast for the data center AI accelerator market size to $40 billion in 2023 and $450 billion in 2027, with a compound annual growth rate of more than 70.
From the current point of view, for the AI market, AMD has come up with a more comprehensive solution than NVIDIA, MI300A as an APU solution, in fact, solves the problems of many manufacturers, terminal manufacturers only need to purchase MI300A can launch OEM AI PCs by themselves;In terms of pure GPU solutions, MI300X is now much stronger than H100, and what AMD has to do at present is to improve its own ecology, so that more manufacturers can choose their own chips. Of course, whether it is MI300A or MI300X, its computing power has long exceeded the bottom line of US export controls, so there is a high probability that these two chips will not be able to be sold in China. After the U.S. Department of Commerce named NVIDIA, AMD may not launch a special edition for the domestic market.