Original author: Animism.
Original text**: The world on the crystal.
In recent years, artificial intelligence (AI) has risen rapidly and become the focus of the global technology and industry community. With the continuous breakthrough of AI technology and the continuous expansion of application scenarios, the market's attention to AI PC, AI phone, and AI car continues to increase, and AI chips, as the core hardware supporting AI computing, have also ushered in unprecedented development opportunities.
According to OpenAI's estimates, since 2012, the amount of computation used for AI training has increased exponentially worldwide, with an average of 3It will double in 43 months. AMD predicts that the global data center AI computing market will exceed 150 billion US dollars (about 1.20 billion yuan.) by 2027095 trillion yuan), with a compound annual growth rate of more than 50%.
Since Nvidia took the lead in dominating the AI large-scale model computing chip market and its market value exceeded the trillion mark, Intel, AMD and Chinese GPU chip companies have also made efforts, eager to get a share of the AI chip market. However, behind the infinite scenery, whether it is an industry leader or a unicorn rookie, they all have their own "difficult scriptures".
There is still a distance between dreams and reality
At present, the AI chip market is still dominated by Nvidia, and according to the latest estimates, its global market share has reached 90%, setting a new record. However, as tech giants, cloud service providers, and startups around the world are scrambling for computing resources to develop their own AI models, Nvidia's chips are in short supply, and the tide is also rising. In order to avoid being controlled by others and save costs, major giants have announced their own AI chips.
One of the recent hot spots in the AI circle is OpenAI's CEO Altman's plan to develop his own AI chips and is in talks with TSMC and Middle Eastern investors.
As the first manufacturer to set foot in self-developed chips, Amazon already has two AI-specific chips - training chip Trainium and inference chip Inferentia; Google has a tensor processing unit (TPU); Meta, the parent company of Facebook, began to work hard ten years ago, and finally officially announced the progress of its self-developed AI chip MTIA for the first time on May 19 last year; Microsoft also announced the Azure Maia 100, an AI chip for data centers at Ignite 2023.
However, Microsoft and Meta have been relatively slow in R&D and still rely on Nvidia's chips. According to the latest report from market research firm Omdia Research, in 2023, Meta and Microsoft tied for first place with 150,000 H100 GPUs purchased. Microsoft's expected release of the AI chip Athena in November last year has also been repeatedly postponed, and Meta recently announced that it will spend nearly $10 billion to buy 350,000 H100s, which also shows that its own research and development is not going well.
Final shipment estimates of Nvidia H100 in 2023, source: Omdia Research
Domestic manufacturers have a difficult winter
In the past 3-4 years, domestic AI chip companies can be said to be in full bloom, with powerful manufacturers such as HiSilicon, Suiyuan, Cambrian, and Horizon struggling to catch up with foreign giants, hoping to break the monopoly pattern of AI chips.
However, the fierce cries have dissipated, and the fate of these unicorns, which had been smash hits in the past, seems to be more and more uncertain.
Recently, Xu Lingjie, an important member of Bichen Technology, announced his resignation, which is the second co-founder lost by Bichen Technology after the resignation of Jiao Guofang, general manager of the GPU product line. This series of high-level changes has undoubtedly cast a shadow on the company's future development.
At the same time, Cambrian, an AI chip company that has attracted much attention in the market, is also in a difficult period. The company has been losing money for seven consecutive years, its market value has shrunk significantly, and the founding shareholders are also in the first shares. The aura of the Chinese Academy of Sciences, which once shrouded us, has almost dissipated, and the bubble in the AI chip market seems to be at risk of bursting. Starting from the second quarter of 2023, Cambrian was exposed to layoffs, and recently its subsidiary Xingge Technology also announced that it will lay off employees on a large scale.
The AI chip factory, which was once popular in China, is like a fire in winter, unable to resist the encirclement of the general environment and external bans and fell into a cold winter and predicament. The capital market's pursuit of AI chips has also turned into a soul torture question: can it survive this winter?
We don't need to pretend that the company is always in danger, we are," Nvidia founder Jensen Huang said in a speech last November.
The pressure first comes from the encirclement and suppression of peers
In terms of products, in addition to those "disloyal" customers, who have developed their own chips, AMD and Intel and two old rivals, every time they release a new chip, they will also compare with NVIDIA. For example, when AMD launched the Mi300X on December 6, it claimed to outperform NVIDIA's AH100. Nvidia then released its own benchmark in response, showing that the AH100 is still better with the right settings. AMD has once again responded to this by releasing new benchmarks that once again show that the Mi300X performs well with the right settings.
In terms of ecological construction, CUDA, the moat of NVIDIA's chip empire, is also under fierce attack, and MLIR, Google, OpenAI, etc. are turning to a "Python language-based programming layer" to make AI training more open.
A "cat-and-mouse game" with the White House
In recent years, in order to restrict China's development in the field of artificial intelligence, the United States has frequently introduced or amended export control regulations, requiring that "high-performance semiconductors" cannot be exported to China, Nvidia's ace chips A100 and H100 were first sniped, in order to keep the Chinese market, Nvidia continues to castrate performance to comply with regulations, and has launched a number of products such as "primary downgrade chips" A800 and H800, "secondary downgrade chips" H20, L20, L2, etc., However, domestic cloud computing companies still have fewer and fewer orders for Nvidia, and they have turned to buy chips developed by local companies such as Huawei.
On January 26, U.S. Secretary of Commerce Raimondo bluntly said that the United States is "doing everything to prevent China from gaining computing power." It can be seen that the United States** will continue to tighten export standards to China, limit the computing power that Chinese enterprises can obtain, and then reduce the efficiency of training AI and maintain its leading position in the field of artificial intelligence. It can be seen that this "cat and mouse game" between Nvidia and the White House will continue.
Precious data that accompanies the loss of the market
As mentioned above, it is becoming a trend for Internet giants or AI companies to develop their own AI chips, and although it is difficult, they have an important resource that Nvidia needs, and that is data.
When the various problems and data encountered in the development of AI algorithms are fed back to NVIDIA, NVIDIA can improve its products and improve the user experience in subsequent iterations. China is at the forefront of the world in the field of artificial intelligence, and what Nvidia loses along with the Chinese market will be the value that cannot be counted by numbers.
Although the development momentum has weakened, thanks to the first-chain and ecological advantages, NVIDIA's dominant position in the field of AI computing chips will not change in the short term. But neither chip companies nor downstream customers want to see "NVIDIA" take the lead.
In the blue ocean of AI chips, competition is becoming increasingly fierce. In the face of such a market environment, what is the driving force for the development of the AI chip industry? How do you discover new competitive advantages?
The integration of storage and computing power increases the computing power by an order of magnitude
Under data-based AI computing, the challenges of "storage wall" and "power wall" of von Neumann's architecture are highlighted. As the computing power increases, the number of processor cores increases, and the available bandwidth per core decreases, limiting the overall speed. Transporting data becomes a considerable bottleneck. Without storage optimization, the computing power provided by the chip will be greatly reduced.
Today's computing processors, such as CPUs, GPUs or AI-specific chips, are designed with the von Neumann architecture, 80% of the power consumption occurs in data transmission, and 99% of the time is consumed in the memory writing process, while the energy consumption and time really used for computing are actually very low.
Therefore, only large computing power on the basis of low power consumption is sustainable development.
The integration of storage and computing is to transfer the computing in the computer from the first-class processor to the memory, and directly perform the operation inside the storage unit, which can alleviate data handling and greatly reduce the data exchange time and data access energy consumption in the calculation process.
The integration of storage and computing is an effective way to achieve high bandwidth, low power consumption, and computing requirements, and how to effectively control the in-memory computing interface is an important challenge. Whoever has an in-memory computing hardware architecture that takes into account both computing density and storage density has the golden key to unlocking energy-efficient computing.
After 2019, most of the new AI chip manufacturers are in the layout of storage and computing integration: according to incomplete statistics from Cirui Insight, there are 20 new AI chip manufacturers from 2019 to 2021, and among them, 10 choose the integrated storage and computing route. On the whole, at present, the integration of storage and computing at home and abroad is in its infancy, and the integration of storage and computing is in a critical period of migration from academia to industry, so this may be another important direction for the development of domestic chips.
Data**: Insight of the Firm.
System-level innovation
AMD CEO Lisa Su has pointed out that according to the current computing efficiency, it will increase by 2 every two yearsBy 2035, to reach 10 trillion levels of computing power, the required power will be as high as 500MW, which is almost equivalent to the output power of half a nuclear power plant. In the face of such unrealistic energy consumption, system-level innovation is even more important. It is no longer limited to a single chip or circuit design, but needs to be considered from top to bottom, including software, system architecture, packaging technology, and other layers, to achieve overall performance optimization.
Similarly, in a joint presentation by the three major European semiconductor research institutes IMEC, CEA Leti and Fraunhofer, system-level innovation was at the heart of the future of semiconductors. As semiconductor processes approach their physical limits, new application requirements can no longer be met by process ramp-up alone. In particular, the requirements for next-generation smart cars and AI applications are more complex and diverse, and they need to be considered and innovated holistically at the system level.
Paradigm change in chip design, **Academician Sun Ninghui Integrated Chip and Chip Conference Report.
System-level innovation or co-optimization of system processes is a new design concept. It starts from product requirements and deduces backwards to various links such as system architecture, chip design, and semiconductor process. Through the collaborative optimization of all links, the product performance is greatly improved. This "outside-in" design idea breaks the limitations of traditional semiconductor design and brings new development opportunities to the industry.
Lisa Su gave a classic case. While using an innovative number system (such as 8-bit floating-point FP8) at the model algorithm level, the algorithm layer is optimized and supported at the circuit layer, and finally the efficiency of the computing layer is improved by orders of magnitude: compared with the traditional 32-bit floating-point number (FP32), FP8 can improve the computational efficiency by as much as 30 times. However, if you only optimize the efficiency of the FP32 computing unit, it will be difficult to achieve an order of magnitude efficiency improvement anyway. This is a testament to the potential of system-level innovation to improve computing efficiency.
Domain-specific computing enables workload optimization to improve performance and efficiency. Source: ISSCC2023 Conference.
Thousands of sails race, innovation pilot
In the era of rapid technological development, innovation is the key to breaking down technical barriers and enhancing competitiveness. China's AI chip manufacturers are facing the dual pressure of international technology blockade and market competition, and only through continuous innovation can they remain invincible in the fierce competition.
In the future, the integration and innovation of memory technologies such as heterogeneous computing clusters, specific acceleration units, advanced packaging technologies, high-speed inter-chip UCIe interconnection, and integrated storage and computing will bring new development momentum and direction to the semiconductor industry. After all, the AI computing power chip track with long slopes and thick snow is difficult to sustain if it lacks core competitiveness and only relies on financing and blood transfusion to maintain the company's R&D investment.
end - Disclaimer: We respect originality and also pay attention to sharing; The copyright of the text belongs to the original author. If there is any infringement of your rights and interests, please contact us in time, we will delete it as soon as possible, thank you!
Original author: Animism.
Original text**: The world on the crystal.