The wave of generative AI technology has brought multi-dimensional and large-scale demand to infrastructure such as cloud computing industry chain and data centers. However, AI is not just a place for data centers, and practitioners are loading AI functions into the edge and terminals. In the near future, AI will sink into billions of devices around the world, benefiting every consumer's work and life.
Traditionally, the AI that people have referred to is server-based AI or generative AI. But we believe that AI can also exist at the edge, in the network, from the traditional data side all the way to the storage and server, and integrate into every computing process. At the ARM Tech Symposia annual technology conference held at the end of last month, Mohamed Awad, senior vice president and general manager of ARM's infrastructure business unit, told reporters.
Mohamed Aad, senior vice president and general manager of the Infrastructure Business Unit at ARM
From the infrastructure side for cloud services and data centers, to smart devices such as mobile phones, to the edge side that brings IoT data closer to data sources and users, Arm has launched solutions at every node in the AI field that has been or is being deployed.
The competition in the AI market is essentially the competition of the developer ecosystem. In order to support the customized and proprietary needs of users in the AI era, ARM empowers the developer ecosystem in an all-round way with a complete and comprehensive computing platform, and becomes an "accelerator" for product launch and industrial innovation.
Bring AI capabilities to the cloud edge
In traditional infrastructure architectures, the center is a standard off-the-shelf CPU, with memory and multiple accelerators connected to the CPU. As a result, each accelerator needs to access memory via the CPU. In the age of AI, it is clear that such an architecture cannot cope with the amount of data and computation required by AI. The majority of cloud service providers and data center infrastructure providers urgently need to customize CPUs so that each CPU core can be directly connected to each accelerator to achieve system-level memory consistency.
Arm is unique in that it enables chip designers to customize chip architectures on demand, and empowers the entire development process based on the powerful ARM software ecosystem to accelerate time to market. This is also why NVIDIA, Amazon, Alibaba Cloud, Microsoft and other leading cloud manufacturers at home and abroad have chosen to develop their own computing power chips based on ARM architecture. The architecture used in NVIDIA's GH200 Grace Hopper superchip was jointly customized by NVIDIA and ARM. The architecture consists of 72 ArmNeoverse cores, combined with NVIDIA's GPU, each core is directly connected to each accelerator, enabling a 10x increase in AI performance compared to x86-based systems.
In the field of smart terminals, technology and scene innovation around mobile phones have entered a white-hot stage. Mobile phones are not only the most commonly used communication and entertainment devices for consumers, but also become the carrier of mobile office and even the deployment target of large models. Mobile phone manufacturers are in dire need of a computing base that can scale according to various consumer needs.
When choosing the configuration of their CPU clusters, mobile phone manufacturers often choose the right CPU configuration according to the user scenario of their ultimate target audience, which is also part of Arm's CPU strategy to enable its partners to choose the right CPU performance for their customers. In the current high-end smartphone market, we can often see many different ideas, and each company has different strategies, which is exactly what Arm expects to see the diversification and differentiation brought about by the flexible configuration of ARM IP. Arm launched the 2023 Total Computing Solution (TCS23) this year, which integrates physical IP, architecture, tools, and software to provide one-stop and simplified technical support for SoC development. As part of TCS23, the Armv9 Cortex compute cluster has achieved double-digit performance gains for the third year in a row. Not only is Arm's flagship GPU Immortalis the first to bring ray tracing and variable-rate shading capabilities to mobile phones, but the new fifth-generation GPU architecture also optimizes the interaction between external memory, CPU clusters, and system-level caches in TCS23 to improve overall performance.
Our ARM comprehensive computing solutions for mobile platforms enable AI to be implemented on mobile devices. In addition, our ARM Cortex-M52 and Cortex-M55 are products that support AI and continue to empower and support AI development. Mohamed Ahad said.
When AI enters the edge, the Internet of Things is not only the collector and transmitter of device information, but also can use AI to achieve advanced maintenance, sensor fusion, industrial control and other functions.
ARM offers a comprehensive IoT solution for the IoT space, simplifying development and accelerating product design by combining hardware IP, platform software, machine learning (ML) models, tools, and more. Arm Helium technology is a vector extension to the Cortex-M processor family, dramatically improving machine learning and digital signal processing capabilities for small, embedded devices. In ARM's latest Cortex-M52, ARM Helium is ARMV8The addition of 150 new scalar and vector instructions to the 1-M architecture family enables the Cortex-M52 to achieve 2 digital signal processing capabilities over its predecessor7x boost and 5 machine learning capabilities6x boost. Facing the characteristics of segmentation and diversification of IoT device scenarios, the ARM Corstone solution provides key IP integration configurations, enabling rapid development of IoT products with different performance requirements, and truly pushing AI computing to the edge.
Empower the developer ecosystem with a comprehensive computing platform
While AI brings unlimited business opportunities to the global computing industry, it also brings severe computing challenges. On the one hand, the exponential growth of connected devices and data traffic is putting heavy pressure on computing infrastructure such as data centersOn the other hand, as advanced manufacturing processes approach their physical limits, Moore's Law development slows down, making it increasingly difficult to achieve a balance between performance and power consumption.
In the face of the latest trends and needs of the computing industry, ARM has transformed from the most well-known IP provider to a computing platform company, focusing on IP licensing business while also focusing on providing partners with flexibility of choice through complete and comprehensive solutions to help them bring products to market faster. From mobile platforms to infrastructure, the Internet of Things, and automotive, Arm has launched computing platforms that simplify the development process while providing chip designers with greater freedom to customize chips for their own scenarios and use cases.
In addition to a range of computing platforms such as the ARM Total Compute Solution, the ARM Neoverse Platform, the ARM Corstone, and SOAFEE, Arm has recently launched the ARM Neoverse Compute Subsystem (CSS) and the ARM Total Design to further activate the power of the ecosystem.
Neoverse is ARM's server and infrastructure product line, and Neoverse CSS provides users with the system management, power management, software, and development tools needed to optimize performance, including Neoverse core, CMN mesh fabric, and system IP, enabling users to deliver custom chips at a lower cost, in less time, and with less risk.
Arm Neoverse CSS saved the engineering team the equivalent of 80 engineers working for one year. One customer reported that after using Neoverse CSS, their project went from concept definition to tape-out in just 13 months. Mohamed Ahad said.
Microsoft's Azure Cobalt 100 CPU is built on Arm Neoverse CSS and contains 128 Neoverse cores. The advantages of the Neoverse CSS and ARM platform software ecosystem give Microsoft more time to focus on unique innovations and optimizations, while saving a lot of development effort. Arm expects that in 2024, more domestic and foreign cloud computing and data center related manufacturers will put the first-generation CSS design into production.
On the basis of Neoverse CSS, Arm has launched the Arm Comprehensive Design Ecosystem project, which enables infrastructure applications including AI, cloud, network, and edge to widely use the Arm Neoverse architecture. Arm will work with Arm to deliver pre-integrated and validated IP and EDA tools from partners such as Cadence, RAMBUS and Synopsys, design services from partners such as AdTechnology, AlphaW**e Semi, Broadcom, Capgemini and Faraday, foundry services from vendors such as Intel Foundry Services and TSMC, and commercial software and firmware from infrastructure firmware** vendors such as Ammann Technologies.
The Cortex product line, which includes terminal and edge-side products, has also been adhering to the ecological construction logic of close coupling of software and hardware to provide overall support for developers. In the case of the Cortex-M52, developers previously had to go through a complex development process combining three compute units, CPU, DSP, and NPU with three different software toolkits to achieve both digital signal processing and machine learning capabilities in a low-power processor. ARM provides a single toolchain for developers using the Cortex-M52 to address traditional computing tasks, digital signal processing, and machine learning workloads with a consistent development process, improving the development experience while significantly reducing time to market.
For ARM, China is an important and high-growth market, and it is also one of the most innovative and potential markets for ARM. ARM has 15 million developers worldwide, 4 million of whom are in China.
On the server and infrastructure side, ARM Neoverse is fully embracing the local ecosystem and open source community building. According to Zou Ting, global vice president of ARM China, ARM Neoverse has many customers in the Chinese market, especially in the infrastructure sector. ARM actively participates in the construction of local ecosystems such as data centers and cloud computing, as well as the construction of open source software communities such as Dragon Lizard, to help these communities better integrate into the global ARM ecosystem. On the device and edge side, ARM also provides one-stop services and rich ecological resources for local developers.
Zou Ting, Global Vice President of ARM China.
Powered by ARM technology, inference running on billions of devices around the world will be transformed. At the same time, it will require the industry to work together to accelerate the sharing of AI training and inference between data centers and devices, thereby improving the efficiency and cost-effectiveness of AI while improving its security, and ultimately making AI ubiquitous.
Author丨Zhang XinyiEditor丨Zhao ChenmeiEditor丨Maria Producer丨Lian Xiaodong