What is shared today is:AIGC seriesIn-depth Research Report:AIGC Topic: AIGC is sinking to the device side, which is expected to lead a new round of hardware innovation
Report Producer: Guohai**).
Report total:Page.
Featured Report**: The School of Artificial Intelligence
As an on-device large-scale model solution, AI PC phones can help solve problems such as data security and privacy leakage. For individual users, their digital twins can be formed by accessing local data, and the large model can become a personalized assistant for users; For enterprise users, the enterprise database can be accessed through the company's intranet to achieve intelligent collaborative office. The general model in the cloud does not have the ability to serve a certain field, and needs to access the data before it has professional capabilities, which means that enterprises must upload the key asset - data to the cloud in order to obtain the industry model, and once the trained model is shared by the whole industry, it means that to a certain extent, the competitive barriers of enterprises are eliminated, which gives rise to the demand for enterprises to train special models in the data wall, and AI PC mobile phones come into being.
In most application scenarios, users have high requirements for the timeliness of inference, and there is inevitably a delay due to the physical distance between inference in the cloud and then transmitting the results back to the end and edge.
Deploying large models on the device side and at the edge may lead to market demand for public clouds, private clouds, and on-premises data centers, as well as a balance of computing power across the cloud, devices, and edges. In the past, the market's attention to computing power was mostly focused on cloud service providers and large model training stages, but the cloud computing power was relatively limited, and it was difficult to meet the inference needs of all users during peak periods. Sharing computing power costs for the cloud through smart terminals (such as AI PCs, mobile phones) and edge devices may balance the demand for computing power for the cloud, devices, and edges.
Based on the penetration rate of AI PC shipments, Canalys expects PC shipments to reach 26.7 billion units, we refer to this data to assume that PC shipments are 24/2.6/2.8/3.000,000,000,00 Similarly, the hypothesis of AI PC penetration is 5%, 10%, 20%, 30%, 40%, 50%, and 60% under seven different scenarios. According to the calculation, the range of AI PC shipments is 012~1.8 billion units. Assuming the price increase level of AI PC, the incremental market space under the two different scenarios of AI PC price increase of 1,000 yuan and 1,500 yuan is 12,018 billion yuan and 18,027 billion yuan, respectively.
With the development of the process, GPUs have begun to be popularized on personal PCs and mobile devices, and even with the exponential growth of their performance, the momentum of independent graphics has gradually increased.
The NPU (Neural Network Processing Unit) can simulate human neurons and synapses at the circuit layer, and directly process large-scale neurons and synapses with a deep learning instruction set, and complete the processing of a group of neurons with one instruction. Compared with the von Neumann structure of CPUs and GPUs, NPU integrates storage and computing through synaptic weighting, thereby improving operational efficiency. NPU is a type of ASIC chip that is currently mainly used in artificial intelligence tasks such as deep learning and machine learning.
Qualcomm released the Snapdragon 8 Gen3 for AI phones and the Snapdragon X Elite for AI PCs at the 2023 Snapdragon Technology Summit, taking the lead in becoming the chip manufacturer that simultaneously seized the two end-side tracks for generative AI applications. As far as the mobile phone is concerned, the phone equipped with the Snapdragon 8 Gen3 runs the stable Diffusion model with only 0An image is generated locally in 6 seconds, which greatly optimizes the inference speed of generative AI on mobile devices compared to the 15 seconds of Snapdragon 8 Gen2. On the PC side, Qualcomm pioneered the Snapdragon X Elite platform, which not only expands the breadth of generative AI applications, but also serves as an important step for Qualcomm to enter the PC market.
The Qualcomm AI Engine used in the Snapdragon 8 Gen3 features a powerful Hexagon NPU for mobile devices, integrating an upgraded hardware acceleration unit, a micro-slice inference unit, and a reinforced tensor, scalar, and vector unit, all of which share a large shared memory with twice the bandwidth. It also supports the mixed precision of INT8+INT16 and all the accuracy of INT4, INT8, INT16 and FP16. According to the Heart of the Machine, its performance is 98% better than its predecessor and its energy efficiency is 40% better. For the first time, the Snapdragon 8 Gen3 supports models running 10 billion parameters, and the scale has reached the scale of 10 billion. At the same time, the time required for stable diffusion to generate images has been reduced to less than 1 second, which is the fastest speed. When running the Meta large language model LLAMA2-7B, the Snapdragon 8 Gen3 can generate 20 tokens per second, which is also one of the fastest on the mobile phone terminal side.
The Snapdragon X Elite processor uses a 4nm process technology and integrates a custom Qualcomm Oryon CPU, which runs twice as fast as Intel's 12-core processor, consumes 68% less power than Intel's competitors, and runs 50% faster than Apple's M2 during peak hours, according to Machineheart. Built for AI, the Snapdragon X Elite uses the industry-leading Qualcomm AI engine with integrated Hexagon NPU, with heterogeneous computing power of up to 75 tops, supports generative AI models with more than 13 billion parameters on the device side, and generates 30 tokens per second for large models with 7 billion parameters, and the AI processing speed is 4% of that of competing products5 times. According to Qualcomm officials, PCs equipped with Snapdragon X Elite are expected to be available in mid-2024.
Report total:Page.
Featured Report**: The School of Artificial Intelligence