It is said that the startup called Groq came with the fastest large model in history and the LPU of its own chip. After testing by netizens, the generation speed of GROQ is close to 500 Tok S per second, which is 40 Tok S of GPT-4.
According to the introduction of Groq's official website, LPU is a chip designed for AI inference. But to train a large model, you still need to buy a GPU. If training must be done to raise soldiers for a thousand days, then the reasoning is to use soldiers for a while, and both are indispensable. The fast output of the GroQ LPU is still inseparable from the data training of the GPU behind it. So look at the total cost of training and inference. For example, artificial intelligence expert Jia Yangqing analyzed that the comprehensive cost of GROQ is more than 30 times that of Nvidia's GPU.
"Have an opinion" message board
@卡卡卡卡西:Everyone knows that large models need more memory-intensive chips, but the current chip manufacturers are based on the chip architecture they have been doing for so many years, and they don't have the courage to start all over again, so OpenAI Ultraman wants to develop its own chips, which is why GroQ has made a shocking debut.
@贰言:GroQ's self-developed LPUs are more powerful than NVIDIA's GPUs, which is a huge challenge for traditional AI chip makers such as NVIDIA. With the increasing popularity of AI technology, traditional manufacturers must consider how to integrate AI technology more efficiently to enhance the competitiveness of their products and services.