The AI model marks a new milestone in the development of artificial intelligence. Since the Dallymouth Conference in 1956, the development of AI can be divided into four stages. With the rise of the Internet and cloud technology, artificial intelligence has gradually transitioned from symbolism to connectionism, showing a wave-like development trend. At present, this wave of artificial intelligence led by AI large models is heralding the arrival of the era of general artificial intelligence. Artificial intelligence
The advent of ChatGPT announced the beginning of the era of AI large models, and since then, multimodal pre-trained large models have become the industry standard.
In the process of extending large AI models to multimodal domains, the field of generative AI is undergoing profound changes. The innovation focus of large models has expanded from single-modality to multi-modality, making multi-modal pre-trained large models gradually become an indispensable tool in many fields such as text, image and ** generation.
Recently, OpenAI launched its first text generation** model, SORA, and the results it has shown are breathtaking. The SORA model can not only generate up to 60 seconds of **content according to the text description, but also accurately grasp the key elements such as color and style, so as to produce vivid ** with rich expressions and full of emotions. SORA's three core strengths have enabled it to make breakthroughs in the field of AIGC, which can be regarded as a major milestone in the field.
AI models are gradually developing from single-modality to multi-modality
Source**: KPMG's AI models can be divided into two broad categories depending on the amount of data type processed:
1) Unimodal model: focuses on processing a single type of data, such as text. They are optimized for specific types of information and excel in the field.
2) Multimodal models: Unlike unimodal models, multimodal models are capable of processing two or more types of data at the same time. This processing method is similar to that of the human brain, which is capable of receiving and processing different forms of information such as text, sound, and images at the same time. Multimodal models provide a more comprehensive and multi-dimensional understanding and expression by integrating data from different modalities.
The concept of multimodality means to express or perceive things from multiple perspectives or senses.
In this context, "multimodal large models" refer specifically to large-scale models that can process text, audio, images, and other different forms of content. These models open up new avenues for the development of AI by fusing multiple types of information to enable higher levels of understanding and generative capabilities.
Multimodal Large Model Framework:
With the continuous improvement of supporting technologies, the ability of AI models to generate multi-modal content after input text is increasing, new products in the industry are emerging one after another, and the speed of commercial application is also accelerating.
June 2020 GPT3The release of 0 indicates that AI has been able to generate text and ** at a high level. Subsequently, in July 2022, Stable Fusion, a landmark product in the field of Wensheng graphics, was launched. By 2023, AI has also made significant progress in the field of generating 3D models and **, and related products have been implemented one after another. In the field of generation, technologies such as Runwaygen and PikalabsPika that use diffusion models continue to promote the optimization and improvement of the results.
With the continuous development of AI technology, both B-end and C-end users can produce images, ** and 3D ** in a high-quality and low-cost way. This will bring huge business value to media industry segments such as film and television, marketing, and games, helping these industries reduce costs and improve efficiency.
Multimodal capabilities can not only improve the interactive experience and content production efficiency, but also optimize the performance of existing AI products in scenarios. Through an in-depth understanding of multiple modalities such as voice, text, and more, multimodal technology can significantly improve the interactive experience and enable AI products to play a greater role in various scenarios.
In terms of technology, multi-modal large models are mainly divided into two types: single-tower structure and double-tower structure. The single-tower structure uses a deep neural network to complete the interactive fusion between images and text, which belongs to the pre-information fusion scheme. The twin-tower structure uses different neural networks to process information of different modes, and carries out information interaction and fusion in the last layer, which belongs to the post-information fusion scheme. This structure has the advantages of strong model independence and high training efficiency.
In terms of business model, the implementation of large models can be divided into three main modes: large models, large models + computing power, and large models + applications. Enterprise users can directly purchase large-scale products or rent large-scale services. At the same time, vendors can also combine models and computing power to sell, or sell upper-layer applications that incorporate large model capabilities to enterprise users, and charge software licensing fees and other fees.
At present, large model + computing power is the mainstream charging model, but with the further improvement of model application and ecology, the proportion of large model + application mode is expected to gradually increase.
At present, multimodality has become an important development direction for many manufacturers in the evolution of AI large models, among which the ability to "speak and draw" has become the focus of major models.
Overseas OpenAI and Google have launched a general multimodal large model with excellent performance by virtue of their extensive layout and advanced technology in the multimodal field, leading the development trend of the industry. At the same time, stabilityUnicorns in verticals such as AI, Midjourney, and Runway are also playing a pivotal role in technological breakthroughs and product innovations.
The close integration of domestic universities, technology and industry has further stabilized the pattern of the large-scale model industry, and generative AI has provided a strong impetus for industrial upgrading. The rise of ChatGPT has stimulated the active participation of major manufacturers such as Alibaba, Huawei, Tencent, JD.com, Byte, 360, SenseTime, and iFLYTEK, making the domestic large-scale model field enter an era of fierce competition in the "hundred-model melee".
As of October 2023, there are 254 domestic manufacturers and universities with large models with more than 1 billion parameters. In this ecosystem, universities and researchers focus on basic research and talent training, providing a steady stream of innovation power for the industry. Large manufacturers rely on strong computing power support, infrastructure construction, and MaaS services to provide a solid guarantee for the training and deployment of large models. Start-ups, on the other hand, are making great strides in the development of large-scale model applications to promote the commercialization of the technology.
In addition, manufacturers that have layouts or have layout capabilities in the multimodal direction include Kunlun Wanwei, Wondershare Technology, Meitu, Xinguodu, etc. With the continuous progress of multimodal technology, AI applications in e-commerce, games, education, marketing and other fields will also usher in new development opportunities, and related layout manufacturers such as Focus Technology, Chinese**, Shengtian Network, BlueFocus, Phoenix Media, Century Tianhong, Palmfun Technology, etc. will benefit from this trend. At the same time, companies such as Arcsoft Technology and Danghong Technology will also benefit from the development of AI applications.
With the continuous growth of the scale of large model data, it is difficult for a single server to meet the growing demand for computing power. Therefore, it has become an inevitable trend to connect a large number of servers through high-performance networks to build large-scale computing power clusters. In this field, manufacturers such as Inspur Information, Sugon, Industrial Fortune Union, and Tuowei Information have actively laid out to provide strong support for the rapid development of the industry.
At present, the development of multi-modal large models is bringing about technical equality, so that C-end content creation can achieve a better balance between cost and quality. This advancement in technology has provided more creative tools and possibilities for the average user, making it easier for them to create high-quality content.
With the further development of multimodal technologies such as images, audio, and 3D assets, we are expected to see the real arrival of the AIGC era. In this day and age, AI will become an important driver of content creation, helping users generate rich and diverse content faster and more efficiently.
This change will bring great opportunities for UGC platforms. In the past, UGC platforms such as Xiaohongshu, Zhihu, Douyin, Kuaishou, etc., have proven the potential of users to create content. Every time the threshold for user-created content is doubled, the amount of user-created content increases tenfold, and the user size of the corresponding platform also increases significantly. This means that with the popularization of multi-modal large models and multi-modal technology, the UGC platform will usher in more creators and more content, thereby further promoting the development and growth of the platform.
Pay attention to [Leqing Think Tank] and gain insight into the industrial pattern!