Economic Observer reporter Shi Zhenfang
On December 15th, Wenge launched Yayi 20 domestic large model (hereinafter referred to as Yayi 2.)0) and publish an open source technical report. Wenge is an artificial intelligence enterprise incubated by the Institute of Automation of the Chinese Academy of Sciences, focusing on complex data analysis and AI-assisted decision-making.
In addition to the Yayi 2 owned by Zhongke Wenge0 In addition, the Institute of Automation of the Chinese Academy of Sciences also has its own large model Zidong Taichu 20。In an interview with reporters, Wang Lei, chairman of Zhongke Wenge, believes that moderately reducing the scale of parameters and reducing the cost of reasoning will be the development trend of large models in the future.
Yayi 20 has independent intellectual property rights for data, models, and applications, and is one of the few native large models in China that is pre-trained from scratch. Based on 240TB (storage unit) multi-source basic data, more than 1,000 data cleaning processes, 265 trillion tokens ("tokens", which are the smallest semantic units used to represent words in a language model) of high-quality training data to ensure the security and controllability of the training data corpus. Yayi 20's Chinese knowledge question and answer ability ranks in many public assessment lists such as agieval, cmmlu, mmlu, c-eval, humaneval, etcThe zero-sample Chinese information extraction ability has won a number of SOTA (Best Performance Performance).
It is based on Yayi 20 independent research and development of the foundation, so that it can be independently trained and fine-tuned for industry application scenarios, the launch of government intelligence and business intelligence industry model system, for security, finance, public opinion, law, traditional Chinese medicine and other fields to build industry model applications.
However, in response to the problem of how to ensure the ability of large models to understand a certain domain while reducing computing power and reducing the scale of parameters, Wang Lei said that it depends on data selection and model training in the training stage. At the same time, in terms of security, the TOB service of Zhongke Wenge deploys the model within the government and enterprises, so as to ensure data security.
Regarding the current situation and future of the current domestic large model, Wang Lei also said that nowadays, the domestic basic native model is extremely scarce, and the independent research and development ability is insufficientThe government and enterprise industry relies on independent, controllable, safe and reliable native models, and the security of open source models is insufficient, and the operability of secondary training is not strongThe next generation of AI technology innovation will rely more on the accumulation of R&D experience in the whole process.