Following the release of Gemini 1 on February 16th5. The front is hard after OpenAI's GPT-4. On February 21, local time, Google launched a new generation of open-source model "Gemma", which directly refers to Meta's open-source model Llama 2.
Google says GEMMA is the "most advanced" family of open models in lightweight, surpassing Mistral 7B and Llama 2 to be the most powerful language model of its kind to date.
GEMMA means "gem" in Latin, and the open model series GEMMA named after it was developed by teams such as Google DeepMind, using the same research and technology used to create the Gemini model, according to Google's official website.
At present, the GEMMA series is available in two scales, namely "GEMMA 2B" and "GEMMA 7B", i.e. 2 billion parameters and 7 billion parameters, to meet the different needs of developers.
In terms of performance, it is ahead of LLAMA 2 in a number of tests, including MMLU, BBH, MATH, etc.
Among them, GEMMA 7B scored 64 in MMLU (Massive Multitasking Language Understanding).3%。In its size, the originally strongest Mistral 7b scored 625%, Meta's Llama-2 7b and 13b are 452% and 548%, which is nowhere near gemma.
In terms of usage, developers can fine-tune the GEMMA model based on their own data, and optimize it across frameworks, tools, and hardware. First of all, Google passed the native keras 30 is compatible with all major frameworks (Jax, Pytorch, and TensorFlow) and provides GEMMA with a toolchain for inference and supervised fine-tuning (SFT).
At the same time, GEMMA supports across multiple AI hardware platforms, including NVIDIA GPUs and Google Cloud TPUs, which allows GEMMA models to run on all types of devices, including laptops, desktops, IoT, mobile devices, and the cloud.
However, compared with closed-source models, the security of open-source models is also a concern for many people.
To do this, Google uses automated techniques to filter out certain personal information and other sensitive data from the training set. In addition, human feedback (RLHF) is leveraged for extensive fine-tuning and reinforcement learning to align instruction tuning models with responsible behavior. The risk profile of the GEMMA model was also assessed.
The launch of GEMMA coincided with the release of Google's new Responsible Generative AI Toolkit to help developers and researchers prioritize building safe and responsible AI applications.
As an open product, Google says GEMMA allows all organizations, regardless of size, to use it commercially responsibly and in compliance with security standards.
In addition, Jeanine Banks, Google's vice president and general manager and head of developer relations, also emphasized that GEMMA is different from Meta's open source, which has licensing terms that prevent its big tech competitors from using its LLAMA 2 open source model, while Google GEMMA has no such commercial restrictions. This wave of Google is indeed on the score!
In fact, in the field of artificial intelligence, if Meta is the originator of open source for large models, then Google can be said to be the originator of open source in the field of artificial intelligence. Whether it's OpenAI or Meta, Google's competitors, or any generative AI company, they can't do without the influence of the Transformer framework.
The Transformer framework was first mentioned in Google's 2017 "Attention is All You Need"**. Written by Vaswani, an AI researcher at Google, et al.
Before Google's release, OpenAI's technical roadmap was mainly focused on convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Due to the proposal of the Transformer architecture, it helps to solve the dilemma of RNN in the problem of long-distance dependency.
Illustration of the main components of the Transformer model in the original.
The Transformer model has achieved remarkable results in machine translation, text summarization, question answering and other tasks, and has quickly become the standard model in the field of natural language processing. Most of today's large language models, including GPT-2, GPT-3, GPT-4, Claude, BERT, XLNet, Roberta, and ChatGPT, demonstrate the ability of Transformer to perform a variety of these natural language processing (NLP)-related tasks.
On the other hand, Google's strength in the field of artificial intelligence has been underestimated before, and it can be said that it even has a greater first-mover advantage than competitors such as OpenAI and Meta.
But since the release of ChatGPT in 2022, OpenAI has represented the world's most advanced generative large model. Google, as a pioneer, has almost "disappeared" in this field.
In response to the challenges of ChatGPT, Google urgently launched Bard in March 2023. In December 2023, the Gemini multimodal large model was released. At the same time, Google has also faced many doubts, and was even accused of presenting a fake clip at the press conference.
In the face of external blows, Google did not stop there. In February 2024, Google will release Gemini Ultra free version and Gimini 15. GEMMA open source three big moves. It can be seen that Google has been proving itself with practical actions.
Recently, OpenAI once again launched the disruptive product Sora, which has put the field of multi-modal artificial intelligence on the agenda. It's stressful for Google, but it's not scary.
As early as the end of last year, Google launched the first generation model videopoet. According to Google, the model can be "zero-shot generated" and supports not only common Wensheng and Tusheng, but also editing, stylization, extension, and soundtrack. Judging from the output ** effect provided by Google, VideoPoet has indeed made a qualitative leap compared with the previous AI ** large model. In the future, it may compete with OpenAI.
At present, in the field of open source large models, with Google's strong entry, a three-legged situation has been formed: Meta, European Mistral AI and Google. In the field of closed-source large models, Google occupies a place against the camp led by OpenAI and Microsoft.
In less than two months in 2024, the "battle of large models" has begun, and it is foreseeable that the competition in the field of large models will further heat up in 2024.
But as far as the domestic market is concerned, there is no one who can take charge of it alone, and we still need to wait and see whether there will be a dark horse sprung up. Of course, we also expect such a dark horse to appear.