Author: Jin Wang.
On February 21, Google officially released the open-source large model GEMMA.
This is a short time since Google's last large model, Gemini 1The release of 5 is less than a week away, and the competition for large models is intensifying.
On February 15, OpenAI released SORA, the ability of SORA Wensheng**, which shocked the entire industry again.
At that time, even the founders of Runway, Pika and other companies that had stirred up trouble in the field of Wensheng had to bow down.
The "viral" spread of SORA around the world once again proves the genius marketing ability of Ultraman OpenAI, and also allows Gemini 1., which was released only two hours before SORA5. Failed to gain the global attention it should have received at the optimal time of dissemination.
Whether the release of the two at the same time was intentional by Ultraman or an accident, the competition between the giants has reached a fever pitch, and Google needs to show another hole card to regain the attention of the market.
Obviously, yesterday's GEMMA is another hole card for Google, but this time Google is aiming for an open source model.
The GEMMA released by Google this time contains two "small" models, GEMMA 2B and GEMMA 7B, which are not large due to the parameter size of 2 billion and 7 billion respectively, and Google classifies these two models as a series of lightweight open models.
It is worth noting that GEMMA uses the same technical architecture as Google's main large model Gemini, and it is also an on-device large model that can be directly delegated to laptops and desktops.
Prior to this, when Google first released the Gemini large model in December 2023, Google CEO Sundar Pichai announced three versions of Ultra, Pro, and Nano in one go, and said, "These are the first models to enter the Gemini era, and they are also the vision of Google Deepmind when it was founded." ”
Among them, the Gemini Nano parameter scale is 18b and 325b, which is used by Google as an end-side model.
In fact, the Pixel 8 Pro released by Google earlier in 2023 has been regarded as an epoch-making smartphone due to its significant AI features, and Google, which has firmly established itself in the trend of AI mobile phones, has put Gemini Nano in the Samsung Galaxy S24 this year, helping Samsung enter the "Galaxy AI era".
Zhiding.com believes that in the year when mobile phone manufacturers, PC manufacturers and chip manufacturers are working together to promote device-side AI, the more important efficacy of the 2B and 7B open-source large model GEMMA released by Google is expected to be reflected in the field of device-side AI.
From the open source perspective, Google Gemma is not the first large open source model.
As early as July 2023, Meta released the free and commercially available large model Llama 2, and this action is actually a masterpiece of cooperation between Meta and Microsoft, with 7B, 13B, and 70B three parameter-level versions of Llama 2, which was regarded as a replacement product for OpenAI's ChatGPT at that time.
When Meta open-sourced LLAMA 2, Yann Lecun publicly said with emotion that the open source and commercialization of LLAMA 2 will greatly change the market pattern of large models.
From a domestic point of view, Ali is another promoter of open source models.
In August 2023, Alibaba Cloud open-sourced the Tongyi Qianwen 7B model, becoming the first enterprise in China to promote the open source of the large model, and then Alibaba Cloud has successively open-sourced 14b, 72b, and 1The large model with 8B parameter scale, the 72B version with the largest parameter scale, even surpasses Llama 2.
The reason why tech giants are willing to open source large models is to accelerate the development of technology with the help of open source.
This is naturally the main purpose of Google's open-source gemma.
Therefore, when Google GEMMA was open sourced, Google also announced GEMMA support for a series of development tools and systems, as well as cross-device compatibility, as follows:
Multi-Framework Tools: GEMMA offers Keras 30. Reference implementations of native PyTorch, Jax, and Hugging Face Transformers frameworks;
Cross-device compatibility: GEMMA models can run on multiple mainstream device types such as laptops, desktops, IoT, mobile devices, and the cloud;
Hardware platform support: Google has partnered with NVIDIA to use NVIDIA GPUs to optimize GEMMA models;
Google Cloud-based optimization: Vertex AI offers an extensive MLOPS toolset with a range of fine-tuning options and one-click deployment capabilities with built-in inference optimization.
Based on this, Google finally launched an open-source model before Meta released a new version of LLAMA, and Google officials even compared the performance of this model on key benchmark sets with LLAMA 2, and came to the conclusion that GEMMA 7B outperformed LLAMA 2 7B and 13B versions.
However, Mobvoi founder Li Zhifei pointed out that "the time is a bit late", "the open source is not enough", and "I feel that this open source is still a passive defense".
He also mentioned that Google often gets up early in the morning to catch up late in the field of AI, and the core component technologies such as VIT, VIVIT, N**IT, and Ma**IT borrowed from this SORA are all the previous ** of the Google team.
Of course, whether it is "attacking" or "defending", the most important thing for GEMMA is to show Google's open source attitude in the field of AI.
In the next 2024, the competition for large models will also intensify.