Earlier this year, OpenAI's rise seemed to herald Google's doom. But the tech giant has quelled the bickering among its AI researchers and has finally gone on the offensive with its latest AI technology, Gemini. Now, the hard part begins.
The release of the first version of the highly anticipated new AI technology Gemini comes almost a year after some critics **OpenAI's ChatGPT could beat Google's dominance in the search space. Google's leadership has succeeded in getting different factions within the company to start working together to deal with OpenAI, which goes against the perception of those who believe that Google has become too indecisive and bureaucratic. Google's dominance in search remains intact. Moreover, Google's share price has been significantly higher this year.
Now, the hard work begins. In the coming months, Google will have to show how to integrate its AI models, collectively known as Gemini, into its products without harming existing businesses, such as search.
According to people familiar with the matter, Google has integrated a lower-level version of Gemini into Bard, a chatbot created by Google to compete with ChatGPT but with limited use so far. In the future, Google plans to use Gemini on almost every product line, from search engines to productivity apps, as well as an AI assistant called Pixie, which will be limited to its Pixel devices. Products could also include wearable devices, such as glasses, that can use AI's capabilities to recognize what the wearer sees, according to a person with knowledge of the internal discussions. The device can then give them advice, such as how to use a tool, solve a math problem, or play an instrument.
Google must do all of this while carefully consulting with regulators. The company is embroiled in two federal antitrust lawsuits involving dominance of its search engine and advertising businesses. Antitrust keeps a close eye on Google's AI efforts, as this could be an example of how Google can use its dominance in one area to win over another, using data from its search index and data from billions of users to train new AI.
The success of AI is also critical to Google's cloud business, as there is an increasing demand for compute-intensive AI models for these services. Microsoft has already reported that its cloud business is growing faster than Google and Amazon.
While ChatGPT is best known for its conversational AI, the technology behind it has proven useful in the business world, with uses including automating customer service and software coding, generating marketing campaigns quickly, and helping Wall Street companies make sense of vast amounts of data. The problem for Google is that OpenAI and Microsoft already have a big first-mover advantage in selling this technology to consumers and businesses, which gives them valuable data and feedback that they can use to improve the product.
It's all about the actual effect – what people are able to build with it. We're just beginning to see that,"Jon Turow, a partner at Madrona Venture Group, said. He was previously responsible for AI products at Amazon Web Services. "But what we've seen is remarkable. ”
Gemini is one of the most important attempts in the company's 25-year history. As Google enters middle age, its core advertising business still generates huge profits that have funded a series of bets on new businesses such as self-driving cars, health reinsurance and biotech by its parent company, Alphabet. However, none of these decade-old bets worked out.
As a result, investors are increasingly pressing for Google's leadership to cut back on this 18The cost of a company with 20,000 employees, which has led to mass layoffs this year, has taken a toll on employee morale. Employees are preparing for more layoffs in the new year, although it's unclear whether they'll be for the entire workforce or specific teams.
AI is another bet that requires a lot of money from the company to pay for everything from personnel to hardware. According to people close to the Gemini team, Google needs a lot of investment to prevent its AI team from leaving because OpenAI is offering millions of dollars in salaries** to Google's engineers and researchers.
Google also wants to dispel the notion that it has done little to contribute other than benefiting from decades ago's innovations. Over the years, the company has invested heavily in AI research through two separate divisions, Google Brain and DeepMind. It even invented the underlying technology called Transformers, which is at the heart of GPT, a series of AI models created by OpenAI.
However, OpenAI's astonishing rise has raised fears that Google could lose out in the tech space like many other venerable tech giants. Executives within the company were particularly annoyed by the fact that the once-clumsy tech giant Microsoft incorporated ChatGPT into its Bing search engine in February, according to a person with direct knowledge of their discussions.
A Google spokesperson did not comment on the matter.
Combat Lab
For years, Sundar Pichai, CEO of Google and Alphabet, has complained to colleagues that he can't get his two AI research units to work together. Demis Hassabis, CEO of Deepmind, which Google acquired in 2014, has long insisted that the company should be independent of its parent company, an arrangement that would better pursue its goal of developing artificial general intelligence (human-like artificial intelligence).
At the same time, Deepmind's sister unit, Google Brain, focused on research implemented in Google's products, incubating important advances in machine learning, such as Transformer, an invention that paved the way for Google and other companies to train more complex models. The team is led by Jeff Dean, an experienced engineer whose early coding efforts at Google helped scale its search engine to billions of users.
The separation between these two units is very deep. Google Brain is headquartered in Mountain View, California, the headquarters of Google, while Hassabis and his team operate DeepMind from their main office near King's Cross Station in London.
As DeepMind grew, DeepMind did everything it could to avoid working with Google Brain, according to one person who had worked there, for example, by apparently opening offices in cities where Google Brain had no major presence at the time, such as Paris and Edmonton, Alberta. DeepMind's researchers have access to ** written in Google Brain, but not vice versa, which some employees see as a sign of DeepMind's excessive secrecy, even among Google employees. When Hassabis wanted to take steps to protect Deepmind's independence, he spoke directly to co-founder Larry Page, who led the acquisition and owned a controlling stake in Alphabet, alongside co-founder Sergey Brin.
Over time, Hassabis wants to completely separate DeepMind from Google as he becomes increasingly concerned about how the sprawling company will use the technology, including selling it to the military, a person familiar with the matter said. He conceived the idea of creating an independent company with Deepmind's intellectual property. However, in 2021, Hassabis told DeepMind employees that efforts to separate from Google, which he called Project Mario, had come to an end because Pichai had pledged more funding for the team, including for AI ethics, according to a person familiar with the matter.
British Prime Minister Rishi Sunak (left) and Google's Demis Hassabis in London in June. **Courtesy of Carlos Jasso Bloomberg via Getty.
Over the years, tensions have been exacerbated by competition for resources between the two teams. Google only has a limited number of server chips available to its AI researchers. These chips have become more scarce as the frenzy caused by AI across the industry has increased the demand for chips.
At the same time, as Google executives become mired in internal politics, the company's high-profile AI researchers are leaving. Some of them set up their own companies, frustrated by Google's bureaucratic culture, which blocked the release of ChatGPT-like services before OpenAI launched its version. Others were acquired by OpenAI, a nonprofit startup founded in 2015 by Elon Musk and other big names, in part because of fears that Google would have artificial intelligence in the future. One of OpenAI's founders is Ilya Sutskever, a key Google Brain engineer who has led advances such as creating artificial intelligence capable of reasoning to solve problems that have not been encountered before.
However, when OpenAI released ChatGPT in November last year, the public's reaction shocked Google. The 400-employee startup outnumbered Google in launching a chatbot that could convincingly answer a variety of questions, calling into question the company's competitiveness.
Still, some Google leaders don't seem to care about this new internet darling. In response to a question about the chatbot at a staff meeting a few weeks after ChatGPT's launch, Dean said Google wasn't just reacting to other startups, a person familiar with the matter said.
But in February, Microsoft announced that it would implement ChatGPT in its Bing search engine. Some investors' arguments that chatbots could erode Google's search dominance have unnerved its executives.
Merge of minds
Google needs to act now.
One solution it came up with was Bard, which pieced it together in a matter of months and released this chatbot in March. Within Google, the effort caused a stir, and Jacob Devlin, a well-known Google researcher, left the company after raising concerns with Pichai and other senior executives about Google's use of ChatGPT data to train AI models. He then joined OpenAI, but only a few months later, he returned to Google for unknown reasons.
Another response from Google was to finally end the internal rivalry between Deepmind and Google Brain. Google selected researchers from these two teams to build a new model: Gemini, a project led by Dean and senior DeepMind researcher Oriol Vinyals. Previously, Vinyals worked with Dean at Brain.
Then, in April, Google announced that it was merging Brain and Deepmind. Hassabis took over the new entity, Google Deepmind, while Dean stepped back as Google's chief scientist. The move came as a shock to many Google engineers, who felt that given Dean's accomplishments and long history with the company, he should be the leader of the unit.
Google's Jeff Dean in 2020. **Courtesy of D**ID Paul Morris Bloomberg via Getty.
The leadership tried to sell the merger as a victory for the consolidated units. Zoubin Ghahramani, VP of Research at Google Brain, visited DeepMind's office in London to explain the restructuring to employees and at the town meeting the week the change was announced. Google Brain held a separate meeting for its employees. Hassabis told employees that Google Deepmind will bring together two of the best AI research teams in the world.
But Google's AI staff quickly realized that priorities were also changing. Google Deepmind's leadership has reduced investment in research projects that aren't critical to building competitive AI products. According to these people, the projects that lost resources include a sequel to the planned multi-model model Gato and a research team called Genrl, which builds AI systems capable of navigating virtual environments, such as the Atari game.
Executives say the changes have had additional benefits: fewer overlapping efforts and cuts back on low-priority projects, meaning employees would no longer fight for chip access for experiments and research.
At Mountain View, where AI employees were scattered across multiple buildings on the company's campus, they moved into a single office located in the core of the campus with the goal of improving collaboration among researchers.
When the onslaught of OpenAI's explosive growth subsided, Google finally had a chance to fight back.
A secret**
However, Google faces a huge obstacle: building a model that is superior to GPT-4.
From the outset, this meant that researchers had to meet aggressive deadlines for milestones in model development. According to a person with knowledge of the job, employees are working day and night to meet tight schedules, a top-down approach that is a stark departure from Google's previous non-interventionist approach to its research labs. A person close to the job said that for some, an 80-hour work week has become common.
Even outside of the company's AI organization, Google's employees are expected to get to grips with the technology quickly. Throughout the year, Google Cloud provided additional material for non-technical roles, such as sales, by requiring employees to improve their knowledge of AI by requiring them to take exams on the topic, according to two people familiar with the matter.
Google's goal is to gain an advantage over OpenAI by enabling Gemini to understand a wide range of different ** (including text, images, ** and audio) so that the AI can interpret and explain the content of complex charts in plain English. Pichai later said that Gemini would train on these types of data from scratch. Pichai is well aware that OpenAI announced similar image recognition capabilities in March, but these features were not initially widely available. This gave Google the opportunity to release an extensive set of multimodal features before OpenAI.
Google has a secret: YouTube. According to two people familiar with the matter, Google's researchers rely heavily on data from this streaming service, which belongs to Google, including images, and audio caption text, which is essential for training AI models.
This gives Google access to a much richer repository of information than competitors like OpenAI and image generation startup Midjourney. This also means that Google must meet the requirements of the legal department, such as ensuring that if a YouTube user deletes**, Google will also remove that content from the dataset used by its model.
Another advantage of Google is computing power. Unlike OpenAI, which relies on Microsoft servers, Google has its own data centers. It has even built its own dedicated AI chip, a tensor processing unit (TPU), to run its software more efficiently. For the Gemini project, it has amassed a large number of these chips - 77,000 *** TPUs, codenamed Pufferfish. In the third quarter, Google's unallocated enterprise costs, including spending on Deepmind, jumped nearly 40% to $1.6 billion.
Through daily meetings with the staff responsible for various parts of the project, Gemini's leadership keeps a close eye on the progress of the researchers. These meetings are held at the end of the day for employees in London and the start of the day for employees at Mountain View. According to a person familiar with the matter, those who presided over the meeting included Dean, Vinyals and Koray K**ukcuoglu, vice president of research.
Google's headquarters in Mountain View, California. The Washington Post via Getty Melina Mara
Senior executives are also personally involved. Dean is responsible for improving the software that helps the company's algorithms process large amounts of data. Co-founder Brin usually distances himself from Google, but works side-by-side with Jimini researchers in Mountain View and often eats lunch with them in the company's cafeteria.
In the process, there were some awkward moments. During Bard's launch demo in February, the chatbot had a factual error about the James Webb Space Telescope, which embarrassed Google just as it tried to catch up with OpenAI. When the mistake became known, Google's stock price rose by as much as 9% two days later.
In May, Google first revealed Jiminny's existence at its annual developer conference. One of the analysts was impressed: the company's share price was more than 4% higher that day.
Be prepared to compete.
Over the next few months, Google got closer to releasing Jiminny. In September, it offered a smaller version of Jiminny to some developers for testing.
But in the same month, OpenAI was the first to launch GPT-4 with vision in terms of multi-modal capabilities, which brought more attention to its technology and new business. According to a person close to the team, Bard's user usage has disappointed some executives internally. Google disclosed in October that its cloud computing division had a revenue growth of just 22% in the third quarter, while Microsoft announced a 29% increase in revenue from its Azure cloud division on the same day. This only adds to the pressure on Jiminy's team to come up with large-scale innovations.
Then, around November, during an executive review of the new product, the state-of-the-art Jimini model didn't work well in languages other than English.
It's a small consolation that OpenAI is facing problems of its own. In mid-2023, OpenAI dropped an important new model called Arrakis because of its poor training. At the end of November, OpenAI's board fired CEO Sam Altman, nearly bringing the company to its knees. Altman returned to OpenAI after this incident and seems to have stabilized the situation so far.
Finally, in early December, Google lifted the veil on Jimini. It published test results showing that the strongest version of Jiminnie, Jimini Super Edition, outperformed GPT-4 in many industry-standard benchmarks, although many researchers questioned these claims. The biggest stain on this high-profile launch was a piece of marketing by Google**, which, according to Google's own claims, exaggerated Jimini's capabilities. The release of this ** frustrated some of the company's rank-and-file employees because they hadn't seen it beforehand. But a series of announcements sent a strong message: Google is ready to compete.
One executive even attacked Microsoft, accusing it of relying on OpenAI to develop cutting-edge technology. Kent Walker, president of global affairs at Google and Alphabet, said at an event hosted by news agency Semafor that the company "does not believe in outsourcing" its AI development.
Now, Google's test is to introduce Jiminny into various parts of its portfolio, just as Microsoft has leveraged OpenAI's technology. But it has an advantage that its competitors don't: a range of Pixel hardware devices, including phones, watches, and headphones, that can benefit from AI. A version of Jimini is specifically designed for use on the Pixel phone, which uses Google's custom artificial intelligence chip.
Pixie is an AI assistant developed specifically for Pixel devices that can boost Google's hardware business at a time when tech companies are racing to integrate hardware with new AI capabilities. Sources with knowledge of the project said Pixie will use information from customers' phones, including data from Google products such as Maps and Gmail, to evolve into a more personalized version of Google Assistant. This feature will likely be available next year alongside the Pixel 9 and 9 Pro.
Eventually, Google wants to bring these features to its low-end phones and watches, among other devices. Companies need more sophisticated models to support all product ideas. But it seems to be struggling to make sure it doesn't get caught off guard again. According to a person familiar with the matter, Google is already training its next big model, the Jiminny 2.
Amir Efrati also contributed to this article.