Along with Gemini's generative AI model, Google released AlphaCode 2 this morning, an improved version of generative alphacode that Google Deepmind Labs launched almost a year ago. Alphacode 2 is actually powered by the Gemini model, or at least some variant of it (Gemini Pro) fine-tuned on top of the programming race data. Google says that in at least one benchmark, Alphacode 2 far outperformed its predecessor.
According to Google, Alphacode 2 (coded in languages such as Python, J**A, C++, and Go) outperformed about 85% of its competitors on average in a subset of coding competitions hosted by CodeForces, a programming contest platform. In comparison, the average score of the previous generation alphacode on the same subset was only 50%.
We selected 12 recent competitions with more than 8,000 participants, either from the second group or from a more difficult one'1+2'Group. This allows us to solve a total of 77 issues"AlphaCode 2's Technology*** is written. "Alphacode 2 solves 43% of problems in 10 attempts, nearly double the original alphacode (25%).
AlphaCode 2 understandably involves"Complex"Programming puzzles in math and computer science theory. DeepMind research scientist Rémi Leblond explains in a pre-recorded video that AlphaCode 2 is capable of dynamic programming, among other rather complex technologies.
Alphacode 2 not only knows when to properly implement this strategy, but also knows how to use it when it is being implemented. Leblond says Alphacode 2 knows not only when to properly implement this strategy, but also when to use it. This is noteworthy considering that programming problems that require dynamic programming were a big stumbling block to the original alphacode.
leblond says:"[alphacode 2] needs to show a certain level of understanding, a certain level of reasoning, and a design of a solution before it can actually be executed to solve [a] coding problem. It can do all of this on problems that have never been seen before"。
AlphaCode 2's solution to the problem is to take advantage of it first"Policy model"series to generate a large number of samples for each question. Samples that don't match the problem will be filtered out, and the clustering algorithm will use the"Semantically similar samples"grouping to avoid any redundancy. Finally, the scoring model in alphacode 2 starts with the 10 largest samples"Clusters", and that's the answer to the question in alphacode 2.
Now, all AI models have flaws, and alphacode 2 is no exception. According to alphacode 2, it requires a lot of trial and error, the cost of scaling is too high, and it relies heavily on being able to filter out obviously bad samples. **Presumably, migrating to a more powerful version of Gemini such as Gemini Ultra may alleviate some of the issues.
Eli Collins, VP of Product at DeepMind, hinted at this possibility in a briefing.
Collins said"One of the things that excites me the most about the latest results is that when programmers collaborate with [Alphacode 2 powered by Gemini], the performance of [the model] becomes better by defining certain properties that ** follows. In the future, we will see programmers utilize high-powered AI models as collaboration tools to assist with the entire software development process, from reasoning about the problem to assisting with implementation. "