In the early hours of this morning, GPT4 was once again overtaken across the board. There is no so-called press conference, nothing flashy**, just a post on X (Twitter).
The new model Claude3 that came out this time is very similar to the Gemini released by Google last year, and it contains three different versions of the medium cup, the large cup and the super large cup in one release.
This model was released by the company Anthropic, which was founded by two siblings who left OpenAI in 2021, so some believe that they are traitors to OpenAI's escape.
Anthropic is currently valued at more than $10 billion, second only to OpenAI in Silicon Valley, and investors include Google and Amazon, apparently trying to use Anthropic to counter Microsoft, which was born from OpenAI. Overall, the founding team of this company does have many of the internal secrets of the GPT model, so the strength of their product should not be underestimated.
Last year, the Claude1 and Claude2 models were often used with GPT35 and GPT4. At the same time, there was a public battle within OpenAI last year, and the board of directors even considered selling OpenAI to Anthropic, but the other party did not agree.
There are three different versions of Claude3: Haiku, Sonnet, and Opus, with low to high performance and a corresponding increase in cost. The evaluation criteria are almost identical to those of Google Gemini and OpenAI last year when GPT4 was released for comparison. Looking at the data, the Claude3's oversized cup version outperforms GPT4 and Google's Gemini Ultra on paper.
The comparison between different models mainly includes several aspects, including basic knowledge, university subject knowledge, mathematics, ability and reasoning ability, etc., which are still representative for the evaluation of a model's ability, which is also a standard widely used by these companies.
Judging from the data here, Claude3 is actually because it has a super large cup, a medium cup and a large cup, so in fact, it seems that only the ability of the super large cup is more than GPT4 and Google's Gemini Ultra.
Another big improvement is that Claude3 is now less likely to reject routine tasks from users, because one of the things that Anthropic values is that they feel that large models are meant to serve humans, so they are very strict about human alignment. One of the characteristics that they have been criticized for in the past is that they will refuse to answer many tasks that are actually very common rather than harmful, and in the new version they have reduced the false rejection rate and improved the corresponding accuracy.
From the point of view, the output of the Claude3 API is $75 per million tokens, which is much higher than GPT4 now. At the moment, its two strongest versions are only available to Claude Pro subscribers, and Sonnet's words are now available for free.
However, while both versions of the Gemini Ultra and Claude3 are now claiming to be better than GPT4 in testing, the GPT4 they use here is still one of the earliest versions of GPT, which they released in March 2023 last year. But in fact, the GPT4 we are using now on OpenAI** is actually GPT4 Turbo, that is, the ability of this new version is actually more obvious than the initial GPT4.
So we can't say for sure whether the Claude 3 and Gemini Ultra are better than the actual online version of GPT4 yet.
After all, most people usually complete more than just the tasks they demonstrate during the test here, so the small percentage point difference in their performance during the test may not be so important to the average person.
So in the end, which one is better to use, I think it still needs most users to experience and feel it themselves, so which one is more convenient for ordinary users or which one is recommended. For example, I have more experience in tuning prompts on ChatGPT, and now the entire ecosystem of OpenAI is becoming more and more complete and rich, so using ChatGPT has met my needs at present.
That's why unless it's Google's big model or Anthropic's model is very strong compared to GPT4, I still don't see that I have to switch to different platforms to use it. For most people, as long as you can pay OpenAI that $20 per month membership fee to keep using it, ChatGPT will still be the best choice for you for a long time.
However, I was very happy to see Anthropic release Claude3, and one of the big reasons was the competition and the comparison in the data.
It is estimated that OpenAI can't sit still anymore and will want to release the next version of their ChatGPT sooner, so I think we can look forward to it.
What do you think of the Claude3 model? You can try it out yourself with the free sonnet, and we're welcome to talk to each other.
**10,000 Fans Incentive Plan