Author |Wanchen, moonshot
Edit |Jing Yu.
In the past eight months, the Baichuan Intelligent team has relied on the accumulation of AI technologies such as search and high-quality data processing to rapidly iterate on the size and quality of the model. Wang Xiaochuan, founder and CEO of Baichuan Intelligence, believes that Baichuan should be half a step slower in the ideal and three steps faster in the landing.
On December 16, 2023, at the Geek Park Innovation Conference 2024, Wang Xiaochuan further shared new thoughts on large models.
As for the evolution direction of the large model, Wang Xiaochuan believes that at present, the large model is learning, but not thinking, and the next direction of the evolution of the model is to combine learning and thinking. In the era of large models, the characteristics of large model technology must be considered, which is the biggest difference between making products in the past and now.
In the past, when I did applications, I always talked about the matching between products and markets - PMF (product market fit), but one word was lost outside the product and market, technology. 」
He believes that the current large-scale model technology is still far from AGI, and it is more important to be clear under the premise of this imperfection: what kind of product is suitable for such a technology, rather than the product manager who has insight into the market and starts to do it when he comes back.
In Wang Xiaochuan's view, under the new development paradigm brought about by the large model, the starting point of the product manager should be from thinking about product market matching (PMF) to thinking about how to match technology and product, that is, TPF (Technology Product Fit).
Wang Xiaochuan believes that the first thing is to provide a 10 times better experience than traditional applications, and users can use it well. To make such an application, the product manager must not only be an avid fan of the model himself, but also have both traditional product experience and the imagination to be able to figure out what the model looks like.
The following is a transcript of the 2024 dialogue of Wang Xiaochuan Geek Park Innovation Conference.
Peng Zhang: You've been to our conference many times, and just now you were listening carefully to the two technical experts, who talked about some of the key factors behind the OpenAI incident some time ago, and even mentioned that large models need the ability to think slowly, I don't know what you think?
Wang Xiaochuan: Yes, this year, we are preparing for the big model to 4 The company was established in January, I mentioned a few keywords, one is called search enhancement, because it is necessary to connect traditional knowledge, and the second I hope is that the large model is to do strong chemical Xi, at that time, I mentioned this point, because I have seen that the large model itself represents a way of fast thinking, like a person, I will give you the answer as soon as I pat my head, and I can say that it has its own shortcomings in learning Xi and applied reasoning, and it is certainly not enough to take the large model as the origin, at that time we thought that strong chemical Xi could be of great help, which is also an area that has been very concerned about in the internal work of Baichuan。
Zhang Peng: Just slow thinking?
Wang Xiaochuan: Yes, it is slow thinking, and compared with slow thinking, today's big model represents fast thinking. Let's talk about two of my own opinions: fast thinking it is not called thinking, slow thinking I think it has so much more thinking, represented by the openai large model, it is called learning, and its knowledge is learned.
I don't emphasize the "thinking" of reasoning, in fact, when people are learning Xi, you may have to think for a long time, this is called thinking, so Confucius said before that learning without thinking is reckless, thinking without learning is dead. 」
Specifically, the large model is learning, it is actually not thinking, it is not like people, it will ponder back and forth, and will open the imagination space to see, what system is thinking?OpenAI has just started the company and what DeepMind is doing, such as doing AlphaZero and playing games, is thinking about it.
But that is the setting of the strong chemical Xi, called the confrontation of multiple agents, alphazero It is not a learning Xi system, it throws away the previous 60 million games (chess training), but it is itself in the confrontation game, in the game to find a new understanding, ultimatum, it is such a thinking.
AlphaZero stops in place after thinking, it is dead, it only does a specific task, and cannot expand it to other fields, so we say that the large model represents learning, and alphazero represents thinking, and these two systems will be very powerful together.
Wang Xiaochuan, founder and CEO of Baichuan Intelligence, and Zhang Peng, founder and president of Geek Park, analyzed the learning and thinking of the large model of Geek Park.
Zhang Peng: So the next important thing is to really learn and think, right?Learning and thinking should be combined.
Wang Xiaochuan: That's right. To be more specific, we think of this scenario, where you ask the large model how to play Go, and it actually can't play. But you ask if you can judge a Go player who loses and wins?A large model can be judged based on what it already knows. Even if you say that you write a ** to determine the winner or loss of this Go, the large model can write this **.
You ask it to write a **, how the state of the move is transferred after each move, that is, the whole process of playing chess, it can also be written.
So imagine that if the large model is strong enough, although it will not play Go directly, it can write a state transition (transaction function) such as playing Go, and finally determine the winner or loss of Go. That is to say, there is a chance for the large model to write an alphago ** and then run it, and after the run, you can play chess, and this thing has a chance to happen.
Zhang Peng: The technology of Europe and the United States is still constantly exploring the boundary, which also makes people feel pressured, how do you think this distance can be measured and shortened?Can you create different value by yourself?
Wang Xiaochuan: Before I went to the United States, I said this in Baichuan, which is to be half a step slower in the ideal and one step faster in the landing. Later, after going to the United States and coming back, he folded the ideal in half, turning it into an ideal that is one step slower and three steps faster when it lands.
Zhang Peng: How do you understand that the ideal is one step slower and the landing is three steps faster?
Wang Xiaochuan: After getting in touch with them, I think the bottom of the thinking of the two sides is different, OpenAI is a non-profit organization, and it just wants to explore the boundaries of AGI, and they really do it.
The last time I talked to them, they wanted to try to put 10 million GPUs together and build a system big enough. What is the concept of 10 million GPUs?Nvidia produces one million pieces a year, GPT-4 is about 25,000 pieces, and we have the GPT-3 as our benchmark today5 is only 4,000 GPUs. When they are thinking about problems, their starting point is not in the same world as us, so we can't compare with them in this regard.
In this case, people and companies have to find their own positioning, and in this soil, we must have a confidence that we have the opportunity to go faster in the application landing.
Maybe as our user dataset becomes larger and the technology accumulates stronger, our application can be good enough, and it can even be used in the United States. In this case, it doesn't mean that you have to reach the stage of GPT-4, GPT-5, and GPT-6 before you have the opportunity to apply it.
Different soils grow different things, and doing application is a traditional strength of China, and it is also an innovation, but I think it is fair. This is also a better opportunity for Chinese companies, especially now that the United States is dominated by OpenAI, companies that make applications have to face OpenAI's technology to make applications, and you can do what kind of applications its technology makes.
However, domestic model companies can make applications themselves, and this kind of end-to-end coherence has the opportunity to run out faster than American companies in a field when the application is implemented.
Wang Xiaochuan believes that China's large models may run faster in the application of Geek Park.
Peng Zhang: Sometimes we are definitely willing to pursue an ideal and a sense of mission, in the big process of AGI, we can join this team, they may be forwards, breaking through boundaries, but we may be a free man or a midfielder, but he also has meaning in the team, for example, I can put the technology down and turn it into something valuable.
Wang Xiaochuan: There will be derivation at these two levels, what Mr. Peng just said is that you, as a global citizen, as a Chinese company, are a division of labor and cooperation in the world, rather than a division of friends and enemies, there is only a competitive relationship, and we respect their inventions, we catch up, but we also have our own unique contributions. And not just: I think I need myself, the world doesn't need me.
Zhang Peng: It's good, I think very clearly, and in this wave of entrepreneurship, I found a point of reconciliation with myself, that is, how do we become a player in a meaningful game in the world, not everyone must be a striker.
That brings us to another problem, everyone is talking about super app today, but they don't see what the future is super app, and Robin Li also said just now that today is not sure. But I think if we want to make a super app, what kind of starting point do we need?For example, in the past, we talked about PMF (product-market matching), but how to do this PMF today
Wang Xiaochuan: Yes, I think this point may be closer and farther away. The reason for this is that when we imagine refactoring the original application, such as refactoring WeChat again, this perspective may limit ourselves all of a sudden, so first, you have to zoom out of this perspective.
Therefore, back to making super apps, a long-term direction is that it represents the fundamental demands of people, and this fundamental demand, I summarize it into three key words, people need to have three things: one is to be creative, the second is to be healthy, and the third is to be happy.
Health and happiness are very easy for everyone to understand, but creativity comes from the fact that people in the world always hope that your existence is different from the world, and you can change the world, so how can you help you change the world is a category that can be separated independently.
When we have this vision, how to do health, how to do entertainment, and how to help you get information and make you more creative, in the long run, these three directions will be there. But on the other hand, we will also throw away some things, such as doing marketing copywriting, helping you do customer service conversations, in fact, the big model is quite good at doing these, but I think these do not return to the fundamental needs of people, which falls into the original so-called reconstruction logic.
Therefore, after having such a sense of three general directions, I will have different ideas, which is a kind of thinking that has been opened, otherwise I will fall into the pit of competition in large factories.
Second, I would like to mention an important word, that is, I just mentioned the word PMF, and I want to use a new word, because PMF always talks about the relationship between products and markets, and I have dropped one word, which is technology. Technology, in the era of AI, it still has a lot of imperfections and uncertainties, unlike when it used to be ** or WeChat.
I think technology is the bottleneck now, but in fact, technical problems can definitely be solved, it just depends on the level of engineers, costs, etc., what you want, can be achieved at the engineering level. However, large-scale model technology, including the illusion and timeliness just mentioned, can only speak in natural language, and this technology itself has limitations and imperfections.
So we are still far from AGI, because of the imperfection of technology, we have to be clear, a technology is suitable for what products, rather than to grab the market first, after looking around the market and start to do it, I think this kind of courage is quite valuable, but how to coordinate between first-principles TP technology and products, how to do it, I think is something to think about now.
A good example is characterai，character.The founder of AI is not actually a product background, he is very familiar with the technology, especially the algorithm behind the product, and he also sees that the technology itself is imperfect and may make mistakes, so he first thought of using it for the entertainment industry. Secondly, what this technology can carry first and foremost is a natural dialogue, which is a persona, so make it a character.
Zhang Peng: So its shortcomings become characteristics
Wang Xiaochuan: Let me first mention two concepts, one is that we used to think that we were making tools, and tools actually represent a lot of certainty, but this time we are not making tools, this time we are building partners, more like new species like people. We humans have to accept its own shortcomings, its strengths. People are hallucinating, and I can use them if they have hallucinations, so why can't machines use them if they have hallucinations?
In the end, a person should match one thing, so in terms of technology matching, we think that we have to change the perspective, not from the perspective of tools, but from the perspective of a person, which is one of my ideas.
Peng Zhang: You just said Technology-Product-Fit, TPF, not PMF.
Wang Xiaochuan: Yes, there must be enough understanding of the technology itself, so that the technology can match related things, which is a requirement for the product manager, or the product manager in the company's No. 1 position should have such a cognition, what the large model is good at and what it is not good at. This process is the creation of people, not tools.
There used to be a story about the king and the painter, the king was blind in one eye and missing a leg, but he was very narcissistic, and if he wanted to paint a self-portrait, he would pull all the painters from all over the country to paint and paint one and kill the other, because the painting was too similar, missing an eye and missing a leg, that is to slander the image. But if the painter paints his eyes with a blazing and heroic image, that is, he will bully the king and kill him as well, and this problem will not be solved. Later, a painter drew a picture of the king hunting, standing on a large rock, curled up with one leg covered, and the king drawing his bow, and the missing eye happened to be closed.
What the technology is good at, what it is not good at, and how to do the matching, this has greater requirements for the product manager, which I call TPF.
Peng Zhang: I think the word TPF is very good. TPF seems to be the starting point, if we stand in the future, we want to make super apps, how can we do TPF well?What is a good TPF?
Wang Xiaochuan: In the past, the product manager wrote more of a document describing the definition and requirements of the function, and could draw a structural design diagram to show the boss, the product looks like this, meets the needs of users, and accurately achieves the functions of each step.
Today's large model is not like this, every time you input a large model, its output is uncertain, and it cannot be finished in one word. At this point, it's hard to explain this with a set of deductive rules. The logic is deductive, and it must be dismantled into a bunch of evaluation sets, and the product manager's requirement is not only to define the product, but to transform the defined product into the later evaluation set. That is, what kind of test set to do for the output of the model under some kind of input.
At this time, the technical counterpart is not the engineering personnel, but the algorithm personnel. The previous working Xi of the algorithm was that you gave me the evaluation set, and I optimized my algorithm to meet the evaluation set. Whether it's through the prompt method, or the SFT, or the post-train method. In this case, the product manager defines the evaluation set, and after the technology gets the evaluation set, it then looks for a dataset or training set to train the system to meet the evaluation set.
Wang Xiaochuan explains how to set up an OKR geek park for a large model.
Peng Zhang: This is to set OKRs for large models.
Wang Xiaochuan: It has a very rigorous mathematical evaluation method. Engineers who have worked on algorithms will adapt to this approach, and finally use the evaluation set and data to speak, which has become a standard working method within us.
Including search companies are also this method, search is an algorithm-driven product, with a review set driven way, but we used to be in the Internet development to the advanced stage, technology is not a problem, or even no longer algorithm-driven, is engineering-driven, this PMF is not wrong, but a layer of TPF is missing, and finally you will find that the product is not unable to meet the market demand, but has been iterating, can not make a phased product.
Peng Zhang: You just explained to some extent a question that I am very concerned about - what is the development of ai-native. Essentially, it's about what we're developing, and you have to set the evaluation set under the set goal, so that the dataset can be effectively trained to meet the requirements of the evaluation set, which is your real development engine.
Wang Xiaochuan: It's called ai-native. If it is agi-native, it is to take the paradigm of AI model capabilities even deeper.
Peng Zhang: This does put new demands on product managers. In the past, we said that PMF is doing a good job, and we have a sense of it, such as the increase in user usage, and the user experience is very good. But now how do you evaluate doing a good job of tpf?
Xiaochuan Wang: TPF first has requirements for product managers.
First, it must be able to transform the requirements into a test set, and the test set will allow the technical engineer to find that the feel is improving when meeting the requirements. And when the demo is rolled out, the distribution of the user's requirements is exactly the same as the distribution of the evaluation set proposed by the product manager, and the results in the evaluation set can meet the user's needs.
Second, PMF will be mentioned when promoting products to see whether the distribution of marketing fit (market fit) on the market is consistent and whether users are satisfied.
Peng Zhang: If users can use the product you developed well, should they use it well or use it well?The cool use is that the number of users has exploded and has become a super app;Using it well is a step-by-step approach. Are we going to pursue a burst of explosions?Or do you solve the problem of the minority first, and then solve the problem of the majority?
Wang Xiaochuan: This is not a contradiction. First of all, good is compared with the original, how much better you can compare yourself with yourself. If compared with the mature large manufacturers, good % is a huge benefit. But for startups, if it is an AI-native native application, it must be used at the beginning, at least for a specific type of specific needs with characteristics, users must feel ten times better perception.
Zhang Peng: Cool is ten times better.
Wang Xiaochuan: It's not better, it's to make you feel surprised. Today, the large model must be selected to do it with bright spots, and the experience must be improved tenfold, and the surrounding demand must be increased by five or three times, so that the peak can be pulled high enough and then gradually widened. I don't think this product is enough if it doesn't make you cool at first, but it's just better than the original.
Zhang Peng: Many people in the venue today are also very concerned about how to participate in this new era promoted by large models. If you want to be a product manager under the new paradigm. How should they go?
Wang Xiaochuan: Looking at the attributes of the company, a company is to do end-to-end, and it must not only do applications but also models. One is a more application-oriented company, which doesn't touch the model much or solves it with a small model. The paths of the two types of companies are different, but there is one thing that must be done first, that is, to use yourself, to experience and feel the difference that this model brings to you, so that you can be curious, feel it, and appreciate it. I'm going to use this model today, just like a friend, you can feel what works and what doesn't.
Zhang Peng: You have to become a superuser of a large model first.
Wang Xiaochuan: I believe that fans of Geek Park are born with such motivation and curiosity. Once you've used it, you'll be inspired, you'll know what it's good at, and then turn it into your idea for the next product.
Zhang Peng: The large model technology is still in the process of rising tide, so we have to follow it first, and get closer to it before we can consider how to apply it.
The company is constantly evolving, and you must be constantly hiring. When you go to choose a product manager, what kind of temperament and experience will you pay attention to, can you open up your selection criteria?
Wang Xiaochuan: Baichuan plans to release a super app next year, and we don't talk about experience, but only some imagination.
We really hope to find people who have previous experience, and if you don't have previous experience and ideas, you say I want to start a business, and in this case, it is quite difficult to start a business. We will ask you to be able to throw the product in its entirety and with a sense of imagery. To be able to imagine what a large model looks like, you also have plenty of motivation, curiosity, and imagination.
Wang Xiaochuan explained in detail the need for product managers in the AI era to both want and geek parks.
At the same time, we also hope that you have previous experience in making traditional products. That is, I hope to have the previous successful experience, but I can also break up my own experience to nourish the large model, and also conceive a new look of the large model.
Today, the environment in China is different from that of the United States, including Baichuan and domestic companies, which are in a state of racing against time, and cannot give you three or five years to explore.
Peng Zhang: If someone has experience in a related field but does not have the technical ability, can he independently explore the application of large models?For example, you are working hard in the field of health, and I have accumulated many years in the field of health, and I also have the temperament you said. Am I joining your company?Or can you also explore it yourself after plugging in someone else's model?
Wang Xiaochuan: Everyone will do the two paths, and there will be people who explore them by themselves, but it is very likely that they will find that they can't walk during the exploration process, and there is a sense of powerlessness, and in the end they still need the support of the model.
Therefore, in China at present, it is more likely to join a model company, because it is not yet to the extent that it can be applied independently. There are articles on the Internet saying that they can adjust the model for application, and this era has not yet come. In the next two years, it is better to join a company that can provide platform-level support to help you break up the original experience and integrate it, so that the probability of success will be much greater, and it may be a super application. It's okay to make a small application, but to do a big thing, try to fully interact with the model company.
Zhang Peng: It sounds like they still want me to join Baichuan.
Wang Xiaochuan: It mainly depends on whether you want to be big or small.
Zhang Peng: If you want to be big, you have to go to Baichuan.
Wang Xiaochuan: Yes.
Peng Zhang: In April, everyone wanted to stay up at night, but now that the big model has been running for 8 months, the excitement at the beginning has almost faded. It is difficult to start a business, after a period of precipitation, how is your mentality of starting a business this time?
Wang Xiaochuan: In the past 8 months, the team has been running fast and growing rapidly. Now that we have come to the period of more precipitation of large models, although we feel that our previous technology, capabilities, products, attention, and experience are enough, we still feel that we are not light enough when we do it.
In the process of jointly exploring the methods of large models, our understanding of how to find out the most effective linkage between models and applications is also improving. I think a good state is to see that you were a fool a month ago, then you are improving again.
When I first started working a few years ago, I was iterating at a weekly rate, and I would find that I didn't have enough ideas, but this time (big model startup) we went back to the monthly unit, and we didn't get to that agile state. After a month, I saw my previous shortcomings, in rapid iteration. In order to participate in the era of large models, our management and product managers are trembling and constantly adjusting their original working methods.
Zhang Peng: This is a state that you enjoy very much.
Wang Xiaochuan: Yes, I am stimulating progress every day, and I still have multi-dimensional growth, even if my ideas can be half a step ahead, but sometimes I find that I have better ideas as I walk.
Peng Zhang: I quite understand this state. In another five years, what will this company look like that you will feel more satisfied?What are the company's goals?
Wang Xiaochuan: In the three directions of helping people create, be healthy, and be happy, we have super apps to explore. I hope it's one to five years, five years really don't dare to think, because five years later, the height of technological development may not be what we can understand now, every day our technicians sigh that there are new ** and development appeared, there is a strong sense of pushing back.
I hope that in two years, we will prove that large models can be used as super applications, and in terms of health, entertainment, and helping people create, it can bring great help or hope to people like the Internet age, and people can experience or use it, I have this belief.
In five years, we may have a whole new way to play, and maybe five years from now, robots on the ground are running, everyone is wearing VR glasses, and everyone's digital avatar is coming out. Five years is too long, and I am very satisfied to think of two years of pictures.