This is the 8th month of Wang Xiaochuan's entrepreneurship and big model.
Many of the common sense and inertia practices of the industry in the past are no longer applicable to the current era of large models, Wang Xiaochuan believesAt present, the application of many large models does not really meet the needs of users, and if they continue to do it, they will be involved in the competition track of large manufacturers.
Once highly prized by Sequoia and A16ZPMF (Product Market Fit).Entrepreneurship standards, because of the iteration and status quo of the technical paradigm, are no longer applicable to entrepreneurship with large model applications. Wang Xiaochuan believes that what needs to be looked for more at present isTPF (Technology Product Fit).Instead of a group of product managers going to investigate the market first, they should first think about what products are suitable for the current imperfect (large model) technology. 」
In the past, people were Xi to developing tools, but what we built with AI is not a tool, but a new species.
At the Geek Park Innovation Conference 2024, Wang Xiaochuan shared his new understanding of the landing of large models after eight months of practice, as well as his thinking and precipitation of entrepreneurship under the new wave of technology.
The following is a conversation between Wang Xiaochuan, founder and CEO of Baichuan Intelligence, and Zhang Peng, founder & president of Geek Park, which has been edited and organized.
The King and the Painter
Fable in search of AI Native
Peng Zhang:Robin also said that I'm still not sure what a super app is in the AI era, so where should we start?
In the past, when we made products, we would first formulate a PMF (product-market fit), but now?
Wang Xiaochuan:There are two levels, one is to pull away and the other is to pull closer.
Pulling away is to refactor, to make changes in the original application, for example, WeChat to reconstruct again, but this perspective will limit our thinking.
So, I want to continue to pull away, if not the market is the long-term goal of this super app, but the satisfaction of people's fundamental demands?
People need these three things:Health, happiness, creativity。The first two need no more talk. When it comes to creativity, people want their existence to make a difference in the world. So how can AI applications help people change the world? Like the concept of the DIKW model (data-to-information-to-knowledge-to-wisdom model), it provides information, knowledge, and even wisdom to people. That's a big vision.
At present, many AI applications, such as marketing copywriting, customer service conversations...In my opinion, these do not return to the fundamental needs of man; Moreover, if everyone still does this, they will also be involved in a competitive track with large manufacturers.
Coming back to Closer, you just mentioned PMF – product-market fit, and now I'd like to come up with a new wordTPF (Technology-Product Fit).In the past, people only focused on the match between products and markets, and lost technology. When we used to do ** and WeChat, although the technology was the bottleneck, as long as the level of the engineer went up and the cost went up, we could basically break through this bottleneck. And now AI technology, it has some imperfections and uncertainties in itself. For example, problems such as hallucinations and timeliness can only be used in natural language, and the technology itself has limitations and imperfections.
Since there are so many imperfections in AI technology, it is not necessary to send a bunch of product managers to think about the market and start working on it after the insight is completed, but to think about what products such a currently imperfect technology is suitable for.
Here I want to talk about characterAI, the founder of their company (Noam Shazeer), one of the authors of Transformer **, is not from a product background. He is very aware of the underlying technology of large models, and knows that it will definitely make mistakes, so he first uses large models to make entertainment products, and then, he thinks that this technology can first carry the form of natural dialogue, so he makes the products into characters and personal designs.
Peng Zhang:In an entertaining scene, shortcomings become characteristics.
Wang Xiaochuan:Before, we were building tools, and tools had a lot of certainty; Now, what we're building with AI isn't a tool, it's a partner—a new species, a more human-like app. It has advantages and disadvantages, just like people. If people have hallucinations, we can use them, so why don't we use machines if they have hallucinations? Finally, it comes back to what needs the specific technology should match.
This is a requirement for product managers, and the company's No. 1 position with product managers should resonate with this technology, what it is good at and what it is not good at.
To tell another story, there was a king who was blind in one eye and lame in one leg, but he was very narcissistic, so he brought in painters from all over the country to paint self-portraits for him, and the result was to paint one and kill the other. Because some people painted too much like, they were killed for slandering the image; Someone deliberately beautifies and is killed for deceiving the king....Until a painter painted a portrait of the king hunting, because the king was standing on a large rock, the defect of his lame leg would be covered, and the king's eye was closed the moment he drew the bow, so that he could neither slander nor beautify, and he would take care of it at once.
So, I thinkToday's product managers should be clear about the strengths of AI technology and the lack of AI technology, and do such a match and consideration as TPF.
PMF assesses the number of usersTPF depends on the test set
Peng Zhang:PMF we will set certain goals, I can see that I have completed PMF on some indicators, and TPF is a starting point today, if we want to do super APP in the future, under what circumstances can we be considered to have done a good job in TPF?
Wang Xiaochuan:In the past, the product manager wrote a document describing the features, definitions and requirements, and drew a structural design to show the boss: what kind of needs the product can meet in this way, and how the functions are accurately achieved at each step.
However, this method is not applicable in large model scenarios. Because every time you give an input to a large model, its output is uncertain, and in the face of this non-unique correspondence, it is difficult for you to do it well with a set of deductive rules.
So what now? You have to break it down and turn it into a set of evaluations, which is a set of tests on what inputs the model can give and what outputs. The product manager not only defines the product, but also transforms the defined product into a review set.
At this time, the product manager takes the evaluation set to the corresponding algorithm person, who optimizes the algorithm through different methods such as prompt, supervised fine-tuning (SFT), and post-training. That is to say,The product manager defines the evaluation set, and the (technical) algorithm finds the dataset (or training set) to train the system to meet the evaluation set.
Peng Zhang:It's like setting an OKR for a large model?
Wang Xiaochuan:Engineers who have worked on algorithms have adapted to such a way of working and giving them a way to both evaluate the set and leave blank space, which has become a standard working method within us. Algorithm-driven products use a review set-driven approach.
In the previous stage of rapid development of the Internet, technology was no longer an obstacle element, and even product development was no longer algorithm-driven, but engineering-driven, just a difference in the speed of execution.
In the era of large models), PMF is not wrong, but we lack a layer of TPF, which will make us finally find that it is not that the market is not satisfied after the product comes out, but that we have been iterating and cannot make a phased product come out.
Peng Zhang:What you just mentioned, setting goals - transforming the evaluation set - so that the dataset can be effectively trained to meet the requirements of the evaluation set, is this the engine that you developed?
Wang Xiaochuan:That's right, that's called AI Native. If it is agi native, it needs to integrate the model capabilities of agi more deeply.
Peng Zhang:Again, judging that TPF is doing a good job, does it mean that the number of users of the product has increased? Or do users rate your product experience as good? What to judge?
Wang Xiaochuan:TPF has two requirements for product managers before a product can be released.
First, be able toTransform requirements into a test setThis test set enables the technical engineer to see that the results are improving while meeting the process (goal).
Second, once the demo is done, you can also find the requirements proposed by the user, which may be the requirements mentioned in a simple sentence, and the distribution of this requirement is consistent with the distribution of the product manager's evaluation set.
A concept of statistical probability is used here, where the distribution of user needs is consistent with the distribution of the product manager's test set, and the results of the evaluation set meet the needs of the user. SoTPF is satisfied in the form of a test set, on the one hand, the internal indicators are satisfied, and then when it is released, the PMF feedback will reflect whether the user's demand feedback is consistent with the distribution of the test set, and whether the user is satisfied.
Peng Zhang:So should users use it well, or do they use it well? The former is to explode and rise all at once, and the latter can be done step by step. Should we pursue a burst of power today? Or do you solve the problems of the few first, and then the majority?
Wang Xiaochuan:In fact, the two are not contradictory.
But what is well done? It is easy for you to compare yourself with yourself, to be better than yourself, and to accidentally fall into the old Xi of the big factory people. If it is a mature large factory, then it is 20% to 30%, and there are already huge benefits; But if it's a startup,The AI native app should be easy for users to use from the beginning。In satisfying the needs of a particular class,Your product must provide 10 times the coolness of the competition, not better, but pleasantly surprised.
Because today, large models are not omnipotent, you can only choose the bright spots, make 10 times better, and the periphery (function) is 5 times, 3 times or even worse, so that your crest is high enough, and then gradually widen it later. If a product doesn't make you cool at first, less than a certain height, just better than the original, it's not enough.
Entrepreneurs in the new eraFirst of all, there are super players with large models
Peng Zhang:Under the evolution of the product paradigm, in the face of new changes (new paradigms), how should entrepreneurs enter the market?
For example, I was so excited just now that the paradigm of making products is different, and we want to be product managers under the new paradigm. How should they get going? Not everyone is like you, done a search, did a great product. We can't everyone put this label on to get in. How should others get in?
Wang Xiaochuan:I think it depends on the attributes of the company. One company is end-to-end, and it (the company) itself is to make both applications and models; The other kind of company doesn't touch the model or uses a small model to solve the problem, but more to do applications. Therefore, I think there will be some differences in the path, but there is usually a necessary premise - to become a large model user is to regard yourself as a fan of the large model era, to experience it enthusiastically, to feel what kind of difference the large model has brought to (you), to be curious, to appreciate, to feel whether (it) is done well.
Peng Zhang:In a way, you have to become a superuser of a large model first.
Wang Xiaochuan:[You] want to use all the products on the market, and the readers of Geek Park are naturally motivated and full of such curiosity. Once you've used it, your inspiration will come to you, you'll know what it's good at, and you'll turn it into an idea for your future product.
I want to make a super-app in the next two yearsI still have to join a large model company
Peng Zhang:In the process of today's technology tide, you may have to follow it first, and you have to get closer to it before you can consider how to use it.
Today, the company is constantly developing, when you choose a person, what temperament will you pay attention to, or what historical experience?
Wang Xiaochuan:Baichuan will release a super app next year, and it is still on the way. I don't think we have achieved enough to achieve our goal today, so we can only talk about some of the experiences and some imagination brought about by the shortcomings we saw in the process.
We really want to choose experienced people, if you really have no (product) experience, that is, a novice, this situation will be more demanding. For example, you need to be able to throw out the full picture of the product.
You need to be fully prepared for what the big model will look like in the future, including what's inside, that is, you have to have curiosity, imagination to push you to make something, and you have to have such an ability. At the same time, we hope that you have previous experience with tradition, and we have to break it up to nourish the big picture.
Many of the product managers we met had a very complete set of thinking paradigms and frameworks. However, when making a large model, he wants to bring the technology of the large model to the original framework paradigm, instead of deconstructing the original things, which will bring great challenges.
Therefore, we hope that (you) have the previous successful experience, but also be able to break up your own experience, nourish the big model, and imagine the new appearance of the big model, which is the stage of both wanting and wanting.
Nowadays, China and the United States are facing a different environment, Baichuan and other domestic large-scale model companies are in a state of racing against time, in this case, the company will most likely not give you three or five years to explore. In the case of the main direction, our requirement is to have both previous experience and to be able to overthrow and integrate.
Peng Zhang:So if I have experience in a certain field, but I don't have the technical ability, can I do the application exploration on my own? For example, I have many years of experience in the health field, and I have the characteristics you mentioned, and I chose to join you? Or can you also use someone else's model to explore this?
Wang Xiaochuan:There are two paths that everyone will do.
There will be people who explore on their own, but in the process of exploration, it is very likely that they will encounter a sense of powerlessness, that is, the support of the model, the optimization of the prompt, and the discovery that they can't walk while walking, so today, I think in the Chinese environment, if there is more opportunity, it is better to join a large model company. BecauseToday, the application has not yet been rolled out independently. Although there are articles that say that you can make your own application by adjusting the model, in fact, this era has not yet arrived.
In my opinion,In the next two years, it is more about joining a (large model) company, which can get platform-level support to help you break up and integrate the original experience, so that the probability of success of super applications is much greater.
Today's big model is Think FastAI needs to think slowly
Peng Zhang:Just now you talked about the key factors behind OpenAI Drama that there may be some technology behind it, and even talked about Q*(Q-Star)) may have slow thinking , I don't know if you have paid attention to this matter?
Wang Xiaochuan:This year, I was preparing for the big model (entrepreneurship) and officially established the company in April, and I mentioned a few keywords at that time, one search enhancement , and the second intensive chemical Xi .
At that time, I put forward this point because I had seen that the large model itself represented a way of thinking quickly, and the transformer was like a person, patted on the head and I gave you the answer, and opened the mouth to speak. It has its own shortcomings in learning Xi methods and applied reasoning methods, (so) taking the large model as the origin is definitely not enough.
So at that time, we thought that intensive chemical Xi would be of great help to this.
As far as slow thinking is concerned, I have always paid a lot of attention to this field in my work at Baichuan. Most of the (technical routes) today represent fast thinking, and it requires slow thinking.
If you talk about two points of your own opinion, one is that thinking quickly is not actually called thinking, and thinking slowly is not called thinking. So I put forward a new word, the large model represented by openai, whose knowledge is learned, and does not emphasize how to think when reasoning. Confucius has a saying that "if you learn without thinking, you will be reckless, and if you think without learning, you will die".
So what system is thinking? On the contrary, when OpenAI first started the company, and what Deepmind did (before) - like AlphaGo and playing games, this is in thought. But that is a strong chemical Xi, or even a confrontation of multiple agents. Alphago is not a learning Xi system, it throws away the previous 60 million chess games, but two alphago their own internal confrontation game, in the game to find a new understanding, ultimatum, so that there is thinking.
But after alphago finished thinking about it, it just stopped where it was, just doing a specific task, and couldn't expand it to other areas. Therefore, we say that the large model (LLM) represents learning, and alphago represents thinking, and if these two systems are combined, it will be very powerful.
Peng Zhang:Um, so the next important thing is to really make learning and thinking come together.
Wang Xiaochuan:Let's imagine a scenario, but that doesn't mean how Q* does it.
You ask how the big model Go is played, but it doesn't actually know how to play and can't do it well. But can a large model determine whether Go wins or loses? It can be judged, and the large model can write ** to judge the win or loss of Go. Even if you ask it to write a paragraph, it can be written after each move to judge the state of the game.
So we can imagine that if the large model is strong enough, although it will not play Go directly, it can write the transaction function of playing Go, and finally determine the winner or loss of Go, that is, the large model has the opportunity to write alphago's **, run (** and then it will play chess, this thing is possible.
So when we think about Q*, we have the opportunity to internally conjecture that the big model has the opportunity to produce some framework for thinking, and then think in the traditional way.
Ideally, one step slowerThree quick steps on the ground
Peng Zhang:The technology over there is still exploring the boundaries in front of us, which makes people feel very pressured, and you also make large models, and the pressure is transferred to you, how do you think this distance can be measured? Can we shorten it, or even say that we can create different value by ourselves in the future?
Wang Xiaochuan:I mentioned before that there is a saying that the ideal is one step slower, and the landing is three steps faster, in fact, it was not said at the beginning, at the beginning it was "half a step slower on the ideal, one step faster on the ground", and then went to the United States (Xi study) and came back to fold the ideal in half, and it became a slow step, and multiplied a 3 on the ground, called "three steps faster".
Peng Zhang:How to understand the ideal of one step slower, three steps faster on the ground?
Wang Xiaochuan:After getting in touch with them, I think the bottom of the thinking of both sides is different.
OpenAI is a non-profit organization that wants to explore the boundaries of AGI, and they really do. Therefore, when they are thinking about problems, their starting point is not in the same world at all, and there is a distance from them to fight for their ideals. In this case, people and companies have to find their own niche. But in this soil, we do have to have a self-confidence, that is, we have the opportunity to go faster in the application landing.
Just like Huawei makes GPU processors, it may not have such good high-precision equipment, but that doesn't mean we can't build things, and even (maybe) be able to run faster locally.
Perhaps with the larger scale of our users and larger data accumulation, the application of technology accumulation is high enough, and it can even be (expanded) to the US market. In this case, it does not mean that you have to wait until GPT-4, GPT-5 or GPT-6 to have a chance to be used, and different things can grow in different soils.
I think that doing application is one of the strengths of Chinese tradition, and it is also innovation. On the contrary, I think this is fair, compared to the United States, we will be weaker than them in terms of moving forward ideally, but we will be faster in application, and Chinese companies will face a better opportunity.
Especially in today's United States, where OpenAI is a dominant company, companies that make applications have to face OpenAI, what kind of technology does it do, and what kind of applications can you make. However, in China, it is the model companies themselves that are making applications, and this kind of end-to-end coherence has the opportunity to land applications (faster than those in the United States) in some fields.
Peng Zhang:What you said is quite inspiring, and sometimes we must be very willing to pursue a great thing with a very ideal and a sense of mission. But if AGI is a big process, we can join the team, they may be forwards, we may be liberos or midfielders in breaking boundaries, it makes sense in the team. For example, we take [the technology] down and turn it into something meaningful. So it's what it's like to be part of a team.
Wang Xiaochuan:Both levels can be deduced in this way.
As a citizen of the world, as a Chinese company, you have your division of labor in the world, and this is not a relationship of friend or foe, but only competition. We respect their inventions, and we should catch up, but we can also have our own unique contributions, and I don't think I need myself, [but] the world doesn't need me."
Peng Zhang:It's good, it seems that this wave of entrepreneurship has found a point of reconciliation with ourselves: that is, we have become a team member in a meaningful (AI startup) game in a world, and not everyone has to become a striker.
Baichuan started his business for eight monthsBegin to precipitate the large model side**
Peng Zhang:The last question, about the mentality of starting a business, we all know that everyone was very excited in April, and now it has been 8 months, and the excitement at the beginning is estimated to have been worn out, and it is still difficult to start a business. Today, after a period of precipitation, what is your mentality for this venture? What about the goal?
Wang Xiaochuan:From April to December, the team did run very fast and grew very quickly. Now, I think it's time to start precipitating the big model. Although we feel that the previous experience in technical ability and product concept is sufficient, when we actually work, we feel that it is not light enough.
Sometimes I find that some teams have relatively few resources, but because they have found a way to fit the large model, they can use the existing model more lightly, help them conceive ideas, make prototypes, and then link up with the technology.
At this stage, I think that we are exploring the most effective linkage between the large model and the application in the joint exploration of the large model method, and our understanding is also constantly improving. I thinkA good state is that every time I look at myself a month ago, I feel like a fool. In the past, when I was working, I was iterating at a weekly rate, but now I am not in such an agile state, and I look at my own shortcomings in a monthly state.
In this, our management and product managers work together to adjust their original working methods to obtain the formula of the era of large models.
Peng Zhang:So this is the state that you think makes you enjoy it.
Wang Xiaochuan:Yes, I am making progress every day, and I have multi-dimensional growth, not only saying that I know these things, but also thinking half a step ahead of things. But sometimes, you'll find that when you walk around, you come up with better ideas.
Peng Zhang:What about the company's goals, and what will you find more satisfied with in the next 5 years?
Wang Xiaochuan:In helping people create, health, and be happy, we have super apps to explore in these three aspects. But it doesn't take 5 years, 5 years really don't dare to think, because the height of technological development may not be what we can understand today.
Our technicians all lamented that the new ** and new development every day make everyone have a strong sense of pushing back. In this case, I think that within two years, it can be proved that the large model can indeed be used as a super application, just like the great help and hope brought to mankind in the Internet era, and help everyone experience and use it within two years. This is a belief that can be held.
In 5 years, I imagine that all the gameplay may be completely new, such as robots running on the ground, everyone wearing VR glasses, and everyone's **atar clone has come out. 5 years is too long, and I'm quite satisfied to think about what it will look like in 2 years.
This article was originally from: Geek Park.