Author |Dexin
Edit |Wang Bo
2023 is a pivotal year for Mobileye's transformation.
Its CEO, Professor Amnon Shashua, said when he first visited China after the lockdown was lifted: "We are moving from a company with front-view cameras and chips as our main products to providing system solutions including perception, driving strategies and sensors. 」
To put it simply, Mobileye is moving from basic assisted driving to advanced intelligent driving.
withZEEKR cooperation of high-speed pilot function NZP delivery, representing its first high-end intelligent driving mass production project run through. In the same year, this veteran intelligent driving company also ushered in a lot of follow-up orders.
The question then arises, how to effectively achieve the large-scale expansion of high-end intelligent driving?
The expansion of high-end intelligent driving has two meanings:
how intelligent driving systems can be used on a larger geographical scale, including more road sections, cities and even national regions; and how to achieve mass production on more models and brands. Lower cost, faster speed, and stronger functions are the most mainstream competitive spirit in the current auto market, and the challenges faced by Mobileye also represent the concerns of a considerable number of leading car companies.
At this year's CES, Mobileye CEO Amnon Shashua routinely shared the company's commercial and technical progress over the previous year. In addition to this, Hiev also had an exclusive interview with Prof. Shay Shalev-Shwartz, CTO of Mobileye, during CES.
In this article, you'll see what they have to say about the following topics:
Under the condition of taking into account cost and development efficiency, how to create a differentiated intelligent driving system for multiple car companies? Mobileye's map strategy, the upper limit of the ability of the generalized engine vision of intelligent driving, and the function that cannot be achieved without pure visionWhen will intelligent driving become a key factor in car purchase? Mobileye currently has five levels of intelligent driving products: Basic ADAS, Cloud Enhanced ADAS, Supervision, Chauffeur and Drive.
Among them, DRIVE is an L4 Robotaxi product, the basic ADAS and Cloud Enhanced ADAS are L2 basic assisted driving products, and Supervision and Chauffeur are high-end intelligent driving products. Among the high-end products, supervision is the main force of mass production at present.
Supervision initially had only one customer model in 2022, which was the Zeekr 001. Another year passed, and Mobileye won a number of fixed points, itsThe number of high-end intelligent driving fixed-point models, including supervision and chauffeur, has increased to about 30
By the end of 2023, the mass production of the supervision project is estimated to be 3.65 million units, and the mass production of Chauffeur is estimated to be 600,000 units. These two figures will be shown later, they represent the speed of Mobileye's transformation, and also represent the profitability of the leading intelligent driving company.
Mobileye's existing high-end intelligent driving designation points are mainly from three car enterprise groups:
Geely Holdings, with a total of 5 models, covering 4 brands: Zeekr, Polestar, smart and Volvo; FAW Group, all six models are concentrated in FAW Hongqi, including pure electric and fuel vehicle series, including supervision and chauffeur systems, and the two sets of plans will start mass production at the end of 2024 and 2025 respectively. The degree of radicalism of the red flag can also be glimpsed from the point in time, and the chauffeur that will be mass-produced in 2025 is even the earliest in the existing public project. A Western car giant, with a total of 17 models, covering multiple international brands, involving the United States, Europe and China, including supervision and chauffeur systems, mass production will start in 2026. The giant has also partnered with Mobileye on the Robotaxi business. Combined with this information, it must not be difficult to guess which car company this is. In addition, Porsche and Indian car company Mahindra and Mahindra have also designated supervisionPorsche plans to launch a model equipped with supervision in 2026
Let's go back and say, why supervision is a key battle for Mobileye's transformation?
In the low-end basic assisted driving, Mobileye's share is taking a hit; The revenue from L4 Robotaxi will not start until at least 2027 and 2028; Supervision and Chauffeur have carried the banner of high-end intelligent driving mass production in recent years, and the mass production of Chauffeur will start at the end of 2025 and early 2026, and Supervision is the main business force that cannot shirk.
In 2023, Mobileye's fixed-point projects are estimated to be based on the whole life cycle$7.4 billion
Amnon Shashua reminds that even more noteworthy is the average system**, which is about $50 for ADAS forward cameras, $1,500 for Supervision, $3,000 for Chauffeur, and $50,000 for Drive. 」
Based on an average $1,500 system**, the estimated revenue from 3.65 million supervision systems is 54$7.5 billion. Compared with Mobileye's revenue data in the past two years, this estimate highlights the importance of supervision.
According to Mobileye's earnings forecast, its full-year revenue for 2023 is about 20$800 million, while 2024 revenue is expected to decline from 2023 to 183 - 19.$600 million.
These numbers just reflect that 2024 is the node where Mobileye's green and yellow intersectThe delivery of more than 30 high-end intelligent driving projects represented by supervisionIt is the key to whether Mobileye can turn the tide of battle and transform smoothly.
Most people have little idea about more than 30 projects. This number of high-end intelligent driving vehicles is a challenge that no Tier 1 in the industry has ever tacked. What's more, by 2024, there will be new fixed-point projects entering the R&D channel.
The dependence on R&D resources for the mass production of high-end intelligent driving has made it possible for even a powerful Huawei to serve only three car companies (BAIC, Changan, and GAC) in the HI mode for a long time.
Amnon Shashua recalls that when the team first worked with a client on the EyeQ6 platform, they received a message from the client1.20,000 parameter adjustmentsneeds. This left the team pondering how to handle this 1 for this customer20,000 adjustable parameters? What will other customers ask?20,000 adjustable parameters? In the end, the team found that the auto customer wanted to adjust far more than 120,000.
Mobileye CTO Shai told us that if we want to set up a dedicated team to support systems like Oversight or Chauffeur for OEMs, it would be necessary to have an engineering team of hundreds of people for each OEM.
According to publicly available data, Mobileye had about 3,500 employees worldwide at the end of 2022. In addition to the aforementioned five product lines, Mobileye is also developing its own EyeQ series of chips for intelligent driving. Building an engineering team of hundreds of people for each OEM is obviously not replicable for all smart driving companies.
With the increasing importance of intelligence in the hearts of car buyers, the mentality of OEMs to hope that they are autonomous and controllable is becoming more and more obvious. On the one hand, automakers do hope that their intelligent driving systems will have differentiated performance in terms of product positioning or regional characteristics, and on the other hand, it is difficult for intelligent driving companies to carry out high-investment customized development for each model.
To help OEMs achieve differentiated development, the usual approach is to:
First, the first company provides a basic perception reference algorithm, and the OEM establishes a complete perception, regulation and control part, which can form differentiation, but the possibility of development failure is quite high. In the past two years, there have been many cases of development failure of domestic independent brands.
The reason why it is easy to fail in development is that perception itself is not perfect, especially for consumer-level autonomous driving, and the system cost must be compromised. The regulatory algorithm needs to be closely integrated with the perception layer, if the perception changes, the regulation must be adjusted accordingly, and if the division of labor between the first vendor and the main engine factory in these two places is separated, the difficulty of integration will be greatly increased.
The second method is to let the vendor handle the perception and planning, and only let the OEM customize the control part, the problem with this method is the lack of differentiation and scalability. Either the OEM gets a set of solutions that are no different from other products on the market, or the OEM will have a large number of needs to find the best supplier, and the first supplier must establish a large support team.
Mobileye has refined the core needs of many car companies, and finally launched a setDriving Experience Platform, or DXP for short。Unlike the previous two approaches, DXP's approach is to set the line between perception and regulation.
To put it simply, Mobileye uses the general parts of the intelligent driving system such as target perception, traffic rules, and safety as the basic platform output, and defines the differentiated parts such as the definition of driving style (such as vehicle speed control, lane change style, acceleration and deceleration control) as adjustable interfaces.
Including the perception technology stack, the car is in the **, the pedestrian is in the **, the lane is in the **, etc., this part of the technology does not need to be differentiated. The complex part lies in the cutting of the regulatory algorithm, in the back end of perception, the intention of the pedestrian vehicle, such as a car parked there, whether it is parked, waiting for the light or blocked, these situations are different, but the state of the vehicle is exactly the same, and the same as the deterministic perception target, these uncertain information is also a general part.
On the basis of the previous general parts, the corresponding planning, control and HMI are the areas where each car company wants to differentiate. For example, some manufacturers want a smoother driving experience, while others need a more aggressive style, and Mobileye provides a large number of interfaces and reference examples.
This design significantly reduces the development risk for OEMs, and from day one, OEMs can get a ready-to-use reference design, so there is no risk of delivery. The only risk, Hai tells us, is whether or not all the desired differentiations are successfully implemented. 」
It is not difficult to see from Mobileye's high-end intelligent driving designation that high-end intelligent driving is the first to break out from the Chinese market, and its two earliest mass-produced car companies are from China, while most of the high-end intelligent driving mass production in the European and North American markets will start in 2026. Tesla's FSD, which is unique in the North American market, and China's new power car companies are the leaders of this wave of demand.
Another sign of the explosion of demand for high-end intelligent driving is that Mobileye has updated its EyeQ product line. In 2022, Mobileye released the EyeQ Ultra, with a single-chip computing power of 176 TOPS, and the EyeQ6 chip, with a computing power of 34 TOPS.
Compared with the hundreds of T intelligent driving computing platform solutions commonly found in China, Mobileye advocates the efficiency and cost advantages of dedicated chips, and is very lean in the application of computing power. Internally, it is believed that judging heroes by computing power is like evaluating a company's technical strength by the number of people.
But two years later, Mobileye made an announcement at this CESEyeQ7H chip, computing power 67 TOPS
The update of the EyeQ series also shows that before the arrival of the fully autonomous RoboXi, there is still a lot of room for high-end intelligent driving to dig up, after all, the Chauffeur based on the EyeQ6 has not yet been mass-produced, and the EyeQ7 is obviously used to support a more capable system.
A Mobileye insider told us that the name of the EyeQ Ultra can be understood as the ultimate chip for autonomous driving, when the demand for high-end intelligent driving market was not so strong, and now as the market changes, Mobileye has further expanded its high-end product line.
Mobileye also plans to work with Zeekr to advance the joint initiativeTesting and pushing of urban NOA
Supervision has gone from 0 to 1, L3 autonomous driving is coming, and the commercialization of high-end intelligent driving is still being explored before the dawn.
The following is a conversation between HIEV and Prof. Shay Shalev-Shwartz, CTO of Mobileye.
why dxp？
hiev: You've just launched DXP, why do you think it's the right time to launch DXP?
shaiAt present, functions such as autonomous navigation assistance have been increasingly applied in car companies.
Our mass production work started with the ZEEKR brand, and now we are working with other brands of the Geely Group, as well as many brands in China and outside of China. Once mass production begins, many OEMs want to differentiate their systems, and we need to find better ways to help OEMs solve the problem while balancing the project resources needed for mass production.
DXP is the key approach we found. On the one hand, this allows Mobileye to build a basic and versatile middle platform for OEMs. On the other hand, OEMs can develop and control the intelligent driving system experience that their customers want according to their own product characteristics and customer needs.
hiev: Can you share the current size of the engineering team, for example, to support the mass production of systems such as supervision or chauffeur, and how big the team usually is? This helps to understand the improvements that come with DXP.
shaiIf every car manufacturer needs a dedicated team to support it, we may need a team of hundreds of engineers. And it's not scalable, because it's not possible to have hundreds of engineers working together to solve every problem. But with DXP, dozens of engineers can support an automaker.
hiev: It's very interesting to provide different OEMs with different intelligent driving system experiences. I remember that a few years ago, I interviewed the person in charge of a leading domestic intelligent driving manufacturer about how different the intelligent driving system can be. His point is that because the investment in the development of intelligent driving systems is too large and requires billions of dollars, the differentiation of cars should not be reflected in intelligent driving.
shaiBut from the customer's and OEM's point of view, they want to have a differentiated experience in terms of smart driving.
We also fully understand this need, because people themselves have different driving styles. People who buy different models have different driving styles. Different living regions also have different driving styles, for example, the driving culture in Shanghai is different from other places in China, it is not the same as Israel, and it is not the same as Europe. In the United States, driving habits in New York are not the same as on the West Coast.
hiev: It is said that your collaboration with Zeekr offers 3 different styles, and Zeekr has chosen the most radical style.
shai: Yes, this method is suitable for his car owner, but not necessarily for other customers. We wanted to make the driving experience more three-dimensional, because it's more than just a simple distinction between gentleness, ordinariness and decisiveness.
We provide a number of interfaces to construct driving styles, including speed, lane change methods, speed settings, and more. Customers can say, I don't want to drive too fast, I don't want to speed, but at the same time I want to get there faster.
The engine of intelligent driving generalization: REM
hiev: How do you solve the scale of scenarios in different regions? Mobileye has 1.5 million REM vehicles deployed in the U.S., but the situation in Europe and China is different from that in the U.S. How to solve the driving problem in different regions with a common system?
shai: This is another advantage of REM. Because in different regions, people's driving habits will be subtle differences in many ways.
Our technology relies on crowdsourcing and understanding the driving styles of local communities. I'll give you a simple example that illustrates the problem very well.
In some countries, for example, Germans never drive beyond the legal speed limit, and the speed limit on this section of the road is 100 kph, so according to the data, no one really exceeds 100. But if you're in other places, like Israel, if you drive at 90, you're going to get the car behind you honking your horn. If you want to adapt to the local situation, you have to drive like a local, and that's exactly what REM provides.
REM gives you the opportunity to adjust the style of the system to the driving habits of the locals. And that's just speed, you can learn a lot from a lot of data, and that's the value that we are able to provide to OEMs. They can choose whether to follow the speed limit of the regulations or the speed of the traffic flow to tune the system according to their needs. For example, we have already done this with the ZEEKR project.
Some elements of the map are universal, such as lanes, road signs, and traffic lights.
But in some ways, there are some problems that can be dealt with more calmly by using swarm intelligence. Another example is the right of way, where there are strict rules on the priority of the right of way, and sometimes traffic signs can tell drivers who has priority, but this is not enough, for example, when two roads are merged (without being explicitly informed of the right of way).
In some countries, when you want to merge lines, other traffic will be normal, and you have to wait for the right time to change in; And in the United States, many times, when someone wants to merge into the lane, the car behind will usually make a lane change to allow the car in front to get in.
Under the same rules, there will be different treatments. We can learn from group behaviour how specific problems are dealt with locally.
hiev: You currently have the most REM vehicles deployed in the United States, so how do you solve the problem of generalization in other regions?
shai: There are also a lot of cars in Europe that use our products.
At present, supervision is only mass-produced in China, and we will introduce supervision to Europe and the United States in the future. We believe that supervision-based data will grow very quickly, because in the past we only had forward perception, but now we can also get data from surround perception. Supervision will contribute more to the iteration of our system than vehicles that used to only have a front-looking camera.
hiev: So the surround-view camera will also be involved in the work of REM technology?
shai: It is very helpful for understanding intersections. If there were only forward-looking cameras, it would require someone driving through different directions of the intersection to establish a complete understanding of the intersection. But with surround view cameras, vehicles only need to drive in certain directions.
hiev: So how did Mobileye scale ODD quickly? Just like there are now 22 cities in China that can use your products on highways, car owners hope to be able to use them all over the country as soon as possible.
shai: We are planning to expand ODD nationwide. The current plan is to Q1 technology to support ZEEKR to achieve all high-speed functions. In some cities, there may not be enough vehicle owners to build the data upfront. The basic principle is to prioritize geographic expansion based on the frequency of usage.
In addition to mapping, we also have completely graph-free technologies, such as RSD (Road Segment Data). Even where RSD is not available, we can still provide NOA functionality, but there will be some differences in the complexity of the scenarios that can be handled.
When you have a graph, you can deal with more complex scenarios. For example, in the case of heavy rain, if there is a map, because there is a map to provide redundancy, our system can remain usable; But the situation becomes complicated by the unplanned rainstorm.
We plan to expand extensively in China soon.
Vision does it all
hiev: What do you think about the ceiling of vision-based systems? I think there's a clear line of boundaries between Mobileye's product line in terms of pure vision and other solutions (i.e., starting with Chauffeur and adding lidar).
shai: Our view is that it is possible to drive anywhere with a pure vision solution alone, without the need for other sensors, and there is nothing that pure vision alone cannot solve, but we need to take into account redundancy.
When you want to achieve a higher level of autonomous driving, such as L3, you need lidar to increase redundancy so that the safety performance of the system is high enough to keep the driver's eyes off the road. But for all cases where you don't need to take your eyes off, a camera alone can solve the problem.
Therefore, our philosophy is to solve all problems with pure vision. You can see this in supervision systems that don't have lidar-equipped equipment.
At the moment, it is not yet available in cities in China, but it is about to happen. It's really a matter of maps, and once we have a map, we can drive on any road inside and outside the city. It's not a problem with cameras and lidar.
In our philosophy, lidar is just about adding redundancy. Of course, if you have lidar, you can also improve the comfort of driving. But it's not necessary, you can drive well without lidar.
hiev: The last time Professor Amnon came to China, he mentioned that FMCW is the real right LiDAR route. Why do you think this is the ultimate route?
shai:FMCW has a lot of uniqueness. First of all, general lidar does not perform well in foggy and bad weather, but I think the main problem is that FMCW lidar brings velocity vectors, while ordinary lidar cannot directly measure speed.
Why is this important? One of the biggest challenges for lidar is the segmentation of adjacent objects, such as a pedestrian attached to the side of a car, and it is difficult for lidar to distinguish between two objects. If the two objects are moving at the same speed, that's fine, but if there's a slight velocity difference between them, it's important to know the change in advance.
With FMCW LiDAR, you can immediately obtain this quantitative change, which you can compensate for with vision + machine learning, but it will be overly dependent on machine learning.
Visual machine learning is a great way to do it, but if you need 8 9's (99999999%), you can't just rely on machine learning, your solution needs to be diversified. You need machine learning as well as a model-based approach, and FMCW provides a great complement.
hiev: Announcing FMCW LiDAR that will only be mass-produced in 2028 is now, which is not very typical of Mobileye's approach.
shai: The importance of FMCW lidar will gradually become apparent. Because we are now planning to deploy more and more vehicles based on products such as chauffeur not only at high speeds, but also in urban areas. In this process, the market will gradually mature.
We don't want to launch a solution when the market needs are not clear. Today's market is too early for mass production of FMCW lidar, as the cost of FMCW was too high for Chauffeur of the original L3.
We think that in the next phase, people will recognize FMCW lidar as a very good product and hope that it will become the standard, and that it will work in urban scenarios. The stronger the demand, the more mature the market for FMCW lidar will be.
For resourcing reasons, we want to launch the right product at the right time. We think imaging mmWave radar will come sooner because for OEMs, imaging radar offers a lot of value at a manageable cost.
What is the impact of large models on autonomous driving?
hiev: How do you see the impact of large models on intelligent driving?
shaiIf you're talking about transformers and end-to-end systems, we think the benefits of these technologies are very clear. We're familiar with these technologies because we started working on them before they became massively popular.
For example, two years ago, we already adopted a transformer-like approach internally, and we evaluated it and felt that it was not a very good approach, so we made some improvements, but this did not affect the overall situation. Overall, we think the big model will get you from 0 to 95% very quickly, but these methods need to be 999999999%, there will be a huge problem.
On top of that, the amount of engineering it brings is also a nightmare. If you find an edge scenario where you need to build a separate solution, that's the big problem. Neural networks are not good at learning new information, or forgetting old information.
On the other hand, today's large language models and transformers make a lot of mistakes, and at the level of 0 - 95%, they have a very big leap, but they are not 100% accurate. To reach 99999999% is very painful.
So we don't think that a single solution can solve the problem, we think that it can be an important component of the system, so in our system, there is a similar technology.
What are the benefits of an end-to-end solution? They understand most of the scenarios that take place on the road. Like usually, two cars don't crash together when you observe.
The perception results of these systems, they never show the situation of a vehicle collision. Normally, the wheels are down and the roof is on and does not turn upside down, but there are exceptions.
End-to-end solutions are very difficult to handle atypical situations, but they are excellent in their own right, so I would say that each approach has its pros and cons.
When will smart driving become a key factor in car purchases?
hiev: We've talked to a lot of developers, and some of them think that smart cockpits are becoming more and more interesting because they need more applications and services when the system frees up the driver; And intelligent driving is becoming boring, because the core driving direction is cost reduction and industrialization. What do you think about the subsequent technological development of intelligent driving?
shai: There is indeed a trend to combine intelligent cockpits and advanced driver assistance systems, but I don't think that's a good direction. This combination creates a lot of complications.
I think it would be important to separate safety-related systems from non-safety elements, so that you can give you more flexibility to change the smart cockpit or make changes to other systems. Therefore, the way of cabin and driver separation is more practical than fusion.
I don't think there is a clear answer yet, telling you what the future direction of intelligent driving is, how to provide more features, or reduce **, we may choose to do both.
All of this discussion is based on the fact that we're talking about a system similar to supervision, and I think chauffeur could be a game-changer. When the system evolves to the point where you can get your hands off and your eyes off, it's a different story. The focus of competition is not to focus on ** and functions, but to save users' time, which is equivalent to users spending money to buy their time back, which will be a new way to play.
Once such a system is delivered, whether it is an OEM or a car owner, we will see a lot of changes in the market. If you just stay at the theoretical level, users will think it's a cool feature, but when it becomes a practical feature, you may realize that I can work while driving, or I can watch TV or movies while driving, which is completely different.
hiev:How do you see the pricing model of Smart Driving in the future? If you look at it from the BOM, it may be less than 5%, but if you can charge a subscription fee, the situation is completely different and can reach 20% or even more.
shai: Yes, as cars become smarter, smart driving will become a very important part of the car.
If the intelligent driving of different cars is different, it can affect the decision of which car the user buys.
Nowadays, people pay more attention to safety, but a small number of users will choose a model based on intelligent driving, and this is only a small part of the consumer's purchase decision.
But one day, you may suddenly find that buying a car with strong intelligent driving ability will save a lot of time, such as 80% of the time on the road, while other cars simply can't do this, which will have a very big impact on consumers, because time can also be exchanged for money.