** Wired, an American technology magazine
Edit |Three-Body Problem.
Jensen Huang:We are all graduates of Stanford University.
Lauren Goode:Yes. I majored in journalism, you didn't go to journalism.
Jensen Huang:I wish I could study journalism myself.
Lauren Goode:Why?
Jensen Huang:One person I admire a lot, both as a leader and as an individual, is Shantanu Narayen, who is the CEO of Adobe. He said he always wanted to be a journalist because he loved storytelling.
Lauren Goode:Being able to tell your own story effectively seems to be an important part of starting a business.
Jensen Huang:Yes. Strategy setting is storytelling, and corporate culture construction is also storytelling.
Lauren Goode:You've said many times that you're not pitching Nvidia's idea through pitch decks, a tool used to present a business plan to investors, partners, or clients.
Jensen Huang:That's true, I really want to tell a good story.
Lauren Goode:So I wanted to start with a story that another tech executive told me. He has pointed out that Nvidia was founded a year earlier than Amazon, but in many ways, Nvidia has a stronger "first day" mentality than Amazon. How did you do that?
Jensen Huang:Frankly, that's a very apt statement. I wake up every morning as if I was on my first day of business, because we are always exploring the unknown and trying things that have never been done before. However, this also means that we are at risk and have the possibility of failure. Just now, I was in a meeting where we were discussing a project that was completely new to our company, and we didn't know anything about it and how to implement it successfully.
Lauren Goode:Anything new?
Jensen Huang:We're building a cutting-edge AI factory, a new form of data center. Unlike traditional data centers, where many users share a cluster of computers and store files, AI factories are more like generators. After several years of careful R&D and construction, we have completed the prototype of the AI factory. Now, we are faced with the challenge of turning it into an actual product.
Lauren Goode:What are you going to call it?
Jensen Huang:This yet-to-be-named innovation project is destined to become ubiquitous. Whether it's a cloud service provider or ourselves, we're committed to building it. Biotech companies, retail companies, logistics companies, and even the automotive companies of the future will have it. It will not only be a factory that makes cars, but also a factory that makes artificial intelligence for cars. In fact, as we talk, Elon Musk is already at the forefront of this trend. His deep insight and forward-thinking into the future of industry left most people in the dust.
Lauren Goode:As you mentioned earlier, you manage a flat organization with 30 to 40 supervisors reporting directly to you, which allows you to be deeply involved in the flow of information. So, what's something particular or an idea that has sparked your enthusiasm lately and made you think, "I should probably go all out and bet the future of Nvidia on this"?
Jensen Huang:In the caveman era, the transmission of information did not need to rely on modern means of communication such as e-mail and text messages as it does today, but more freely flowed within the community. Today, with the rapid development of technology, the flow of information is far faster than it used to be, which means that the traditional, layer-by-layer information structure is no longer suitable for today's needs. The flat organizational structure, with its efficient information transfer and decision-making speed, allows us to quickly adapt to this ever-changing world.
Taking NVIDIA's technology development history as an example, the iterative speed of Moore's Law has allowed us to witness the exponential growth of technology. In the last decade alone, the capabilities of AI have increased by a factor of about a million, which is far beyond Moore's Law. Therefore, in such a fast-moving era, the flow of information must be equally efficient and swift, ensuring that employees at every level can access and respond to new information in a timely manner.
Lauren Goode:However, I am eager to learn more about what kind of grand picture you envision the Roman Empire. What are some of the exciting visions depicted in today's Transformer (T in ChatGPT, an algorithmic architecture for the underlying AI learning)? Do you think that the historic changes that are taking place today have the potential to upend all the perceptions and assumptions we are familiar with?
Jensen Huang:There are several transactions that are worth it. One of them has not yet been officially named, but it is a breakthrough in the field of basic robotics. If machines can generate text and draw images, can they also generate actions? The answer seems yes. Once machines have a grasp of how actions are generated, they may be able to gain insight into the intent behind them and create generic expressions. In this way, the arrival of humanoid robots may be just around the corner.
I am convinced that the study of state-space models (SSMs) will enable us to learn extremely complex patterns and sequences without the burden of quadratic growth in the computational process. This could be the prototype of the next generation of converters.
Lauren Goode:What does this bring? Can you give a real-life example?
Jensen Huang:You can have a conversation with a computer that lasts a long time, but the context is never forgotten. You can even change the theme temporarily and then go back to the previous topic, and you can keep the context of that context. You may be able to understand the sequence of a very long strand, such as the human genome. Just look at the genetic code and you can understand what it means.
Lauren Goode:How far are we from such a future?
Jensen Huang:It only took five years from the birth of Alexnet to the rise of its remarkable version, AlexNet. Today, robotic base models are emerging to signal a technological revolution on the horizon. I expect that maybe sometime next year, we'll launch it. And five years later, you'll be witnessing a series of jaw-dropping miracles.
Lauren Goode:Which industry would benefit the most from a widely trained model of robot behavior?
Jensen Huang:Heavy industry undoubtedly occupies a central position in the global industrial sector. Although it has become relatively easy to move electrons in modern technology, moving atoms is extremely challenging. Transportation and logistics, as an important pillar of the global economy, involves moving heavy objects from one place to another efficiently and safely, which requires a deep understanding of microscopic elements such as atoms, molecules, and proteins. These are all large industries that AI has not yet impacted.
Lauren Goode:You mentioned Moore's Law, is it becoming irrelevant now?
Jensen Huang:Moore's Law has evolved into a systemic problem that encompasses a wide range of fields, not just chips. It focuses more on connectivity and collaboration between multiple chips. About 10 to 15 years ago, we embarked on the journey of computer disaggregation, which allowed multiple chips to work together seamlessly.
Lauren Goode:That's why you acquired the Israeli company Mellanox in 2019. Nvidia said at the time that modern computing places huge demands on data centers, and Mellanox's networking technology will make accelerated computing more efficient.
Jensen Huang:That's right. We acquired Mellanox to expand our chips and turn the entire data center into a single superchip, which makes modern AI supercomputers possible. In recent years, people have begun to recognize the limitations of Moore's Law and realize that if they want to scale computing power further, they will have to do it on a larger data center scale. We delved into the mechanism by which Moore's Law was formed, and found that it didn't really limit the development of computers. Therefore, we must break free from Moore's Law and think in a new way about how to achieve the continuous expansion of computing power.
Lauren Goode:Mellanox is now considered a very smart acquisition move by Nvidia. Recently, you tried to acquire ARM, the world's foremost chip intellectual property company, but were blocked by regulators. What are some of the specific aspects that you consider when considering an acquisition now?
Jensen Huang:Indeed, the design of operating systems for large systems faces unprecedented complexity, especially when tens of millions, hundreds of millions, or even billions of tiny processors need to be reconciled. We are more than willing to work with any team with research in this area to push the boundaries of operating system technology. At the same time, we will continue to increase R&D investment and explore more possible solutions.
Lauren Goode:You've said that it's important for Nvidia to have an operating system and make it a platform.
Jensen Huang:We are a platform company.
Lauren Goode:Being a comprehensive platform does mean taking on more responsibility, especially when it comes to key areas such as autonomous vehicles, medical devices, and AI systems. How well self-driving cars are performing, what is the margin of error for medical devices, and whether AI systems are biased. How do you solve this problem?
Jensen Huang:We're not a pure application company, we're committed to serving a specific industry. In healthcare, although drug discovery is not our specialty, we have excellent expertise in computer technology. Similarly, while we don't manufacture cars directly, we have a significant advantage in providing the automotive industry with advanced computer solutions that enable vehicles to excel in artificial intelligence. Admittedly, it's almost impossible for a company to excel in all areas, but we strongly believe that we can do our best in the field of AI computing.
Lauren Goode:Last year there were reports that some customers waited months to get their hands on your AI GPUs. What's going on now?
Jensen Huang:I think we're going to continue to be limited this year. This year, and even next, it will not be able to meet the demand.
Lauren Goode: What is the wait time now?
Jensen Huang:I don't know what the current delivery time is. But, as you know, this year is also the beginning of our new generation of processors.
Lauren Goode:Do you mean Blackwell, the rumored new graphics processor?
Jensen Huang:Yes, a new generation of GPUs has arrived, and Blackwell has broken records for performance. It's going to be incredible.
Lauren Goode:Does this mean that customers need less GPU?
Jensen Huang:That's what we're aiming for. We want to dramatically reduce the cost of training large models, and then people can scale up the models they want to train.
Lauren Goode:Nvidia has invested in a lot of AI startups. Last year it was reported that you invested in more than 30 companies, were those startups being pushed out of the queue while waiting in line for your hardware?
Jensen Huang:They are facing the same challenges as many enterprises, and since most companies rely on public cloud services, they have to negotiate directly with public cloud service providers. However, they can take advantage of our AI technology, which means they can use our engineering capabilities and special technologies to optimize their AI models. We're committed to helping them be more efficient, and if they get a five-fold increase in throughput, it's essentially an additional five GPUs of computing power. That's exactly what they can actually gain by choosing us.
Lauren Goode:Do you consider yourself a kingmaker in this regard?
Jensen Huang:No. We choose to invest in these companies because of the impressive work they do. It's a privilege for us to be able to financially support them, not for them to rely on us. These companies bring together the best talent in the world, and they don't need to rely on our name to enhance their credibility.
Lauren Goode:What happens when machine learning shifts more to inference than training? If AI work becomes less computationally intensive, will this reduce the need for GPUs?
Jensen Huang:We are passionate about expanding our inference business. If I had to make a rough estimate, I'd believe that NVIDIA's current focus is 70% on reasoning, with the remaining 30% focused on training. This shift is undoubtedly cause for rejoicing, as it represents a quantum leap forward in AI technology. If NVIDIA's business is still dominated by 90% of training and only 10% of inference, then we can assert that AI is still in the research stage. This is exactly the situation in the field of artificial intelligence seven or eight years ago. However, today, when we input a command in the cloud, it can quickly generate all kinds of content - whether it is **, images, 2D, 3D graphics, or text, which often hides the strong support of NVIDIA GPUs.
Lauren Goode:Do you think at some point, the market demand for your AI GPUs will decrease?
Jensen Huang:I firmly believe that we are at the beginning of the generative AI revolution. In today's world, most computing tasks still rely on traditional retrieval methods. Retrieval, which means that when you tap the screen of your phone, it will send a signal to the cloud seeking a piece of information to be retrieved. The cloud may use various technologies, such as J**A, to combine this fragmented information into a single response that will eventually be presented on your phone's screen. However, future computing will increasingly rely on RAG – Retrieval Enhanced Generation. It's a revolutionary framework that gives large language models the ability to extract data beyond their regular parameters. In this new computing paradigm, the retrieval part will gradually decrease, while the personalized and generated parts will dominate. These generation tasks will undoubtedly be completed by efficient GPUs all over the world. So, I think we're at the beginning of a revolution in retrieval augmentation, generative computing, where generative AI is going to be an integral part of almost everything.
Lauren Goode:The latest news is that you've been working with the U.S. to develop chips that comply with the export ban and ship them to China. My understanding is that these are not the most advanced chips. To ensure that you can continue to do business in China, how closely do you work with **?
Jensen Huang:The U.S. has determined that Nvidia's technology and this AI computing infrastructure are strategically important to the country and will impose export controls on it. We have been in compliance with export controls since the beginning (August 2022). The U.S. added more export control provisions in 2023, which led to us having to redesign our products. We are developing a new suite of products that comply with today's export control regulations. We work closely with ** to ensure that the recommendations we make are in line with their ideas.
Lauren Goode:Are you concerned that these restrictions will spur China to launch more competitive AI chips?
Jensen Huang:China has a lot of very competitive things.
Lauren Goode:Truly. Huawei's Mate 60 smartphone, launched last year, has garnered a lot of attention for its self-developed 7nm chip.
Jensen Huang:Huawei is a very good company. Although they are limited by existing semiconductor processing technologies, they can still build very powerful systems by bringing together many chips.
Lauren Goode:In general, are you worried that China will be able to compete with the United States in the field of biointelligence?
Jensen Huang:The implementation of this regulation will undoubtedly impose restrictions on China's ability to acquire state-of-the-art technology. This means that on the technological track, those Western countries that are not subject to export controls will likely gain more advantages, thereby accelerating their progress in the field of science and technology. For China, this undoubtedly increases the cost and difficulty of obtaining its technology. Technically, China may be able to compensate for this by aggregating more chip-making systems, but such a solution would undoubtedly increase unit costs and increase production complexity.
Lauren Goode:You're producing chips that meet the standards so that you can continue to sell them in China, has this affected your relationship with TSMC?
Jensen Huang:No. The rules are specific, and it's no different from the speed limit on the road.
Lauren Goode:You've said many times that out of 35,000 parts of your supercomputer, 8 come from TSMC. When I heard this, I thought it must be a small part. Are you downplaying your dependence on TSMC?
Jensen Huang:No, not at all! I want to emphasize that building an AI supercomputer is a comprehensive project that involves the integration of many components and technologies. In fact, almost the entire semiconductor industry has worked with us on the AI supercomputer projects we're working on. We have established strong relationships with industry leaders such as Samsung, SK hynix, Intel, AMD, Broadcom, Marvell, and more. This cooperation model not only promotes the sharing of resources and the exchange of technology, but also lays a solid foundation for our common success. When our AI supercomputer achieves a breakthrough and success, it also means that the large group of companies we work closely with will also have great success together. We're happy about that.
Lauren Goode:How often do you talk to TSMC's Zhongmou Chang (founder) or Deyin Liu (CEO)?
Jensen Huang:We are always in touch and have never been interrupted.
Lauren Goode:What is your conversation about?
Jensen Huang:These days, we've taken a deep dive into advanced packaging technologies and looked ahead to where storage capacity and computing power would evolve in the next few years. Among them, CODOS technology, TSMC's unique innovative approach to integrating chips and memory modules in a single package, has become the focus of our attention. However, to achieve large-scale production of this technology, we need new factories, production lines and corresponding advanced equipment. Therefore, it is important to have the support of our partners.
Lauren Goode:I recently spoke with the CEO of a company focused on R&D generative AI technology. I asked Nvidia who its future competitors would be, and this person was talking about Google's TPU. Others also mentioned AMD. I guess it's not that simple for you, but who do you think is your biggest competitor? Who keeps you up at night?
Jensen Huang:Today, the entire tech industry is scrambling to develop and optimize its own chip technology, whether it's the TPU team, the AWS Trainium and Interentia teams, Microsoft's MAIA project, and China's major cloud service providers and startups. It's a lot of competition, but for me, it's not going to keep me up at night.
I know that as long as our team gives our best in our work and is always efficient and innovative, we are able to cope with the fierce competition in the outside world. It's something I can control, and that's what I've always believed in. However, what keeps me motivated to wake up every morning is that we must continue to deliver on our commitments. That is, we want to be the only company in the world that can attract everyone to work together to build a data center-scale and full-stack AI supercomputer.
Lauren Goode:I have a few personal questions for you. I once asked ChatGPT a question about you. I want to know if you have a tattoo because I plan to give you a tattoo when we meet next time.
Jensen Huang:If you get a tattoo, I get a tattoo too.
Lauren Goode:I already have one, but I've always wanted to get more tattoos.
Jensen Huang:I also have one.
Lauren Goode:I've learned from chatgpt. According to it, when Nvidia's stock price rose to $100, Huang tattooed the company's logo. It then went on to read: "However, Huang said he was unlikely to get another tattoo, noting that the pain of getting a tattoo was more intense than he thought." "It said you were crying at the time. Are you really crying?
Jensen Huang:It hurts a little. My advice is that you should drink a glass of whiskey before getting a tattoo, or take painkillers. I also think that women can take more pain because my daughter has a rather large tattoo.
Lauren Goode:If you want to get a tattoo, I think the triangle might be good. Who doesn't love triangles? They are perfect geometric shapes.
Jensen Huang:Or tattoo the silhouette of the Nvidia building! It is also made up of triangles.
Lauren Goode:I wonder, how often do you personally use tools like ChatGPT or Bard?
Jensen Huang:I've been using Perplexity and I love ChatGPT too. I use them almost every day.
Lauren Goode:What is it used for?
Jensen Huang:Study. For example, computer-aided drug discovery. Maybe you're wondering about the latest advances in computer-aided drug discovery. So you want to build the whole theme so you can have a framework. From this framework, you can ask more specific questions. I really like these large language models.
Lauren Goode:I've heard that you've done weightlifting before, are you still doing it?
Jensen Huang:No. I would try to do 40 push-ups a day, which would only take a few minutes. I'm a lazy exerciser. I do squats when I brush my teeth.
Lauren Goode:Recently your comments on the acquired podcast have gone viral. At that time, the host asked you: If you were 30 years old now and you wanted to start a company, what would you do? You say you're not going to start Nvidia at all. Do you have any new thoughts on this?
Jensen Huang:This question can be answered in two ways. And my answer was: If I had known all the things I knew now, I would probably have been too scared to do it.
Lauren Goode:You have to be a little paranoid to start a business.
Jensen Huang:That's the benefit of ignorance. You don't know how difficult it will be in the future, you don't know how much pain and suffering there will be. Nowadays, when I meet entrepreneurs and they tell me how easy it is to start a business, I support them very much. I'm not trying to shatter their illusions. But I knew, deep down, I thought, "Oh my gosh, it's not going to be as simple as they thought it would." ”
Lauren Goode:What do you think is the biggest sacrifice you've made while running NVIDIA?
Jensen Huang:Behind success is often a myriad of sacrifices and hard work. For entrepreneurs, this road is full of challenges and hardships. Working hard for long hours, facing doubts and denials from the outside world, insecurity, fragility, and even sometimes humiliation are all realities that entrepreneurs have to face. CEOs, entrepreneurs, and others are just as human as everybody. It's embarrassing when they fail in public.
So when someone says, "Lao Huang, you already have so many things today, you won't start from scratch again". But if I had known then that Nvidia would be what it is today, would I still start this company? Are you kidding? I sacrificed almost everything for this!
Disclaimer. We respect the hard work of every original creator, and **indicate the author of the article**. If you have any copyright issues, please contact us. We will deal with it as soon as possible.