In July of this year, we made a review of:AI Security & RegulationThere was a big debate.
In the program, Deng Jiawen, a postdoctoral fellow in the COAI research group of Tsinghua University under the guidance of Huang Minlie, represents the positive side - agrees with the AI threat theory;Gan Ruyi, head of the IDEA Research Institute and the open source team of the Fengshenbang large model, represents the opposing side - opposing the AI threat theory.
The two sides agreed on whether the AI has "criminal capacity";There will be a debate on three levels: whether there will be a "criminal motive" and whether it is necessary to restrict the use of AI. Overall, the main point of disagreement between the pros and cons is the time dimension, i.e., the AI threat faction is more from a long-term perspective, while the AI optimists are more from a short-term perspective.
So, at the end of the show, there is a problem left to solve:In the distant future, will AI threaten the survival of humanity or even extinct it?
Sorting out the views of previous AI optimists, Yann Lecun believes that the development of AI is far from posing a threat to mankind, and puts forward the classic metaphor of "no car makes any brakes";Andrew Ng argues that exaggerating AI risks could limit open source and innovation.
In fact, the threat faction is even more powerful than the lone optimists. For example, in May this year, a joint letter "calling on all AI labs to immediately suspend work for six months";In November this year, the first Global AI Security Summit was held for China, the United Kingdom, and the United States to sit down and talk.
News from far and wide seems to tell us that the tension brought about by artificial intelligence is comparable to nuclear proliferation. But who still remembers, it's only been a year since ChatGPT came into the public eye.
Such a strong sense of disconnection is also puzzling – is there any other explanation for the AI threat theory?
In a recent interview with the Financial Times, Andrew Ng shared his views on the development of AI.
First of all, Ng admits that the current LLMS already have the rudiments of a "world model" that can understand how the world works.
But at the same time, Ng argues that we can't prove whether AI will cause humanity to perish, just as we can't prove whether the radio waves emitted by Earth will attract aliens to wipe out humanity. The probability of AI destroying humanity is so small that we shouldn't waste a lot of resources just to guard against such a tiny danger.
This is reminiscent of the previous conversation between Hinton and Lecun.
Hinton: If AI is not strictly regulated, what is the probability that it will become extinct in the next 30 years?
Lecun: Actually, AI might save humanity from extinction, what do you estimate to be the odds?
From this point of view, many more people see only the "threat" and not the "rescue".
Eventually, the AI threat theory has become a philosophical question – will the world be a better place for a long time because there is more "intelligence"?
Ng's view is: Yes.
However, Ng also affirmed the need for regulation. For example, "watermarking" AI-generated contentPut some restrictions on the tech giants;Pay attention to the supervision of high-risk applications such as finance and medicine. In October this year, the White House signed the first executive order on the regulation of artificial intelligence, which will lead to the development of artificial intelligence on the path of stifling innovation.
In addition, for China's "lack of core" dilemma, Ng Anda's answer is: it will have an important impact, but it remains to be seen how everything will develop in the future. And he did see that Chinese companies have made a lot of innovations in LLMS inference with low-power chips without advanced NVIDIA and AMD chips.
The following is the content of the interview.
01 LLMS has the prototype of a "world model".
When Financial Times reporter Ryan McMorrow visited Andrew Ng's office in Palo Alto, Silicon Valley, he pulled out a laptop without internet connection and demonstrated a large open-source model from French artificial intelligence startup Mistral. Running the model can be cloud-independent. "The model is stored on my hard drive, running through the GPU and CPU on my laptop," ng said.
When you ask the model "what AI questions should journalists interview Andrew Ng?" it provides background information about Ng and his work, which is not far from ChatGPT's response.
rm：Under what circumstances would you use these open source models?
an：I have multiple models running on my laptop — including Mistral, Llama, and Zefa, and I use ChatGPT a lot. But for those sensitive information that I don't want to upload to the cloud, I prefer to use them on my laptop. For example, when it comes to the idea of a highly confidential project, or the document I want to write with sensitive financial data, I choose to use the open source model. In fact, the open source model is already performing quite well right now.
rm：Many tech companies are in dire need of NVIDIA chips to run artificial intelligence. If the mistral model on your laptop is up to the task, then why would they (tech companies) go to the trouble of getting these chips?
an：Mistral is a smaller language model: with only 7 billion parameters, it can't compete with GPT-4 for complex inference tasks. But for simple ideation and basic facts, mistral is sufficient. As you can see, it's very responsive at times.
However, it is simply impossible to train a model from scratch on my laptop – it would cost tens of millions of dollars, be computationally intensive to train, and do inference on very large models, which is beyond what I could do in a laptop.
Actually, I also tried to run a 70 billion parameter model on my laptop, but it was just too slow. If you take a 175 billion parameter model (GPT-3 scale) on my notebook it won't work either. Inference on large models still requires data center-level resources. Now open-source software has become simple enough that most people can easily install and use.
I'm not obsessed with regulatory issues – but if some regulators get their way, it's harder to keep the open source model up to date.
rm：How can regulation disrupt open source models?
an：For example, some proposals require LLMS to get a report or even a certificate of permission. Big tech companies have the ability to handle these complex compliance requirements, but smaller businesses don't.
For example, when a medium-sized company wants to release an open source model, the in-house counsel will say, "If you do this, there may be various legal risks". So in this case, I think there will be fewer companies willing to take that risk. Regardless of what is being regulated, we will see a decrease in the number of models.
An open-source model is a general-purpose technology: it can be used to build medical applications, customer service applications, financial services applications, and more. So if you're policing the core technology of the open source model, you're actually slowing everything down, and you're probably not substantially improving security.
rm：When discussing the regulatory framework, it is important to understand the current capabilities of AI. In June of this year, you had a conversation with Hinton, and he warned about the dangers of artificial intelligence. You talked about whether AI models really understand the world, and at the time you didn't seem to fully believe that they could do it. And now?What do you think?
an：I think AI models do have this capability. The problem is that there is no clear definition of terms like "understands", or "conscious" or "sentient". As a result, we don't have a universally accepted standard for testing when something actually understands something, rather than just acting as if it does.
But based on the scientific evidence I've seen, AI models are really building their worldview. So, if an AI has a "world model", I tend to think that it understands the world — based on my personal interpretation of the word "understands."
rm：What do you mean by a "world model"?
an：If you have a "world model", you have an understanding of how the world works, and you can develop in different situations in the future. Scientific evidence suggests that LLMS trained on large amounts of data do indeed build a model of the world.
The researchers had an LLM simulate the next move of the Othello board game — C4, D5, B3, and so on. After that, they asked "did the LLM learn the layout of the board, did they learn the rules of the Othello game?" The researchers found that the neural network actually seems to have built a chessboard model inside the neural network that can play chess. Based on this and other similar experiments, I believe that LLMS does build some kind of world model. I can honestly say that they do understand the world.
rm：Do you think LLMS are "conscious"?
an：I might avoid the question of "conscious." Because for me, it's more of a philosophical question than a scientific one. Philosophers say that we assume that others are conscious. I think that's a polite way to put it. Because how do you know I'm conscious?Maybe I'm just a zombie who looks conscious. Therefore, I think that consciousness is untestable, which is why it is a philosophical question rather than a scientific one.
rm："Conscious" aside, do you think LLMs are capable of "thinking for themselves"?
an：I don't know exactly what the term "think for yourself" means. I'm inclined to say yes, but the lack of a clear definition of "think" makes this argument hardly tenable. In other words, can the relay switch in the ceiling lamp "think for itself"?I'm inclined to think that it can, but I think it's hard to justify this in a rigorous way.
rm：When does an LLMS be able to "understand"?
an："Understanding" is achieved gradually, not overnight. For example, as LLMS evolved, we had GPT-2, GPT-3, ChatGPT, and I felt like their level of understanding was constantly improving — until I was fairly confident that LLMS did "understand" the world in some way.
02 There is a risk but the probability is small, the theory of AI extinction and the theory of alien extinction
rm：If everyone agrees that LLMS are capable of "understanding", then the debate about AI seems to boil down to optimists like you, who focus more on what the technology is capable of. Doomsayers, on the other hand, are more likely to witness the impact of exponential advances in artificial intelligence on the future. Do you think the doomists' reasoning is reasonable?
AN: I don't agree with this description because many optimists are also looking ahead to a better world that AI will build in a few decades. In fact, when it comes to the topic of "AI will exterminate humanity", the concerns of people I talk to seem very vague about it, and no one can say exactly how "AI will lead to the demise of all mankind".
I can't prove that AI won't cause humanity to perish, just as I can't prove that the radio waves emitted by Earth will attract aliens and wipe out humanity. But I'm not going to worry too much about radio waves causing the extinction of the human race, just as I don't see artificial intelligence causing the extinction of the human race.
rm：However, there are some respected scientists who believe that there is a risk of extinction caused by artificial intelligence. What I want to ask is, how can we, as human beings, ensure that the development of artificial intelligence does not lead to the extinction of humanity?
an：The possibility of radio signals attracting aliens and exterminating all of humanity is non-zero, but the odds are so small that we shouldn't waste a lot of resources to guard against such a tiny danger. What I'm seeing is that we're investing a disproportionately large amount of resources in the face of almost zero risk.
rm：So from a regulatory standpoint, what measures, if any, do we need?
an：We need good regulation. For example, when we build critical applications with AI, it is absolutely essential to ensure that they are inherently secure, and to protect the regulation of consumers. But what I'm seeing is a lot of bad regulation, and we don't need more of that.
rm：In simple terms, what is "good regulation" and "bad regulation"?
an：If someone is building applications for healthcare, financial accounting, or self-driving cars, we want them to be safe and unbiased. Taking a layered approach to risk management – thinking about the actual risks of the application and policing the potential for undesirable outcomes – would be "good governance".
But to me, claims like LLMS have "systemic risk" don't make any sense. For example, some ** only claim that LLMS is more risky, but people can also use other small or large language models to build potentially dangerous medical devices. Similarly, both small and large models can be used to spread disinformation.
Therefore, the size of the model is not a valid measure of risk. A more reasonable measure would be: What is the essence of the application?Medical applications, for example, are inherently riskier. Another measure is the reach of the application. If a social media platform has hundreds of millions of users, then the risk of spreading false information is obviously higher than that of a small forum with only 100 users. As a result, we should impose stricter regulations on big tech companies.
Such an approach is not without precedent. For example, the U.S. Department of Labor's OSHA-Work Safety and Health Regulations require more from large employers than from smaller employers, and this balance between worker protection and the burden on small businesses is worth emulating.
03 The "voluntary commitments" of the Open AIs are in vain
rm：Last October, the White House issued an executive order aimed at strengthening the regulation of artificial intelligence**. Is this going too far?
an：I think we may be going in the wrong direction. If we can establish in the constitution – the limits on the development of AI technology that stop here – will not go any further. I might be okay with that. But the current situation is that various ** departments are assigned to imagine more obstacles to the development of artificial intelligence. I think this will lead us down the path of stifling innovation and creating very anti-competitive regulation.
rm：All we know so far is the outline, but we don't know what will happen nextFor example, if a company develops any underlying model that may pose a risk to ***, it should notify the federation and must share all red team security test results. We don't know the underlying model described above, the specific criteria for defining the information shared.
an：Now a lot of lobbyists are busy shaping their positions. The White House Executive Order's initial mandatory notice threshold is required to calculate, and I think that's a very simple way to measure the risk of a model. (Note: The Executive Order requires that if a new AI model exceeds a certain threshold, the company developing the model is required to notify the federal** when training the system.)
We know that today's supercomputers may be tomorrow's smartwatches. As startups scale up and more computing resources become generally available, we'll see more and more organizations hit this threshold. Setting computing resources as a threshold is like saying that a device that uses more than 50 watts of electricity has a greater systemic danger than a device that uses 10 watts:While this may be true, it's a very simple and crude measure of risk.
rm：If we don't use the amount of computation as a threshold, then what is a better way to measure risk?
an：When we look at applications, we can understand exactly what means safety and what means danger, and make appropriate regulation accordingly. Because technology is so versatile, if the technology is regulated, it will only slow down the pace of technological progress.
The fundamental question is: Do we think the world will be a better place because of the increase in "wisdom"?"Intelligence" includes "human intelligence" and "artificial intelligence".
There is no doubt that "wisdom" can be used for undesirable purposes. But over the centuries, society has progressed as humans have become smarter and education has become more universal. I believe that if the world had more "wisdom", whether human or artificial, it would help us solve problems better. So putting up regulatory barriers just because "wisdom" might be used for some undesirable purpose is, I think, going society backwards.
rm：How do we ensure that "wisdom" is not used for undesirable purposes?
an：Of course, the undesirable uses of "wisdom" should be identified and guarded against. If we think about the idea of AI exterminationism, we will find that the scenario it describes is so vague and exaggerated, I don't think it's realistic and difficult to prevent.
But there are risks that we don't want to be biased in the application of insurance accounting, so we can have rules for auditing the application of insurance accounting and assessing the fairness of the application – which would be a positive change. As for social** or chatbot companies that affect a large number of users, there is a risk of misinformation or bias. At this point, I think transparency makes sense.
All in all, if we regulate high-impact, higher-risk businesses, we shouldn't also hinder the growth of small start-ups, which deprives them of the opportunity to grow into large enterprises.
rm：AI-generated fakes and other content may proliferate, should it be managed?
an：This is a tricky question. I thinkWatermarkProbably a good idea. In July, companies including Google, Meta and OpenAI pledged to the White House to adhere to three basic principles when developing AI: safety, security and trust. But if you read these promises carefully, I think everything is empty words other than "watermarking". Because the company can completely go against the law.
To guard against the risk of spreading disinformation on a large scale through ** or texts – especially texts, I thinkWatermarkis a measure that should really be considered. Unfortunately, since the aforementioned White House promise, I've seen some companies regress in watermarking textual content. So, I think the use of voluntary commitments as a regulatory tool is a failure.
04 Bad regulation is better than no regulation
rm：We may feel the louder voices calling for stricter regulation. So, can you feel like you're alone in your position?
an：Actually, I'd very much like to be able to get out there and come up with some good rules, rather than the bad regulatory proposals that we're seeing right now. So, I'm not advocating letting it go. But between bad regulation and no regulation, I'd rather have no regulation. However, some powerful forces, including giant tech companies, have exaggerated the risks of AI. Admittedly, large companies want to avoid competition with open-source AI. Their strategy is often to exaggerate fears and then implement rules to slow down innovation and open source.
What I showed on my laptop proves that there is a lot of competition for open source software and some proprietary models - although they may not be the best, they are good enough to be compared to some of the earlier versions. And for some companies, if there is no competition from open source software, it will be unimpeded.
rm：Can you point to some of the companies that are pushing the AI threat discourse?
an：You know who they are. Several companies are emphasizing the threat narrative. It's a motivator for larger enterprises that prefer to avoid competing with open source;It's also uplifting for some nonprofits to exaggerate fears, create illusions, and then raise money to fight the illusions they themselves create;Some individuals can exaggerate the fear of getting more attention and higher speaking fees. Still, I think there are some people who are sincere – they may be misguided, but they are genuine – but on top of that, there is a motive to exaggerate fear for some benefit.
I don't think I'm the only one who thinks so. Bill Gurley (Partner at Benchmark) thinks deeply about regulatory capture. One of his lectures on YouTube was brilliant and accurately encapsulated a lot of what's happening right nowRegulatory captureLet's go. Yann Lecun also commented on this issue. I think there are actually a lot of people who have very thoughtful views on this. (Appropriate: Regulatory Capture - Regulators are "captured" by the supervised objects in various ways and become tools of a few interest groups.) ）
rm：It's just that it seems that the other side is louder.
an：Frankly, you guys are **, so you can amplify your voice. Bloody headlines are always more popular, and so is fear. When a lot of people sign a statement saying that AI is as dangerous as nuclear, a lot of attention is given. And when more rational statements emerge – for example, Mozilla says open source is a good way to keep AI safe – there is little coverage.
I think the measures of the Center for Artificial Intelligence Security (CAIS) are very poor. Because many regulators don't understand AI, when the statement that equates AI to nuclear ** spreads misinformation very clearly, it distorts the thinking of regulators. I have already seen the impact of this statement on the United States**. However, it is absurd to compare AI to nuclear **. Artificial intelligence can help us make better decisions;And nuclear ** can destroy cities. What can be the connection between the two?
Today, more countries are aware of the risks of regulatory capture, as many of the top talent in generative AI is currently concentrated in the United States. If rules are introduced that hinder the spread of open source models, guess who will be left behind?Almost all countries except the United States.
Of course, some of the forces in the United States also want to see this situation, because they believe that American competitors are benefiting from open source software. No one wants to see AI used in an unjust war, but the cost of slowing down global innovation and degrading the quality of intelligence and decision-making around the world is too high.
I hope that regulators in Europe will also be aware of this. Frankly, if we slow down the pace of open source, then who will be left behind?
05 At the software level, all countries are equally restricted
rm：After reading Kai-Fu Lee's book AI Superpowers (2018), I was convinced that China would lead the wave of AI development. But that doesn't seem to be happening. What do you think is the reason?
an：Kai-Fu Lee discussed the advantages of data acquisition in China, but data is a very vertical thing. Google, for example, has a lot of web search data, but it doesn't have a direct impact on logistics, smartphone manufacturing, or drug development.
Data that different countries have in different verticals can create their own advantages. China temporarily leads the U.S. in surveillance technology, thanks to the rise of digital payments in China. But I think the U.S. has a lot of data and an advantage in areas where it is strong, such as web search, drug development, pharmaceuticals, and so on.
Technological development is phased, and China has only missed the starting point of the generative AI wave. Much of the early, and even now, generative AI work was done by two teams: my former team, Google Brain and OpenAI, both of which are in Silicon Valley. Sometimes these team members leave and start anew, which is why generative AI talent is now in Silicon Valley. Although there are also talent clusters in the UK, Canada, and China, the concentration is actually far less than that of Silicon Valley. Even Seattle and New York are not as good as Silicon Valley.
rm：China's regulatory approach to AI is quite aggressive – any company that wants to publicly release an LLM must be approved, and the authorities must review the datasets used in their training. Do you think this will stifle innovation?
an：I'm not an expert on this. I think to a large extent, how exactly a stipulation is implemented affects whether it stifles innovation or not.
rm：Recently, both Tencent and Alibaba have expressed their limitations in AI development due to the lack of NVIDIA chips. What do you think of the American approach?
an：U.S. export controls on chips have undoubtedly had a substantial impact on China's AI development. But I've noticed that China has made a lot of innovation in LLMS inference with low-power chips without advanced NVIDIA and AMD chips. At least for now, there are enough loopholes in export controls, and I think Chinese companies are also trying to find overseas computing resources. It remains to be seen how everything will play out in the future.
rm：Do you think the U.S. should do that – basically trying to weaken China's AI industry?
an：I have somewhat mixed feelings about this issue, I'm not an expert on geopolitics. But it's clear that the U.S. is trying to regulate the underlying model — and there is a faction that influences the White House's executive order because they fear that U.S. adversaries will exploit open source software. So, at the software level, I think the U.S. itself is subject to as many restrictions as other countries, and that's a mistake. As for the hardware, I don't know much.
rm：It sounds like you think Chinese companies can find ways to get around chip restrictions.
AN: yes, I've seen a lot of innovations that get the job done despite chip limitations.