Jiang Xufeng (senior financial person).
Recently, the American artificial intelligence research company OpenAI's large text generation model SORA has attracted much attention. But at the same time that it is believed that new advances in artificial intelligence technology have been made, concerns have also arisen: if criminals use SORA to fake and spread rumors, how can ordinary people identify them?
The current sora is not perfect, and some ** can be easily identified because of logical errors - the red wine glass fell on the table, the glass did not break, and the wine had spilled on the table;Archaeologists excavated a chair, which began to run ...... on its ownBut judging from the many samples released by OpenAI, there are more cases of "confusing the real with the fake", including the most discussed fashion girl walking on the streets of Tokyo, whether it is the city in the background, or the freckles on the heroine's face, the reflection in the sunglasses and other details, it is almost impeccable, and it is completely impossible to distinguish whether it is filmed or generated by artificial intelligence with the naked eye.
As a result, some people put forward a hypothesis: If you use artificial intelligence technology to look at Sora's **, can you identify it?
One example is the Gemini 1., a large AI model released by Google5** and analyze the ** generated by SORA. After finishing a section with the theme of "Cherry Blossom Snow Scene", Gemini judged this paragraph to be false for the following reasons: cherry blossoms generally bloom in spring, when it does not snow;The snow falls too evenly;Although it was a snowy day, the characters in ** were dressed very thinly.
Gemini's reasons may not all be tenable, but it offers another way of thinking: in the face of AI fraud, technical anti-counterfeiting has become more and more important.
"There is ** and there is truth" has become history
The release of SORA means that "fake news" and "fake news" are constantly iterating. Therefore, for those netizens who are accustomed to getting information from the short ** platform, they really can't trust those ** at will.
At present, many people pay attention to the application of SORA in the production of entertainment content such as movies, but in fact, it will also change the way and process of news production. At present, the "fake news" concocted by technology is emerging in an endless stream, which has changed many habitual concepts: **era, people think that "there are pictures and truths", and it turns out that pictures can p;In the times, many people feel that "there is" there is a truth, but in terms of fraud methods such as transplantation, there is now a sora, which is directly generated by artificial intelligence, and even the original material is not needed.
According to this, Guo Quanzhong, a professor at the School of Journalism and Communication of ** Minzu University, believes that the emergence of SORA has led to a great reduction in the production threshold and a great impact on news ethics.
Guo Fengjun, director of image algorithm research and development at INTSIG, also pointed out that with the development of artificial intelligence technology, the governance crisis brought about by deepfake will become more obvious. The so-called "deep fake" refers to the creation or synthesis of audio-visual content based on machine learning methods such as deep learning, such as speech simulation, images, audio-visual content, text, etc. Therefore, whether it is AI face swapping or SORA, which has appeared, it can be regarded as a typical application of deep fakes.
Although new technologies and applications can help many industries get rid of simple or repetitive work, or make innovation ideas easier to achieve, they will also have certain negative effects. For example, deep fakes can more easily steal other people's identities, fabricate relevant information, and commit illegal acts such as commercial defamation, extortion, cyber attacks, and crimes. Another example is that criminals use deepfake technology to spread falsehoods, intensify social contradictions, incite violent actions, and threaten public safety.
It can be seen that SORA is a double-edged sword. The management department and the industry should pay attention to the supervision of relevant technologies, and for ordinary people, they should also be vigilant: "there is a picture and the truth" and "there is the truth" have become history.
Does "tagging help"?
If the naked eye can't distinguish the authenticity from the fake, then what is the way to distinguish it? As far as the current exploration of the industry is concerned, "labeling" seems to be the simplest and most direct.
A number of social platform practitioners said that there are no existing laws and regulations at home and abroad that prohibit AI-generated content from being disseminated on social platforms. In fact, from a technical point of view, platforms may not be able to determine whether the content is "AI-generated" or "filmed for real". Therefore, the current common practice of various platforms is to require publishers to label this kind of generated content, and not only involve large-scale model-generated content, but also those works that are posed, screenplayed, and edited.
As developers of large models, Google and OpenAI are also working on "tagging" - watermarking all content generated by their large models through the network through backend settings to inform users.
However, "labeling" or "watermarking" does not fundamentally solve the problem of deepfakes.
On the one hand, according to the current governance of social platforms, a considerable part of the "tagging" behavior depends on the publisher. Judging from the rumors clarified by the Shanghai Rumor Refutation Platform, there are many works that are taken out of context, transferred, and posed, but the photographer and publisher did not label them. It can be seen that in terms of governance results, "labeling" does not wipe out all fraudulent behaviors. Although some ** have been marked and deleted after platform autonomy or netizen reports, there are still a large number of fake** and fake news.
On the other hand, even if the large model development company uses technical settings to make the text and ** generated by the large model be forced to "label" and tell the public that they come from artificial intelligence rather than reality, from the perspective of reality communication, relevant labels and watermarks may be deliberately erased in sharing, and screenshots, screen recordings, secondary editing processing, etc., can easily remove labels and watermarks, making it more and more difficult for the public to identify.
In this way, "labeling" is only a preliminary or basic means to prevent deep fraud, and the effect may not be ideal. Perhaps because of this consideration, OpenAI also admitted when it publicly introduced SORA that the model still has some security risks, so it will not be open to the public for the time being.
"Defeat magic with magic".
So, is there any way to identify deepfakes more accurately?
Gemini's judgment on the authenticity of SORA-generated content provides another way of thinking - using artificial intelligence to identify artificial intelligence. Many in the industry have likened this to "defeating magic with magic", arguing that through technical means, there is an opportunity to identify deepfakes at the root and reduce the associated risks.
OpenAI also said that it is conducting relevant research, including the development of text classifiers and image classifiers that can detect misleading content, "In OpenAI products, our text classifiers will check and reject text input that violates the usage policy, including content involving extreme violence, sex, hatred, celebrity portraits, other people's IPs, etc., and make relevant prompts." We've also developed a powerful image classifier that reviews each frame generated to ensure that it complies with our usage strategy before it is shown to the user. ”
However, these can still be seen as self-regulatory behaviors of enterprises. The "magic" from third parties is just as important for the industry as a whole – because the "norms" of a content generation company are based on the value of the business itself. If the R&D enterprise itself "does evil", how can it be expected that "self-discipline" behavior can prevent risks?
Because of this, many third-party companies and institutions have started the research and development of "deepfake" technology.
Guo Fengjun introduced that there are many domestic and foreign enterprises focusing on identifying artificial intelligence tampering, including a large number of Chinese enterprises, such as China Telecom and other central enterprises, technology companies incubated by universities and research institutes such as RealAI and Zhongke Ruijian, as well as technology companies such as NetEase and Hehe that have been deeply involved in the artificial intelligence industry for many years. Overall, the achievements of domestic scientific research teams in the identification of deep fraud have been at the world's advanced level, and many domestic research teams have won good results in internationally renowned tampering detection competitions.
A more positive result is that "defeating magic with magic" has been implemented in China. For example, in the financial field, many financial institutions have used artificial intelligence independently developed by domestic technology companies to identify fake face images to determine whether the relevant face images are real or synthetic, or have been replaced by AI. This technology is mainly used in the banking and financial industry to detect the authenticity of customer images and fraud, and to protect property security.
However, industry insiders also pointed out that "defeating magic with magic" still has a long way to go. With the continuous advancement of technologies in the fields of deep learning and computer vision, tamper detection technology needs to be continuously upgraded and the application scenarios need to be expanded. In this process, more enterprises and social platforms need to participate and work together for "technology for good".