Tang Xiaoou, the founder of SenseTime, passed away, and his triple AI life

"Insist on originality and let artificial intelligence lead human progress. ”

On December 16, SenseTime released an obituary, announcing that Tang Xiaoou, the founder of SenseTime, passed away on the evening of December 15, 2023 at the age of 55. Zhang Yaqin, a chair professor of intelligent science at Tsinghua University, said in a WeChat group that he learned that Tang Xiaoou had died in his sleep due to apnea.

Tang Xiaoou is a world-renowned artificial intelligence scientist who has experienced three identities in his career: an AI researcher, an educator who has cultivated a group of AI talents, and an entrepreneur who founded the AI company SenseTime.

The business world is concerned about the ups and downs of SenseTime, founded by Tang Xiaoou: its once-fast-growing valuation, huge funding and ongoing commercialization explorations.

The academic community laments the loss of a mentor who has carried many younger generations. "Mr. Tang's greatest asset is that he has cultivated a group of very good people like He Kaiming, which has greatly increased the influence of Chinese computer vision in the world. One of Tang's postdocs said.

Xu Li, co-founder, chairman and CEO of SenseTime, mourned in the circle of friends: "When we met you, we had this best time. Xu Bing, a senior executive of SenseTime and a student of Tang Xiaoou, said: "He is not only the soul of SenseTime, but also a beacon in the hearts of countless students. ”

In the five years before his death, Tang gradually retired from SenseTime and returned to education and research, serving as the director of Pujiang Laboratory, the director of Shanghai Artificial Intelligence Laboratory, and a professor at Chinese University of Hong Kong.

His last major public event was at the World Artificial Intelligence Conference in Shanghai in July. He summed up his research career with the research stories of three students, saying that his team's 18 work was the first time in the world that the Xi of depth science was applied to a visual problem, which is equivalent to "ringing the doorbell 18 times at the door of the Xi of depth learning."

Tang Xiaoou talks humorously and likes to tease. Talking about the backbone of scientists, he said that although he would not bend his waist for "five buckets of rice", "if it is six buckets...It's one more bucket than five buckets." Some people call him a "joker" among scientists.

He said that humor is to see things as they are: "There are two sides to the essence of things, one is ridiculous and the other is more ridiculous".

At the end of the speech in July this year, he mentioned that he often listened to Yu Qian's cross talk and slept, thinking: "How can a machine surpass such an interesting soul?".I don't believe it. ”

Now, the world is missing another interesting soul.

Scientist Tang Xiaoou: "Looking for light in dark primary colors".

Tang Xiaoou was born in Anshan, Liaoning Province in 1968 and was admitted to the Department of Precision Mechanics and Precision Instruments of the University of Science and Technology of China at the age of 17 from Anshan No. 1 Middle School. After graduating in 1990, he went to the United States to study, where he spent 1 year earning a master's degree at the University of Rochester and then pursuing a Ph.D. at the Massachusetts Institute of Technology.

During this time, his research interests shifted to the processing of images with computers. The earliest of Tang Xiaoou's publicly available articles was published in 1994, comparing three methods of classifying sonar images based on texture. The collaborator of this ** was one of the first deep-sea archaeological scientists to board the Titanic.

After graduating in 1996, Dr. Tang went to Hong Kong to teach, and spent more than 20 years turning Hong Kong Chinese University into a computer vision research center. In the beginning, his research continued in the direction of his PhD, such as how to make machines automatically identify plankton on the seabed.

After establishing the MMLAB at the University of Hong Kong Chinese in 2001, Tang's research focus began to shift to image processing that is closer to daily life, such as face recognition. In the next three years, Tang's team published more than 50 articles in various academic conferences and journals**.

This is an achievement that most researchers find difficult to make in their lifetimes, but for Tang Xiaoou, it has just begun.

In 2005, he was also the head of the Visual Computing Group at Microsoft Research Asia, which led to the peak of the premier academic conferences**. Tang Xiaoou attributes the driving force of scientific research to life. He later mentioned in an interview: "I have always wanted to use computer vision and artificial intelligence in daily life. He did it himself.

When he took over the work of Microsoft Research Asia, his son was only two years old, and Tang Xiaoou had to travel between Beijing and Hong Kong. "I always feel that I have too little time with Ming Ming (Tang Xiaoou's son's nickname), and I want to record every minute. He wrote in a self-report. Every time he sees his son, Tang Xiaoou will take a large number of **, and he has saved tens of thousands of photos in less than two years, and he can't sort them out at all.

Tang Xiaoou's solution is to "selfishly call on everyone to do ** management research", such as how to quickly find imagesHow do I split a portrait from multiple images?

In fact, these are rare high-quality data in computational vision research. During his time at Microsoft Research Asia, Tang's team published more than 60 articles in various top academic journals, and he teased his son as "the world's first name".

Tang Xiaoou often compares doing research to a competition of martial arts, and believes that it is necessary to focus on the goal of top-level academic conferences. "If you have to go to Taihang Mountain to discuss the sword and advance to Dabie Mountain, others can only treat you as a guerrilla. ”

In 2009, Tang Xiaoou ushered in the highlight of his academic career. This year, he was named an IEEE Fellow, one of the highest honors in the field of computer information. He and his student Kaiming Ho, then a scientist at Microsoft Research Asia, and Jian Sun, were named the best in CVPR. This is the first time in CVPR's 25-year history that an Asian team has won the top award.

A year later, Tang Xiaoou returned to his alma mater, the University of Science and Technology of China, to give a lecture and talked about this article. They discovered a fundamental property of nature images known as "dark channel prior" – within any patch of color in any normal image, there is always at least one pixel with red, green, or blue values close to zero. With this discovery, they were able to remove smoke, haze and other elements from the image almost perfectly, and restore the real scene.

Tang Xiaoou said that natural images have been studied for decades, and new basic characteristics of images can still be discovered, and researchers should adhere to this attitude: look for light in dark primary colors.

CVPR is one of the representative works of Tang Xiaoou's academic career, and it is also a reflection of his research philosophy.

In 2010, Tang Xiaoou gave a lecture at the University of Science and Technology of China. **From University of Science and Technology of China.

Mentor Tang Xiaoou: Aiming at the in-depth learning Xi in advance, bring out the technical leaders of half of China's AI industry

At this year's World Artificial Intelligence Conference, Tang Xiaoou proudly introduced three of his students: Wang Xiaogang, He Kaiming and Lin Dahua.

In 2014, as the co-founder of SenseTime, Wang Xiaogang led the team to develop the GaussianFace algorithm, surpassing the human level in face recognition for the first time.

In 2015, Kaiming Ho published Deep Residual Networks (Resnets)** large models based on the Transformer architecture, which are now commonly used by Ho Kaiming.

In 2014, he led the launch of OpenMMLAB, now the world's most influential open source project for computer vision.

In his early years, when he talked about these genius students, he didn't forget to add a few words of ridicule: "Xiaogang is the first genius student I have ever met..."His talent and character were so outstanding that I didn't hesitate to marry my sister to him. Later, another of my genius students, Dahua, published more articles, but I no longer had a sister to remarry. ”

Tutor Tang Xiaoou brought out far more outstanding students than these three. Xiaogang Wang, Kaiming He, Dahua Lin, and many more AI scientists have all studied or worked at MMLAB, founded by Xiaoou Tang.

One of the characteristics of a good mentor is that he or she has a keen grasp of the technical direction and asks the right questions.

In 2010, Tang Xiaoou began to pay attention to the deep learning Xi genre in artificial intelligence. This was a few years before the Xi method of depth science shocked the academic community in 2012 at ImageNet, a machine vision recognition competition sponsored by Stanford University.

Soon after that, Tang Xiaoou decided to shift the focus of MMLAB's research to deep Xi. In the years that followed, mmlab was a major leader in using deep learning Xi methods for computer vision.

From 2011 to 2013, 14 of the 29 in-depth Xi** in the CVPR were from MMLAB. For example, in 2012, the only two in-depth Xi** articles in CVPR were all from mmlab.

In 2016, mmlab was selected as one of the world's top 10 AI pioneer labs, the only one in Asia.

One of Tang's students said, "Teacher Tang is a calm, patient, and guided" mentor. Even when students who leave the lab meet Tang Xiaoou at a meeting, he will take time to communicate with them: "He is very good at guiding students to amplify their abilities, so he has cultivated a large number of outstanding talents." ”

In addition to the above qualities, Tang Xiaoou also has abilities that most professors do not have: strategy, understanding of human nature, and problem-solving skills.

When Tang Xiaoou returned to Hong Kong in 1996 to teach, there was a tension between professors and students at Hong Kong universities: at that time, Hong Kong ** was pushing Hong Kong's universities to shift from education to research, and a group of senior European and American scientists were hired with high salaries, and these people wanted to train their own PhDs in Hong Kong;However, students who come to Hong Kong generally only want to study for a master's degree in Hong Kong and then go to Europe and the United States for further study.

After starting mmlab and recruiting students, Tang Xiaoou did the opposite: he didn't mind students using mmlab as a springboard, and even "encouraged them to do so."

Tang Xiaoou believes that the core of being a teacher is to find good students, not the name of doctoral supervisor. When he gave lectures at universities in mainland China, such as Tsinghua University and University of Science and Technology of China, he would tell the candidates: MMLAB doesn't mind that you come here and only study for a master's degree, but before you graduate from the master's degree, you must work hard to make resultsIf you want to study in Europe and the United States after that, "I'll help you apply together".

Wang Xiaogang, Lin Dahua, Li Xuelong, Yan Shuicheng ......It was all those times that I went to the lab, and then to MIT, Stanford, and Oxford.

The word of mouth between the brothers and sisters has attracted more talents, including the top few in the relevant departments of Tsinghua University and USTC, and He Kaiming is the top student in the college entrance examination in Guangdong Province in 2003, and he studied in the basic science class of the Department of Physics of Tsinghua University as an undergraduate.

The first article published by Kaiming He during his master's degree in MMLAB was the article that won the best article in the CVPR 2009 of the world's top computer vision conference.

The strategy of following the trend has allowed Tang Xiaoou to achieve the goals that some of his mentors have not been able to ask for: some talents choose to stay in Hong Kong to complete their PhDs. Ho Kai-ming is an example, he originally applied for a master's degree in Chinese in Hong Kong, but later switched to a doctorate, and completed the master's and doctoral studies in 4 years. The results of MMLAB have given him the confidence that he does not need to rely on school to increase his aura.

Tang Xiaoou and He Kaiming. **From Chinese University of Hong Kong.

The students who went out of MMLab later enriched the research and industry of AI in China, especially in the field of computer vision, which is known as the "Whampoa Military Academy of Computer Vision": in addition to the three students mentioned in Tang Xiaoou's speech this year, there is also Yan Shuicheng, who has served as CTO of YITU, CTO of Shopee, and is now the co-CEO of Kunlun Wanwei Tiangong IntelligenceGao Xinbo, President of Chongqing University of Posts and Telecommunications;Xu Chunjing, Director of Noah's Ark Computer Vision Lab of Huawei;Zhao Deli, the former head of basic vision of Ali Dharma Academy;Xu Bing, co-founder of SenseTime;Qiao Yu, Director of the Institute of Advanced Computing and Digital Engineering, Chinese Academy of Sciences;Tao Dacheng, the former top technical scientist of JD.com;Zhao Cong, former head of AI vision at DJI, etc.

The high density of talent is also the biggest advantage of SenseTime when it starts. After the outbreak of deep learning Xi and the attention of the industry, Tang Xiaoou will open another identity: entrepreneur.

A person who understands the organizational method of MMLab has a different view of Tang Xiaoou's entrepreneurship: "Professors are generally not optimistic about entrepreneurship. But Tang Xiaoou really has the potential to succeed in starting a business. ”

Entrepreneur Tang Xiaoou: Founded SenseTime, but not just the founder of SenseTime

In June 2014, Tang Xiaoou's team released a face recognition model called DeepID, which had a recognition rate of more than 99% on the face recognition database LFW (Labeled Faces in the Wild), beating Facebook's Deepface.

At that time, the global business ** that was attracted to "face recognition" by Facebook found that Deepface, which Facebook vigorously promoted, actually lost to a laboratory from China.

Coupled with the promotion of IDG investor Niu Kuiguang, Tang Xiaoou founded SenseTime at the end of the same year. Compared with Megvii, which was established in 2011 and YITU was established in 2012, SenseTime's entrepreneurship started a little later. But it has since become the No. 1 company in Chinese's artificial intelligence field in terms of talent density and valuation.

Tang Xiaoou has formulated a lofty company mission for SenseTime: "Adhere to originality and let artificial intelligence lead human progress." ”

SenseTime's initial choice of model is also a way that seems to be able to efficiently lead the progress of various industries: to be a technology platform, that is, SenseTime mainly makes general software technology for computer vision, so that downstream applications or partners can apply it to their own scenarios.

The positive atmosphere after 2014 supported SenseTime's early boom, especially after AlphaGo's victory over Li Shiqi in 2016, which set off a new wave of AI boom. A group of investors have gone from paying for traffic growth in the past to paying for the best quantity.

At its peak around 2018, SenseTime had nearly 300 doctors, so much so that there was an internal joke: "If you are a threesome, you must have a doctor".

This year, Tang Xiaoou released at the press conference the number of comparison charts released by various institutions around the world at top computer vision academic conferences in the past few years, and SenseTime ranked third, second only to Microsoft and Carnegie Mellon University.

BAT all say that it is an AI company, but internationally, only SenseTime exists. Tang Xiaoou said.

SenseTime was once the world's most-funded AI company, raising a total of $5.2 billion in funding and valuing it at $12 billion before going public at the end of 2021.

But over time, the idea of a "platform" based on technological ideals has suffered a setback in the commercial competition.

The competitiveness of AI algorithms is not easy to maintain due to the rapid proliferation of software technology;Large-scale manufacturing capabilities, business relationships, or industry awareness held by companies in the industry are deeper barriers. At that time, computer vision technology itself was not universal enough, and it could not be migrated at low cost in many scenarios.

Companies in the industry, from Hikvision to Byte to Tesla, are reaping the benefits of AI technology. The object that SenseTime wants to empower itself is to empower itself, which is a common commercialization dilemma faced by a group of pure AI companies.

A SenseTime executive said he initially believed that "science and technology are the primary productive forces", but later found that this did not apply to all scenarios and stages. In the Chinese market, in order for technology to generate revenue, it must either sink down into infrastructure or tie up the application.

In recent years, SenseTime has adjusted its business strategy. It includes the scope of "empowering all industries" to include smart cities, smart cars, smart businesses and smart lives"Vertical integration" not only does software algorithms, but also does computing infrastructure and applications downward, such as the establishment of a large computing power device with more than 30,000 GPUs in ShanghaiGrab the opportunity for a big model.

SenseTime's commercialization exploration continues. According to the financial report, SenseTime's revenue in the first half of this year was 143.3 billion yuan, gross profit of 64.8 billion yuan, both of which increased slightly year-on-year;Net loss 314.3 billion yuan, a slight decrease of 2% year-on-year;Accounts receivable in the first half of the year were as high as 77$2.6 billion, five times the revenue.

A number of SenseTime executives said that in the entire entrepreneurial process, Tang Xiaoou's management style is to grasp the macro level, "just manage the co-creation of several SenseTime". However, some SenseTime people said that Tang Xiaoou would care about the company's external image and personally review the company's promotional materials.

A former SenseTime employee told LatePost that Tang Xiaoou is also a mentor in SenseTime, and everyone calls him "Mr. Tang", and he will not talk about the company's development strategy at the company's annual meeting, but about life and share the good books he just saw recently.

Tang Xiaoou advocates the "black sheep culture" in SenseTime: employees are required to have empathy, but they must not fall into the "herd effect", and do things that others have not done, or even things that others can't think of.

Around 2019, Tang Xiaoou gradually faded out of SenseTime, and only retained his status as an executive director. In the later ** report, Tang Xiaoou's news related to SenseTime was basically only the decline in the market value of SenseTime, which reduced his wealth.

In the four or five years before his death, Tang Xiaoou was more active in the industry as the director of the Shanghai Artificial Intelligence Laboratory. The institution was established in July 2020, with Tang Xiaoou as the director of the laboratory, and the leading scientists also include Academician Yao Chizhi and Academician Chen Jie. In addition to AI R&D, the Shanghai Artificial Intelligence Laboratory also cooperates with many well-known universities in China to train doctoral students.

Tang Xiaoou has returned to his familiar role as a researcher and mentor.

In the past few years, the Shanghai Artificial Intelligence Laboratory has made achievements in many fields such as machine vision, autonomous driving, and machine learning, and Xi: in 2022, the laboratory's autonomous driving team open-sourced the Bevformer architecture;In June this year, the Shanghai Artificial Intelligence Laboratory, Wuhan University and SenseTime jointly proposed a general model for autonomous driving**, which won the best in CVPR 2023**, which is also the first best article in the top computer vision conference in the past decade with a Chinese academic institution as the first unit**.

The World Artificial Intelligence Conference in Shanghai in July 2023 was Tang Xiaoou's last public speech. At the end of his speech, Tang Xiaoou once again thanked his students.

He borrowed a line from the movie "Good Teacher": "I didn't meet you at the best time. It's when I met you that I had the best time. ”

At the World Artificial Intelligence Conference in July this year, Tang Xiaoou borrowed the line from "Good Teacher" at the end.

Title image**: Visual China.