People or organizations are faced with various decisions all the time, and big data brings great help to the optimization of decisions. In order to turn non-speaking data into a helper for decision-making, it is necessary to use the method of optimization modeling, which belongs to the research of Yuan Yaxiang, a famous mathematician and academician of the Chinese Academy of Sciences.
On the morning of December 16, Yuan Yaxiang gave a popular science lecture in the "Science and Technology Innovation Academician Lecture Hall" of Shenzhen Institute of Innovation and Development with the title of "Big Data and Optimization Behind "Intelligent Decision-making". The whole lecture was simple to understand, which not only made the audience feel the "beauty of mathematics", but also triggered a lot of thinking about the application of big data.
Yuan Yaxiang graduated from Xiangtan University in 1981 and received a doctorate degree from the University of Cambridge at the age of 26He is currently the vice chairman of the China Association for Science and Technology and the chairman of the International Union of Industrial and Applied MathematicsHe mainly researches nonlinear optimization calculation methods, and has made important contributions to the trust domain method, quasi-Newton method, nonlinear conjugate gradient method, subspace method, etc. His research results in nonlinear programming have been internationally named "Yuan's lemma".
The big data problem is the optimization problem.
I'm not doing big data research, but many areas of mathematics are related to big data, and the more traditional ones are statistics, computation, and optimization. "Optimization is my research field, so I want to talk about the relationship between optimization and mathematics - big data can be modeled with optimization. ”
The data tends to have an observation. For example, if an enterprise has a lot of data, what problems are found and what conclusions are drawn from it, this is the observation value of the data, and it is also the essence of the data. From the point of view of mathematical research, it is the correspondence of data, that is, functions. The essence of studying data is to find the correspondence, and by writing a function, the error of the two things is minimized, which is the problem of optimization. Yuan Yaxiang further explained.
For example, Yuan Yaxiang said that people want to take photos with as much clarity as possible and as little storage space as possible, which requires optimization. This problem is an engineering problem, but in fact it can be converted into a mathematical problem, into a problem for solving a system of linear equations. For another example, the same type of audience will give similar ratings to the movie, and by analyzing the mathematical language matrix, the scoring results of different audiences can be inferred, so as to guide the production of the film, which is also the application of big data in optimization.
All in all, a lot of big data problems boil down to optimization problems. Optimization is everywhere, and in layman's terms, it's about picking the best of the best. Yuan Yaxiang said.
The most famous example of the Chinese's use of optimization and operation planning is the ancient Tianji horse race. The mountain is still the same mountain, the horse is still the same three horses, and different matchups can get completely different results. Yuan Yaxiang then gave the audience a bowl of "chicken soup": "In life, each of us will complain about our lack of resources, in fact, optimization and combination will make the results of the essential difference, and many problems depend on whether we have made the best decision." ”
Yuan Yaxiang, a representative figure in the application of optimization theory in modern China, believes that the mathematician Hua Luogeng is the first to be recommended. Hua Luogeng once vigorously promoted the optimization method to factories, hospitals and mines across the country, and made important contributions to the country's economic construction at that time.
Optimizer**.
Regarding the optimization method, Yuan Yaxiang popularized science for the audience one by one.
The first is the gradient method, "just like climbing a mountain, optimization and climbing are both seeking, the 'most' value, climbing the mountain is to reach the 'highest' altitude, and the optimization is to require the result to be 'the best', and you can climb to the highest point along the steepest direction." Yuan Yaxiang explained vividly.
Optimization can also take the "alternate direction" approach. "There are many variables involved in the problem you want to make a decision, and the so-called alternate direction is to take only one variable at different stages and at different times, and take turns to solve the problem. Just like a large enterprise, involving many branches, the boss only pays attention to one of them at a time and solves a problem. Yuan Yaxiang explained.
If a problem involves two parts, and the two parts are not coupled to each other, a separable optimization method can be used. Yuan Yaxiang gave an example, a Go board is full of irregular black and white chess pieces, and the fastest way to count the total number is for two people to count the chess pieces of one color at the same time. "It's equivalent to an optimization problem with two parts to find 'minimum', these two parts are unrelated, they can be done at the same time, and each part becomes a smaller problem. Yuan Yaxiang said, "This is also the case in real life, a large project can be divided into two sub-projects, and the two sub-projects are unrelated and become two small problems, so the big problems can be converted into small problems, and no matter how small the problems can continue to be decomposed." ”
Subspace is also a common way to deal with optimization problems, Yuan Yaxiang said: "Big data problems and highly complex problems are ultra-high-dimensional problems, the so-called subspace method is to convert large-parameter problems to low-dimensional space to solve, so that the problem is simplified." ”
At present, artificial intelligence, machine science, Xi a very widely used optimization method is the stochastic gradient method. Yuan Yaxiang explained, for example, it is best to manage a city and meet the average demands of each citizen. But it is impossible to add up everyone's demands to satisfy, so you can pick some randomly, take the average, take the subset, and replace the whole with parts, which is the basic idea of the stochastic gradient method.
In addition, stochastic techniques and multi-objective optimization are also commonly used optimization methods in life. "Any decision in the world is an optimization problem, although there is generally no need to use mathematical formulas to deduce how to make decisions, but to have this awareness, to arm our minds with the idea of optimization. Yuan Yaxiang said.
Confronting the tech gap.
Some listeners asked, what are the challenges of optimizing the implementation of theories and innovation?In this regard, Yuan Yaxiang replied that the application of optimization theory to practice can produce huge benefits, but it is still far from the real implementation. "This also requires us to work more closely with those who do scientific research with those who do technical research and even with entrepreneurs. Yuan Yaxiang said, "The state is now encouraging more scientific and technological workers to do the landing, including the Chinese Society of Industrial and Applied Mathematics also very encouraging scientists to do the landing." On the one hand, it is hoped that scientists will get out of the laboratory and cooperate with enterprises and productsOn the other hand, it is also hoped that entrepreneurs will take the initiative to talk to scientists when they encounter problems in the development of enterprises. The joint efforts of the two parties can promote the optimization of cooperation between enterprises and industries. ”
Another audience member asked what are the advantages and disadvantages of China compared with developed countries in the era of digital economy, where the competition between algorithms and computing power is very crucial
Yuan Yaxiang used an abacus familiar to Chinese to answer this question. He said that our country's ability to build computers has always been in the forefront of the world, which is equivalent to our abacus being built well, but whether the plan is also very good is not at present. The mantra of the computer is the algorithm, and the computer application, big data processing, large model, and artificial intelligence ultimately need the support of the algorithm.
Yuan Yaxiang further answered that China's research on algorithms is relatively advanced in the world, but it is not the most advanced. "We are not weaker than Europe and the United States in terms of theoretical research and algorithm construction, but we are lagging behind in turning algorithms into software. From methods to algorithms, from programs to software, there is a gap in between." Yuan Yaxiang said.
Yuan Yaxiang believes that compared with developed countries, China is not much behind in science, but it is far behind in technology, and it is even more backward in engineering, which may be related to the scientific and technological system. On the one hand, the evaluation system of state-owned research institutes does not encourage it, and on the other hand, enterprises do not have the ability and policy support to do it. This is an important question that deserves to be pondered and faced. Yuan Yaxiang finally said.