The boom in AI large models has further accelerated the scarcity of intelligent computing power, and in this context, how to find more efficient computing power solutions has become a difficult problem for many banks. What is the root cause of the shortage of intelligent computing power in banks?Can the "computing power pool" built in some places and the AI agent that has been implemented in the first form provide "underlying computing power support" for the large bank model?
Since the beginning of this year, more and more domestic banks have actively embraced large-scale model technology and widely applied it in many financial scenarios. At the same time, the bottleneck of computing power is becoming a major challenge for the layout of large model technology.
The foundation of computing power is advanced AI chips. However, due to the restrictions on the export of advanced AI chips (including GPU chips) to China by the United States, domestic banks have generally encountered computing power bottlenecks in the development of large financial models. A person in charge of the IT department of a joint-stock bank said.
It is understood that in order to solve the challenge of computing power bottleneck, many banks have shown their own talents.
Specifically, large banks simply build their own computing power, although the cost of this approach is quite expensive, but the advantage is that banks have a high degree of autonomy and security, and can also develop large financial models according to their own business needs.
Small and medium-sized banks have been testing the waters of hybrid deployment of computing power, that is, on the basis of the public cloud, they first call the large model computing service interface of large Internet enterprises or telecom equipment service providers, and then carry out the pre-training of large models of their own data through privatized deployment.
At present, small and medium-sized banks have formed a relatively mature cloud + computing power service solution, which mainly introduces the computing resources of large Internet enterprises and telecom equipment service providers to solve the pre-training and optimization of large model data of the former. A director of the information technology department of a small and medium-sized bank introduced.
Differentiated solutions for banks of all sizes.
The survey found that at present, the main ones that have the ability to build their own computing power are large state-owned banks and some joint-stock banks.
The reason for this is that these banks have reserved a certain number of advanced AI chips, which are widely used for the pre-training of their own financial large model data.
Considering that a single advanced AI chip** is quite expensive, it would be a significant expense to reserve a certain number of chips, which only large banks and some joint-stock banks can afford to do. The head of the information technology department of the above-mentioned small and medium-sized bank said.
Correspondingly, large banks are making faster progress in developing their own financial models.
According to a source from the IT department of a large bank, they have established a heterogeneous GPU computing power resource pool and realized flexible scheduling of computing power based on cloud service deployment.
This allowed us to quickly build a basic model, a financial industry model, and a scenario model based on computing algorithms. The IT person said bluntly. At present, these large models have been widely used in scenarios such as intelligent customer service, text generation, auxiliary R&D, robo-advisory, intelligent collection, and big data risk control.
The above-mentioned IT people admitted that due to the impact of the United States' restrictions on the export of advanced AI chips to China, coupled with the acceleration of the iteration of the bank's internal financial model, they also found that their GPU chips are being "consumed" rapidly. Therefore, they also try to carry out computing power cooperation with some large Internet companies, such as jointly building underlying infrastructure such as computing power clusters and computing network storage, and further use external computing resources to carry out the optimization and iteration of financial large models.
In contrast, the vast majority of small and medium-sized banks basically use computing power leasing or hybrid deployment of computing power to solve the computing power bottleneck of their own research and development of large model technology.
At present, we mainly carry out the pre-training of large models of internal data of banks by leasing the computing resources of large Internet enterprises (that is, calling the computing power of the latter through cloud deployment), but this computing power investment still reaches millions of yuan every year. The head of the information technology department of the above-mentioned small and medium-sized bank said.
According to the head of the information technology department of the small and medium-sized bank, in order to reduce the cost of computing power as much as possible, they are considering the development of lightweight large models, that is, optimizing and adapting to scenarios on the basis of large model products developed by third-party financial technology platforms, and then combining the pre-training results of the bank's internal data to develop large models suitable for specific tasks or specific scenarios, which can not only effectively reduce the workload of data pre-training and computing power investment, but also save a lot of "repetitive work" in large model research and development.
The survey found that in order to further save computing power investment, some small and medium-sized banks also adopted the "large model + small model" approach to promote the application of large model technology in financial scenarios. Specifically, the first is that the large model plays the role of task decomposition and scheduling, and through the scheduling of small models by large models, more intelligent work can be achieved and the computing power investment caused by massive data pre-training can be reducedThe second is to empower the feature vector by the large model, which can replace the feature engineering work required by the small model, which can also reduce a lot of computing power investment.
In the view of many bankers, although many banks have taken many measures to solve the bottleneck of computing power, under the trend of faster iteration of financial large models and the rapid development of large model technology, the best way to truly solve the bottleneck of computing power is to actively embrace domestic AI chips.
At present, some banks have tried to introduce domestic GPU chips as a new computing resource support for their further research and development of financial models.
However, due to the differences in the technical route and software ecology adopted by domestic GPU chips and advanced AI chips in the United States, and the fact that domestic GPU chips are still further improving computing performance, it still takes a certain amount of work and time to adapt domestic GPU chips to the existing technical architecture of financial model research and development.
Test the water in the first computing power pool.
In order to solve the computing power bottleneck of many enterprises to develop large models in the industry, many places have been actively building "computing power pools" recently.
This has also aroused strong interest from a number of banks.
If we can solve the problem of data pre-training through the local computing power pool, it may speed up the iterative upgrading process of our financial large model, so that the large model technology can be better applied in more financial scenarios. The head of the information technology department of the above-mentioned small and medium-sized bank said.
But he soon discovered that the biggest challenge was how to "keep data from traveling."
In the past, when they cooperated with large Internet companies to carry out computing power leasing, the latter deployed the computing resources required for the pre-training of large model data in small and medium-sized banks, so that when the bank used the locally deployed computing resources for massive data pre-training, there was no trouble of "data travel". Nowadays, how to introduce the computing resources of the local computing power pool under the condition of "data does not travel", they are currently studying the feasibility of compliance operation plan.
Some small and medium-sized banks from the IT department suggested that the relevant financial regulatory authorities may wish to establish a computing power pool for banks and other financial institutions for banks of different sizes to carry out pre-training of financial large model data, which can not only solve the "computing power bottleneck" commonly encountered by many banks, but also minimize the financial service "gap" between banks brought about by large model technology. However, who will fund the purchase or lease of computing power, how to allocate computing power, and how to build the underlying infrastructure that matches the transfer interfaces of different banks all face certain practical challenges.
AIagent has taken shape.
Towards the end of the year, the AI industry once again offered the "king bomb".
Recently, at the first OpenAI developer conference, OpenAI released the initial form of AIAGENT GPTS, and launched the corresponding production tool GPTbuilder. Users can generate their own GPT by simply chatting with GPTbuilder and describing the desired GPT function.
In this regard, a person in charge of the IT department of a city commercial bank revealed that they have also noticed that the AIAGENT business is becoming increasingly hot, and the bank's senior management also believes that the AIAGENT large model technology can play a more intelligent service effect in many financial scenarios, but due to the bottleneck of computing power, they are unlikely to carry out exploratory research on this cutting-edge large model technology.
AIagent, also known as AI agent, belongs to two completely different concepts from LLM, LLM is essentially a large language model, including GPT, GLM and other generative pre-trained large models based on natural language processing and a huge number of parameters, but AIAGENT focuses on AI interaction with the environment, and according to the characteristics of the current environment, AI makes decisions and takes actions autonomously.
In a way, LLMs are a key link to AIagents. However, at present, aiagent is not yet mature, and there are many errors and problems. However, it is foreseeable that if AIAGENT technology matures and forms various practical applications, it can help financial institutions use AI technology more efficiently and at low cost.
A number of small and medium-sized banks bluntly said that with the increasing maturity of AIAGENT technology and the continuous progress of data encryption and retrieval technology, they may introduce AIAGENT large model technology in the future and try to apply it to more financial scenarios, because AIAGENT can indeed play a more powerful role in intelligent services and cost reduction and efficiency increase. However, a major prerequisite for whether this work can be carried out is whether banks can build independent, secure, stable, and powerful computing resources to support the R&D and application of cutting-edge large model technologies such as AIAGENT. Chen Zhi.
*: City Finance News.