During the two sessions of the National People's Congress in 2020, Tan Jieqing, director of the Institute of Applied Mathematics of Hefei University of Technology, mentioned in an interview: "Domestic databases only occupy less than 7% of the market share, especially in the core transaction business of the database, there are few products that can compete with Oracle on the same stage and achieve replacement." ”
At that time, the wave of "de-ioe" had risen for more than ten years, but due to objective factors such as performance and stability, traditional businesses such as banking, energy, and telecommunications still had an attitude of "not daring to use, unwilling to use, and not wanting to use" for domestic databases, which virtually created a ceiling for the domestic database industry.
Now, that vision is becoming a thing of the past.
Huawei Data Storage and Youxuan Software recently jointly released the "Data Inventory Computing Separation Joint Solution" and launched a highly reliable and high-performance shared storage multi-write-multiple-read cluster database solution, aiming to meet the database replacement needs in traditional business scenarios such as finance, carriers, energy, manufacturing, and government affairs through advanced storage technology and innovative design.
In the context of digital and intelligent transformation, what does such cooperation mean, and can it break the ceiling of the domestic database industry?
Compared with basic software such as operating systems and middleware, databases are the market with the most rapid domestic substitution. However, the outside world's impression of the domestic database does not seem to be too optimistic, in one sentence to describe it: the number of flowers is blooming, but the market competitiveness can only be said to be "many but not strong".
Why is this happening?There has been a lot of discussion from the outside world, and two mainstream views have emerged.
One theory is that the starting point of domestic databases is relatively late.
As early as 1978, Oracle launched the first version of the database, when China's information transformation process was not yet possible. Around 2000, the first batch of database companies appeared in China, but the global database industry has completed the process of competition, mergers and acquisitions, and exit, forming a typical IOE pattern, I refers to the server provider IBM, O refers to the database software provider Oracle, and E refers to the storage equipment provider EMC.
After 2014, driven by policies and market dividends, the domestic database industry has blossomed, but it has failed to change the pattern of leading markets such as Oracle and IBM, and domestic databases can only occupy some niche markets. The number of domestic databases is increasing, but the result is a high degree of fragmentation.
Figure: The number of monthly entries in the 2023 Motianlun China Database Rankings.
According to the statistics of the Academy of Information and Communications Technology and Mo Tianlun, there are currently more than 280 database products in China. More than 60% of domestic database manufacturers have less than 100 people, and less than 10% of enterprises with more than 500 people, coupled with the lack of collaboration, the relatively weak R&D capabilities cannot form a joint force, and it is difficult to enter the business scenarios such as finance and energy, and the survival environment has been relatively difficult.
Another explanation is the architecture of domestic databases.
Around 2008, under the advocacy of Internet giants such as Alibaba, the wave of "de-IOE" was in full swing. At that time, China's Internet had entered a period of rapid growth, and there were scenarios such as the Double 11 shopping festival with a surge in data volume and user volume, and the IOE architecture lacked horizontal expansion capabilities to meet the surging performance and flexible expansion requirements.
In terms of database architecture, in order to eliminate unnecessary data migration delays and power consumption, the integrated storage and computing architecture, which seems to improve efficiency and reduce costs, is gradually sought after by Internet companies. The advantages of the integration of storage and computing are obvious, and the shortcomings are equally obvious. In order to achieve high reliability, a master, many-slave architecture is usually adopted, and multiple slave nodes are idle most of the time, resulting in extremely low CPU resource utilization. In addition, after a server failure, it cannot be switched automatically, and a lot of manpower and time need to be invested in manual data recovery.
Even if old domestic database manufacturers such as Dameng and Nanda General are still insisting on the separation of storage and computing architecture, when the attention of the entire market turns, one or two companies cannot influence the direction of the industry. Traditional businesses such as banks, which require strict stability, have to hand over orders to foreign manufacturers because the localized database cannot meet the demand.
When the integrated storage and computing architecture has been criticized by more and more people, and the shortcomings of data transmission performance have been filled, the architecture of storage and computing separation has been pushed to the front again. The cooperation between Huawei Data Storage and Youxuan Software is a narrative in the new context, trying to break a new path with a combination of software and hardware.
The concept of storage and computing separation is not difficult to understand, simply put, it is to build a computing resource pool and a storage resource pool separately, share a share of data globally, and some unnecessary consumption can be avoided, further improving the performance of the database, even if a server fails, it will not lead to data loss.
Under the architecture of storage and computing separation, Huawei Data Storage and Youxuan Software jointly released the "Data Inventory and Computing Separation Joint Solution", which mainly includes three sub-solutions:
The first is the active/standby cluster deployment solution, which adopts a one-master-one-slave database architecture to ensure high business availability, and has the characteristics of easy deployment and management, which is suitable for business systems such as OA, portal, mailbox, and order management
The second is the read/write splitting cluster deployment scheme, which adopts a master-multi-slave architecture, and ensures that the slave node is readable through the strong consistency technology of master-slave data through storage-computing separation, which has the advantages of high performance, easy scalability, and high reliability, and is suitable for medium and large-scale key transaction applications such as financial accounting systems, ERP systems, CRM systems, manufacturing, and R&D systems
The third is the multi-write and multi-read cluster deployment scheme, which adopts a multi-master architecture, ensures strong read/write consistency of global node data through shared storage SRAC technology, and achieves the effects of multi-write and multi-read, load balancing, and split-brain control, etc., with extremely high reliability and performance expansion potential, and is suitable for large-scale core trading systems with high requirements for availability and performance in industries such as finance, telecommunications, energy, transportation, finance and taxation, and manufacturing.
The three sub-solutions correspond to different business scenarios, among which the most notable is the multi-write and multi-read cluster deployment solution, which is largely related to whether the domestic database can replace Oracle RAC in the core transaction business.
Take a common payment scenario as an example: when a bank rejects a user's payment request, it needs to quickly query the user's past payment habits to determine whether the payment request is risky, and at the same time provide a risk warning in the form of a pop-up window. This requires the database to have a high ability to handle complex transactions, and business continuity requires high and absolute high availability, business and data consistency, and a certain degree of scalability.
The substitution of RAC in China has been around for a long time, and most of them adopt three methods: middleware simulation, distributed database and technical routes similar to RAC. Youxuan Software's "multi-write and multi-read cluster deployment solution" adopts the RAC route, which can be directly replaced by domestic production.
In addition to the high-pressure investment of Youxuan Software for 10 years, storage performance has played a crucial role in overcoming the problem of multi-write and multi-read clusters.
With the ultimate latency of 50 seconds, 21 million IOPS, and the extremely stable SmartMatrix fully interconnected architecture, Huawei's Oceanstor Dorado all-flash storage delivers an extreme latency of 50 seconds, 21 million IOPS, and a highly stable SmartMatrix fully interconnected architecture, improving overall performance by 30% compared with ordinary storage with the same configuration, meeting the requirements of different types of transactional services.
The database system follows the "barrel theory", and any shortcomings in hardware and software will restrict the development of the database. The cooperation between Huawei Data Storage and Youxuan Software undoubtedly provides a new solution for the entire database industry: after the integration of the software advantages of Youxuan Database and the hardware advantages of Huawei's OceanStor flash storage, the "curse" that was originally on the head of domestic databases has been quietly lifted.
If we say that in the wave of "de-IOE" more than ten years ago, too much focus was placed on the needs of Internet services, and the demands of traditional services were selectively ignored. The purpose of the joint innovation between Huawei Data Storage and other database vendors is to continuously narrow the gap in performance and reliability between Huawei Data Storage and other database vendors in key industries, and improve the overall competitiveness of domestic databases.
In particular, in terms of the "storage and computing separation" architecture, not only Youxuan Software, but also Huawei Data Storage has launched joint solutions with many database vendors, achieving leapfrog upgrades in many fields.
For example, the "Storage-Compute Separation & Multi-Master Architecture Joint Innovation Solution" jointly released by Wanli Database and Huawei Data Storage Co., Ltd. breaks through the industry problem of multiple reads and multiple writes of databases, greatly improves database performance, and reduces system construction costs. Taking performance as an example, the database cross-node cache pooling technology implements the concurrent read/write and transaction processing capabilities of global tables, which can improve the performance by 10 times in different scenarios compared with the traditional primary/standby and sharded databases.
Another example is the "Financial Core Database High Availability Solution" jointly released by Huawei Data Storage and NTU General Computing, which provides a high-performance and high-availability database solution that meets the requirements of financial core systems based on the storage-compute separation + shared storage architecture, combined with GBASE and Huawei OceanStor flash storage.
One of the innovations that cannot be underestimated is the dual disaster recovery mechanism, which implements database disaster recovery based on logical replication and readable backup database at the application layer, and relies on the Hypermetro A-A active-active capability of OceanStor flash storage at the storage layer to ensure that data is efficiently and completely replicated to the disaster recovery site without affecting the performance of the working site, ensuring RPO=0, ensuring that data is not lost, and business recovery is fast to meet the business requirements of the core financial system.
In the past, domestic databases were frequently criticized for fighting their own battles, and compared with the "best combination" such as IOE, the domestic database market can be described as a plate of scattered sand. The lack of close cooperation between different manufacturers makes it impossible to build a benign ecosystem and break away from the ecosystem dominated by foreign brands, resulting in a market pattern that is more but not stronger.
Youxuan software, Wanli database, Nanda general, massive data ......Huawei's data storage is like a link, bringing together different database vendors, and the ecosystem collaboration may not be particularly close at this stage, but "storage and computing separation + shared storage architecture" is already an industry consensus.
To borrow a quote from a database practitioner, there is no shortage of excellent engineers in the domestic database industry, and the important thing is to find the right problem and the right direction to exert force.
Following this logic, as more and more database vendors choose Huawei as a partner to innovate joint solutions with Huawei data storage, a benign ecosystem conducive to the rise of the domestic database industry has begun to take shape. China's database industry, which has been silently working the road of independent innovation for more than ten years, is crossing the hills, and what awaits them will be a prosperous database ecology.
In the 2022 "Database China Market Guide Report" released by Gartner, a market consulting agency**, by 2025, only 30% of China's analytical database market will come from overseas vendors, and only about 50% of the overseas vendor market will be left in the transactional database market.
Perhaps more than a year ago, many people would have expressed doubts about Gartner's ** data, after all, IOE is still irreplaceable in the transactional database market. With the continuous innovation and breakthrough of "storage and computing separation + shared storage architecture", Gartner's ** is getting closer and closer to reality.