On December 20, the new batch of ** was announced at VLDB2024, the international top database conference, and Alibaba Cloud's new technology Pilotscope was successfully shortlisted, which can realize the "one-click deployment" of AI algorithms in the database, greatly reducing the application threshold of AI algorithms in the database, and opening up a new space for database intelligence. On the same day, Alibaba Cloud announced that all PilotScope technologies would be free and open-sourced.
Caption: The top of the database will vldb2024
Database is a basic software technology related to the national economy and people's livelihood, and the update of database technology affects thousands of industries in the digital age, and one of the frontier areas is database intelligence (AI4DB, AI for database). Today's database systems are very complex and require high stability, and even if a single AI algorithm is matched and debugged with a single database, engineers from both sides need to work closely together for weeks or even months, resulting in low efficiency and poor effect, resulting in the industry's delay in applying the ever-changing AI algorithm to the database.
Caption: Alibaba Cloud Pilotscope architecture diagram.
In response to this problem, Alibaba Cloud proposed a new solution: by abstracting and generalizing modules and interface definitions at the database and AI system level, Alibaba Cloud has developed a new middleware system platform, Pilotscope, which enables AI algorithms to be deployed in the database within hours or even minutes. The VLDB review believes that PilotScope's pioneering system design based on application scenarios will open a new direction of database intelligence.
It is understood that Pilotscope has preset more than 10 AI algorithms for mainstream database tasks such as parameter tuning, index recommendation, cardinality estimation, and query optimization, and has completed the adaptation and proofing of two mainstream open source databases such as PostgreSQL and Spark. Experimental data shows that the use of Pilotscope to embed AI algorithms into the database is 1 to 2 times faster than the traditional "hard implantation" method, and the additional cost of PilotScope itself is basically negligible.
Caption: Pilotscope renderings.
PilotScope is like a database AI 'super administrator', through which AI engineers only need to focus on designing common algorithms, which can realize the deployment and application of different databasesDatabase users can use AI as easily and efficiently as they can call APIs. Zhu Rong, the person in charge of the project, said that PilotScope has "zero intrusion" into the database, and has designed intelligent detection, fallback, isolation and other mechanisms to reduce the risk of AI illusion and achieve intelligent improvement on the premise of ensuring the stability of the database.
At present, pilotscope has been piloted within Alibaba Cloud, and related technologies are also free and open source through GitHub and ModelScope community.
It is understood that VLDB is one of the three top international conferences on databases, and only includes new achievements that have an important impact in academic and industrial practice every year, and is an authoritative database technology vane. The 50th edition of the VLDB is expected to be held in August 2024 in Guangzhou, China.
Correspondent Shirui.