On December 15th, Aloudata, a leading provider of automated data management software in China, held a Noetl!Reconstructing Data Productivity Strategy and Product Launch, Zhou Weilin, CEO of Aloudata, and the founding team officially released the future-oriented NoetL data engineering architecture, and released three disruptive innovative products, Aloudata Air Logical Data Platform, AlouData Big Active Metadata Platform and AlouData CAN Automated Indicator Platform, providing new ideas and solutions for enterprises to solve the difficulties and pain points in digital transformation, and providing data solutions The implementation of the Fabric data architecture concept in China has opened up a clear path. The event also invited Zhang Junjun, head of the data intelligence platform of China Merchants Bank's head office, to share the bank's agile data analysis practices and prospects for future trends.
With the penetration of digital intelligence into all walks of life, the demand for data use, the number of people and the data source are growing at the same time, which brings great challenges to the data engineering efficiency, cost and orderly management of data of enterprises. How to bridge the gap and achieve a hundredfold increase in data productivity, data management urgently requires new thinking, new architecture, and new technologies. Aloudata pioneered a new paradigm of data engineering that eliminates manual ETL through automation, which is the concept of "NoETL".
At its core, Noetl is the driving force behind transforming data production. While traditional ETL is driven by ETL engineers, NOETL is driven by ETL agents, which is equivalent to providing each enterprise with a group of 24/7 ETL experts, so as to achieve "autopilot" of ETL tasks. NoETL is highly consistent with the data fabric architecture concept that has been gaining traction in recent years, aiming to reduce or eliminate traditional manual-based ETL jobs, achieve more efficient data delivery, use, and governance through a high degree of automation of the data management process, and reconstruct data productivity. There are 4 main features of the Aloudata Noetl data engineering architecture:
1.no pipelines: Go to the pipeline, no need to care about the location of the data;
2.no tasks: O&M, no need to worry about task O&M
3.no cubes: self-optimizing, no need to worry about query performance;
4.Active Metadata: From passive to active, data management is "autopiloted".
Relying on Noetl data engineering architecture ideas, Aloudata has achieved three core technology breakthroughs: data virtualization engine, data semantic engine and active metadata engine, which bring new data integration solutions, new data interaction interfaces and new data management models, which constitute the cornerstone of Aloudata's product system.
Xiao Yuhong, CPO of Aloudata, introduced Aloudata Air, the first data fabric logical data platform in China. It uses the data virtualization engine, AI-enhanced policy engine, and active metadata engine to realize the logical integration and integration of multi-source heterogeneous data, adaptive data acceleration, automatic data link orchestration, centralized data governance and control, and other functions, helping enterprises easily break through data silos, improve data delivery efficiency by 10 times, and achieve storage and computing cost savings of more than 50%. Aloudata Air's agile data preparation solution has been implemented and verified in China Merchants Bank, realizing unified access and management of global data, significantly improving business efficiency and data management flexibility, and greatly reducing storage and computing costs and the burden on the ETL team.
Zhou Quan, CTO of Aloudata, introduced Aloudata Big, the world's first active metadata platform that implements operator-level lineage resolution. With fine, accurate, and comprehensive lineage analysis capabilities and active metadata management capabilities, Aloudata Big can solve the difficulties of "unclear" and "uncontrollable" in enterprise data management, make the data management process more automated, and easily cope with the needs of exabyte-level big data management. Based on semantic analysis technology, Aloudata Big realizes the operator-level automatic parsing of SQL scripts and the construction of operator-level lineage map, realizes the accurate description of the calculation semantics of global fields, and provides users with high-confidence suggestions or design solutions based on in-depth analysis and mining of metadata. Aloudata Big has been verified by many large banks, helping customers quickly locate link anomalies, improve model governance by 100 times, and achieve efficient and reliable full-link data quality assurance.
In addition to data integration and integration, enterprises often face the dilemma of indicator management in the actual use of data: chaotic indicator caliber, slow delivery efficiency, high cost of repeated development, inflexible dimensions, and difficult interpretation. In response to the above problems, Du Xuefang, Chief Business Architect of Aloudata, introduced the Aloudata CAN automated metrics platform designed with the concept of Headless BI. The product realizes the one-time definition and multi-use of indicators through an independent index semantic layer, and truly realizes the "definition is production" and "utility integration" of indicators. In the case of a first-class industry customer, AlouData CAN perfectly solves the problem of inconsistent business indicators, shortens the indicator development cycle from weeks to minutes, and realizes the precipitation of unified data assets, forming a top-down unified data language.
As the concrete implementation of the Noetl architecture, these three products together form Aloudata's "autonomous driving" solution for data engineering and management. Zhou Weilin emphasized that as a company with the belief of "Noetl reinventing data productivity", Aloudata's original intention and mission is to make data ready and available on demand.
As one of the leading enterprises in China's digital transformation, China Merchants Bank believes that improving the accuracy of management and marketing through data insights, and making operations and services refined and personalized, is the core driving force to participate in the next stage of digital banking competition. After years of exploration, the integration of China Merchants Bank's digital technology and business scenarios has gradually entered the deep water area, and the flexibility of business requirements has brought new challenges to the data model in terms of spatial and temporal complexity, which is mainly manifested in the response speed of report query and self-service analysis, the efficiency of data delivery between headquarters and branches, and the semantic consistency of data synchronization and sharing. Zhang Junjun introduced the business intelligence team of China Merchants Bank and Aloudata to jointly explore and practice agile data solutions to meet the above challenges. The solution realizes the innovation of data demand delivery mode through self-service data exploration preparation tools and massive data adaptive acceleration engine, and provides new ideas for massive data analysis. At the same time, the commercialized "dataset indicators" support the reuse of BI tools, branches and branches, and business lines, effectively promoting the precipitation of enterprise-level business logic and the consistency of use.
Zhang Junjun said that China Merchants Bank has implemented a set of data fabric agile data service system, based on the data virtualization engine, connecting multiple sources and heterogeneous data sourcesOn this basis, tools such as data preparation, analysis, and report making are providedProvide users with unified data semantic services, which can not only quickly deliver data, but also realize semantic sharing. In the future, China Merchants Bank will continue to work with Aloudata to explore the intelligent path of data analysis and promote the upgrade of business from self-service data usage to intelligent data usage.
The Noetl-driven data engineering architecture fully demonstrated by Aloudata at this conference relies on the company's team's 20 years of big data technology practice accumulation, and through the commercialization of three products, it is the first to realize the best practices of data fabric in China. Named one of Gartner's Top 10 Strategic Trend Technologies for 2022, Data Fabric is internationally recognized as a trend solution to hyperscale data management and unlocking the full value of data. However, because the technical system has not yet formed a clear understanding in the domestic market, the landing path was not clear enough. Aloudata's product and business achievements represent the most cutting-edge progress in the entire data fabric segment in China, and also provide practical experience and construction goals for the development of digital intelligence construction of Chinese enterprises in the next five years.
In the face of more complex industry scenarios, as a leader in the future-oriented data fabric data architecture concept, Aloudata will accelerate the release of enterprise data potential with advanced architecture ideas and technical advantages, and bring profound changes to the data management of the entire industry.
About Aloudata
With the mission of "making data ready at any time", Aloudata is committed to eliminating the bottleneck of data management technology, improving the level of ETL engineering automation, and helping enterprises smoothly upgrade to the next generation of big data infrastructure.
Aloudata's self-developed Aloudata Air logical data platform supports the logical integration, integration and query of heterogeneous data, and achieves second-level query response and saves more than 50% of storage and computing costs through adaptive materialization acceleration and automatic ** technologyBased on the world's original operator-level lineage analysis technology, the Aloudata Big active metadata platform enables complex data links to be seen, managed, and managed, and implemented more refined and intelligent data managementAlouData CAN is an automated indicator platform that changes the traditional model of "IT development of business requirements", supports business personnel to flexibly analyze indicators from any granularity and any dimension, and realizes "definition as development and definition as service" of indicators.