Text: Li Lei, Fintech Department, China Everbright Bank, Shi Xinli.
In recent years, with the digital transformation of commercial banks, the rapid development of artificial intelligence technology, and the deepening of information technology application innovation, the financial industry has been accelerating the pace of product research and development to promote the innovation and product upgrading of its business model. In this process, how to ensure the high quality and rapid iteration of products and improve the efficiency of product research and development has become an important topic of financial technology. To this end, CEB actively explores and creates a new model of R&D and testing, and builds an independent and open automated testing ecosystem.
In the context of the digital transformation of testing services, the traditional automated testing mode has shortcomings in terms of scenario coverage, automation capabilities, and openness, as follows.
1.Automated testing applications are uneven. In the financial industry, the testing business consists of multiple stages: development joint debugging and testing stage, system testing stage, acceptance test stage and post-production business verification test, involving multiple roles of business, development, testing, and operation and maintenance.
2.The full-process automated testing capability is insufficient. The whole process of testing includes test analysis, test design, test execution, and test result traceability, and the traditional automated test is mainly used in the test execution stage, and there are still gaps in the test analysis, test design, and test result traceability stages.
3.The automated testing model needs to be improved urgently. Most of the traditional automated testing models are "workshop-style" construction models, relying on a small number of automated testers, and lacking top-level design and unified standards, which makes it difficult for automated testing to develop in the direction of scale and differentiation.
In order to support the digital transformation of the testing business and build an automated testing value chain, the Bank has started to promote the construction of automated testing capabilities from the following two aspects: one is to build an automated testing system, and the other is to integrate cutting-edge technologies to build an integrated automated testing platform.
Adhering to the vision and goal of digitalization of testing business, we have formulated a unified automated test management specification to form a standardized application promotion concept and promote the application upgrade of automated testing in our bank from top to bottom. Led by the innovation of testing technology, process and process, we provide digital decision-making support to help our bank achieve agile and digital testing transformation.
Figure 1 Automated test system.
1.Establish organization-level management norms with the goal of improving test efficiency. Establish automated test management specifications at the organizational level, promote the deep integration of automated testing and the whole life cycle of testing business, and make automation run through the whole testing process. Formulate automated test case management standards, standardize case design, manage the whole life cycle of cases, and ensure the effectiveness of case assets. Relying on management norms and management standards, we formulate a feasible promotion plan according to the current situation of our bank's system, and carry out "hierarchical, graded and phased" application promotion, which can not only ensure the comprehensiveness of automated test application, but also reflect the differences according to the characteristics of the system. At the same time, a sustainable measurement mechanism is designed, and a tracking and measurement system is established before, during, and after the event, so that the application effect can be visible and tracked in real time, forming a continuous driving force for automated testing.
2.Promote the comprehensive coverage of business systems by automated testing in a "scenario-based" mode. In order to promote the value transformation of automated testing and meet the needs of shifting left and right of testing, we adopt a "scenario-based" model to promote the comprehensive coverage of automated applications. When testing is in the whole life cycle of product development, continue to promote the left and right shifts of automated testing, explore automation application requirements, and form automated testing scenarios. In accordance with the idea of "incubating a scenario and promoting a scenario", we will continuously expand the application and value transformation of automated testing, so that automated testing can penetrate into all aspects of the testing business. The scenarios incubated in this process include general-purpose automated smoke testing, automated regression testing, manual integration testing, business product testing, business verification automation, etc., as well as full-process automation application scenarios for some specific systems.
3.Governance of the entire life cycle of automated test assets to improve asset reuse rate. Automated assets, like other business assets, need to be managed to maximize their value, and need to have the characteristics of "comprehensiveness", "accuracy", "updateability" and "reusability" according to the attributes of the assets. To this end, the Bank has established the construction and measurement standards of automated assets, carried out effectiveness testing and full life cycle management of automated assets, and provided quantifiable and traceable ways to promote the maintenance and update of automated assets, so as to promote the effective accumulation of effective automated testing assets, build a solid foundation for automated testing, and realize the value transformation of automated assets.
4.Build a new form of automated testing with new technologies and existing resources. Combined with intelligent technologies such as natural language processing, artificial intelligence recognition, precise testing technology, and big data analysis of test behavior, it supports all aspects of the test life cycle, such as test analysis, test design, test execution, test process traceability, test result evaluation, and test feedback. Through intelligent technology, the accuracy of test analysis is improved, the cost of test case writing is reduced, and the streamlining and effectiveness of test cases and the fast and accurate location of test defects are realized. Provide digital guidance and test acceleration for all aspects of the whole test process.
In order to meet the needs of automated testing under the new situation, our bank has made great efforts to build an integrated intelligent test platform. The platform is divided into three layers: front-end, middle-end and back-office. The front desk is responsible for providing test service scenarios on demand; The middle office is responsible for providing general capabilities, including unified management of interface models, automated case design, data design, general intelligence capabilities and measurement and analysis capabilities. The backend is responsible for the centralized management of various types of test tools and centralized scheduling. The access center integrates the bank's management, asset, R&D and analysis tool platforms for technical capability reuse and automation capability output. Through the above three-tier architecture, the all-in-one intelligent test platform plays an active role in improving user experience, optimizing user journey, and promoting business transformation and development.
Figure 2 All-in-one intelligent test platform.
1.Standardized case design and management. The all-in-one intelligent test platform provides standardized case structure design guidance, and supports the hierarchical design management of cases and the full life cycle management of cases. Users can use the basic atomic library - composite case - test scenario ** mode to build their own case scenarios, different levels can be synchronized between the structure or data, when there is a business change, users only need to modify the atomic case to achieve the change of all related cases. In addition, the platform can automatically analyze the effectiveness and freshness of cases, and manage the whole life cycle of case creation, use, and death. The standardized case structure, hierarchical case design, and full life cycle management of cases have greatly promoted the accumulation of "living assets" of automated testing, improved the sharing and reuse of automated testing assets between testing, development, and O&M, and revitalized automated testing at all stages of the R&D process.
2.Intelligent case writing and generation. AI intelligent technology and self-developed case generation algorithms are used to write and generate cases. Through NLP technology, users can directly use natural language to write automated test cases, so that they can write automation cases after understanding the business, simplifying user operations. Through intelligent recognition and detection technology, AI recognition and control recognition are driven by two engines, improving the adaptability of automated test scenarios. In terms of case generation, combined with years of testing experience, the self-developed case derivation algorithm, derived based on custom variable factors, and verified by comparing with the results of benchmark cases, avoids the problem of invalid cases or incomplete coverage, not only provides a reasonable range of case selection, but also ensures high coverage of business testing, reduces maintenance costs, quickly locates problems, and realizes efficient test delivery.
3.Long-link test execution at scale. Due to the high complexity of the bank's business process, a complete business process needs to run through multiple heterogeneous systems, in order to ensure the automatic testing of the complete business link, the Bank has developed a long-link test model, through which the transformation of the business link to the long-link automated test is realized, and the timing of the execution of the long-link automated test in the heterogeneous multi-terminal scenario, the problem of parameter transmission in the heterogeneous environment and the continuity of execution are solved. At the same time, the automated test platform manages various types of cloud executors, and dispatches different types of executors through the dispatch center to achieve large-scale concurrent testing of long-link cases.
4.One-stop testing service scenarios. The construction of the integrated intelligent test platform has realized the evolution from "tool" and "middle platform" to "service", and through the integration with a variety of tool platforms, the user journey is optimized and one-stop intelligent testing services are provided. Taking a test requirement as an example, after the test corresponding to the requirement is proposed, an automated test task is automatically constructed, and the test design is covered in the following two ways: one is case automatic recommendation, which automatically recommends automated cases related to test requirements through business change impact analysis, risk identification analysis, and change impact analysis of the requirements. The second is automatic case generation, which automatically generates positive and negative cases that cover interface attribute information through the case expansion function, expanding the coverage of interface attribute information and business logic. After the automated test set is built, you can start the automated test execution, and check the test coverage of the change scope in real time during the execution process. For cases of failed test execution, users can be assisted in locating and analyzing defects through automatic failure log extraction, running trajectory analysis, etc., and the confirmed defects can be uploaded to BMS with one click. The one-stop testing service scenario builds a new testing ecological chain, enabling users to quickly access the capabilities, tools, and services required for testing, and accelerate the transformation and upgrading of testing.
Under the new form of automated testing, an independent and open testing ecosystem has been built, and the left, right, and in-depth development of automated testing has been realized, forming a new automated testing model of "automation for everyone" and "automation everywhere", which has helped the digital transformation of our testing business.
Up to now, the platform has achieved 100% automated test coverage of the bank's important systems, with a transaction coverage rate of more than 80%, accumulating 100,000 effective automated test cases and more than 500,000 executed test cases. After the introduction of intelligent testing technology, the efficiency of automated case writing has been increased by 2 times, the maintenance workload has been reduced by 60%, and more production capacity has been released. Accurate demarcation analysis enables test risk to be pre-empted, test coverage increased by 30%, and test requirement analysis accelerated by 10%. The efficiency of automated test problem analysis after integrating agile troubleshooting solutions and abnormal alarms is improved by 25%.
In the future, the Bank will continue to focus on the development vision of "accelerating digital transformation and building digital Everbright", continue to explore new technologies and innovate business models, temper digital competitiveness, and empower business development.
This article was published in the first half of January 2024 of Electronic Finance).