National Health Service Case Study
Rapid implementation of service layer test automation using an IDS interface automation strategy powered by Axe and Intelligent Delivery Services iData. Learn how IDS’ Technical Testing services delivered a repeatable regression testing solution for a Health Service programme in the United Kingdom.
If you require more information on Intelligent Delivery Services visit their website.
The Challenge:
Testing was struggling to keep pace with the delivery of the application. IDS were tasked with Developing a Leaner, More Cost-Effective Testing Process
The Test, Release Management and Domain Leads on the programme, recognised the importance of completing regression testing of the application quickly, but the traditional manual user interface focussed approach they adopted was becoming a costly bottleneck.
The requirement from the client was to recover the lost time in testing due to unforeseen blockages.
IDS used an automation process to data drive test scripts from an external excel spreadsheet into a predefined request and response XML service definition forming the processing and validation elements of the testing process.
Compounding the inherent challenge of manual testing was the growing complexity of their testing coverage. The team had an overhead of 40-man days testing to complete for each deployment of the application. This, coupled with the heavy involvement of manual related results validation, exposed the manual testing approach to heavy timescales and potential human error which could be catastrophic for the organisation.
Faced with the increasing cost and complexity of their legacy testing approach, the programme benefited from the support of IDS to help deliver a leaner, more cost-effective process through IDS specialist automated testing and continuous delivery capabilities.
The requirement from the client was to build an automation process that would scrape data from an Excel exported XBRL file into a 2D table. Then it would use a 2D table setup and pivot tables in Excel, which connects to Cube (Microsoft SQL Analysis Server), to compare the two tables and report any failures.
On top of the strain that manual testing creating was the growing complexity of their testing coverage. The team had an overhead of 15 man days to complete for each monthly deployment of the application. Coupling this timescale with heavily involved data related validation of results exposed the manual testing approach to human error which could be catastrophic for the organisation.
Faced with the increasing cost and complexity of their legacy testing approach, the bank has requested the support of our partner organisation to help deliver a more cost effective process through automated testing and continuous delivery.
The Solution:
Automated Regression Testing
Implementing the solution
Upstream and downstream applications had been subject to delays and were not available. Given that the project had already been subject to significant delays, it was imperative that testing was completed on time, without any reduction in quality and coverage.
Knowing the architecture design, we decided that the most efficient approach would be to focus on the service layer and API for testing. We didn’t have to wait for the availability of the application as we could create our test assets ahead of time.
Using the applications WSDL’s (Web Service Definition Language) and XSD’s (XML Service Definitions) we set about creating our test templates using the Harness provided the Axe automation platform tool. This followed with a test data management phase which coupled test data and test scripts together.
As this testing had previously been completed manually, it would take up to 3 weeks to complete. With this solution, it has been calculated that the entire test process could be completed in just 1 day and without the need to transfer knowledge to new onboarded staff.
If the client had chosen to execute the same level of automation using manual techniques the cost would be £72,000. In a like for like comparison they are making an immediate yearly saving of £67,200 using the solution.
With a clear understanding of the costs and return on investment opportunities available, we began the process of establishing an automated solution for the execution and re-execution of tests that will contribute to the overall Application Lifecycle Management (ALM) process.
There had been limited success with automation previously, but with our background in automated testing, it was felt that using specialist help to define and implement a standardised approach would prove decisive.
The Outcome:
Eliminating the testing bottleneck, cutting costs and increasing quality
- Our solution was to make use of Axe Harness in its simplest form with no customization
- Building test assets ahead of time enabled us to be ready when interfaces became available
- Significant time was saved and faster feedback on the state of application quality
- Reporting of the executed suite was done within the Axe tool
- We were able to expand on the test assets (Object Mapping and Subtests) prior to having access to and/or the application being available.
- The client now has the benefit of a fully functioning, repeatable regression test pack working as part of a continuous delivery model
- Enabling the client’s development teams to Build-Deploy-Test as often as required, resulting in much more robust build, deploy practices, and increased confidence in the application being deployed
Having achieved ALM, the bank have increased the volume of testing as well as improvements in:
- Build & deployment reliability
- Overall Quality
- Better on-demand reporting metrics
- Efficient testing
- Testing practices
Through the specialist implementation of a test automation solution powered by Axe, the bank now benefits from a considerable saving in staff utilisation, freeing up skilled resources to focus on manual and exploratory testing or other areas of concern.
The quality of testing is also greatly improved with the client having greater confidence in the application under test as the possibility of human oversight is eliminated.
