Is Automated Regression Testing (ART) the saviour of your business?

December 3, 2014

There’s an old adage, ‘if it ain’t broke don’t fix it,’ but what happens when you want to enhance your existing software? ART could just be exactly what you need.

ART is an automated suite of software regression tests that are carried out to ensure that the changes or enhancements that have been made to the software haven’t broken its existing functionality.

It offers a range of benefits, including:

  • Preventing reputational loss – If the new version of your software is not thoroughly tested and an existing functionality is broken, your reputation is at significant risk.
  • Preventing financial loss – A broken existing functionality can result in huge financial losses with system defects undermining your business.
  • Preventing loss of time and human effort – There is a high risk of error where regression testing is carried out manually, and typically a lot more effort and lead-time is required for each release.
  • Accelerating your business – ART ensures faster, safer improvements to your business.

ART clearly offers huge benefits, but how easy is it to build, particularly for legacy systems and/or software that customers are already using? In these instances, ART needs the following:

  • A clear set of requirements.
  • Time and involvement from the developers as well as the application owners.
  • In some cases small adaptations (like change in field object properties) in the applications.

Most of the legacy applications will have limited or no requirement documentation, and application owners are likely to be very busy with continuing production and maintenance issues. It is likely that the project team will be hesitant to perform even small changes at short notice because they have to analyse the impact of changes in advance. More than 90 per cent of the applications that are critical to business are legacy systems. Reflecting this, the challenge of implementing ART can be significant and can cause difficulties in aligning client expectations throughout the project to ensure that they get quick return on investment. As Brickendon has discovered however, with the right approach, these challenges can be addressed and the fruits of ART fully realised.

Ground Zero – A working example

A large European investment bank had a legacy software application in the form of a customer-facing trading system. Every time a change was made to the application, a huge testing effort was required to ensure that the new release had not affected the system’s existing functionality. Failure to do this correctly could severely compromise the bank’s reputation, in turn leading to huge financial losses. Manually, this testing process was time-consuming, unreliable and was costing the company considerable time and money. The challenges included:

  • Starting the whole testing project from scratch because there were no business requirements documents in existence.
  • Ensuring that all requirements were forecast in advance and that all issues were highlighted upfront.
  • Ensuring that enough time was allocated to accommodate automation requirements because the application owners were themselves very busy in production maintenance.
  • Managing client expectations and ensuring that their requirements were reasonable, particularly given that this was the department’s first automation project.
  • Building all test cases from scratch because the application had no existing manual test pack.
  • Ensuring accountability and clarity at all stages of the project.
  • Adhering to the client’s expectation of a quick return on investment.
  • Providing enough time for the stakeholders to adapt and perform small changes (like object properties) in the legacy system to successfully complete the project.

Resolving the challenges

To resolve these types of challenges, one approach is to implement an effective test checkpoint system to ensure that the test scope, automation requirements and accountability are clear to testers, developers, other stakeholders and sponsors. This approach makes the process clear to everyone involved, ensuring that all parties are on the same page from the very start.

It is best to get all stakeholders involved in the process from an early stage, a move that fundamentally helps to meet client expectations. This can be achieved by providing the stakeholders with a defined project scope and/or brief as well as clear timeframes for the project. Any amendments sought by the client can then be made before the actual test implementation. This in turn, eliminates the need for additional reviews and reworks, which have the potential to prolong the project and delay the processes.

To provide the client with further benefits, an efficient move is to implement a reusable test automation framework and a mechanism to obtain maximum test coverage with minimal code. It is also a good idea to implement a methodology to integrate all the test assets. This helps to reduce the manual intervention in the testing process, improve maintenance and achieve faster test results analysis.

To ensure that the project deliverables are delivered on time and transparency with the client is maintained, it is best to modularise the deliverables and implement a three-dimensional level tracking system.

According to Bala Ethirajalu, senior manager at Brickendon, the firm has set about solving these issues by developing “a set of innovative methodologies that are not available off the shelf”. Ethirajalu goes on to explain that: “These included our Time Check Point System (TCPS), Functional Test Automation Solution (FTAS) and ITTA (Integrated Test Tools Approach). These methodologies helped us to deliver the project on time with highest quality. Through the three-dimensional tracking approach, we were able to efficiently manage all project issues and changes, and still deliver the project on time. Most importantly, we won the confidence of the project stakeholders. We were able to reduce the testing efforts to just overnight from a three-week testing time lag for every release.”

Notably, banks have started favouring horizontal system implementation to reduce overheads. With cross-asset integration however come additional complexity and a higher regression-testing burden. These factors mean that now more than ever, the sector needs a structured approach to ART. Without this there will be more outages and a much longer time to market.

Become a Brickendon Change Leader

What can we help you achieve?