Test Metrics Solution

February 18, 2015

One clear goal of a project lifecycle is to identify and address test issues and defects prior to the go-live. Brickendon Consultants identified the ‘bottleneck’ problem, where an incomplete resolution process leads to delays in the test and project cycles.

What happens during the Project Lifecycle

Screen Shot 2015-06-11 at 10.07.58Within the testing stages of a project, the Test Cycle includes Test Design, Test Execution and Defect Resolution Management stages.
As information on issues or defects is exposed through the testing process, they are evaluated according to their nature, severity and priority levels and a resolution agenda is established. Business Analysts analyse these issues or defects and Developers resolve them, allowing the Test Analysts to confirm resolution completion by running a re-test procedure.

Problems exist when issues and defects are not addressed in a timely manner in the project cycles due to an obstruction in the process, known as a bottleneck. The challenge in Project Management is the identification of bottleneck areas and the need to prevent or swiftly resolve these issues.

The Test Metrics Solution (TMS)

Screen Shot 2015-06-11 at 10.08.44
Brickendon Consultants constructed and validated a mathematical formula to systemically capture the conditions of bottleneck points, based on priority and established dependencies. The TMS provides a transparent progress measurement tool by generating a relatively holistic perspective of how the Project Lifecycle will evolve. This greatly increases the ability to effectively and efficiently coordinate the Project Lifecycle and prevent bottlenecks from occurring.

The TMS mechanism works in three stages: Data Collection, Analysis and Diagnosis (DcAD). The DcAD elements of the TMS encompasses the Test Design, Test Execution and Defect Resolution Management stages in a Test Cycle. The process begins with the collation of planned and actual resource data through the Design-Execute-Evaluate (DEE) stages of a Test Cycle. The TMS then analyses this data using the mathematical formula, examining the variance and observing any underlying patterns. Results highlight bottlenecks well in advance, allowing the reorganisation of resources and reformulation of the Test Design, Test Execution and Defect Resolution Strategy in a Project Lifecycle.

Design
  • Test Case Design Delay
  • Accuracy of Test Design Estimation
  • Test Design Completion
  • Rate
Execute
  • Test Case Execution Delay
  • Accuracy of Test Execution Estimation
  • Execution Completion Rate
  • Execution Pass Rate
  • Execution Failure Rate
Evaluate
  • Average Defect Analysis Efficiency of a Defect
  • Defect Fixing Progress
  • Business Impact of Defects
  • Average Fixing Turnaround Efficiency of a Defect
  • Average Retest Efficiency
  • Quality of Defect Fixes Defect Density

Strength of the TMS
This TMS provides a reliable tool that highlights underlying bottlenecks which frequently go unnoticed. Variance analysis is key in enabling bottleneck identification during the Project Lifecycle. The comprehensive metrics provide sufficient information for project teams to efficiently collaborate for test design, execution and defect resolution.

Once implemented this methodology can be used throughout the Project Lifecycle.

Client Benefits
  • Benefits to a client organisation include:
  • Early detection of issues in design and execution stages
  • Timely Test Issue/Defect Resolution;
  • Enhanced transparency in ensuring project quality;
  • Evidence-based Performance Management.

Become a Brickendon Change Leader

What can we help you achieve?