MEASURE YOUR TESTING
I was always a numbers guy, who goes in for showing everything with quantities and referencing outcomes to objective measures. In private life, it is sometimes hard and sometimes very easy. In order to show my love and dedication to my wife, I always remind her with numbers. Sometimes these can be the dates of our anniversaries and sometimes they can be the number of days we haven’t seen each other.
In testing, these measurements are easier to extract and present. When you are doing any unmeasured or unquantifiable testing activity, you need to be very careful. Without any metrics and measures, software testing becomes a meaningless and unreal activity. Imagine, in any project, that you are not aware of the total effort you have expended and total number of defects you have found. Under these circumstances, how can you identify the weakest points and bottlenecks?
I always remember and respect the famous saying: “If you want to improve something, you need to measure it first.” And I truly believe this. If you want to show the bad as well as the good, you need to measure your testing and present quantifiable results. In general, logical and clever managers require numerical evidence; they are not satisfied with any nonnumeric information.
Let me list some of the favorite test metrics that I love to capture and display. I believe some of them should be very easy for you to get, and for the rest you may try your chances. They are:
• Total Test Effort and Test Efficiency (including schedule and cost variances)
• Test Execution Metrics (# of Pass, Fail, Block, Inconclusive test cases)
• Test Effectiveness (# of defects found in system test / total # of defects found in all phases)
• Total # of Defects, Priorities, Severities, Root Causes (system test, regression, UAT, and live)
• Defect Turnaround Time (time required for developers to fix defects)
• Defect Rejection Ratio (ratio of false/invalid defects)
• Defect Reopen Ratio (ratio of unfixed/failing “ready to test” defects)
• Defect Density (per development day or line of code) and Defect Detection Ratio (per day or per testing effort)
• Test Coverage, Requirement Coverage, Requirements Stability, and some others you may like more…
If you collect and analyze the above metrics accurately, you will be several steps ahead of others who don’t.
IT professional with 15+ years of experience as IT Consultant, Software Engineer, Software Developer and Software Tester for many different organizations. Published a number of papers and books within the Software Engineering profession and contribute to the field by regularly attending conferences as a public speaker.
- Think Wider, Test Better!
- Mobile Testing is a Challenge, Meet It!
- Convincing Your Manager To Fund Testing
- An Overview of Test Design Techniques
- User Acceptance Testing Has Many Meanings
- Testers Never Assume; Testers Ask Questions
- Performance Testing Tools Need Calibration!
- Testing in a Cross-Cultural Team
- Be “A Little” More Flexible in Hiring Junior Testers
- Buy Tools if You Really Need Them
- Mobile Testing & The Importance of Test Metrics
- Improve Your Test Processes by Improving Your Test Data
- Negative Testing Can Be Really Positive
- 20 Practical Advice To Help You Better Test Your Mobile Apps
- Testers Can (!) Stop Bad Software
- Mobile Usability Testing is Not Rocket Science!