NEGATIVE TESTING CAN BE REALLY POSITIVE
I have not failed. I’ve just found 10,000 ways that won’t work.
Thomas A. Edison
Three things cannot be long hidden: the sun, the moon, and the truth.
The only true wisdom is in knowing you know nothing.
Let me tell you first what negative testing means to me. It can be any kind of test execution activity based on improper or inconvenient scenarios. It may show a system is properly working or not—so it is not different from any other test activity from this perspective—but the type of actions taken by the tester are negatively driven.
I observe some misinterpretations of the definition of negative testing. Surely, I respect every opinion, but the negativeness of this test activity is not coming from the aim of showing that some system is not functioning, rather it is coming from the way we approach the system. In this case, testers are negative, not the systems. They are basically entering invalid inputs or executing some unexpected user behavior.
Hopefully, we are on common ground now. (If not, you can still read my ideas. At least they prove that your ideas are better!)
Let me move on to the next topic. People say that testers are pessimistic by nature, but their scenarios are by default positive! How can this happen?
The answer is clear: it is all because of the test basis. To be more precise, we naturally observe requirements, analysis deliverables, business process diagrams, and functional documents all written and prepared in positive manners. It sounds logical, because we do not expect directly negative expressions in requirements. In other words, requirements generally tell us what a system will do rather than what it won’t do.
As a consequence, test scenarios tend to be positive from the nature of the basis on which we wrote them. What, then, can we do?
In my work environment, I encourage people to write at least 20–25 percent of their test cases negatively. To be honest, I don’t like such arbitrary numbers but feel that this is necessary in order to ensure that sufficient attention is given to negative testing.
Let me relate an experience I had:
In a very complex transformation project, we were about to finish functional testing. Lots of bugs were found and fixed, and some modules seemed to be mature enough for user acceptance tests (UATs). Just before we officially announced the UAT phase, some colleagues told me that they had found many defects while they were trying some unwritten negative test scenarios.
Until that moment, we were neither concentrating on nor encouraging any negative testing activity. Just after that, I did some research into negative testing, and at first, I was not thrilled. I said to myself, this cannot be that effective.
When I returned to the office, I asked the guys to show me their work. They showed me some negative scenarios, like entering five characters into a field where it should be a minimum of eight, typing letters in numeric fields, concurrently logging into the system from several different browsers and checking the systems authentication/authorization behavior, checking data limitations and attachment size controls, and so many other invalid actions.
Frankly speaking, all the test cases failed! It was really impressive to see such intense negativity in a single run, but the things I saw completely changed my mind. From that moment on, I was a fan of negative testing.
And if you wonder what happened to the project, it was postponed for three months…
IT professional with 15+ years of experience as IT Consultant, Software Engineer, Software Developer and Software Tester for many different organizations. Published a number of papers and books within the Software Engineering profession and contribute to the field by regularly attending conferences as a public speaker.
- Think Wider, Test Better!
- Mobile Testing is a Challenge, Meet It!
- Convincing Your Manager To Fund Testing
- An Overview of Test Design Techniques
- User Acceptance Testing Has Many Meanings
- Testers Never Assume; Testers Ask Questions
- Performance Testing Tools Need Calibration!
- Testing in a Cross-Cultural Team
- Be “A Little” More Flexible in Hiring Junior Testers
- Buy Tools if You Really Need Them
- Mobile Testing & The Importance of Test Metrics
- Measure Your Testing
- Improve Your Test Processes by Improving Your Test Data
- 20 Practical Advice To Help You Better Test Your Mobile Apps
- Testers Can (!) Stop Bad Software
- Mobile Usability Testing is Not Rocket Science!