USER ACCEPTANCE TESTING HAS MANY MEANINGS
If you are irritated by every rub, how will you be polished?
—Mevlana C. Rumi
As is evident from its name, a user acceptance test (UAT) is an evaluation of a software product carried out by its target users. In order to see that the software product is functioning as required, user acceptance tests follow patterns for validating the user requirements especially.
After this formal definition, I want to expand on UAT’s meaning by adding more perspectives. Is it an activity of finding bugs, or is it an activity of gaining confidence that the product meets business needs?
From an IT test department’s perspective, it is an additional test phase for finding bugs that were missed in the earlier test levels. On the other hand, if you ask other IT departments (also the business departments), they would say that UAT is an activity of gaining confidence. Both views seem logical, but test departments need to know that UAT is not specifically done for correcting their deficiencies. A test department’s main goal should be that all the test levels before UAT are to be carried out with perfection, and after system testing these people should feel ready to go live without UAT.
The main purpose of a UAT should be having users’ real feedback and vision about a software product and allowing them to interact with the system before it goes live. Uncovering the unidentified defects from previous test activities is a nice-to-have feature of a UAT. That’s why we ask users to prepare their own user test scenarios and to not repeat the ones that were carried out before by the IT test department.
This is very important because if you rerun system tests in UAT, there is no point at which the users are involved. If these people reflect their daily habits and points of view on the test scenarios, then we can talk about a valuable UAT. Otherwise, UAT will be an just extension of system testing.
When we investigate the users’ behaviors who are participating in the UATs, it’s another story. We see several patterns. Some users attend UATs only to recommend new features, others attend to just manipulate/change how a system is designed, and others attend for nothing but to follow the test cases distributed by the IT test departments. All these user types are common in any UAT, and IT test departments need to be careful about selecting, handling, guiding, and monitoring these people. I accept that IT test departments’ responsibilities in a UAT can be seen as supportive responsibilities rather than primary responsibilities, but still we are the ones who should take care of this activity and finally receive the UAT sign-offs. This way we can roll out the software product and hand over our responsibilities to operations departments.
Before I forget, another special user type you may observe in a UAT is the emotional user. These users are attending UATs for “liking” the products instead of “accepting” them. Since liking is more intuitive and feeling based, it harder to “like” anything than “accept” it. If any product meets your basic needs, you may accept to use it, but this doesn’t mean that you will like it. Liking is an emotion that is harder to achieve and easier to lose. So if you have emotional users, explain to them the real meaning of user “acceptance” tests and guide them to be more process-driven, analytical, neutral, and objective in their test activities.
You should always remember that UAT users generally do not have any software background or solid technological knowledge. They are not 100 percent booked for UATs, and they have their ongoing business responsibilities. Testing is not their main responsibility, and they can be unaware of any testing techniques and tools or any defect-tracking software. And finally, they are outsiders! They are not directly reporting to you (to testers), and they represent the customer side.
Under these circumstances, you might as well prepare yourself for the UAT battle. Get ready for facing different perspectives, different attitudes, different targets, different motivations, different skills, and different outcomes.
You may answer the following questions first:
- Who will attend the UAT?
- What are their backgrounds/skills?
- What is the scope of the UAT?
- Which mobile platforms will be tested?
- Which mobile devices will be tested?
- How will the test devices be managed?
- What deliverables are expected to be produced?
- What is the entry and exit criteria of the test?
- What kind of test data will be used and how?
- Which tools will be used for test management and defect tracking?
- How will the users be trained for using the tools?
- Who is the decision maker who will give the sign-off?
- Who will coordinate the UAT members?
- How will the UAT reporting be done? Who will do it?
- What metrics will be included in the test report? How are they collected?
- What is a defect? What is a CR (change request)?
- Who will prepare the test cases (IT testers or business users)?
IT professional with 15+ years of experience as IT Consultant, Software Engineer, Software Developer and Software Tester for many different organizations. Published a number of papers and books within the Software Engineering profession and contribute to the field by regularly attending conferences as a public speaker.
- Sourcing Approach for Mobile Testing
- Test Execution Techniques
- Your Users Will Be Your First Guideline
- Think Wider, Test Better!
- Mobile Testing is a Challenge, Meet It!
- Convincing Your Manager To Fund Testing
- An Overview of Test Design Techniques
- Testers Never Assume; Testers Ask Questions
- Performance Testing Tools Need Calibration!
- Testing in a Cross-Cultural Team
- Be “A Little” More Flexible in Hiring Junior Testers
- Buy Tools if You Really Need Them
- Mobile Testing & The Importance of Test Metrics
- Measure Your Testing
- Improve Your Test Processes by Improving Your Test Data
- Negative Testing Can Be Really Positive
- 20 Practical Advice To Help You Better Test Your Mobile Apps
- Testers Can (!) Stop Bad Software
- Mobile Usability Testing is Not Rocket Science!