User Acceptance Testing Challenges

Client UAT (user acceptance testing) is a constant challenge for a services firm like AWH. UAT challenges are not reserved for firms like ours, however.

Clients often underestimate the time, energy, and work needed for user acceptance testing. This is true for external clients, but it is also true for internal clients at a company.

User acceptance testing isn’t new, and clients who have worked with product teams previously to create or evolve products should be and in some cases are better at it, but most still struggle to fully value and get good at it.

Our challenge as a services firm working with clients who don’t create software products frequently, if ever before, is that they don’t understand and value user acceptance testing. This causes clients to discount their focus and commitment to doing it. Clients will often assign user acceptance testing to a team member or a few team members who already have full-time roles. Therefore, the user acceptance testing gets deprioritized, done last minute or after hours, causing it to be done inadequately, incompletely, or untimely. User acceptance testing is a crucial part of the process, and when it isn’t performed well and promptly, it causes unnecessary thrashing.

So how can clients work in their and a product’s best interest around user acceptance testing? Here are some tips:

• Dedicate team member time: It doesn’t work to have UAT performed as an additional responsibility for team members who already have full plates and schedules. UAT will not be performed well, and the product will be viewed negatively by those who are asked to perform UAT in addition to their regular work, especially if it means after-hours work.
• Single point of contact: Multiple team members can be doing UAT, but the feedback and findings need to flow through a single person. Too many people trying to communicate and collaborate with the product team on issues or clarity of functionality creates confusion. One person should be responsible for distilling and communicating with the product team on any UAT issues.
• Context: One of the reasons to have a single person manage UAT findings is that this person should have the context of how the functionality of a product is expected to work and why. Not just at a basic functional level but at an experiential level too. This helps speed the communication and remediation of an issue with the product team. It will also allow the person to prune the list of issues before involving the development team.
• Timely: UAT happening promptly is critical because the more time elapses, the more it impacts a product's timeline, the more context gets lost, and the more expensive the effort becomes. UAT that happens randomly or at unexpected times isn’t accounted for as part of a production schedule and roadmap, so it will cause the product team to have to adjust. It will also create workflow and staffing challenges for a product team expecting UAT to happen at a particular time and pace. Unexpected timing and workflow from UAT send product teams into reactive mode instead of a planned mode. This causes friction around designer, developer, quality assurance, and DevOps involvement and scattered work rather than intentional.
• One place to track findings: All UAT findings should be tracked in one place. It doesn’t matter where so long as the UAT lead and product team agree on where and how. The tool for tracking UAT findings matters less than it happens in one place in an agreed-upon manner. Too often, UAT findings are being tracked and documented differently by team members performing UAT. The UAT lead needs to enforce where and how UAT findings are captured. This is another reason to have a UAT lead.

Product Managers need to ensure they have a sound and cohesive UAT plan with a client (external or internal). The plan needs to align what, who, how, and when UAT will get performed. It also needs to be discussed and agreed on how UAT will be prioritized against other work for the testing team members. Team members performing UAT should never be unclear about its priority compared to other work. They should also never conflict with UAT versus other work or personal matters. UAT will lose to competing work and personal time every time.

The importance of timely and well-executed UAT must be reinforced before and throughout testing. Too often, it gets treated as an afterthought by product teams and testers. The product teams want to release, and testers wish the work to go away and the associated annoyance. The push to release is essential. Everyone involved in a product should be operating with a sense of urgency, but the push to release without reasonably practical and thorough testing doesn’t serve anyone well.