As testing is executed by the Applause Community, the testers will be submitting issues when encountered. The Value of the issue is logged during the triage upon approval, by your TTL and later potentially other team members and even you. Having an accurate estimation of the value of issues is purposed to help you to provide a clear feedback to your Applause team on their work, and, ultimately, the value you see from implementing Applause’s crowdtesting model. Over time, trends in the overall value of issues submitted may draw your attention, specifically when you see a significant decrease. Such shift in value is likely to negatively reflect on your build’s Applause Quality Score (AQS), and you may find yourself asking why – and what to do next.
First, it is important to acknowledge that the submitted issues are the symptom, not the cause. The value of submitted issues is commonly linked to the way your Applause team understands your product offering, the intended users and use cases, and their impact on the quality of the tested product, website, app or software build. It is always important to understand your test results, and it is especially critical to do so when there is a decrease in the overall value of issues.
Here are a few directions to follow up on when investigating the decrease in overall issue value:
- Were there any recent changes to the development process that may have resulted with improved quality? Such changes may be in processes, personnel or tools, and are clearly desirable. While such changes tend to have a more gradual impact, a sudden reduction in reported issue value is certainly a possible result as well.
- Was there a change in the way testing instructions were composed and/or delivered to the testers? Lacking, unclear or confusing instructions not only prevent testers the full understanding of your product, use cases, and their impact, but may also discourage the testers from participating in future testings of your product.
- Was there a recent change in personnel – specifically in the interface between you and the Applause team – that may have resulted with lost knowledge or misaligned perspectives? Changes are inevitable, yet documentation and knowledge transfer are key to ongoing success.
- How clear are the release notes provided to the Applause team in describing how the product and/or new functionality will be used by the end users? Value is often subjective, and members of your Applause team need to understand what you care about.
Whatever the issue, you are advised to collaborate with your Applause team to further troubleshoot the causes for the decreased overall issue value, and identify improvement opportunities in your processes and testing strategy.