A Useful Guide to Judging the Value of a Bug

Ask yourself two questions:

• Does this bug fall in line with the expectations of the cycle?
• Does this bug impact my application?

When the answer to the above two questions is generally yes but you are either unable to reproduce the issue yourself or cannot prioritize the issue to be fixed, designating the bug as “Won’t Fix” will still credit the tester for their work.

Use Your Test Team Leader (TTL)

• The goal of the TTL is to speed up your triage process and manage the testing group
• TTLs are asked to review and replicate every issue within 24 hours of filing
• For each issue you should see a recommendation broken down into the following: 
  • Could they reproduce, was it a duplicate, is it in scope, do they recommend approval or rejection, and what value

• The TTL is asked to verify that every report:
  • Has all the information you requested
  • Has provided clear steps
  • Follow our reporting standards
• The TTL may also leave a comment under the recommendation section to explain it further.
• The TTL is your immediate source to help redirect testers or modify instructions. Feel free to reach out to them if something needs to change during a cycle.

Triage Early in the Cycle

When you start approving issues, testers can learn what is valuable to you, and what is being approved and rejected. Doing this early while the cycle is active is a means of keeping them focused and engaged.

Do Not Fix Bugs Before Approving Them in the Platform

This will make it harder for you to keep track of the status and progress of issues, and it may also confuse things for the testers, especially if you attempt to run a bug fix verification cycle later on. 

The smoothest flow for you would be to review the triaged bug results in the platform from the TTL, make your final bug value determination, push the results to your BTS and then assign to your developers to fix. Once fixed, we can run a Bug Fix Verification cycle for testers to confirm the fix was successful.

Triage Bugs Before or Shortly After Exporting Them to BTS

Integration with your Bug Tracking System will save you tons of time and energy in sending discovered issues to your developers. Although you can export bugs into your BTS prior to approving them in the Applause platform, keep in mind that testers typically get compensated for their bug finding efforts, so it’s important to keep the community engaged and interested in your product by going back and approving those bugs shortly thereafter.

Bug Severity vs. Bug Value

• There are times when the issue itself may not be that severe to the site because of limited scope or a mature product, but the tester has found something directly related to the task given in scope. In those situations it may be appropriate to evaluate the bug based on being in scope rather than the priority level it will be assigned.
• Occasionally you will see some high quality testing that requires a lot of effort but the value of the issue is not that big a concern. Feel free to reward good testing/reporting as well as high priority issues. This keeps them going for more.

Balancing Somewhat Valuable and Very Valuable bugs

Testers are paid according to your value of the bug and the scale is significantly weighted to incentivise them to look for high value bugs. Approving bugs as very or exceptionally valuable is telling the testing pool “this is what I want to see more of” – when it is appropriate.

• Approving too many bugs as Somewhat Valuable - This might discourage testers from testing since Somewhat Valuable is the lowest value and might not help increase a tester’s rating.
• Approving too many bugs as Very/Exceptional Valuable - If too many tickets are overvalued, you will see testers throw a lot more against the wall and you may see them “over reporting” the same issue in different locations in order to garner another high payout.