In addition to following the [ilink url=”https://help.applause.com/bug-triaging-best-practices/”]Bug Triaging Best Practices[/ilink], in order to get the most out of your cycle we also recommend the following tips:
- If you make an update to your app during testing, inform testers immediately. Use the “Announcements” portion of the cycle chat to let your testers and PM know in real time if anything major has happened or changed during the active testing time period that may effect testing efforts.
- Think carefully about making changes to scope during active testing. Once the scope has been established and testers have had a chance to review it and accept the cycle, it is not recommended to make scope changes. If it is necessary, it should only be done for minor changes. If you have a need for a major scope change, talk to your PM, as spinning up a brand new cycle may be a more appropriate course of action than modifying the existing cycle.
- Discuss expectations with your PM and TTL upfront. Typically you will see bugs start to come through after the first few hours of a cycle’s activation, and TTLs will usually start triaging bugs after about 24 hours. This could vary by project, so it’s important to discuss this process and your expectations in advance.
- Raise a red flag right away if you’re not seeing what you expect to in a cycle. Let your PM know if you’re not seeing as many bugs as you expected, if you encounter a questionable tester, or just generally if you think something is off. We can quickly act to correct this, swap out testers, clarify testing focus, etc.
- Get to know your testers! Don’t be a stranger – they are real people, too. You can learn more about your testers by checking out their uTest profile — simply click on the tester’s name either in the Dashboard, the Testers tab in the left navigation bar, or directly within the bug report.
- “Favorite” your best testers. Have you noticed a rock star tester in your cycle? Would you like to see them participate in future cycles? By [ilink url=”https://help.applause.com/favorite-tester”]favoriting a tester[/ilink], they are more likely to appear in your future cycles.
- Don’t be afraid to ask testers to provide more information. Testers are contractors for uTest and are testing on their own will, but that doesn’t mean you can’t ask them for clarification or push them to expand (as long as it’s within scope). Treat them like they are just an extension of your own team and reach out to them through the Platform directly when you need to.
- Triage early and often! If you’re able to triage bugs early on in the cycle, and with consistency, you may notice a big difference in the quality of results you are seeing over time. Assigning values to bugs quickly will show active testers exactly what you’re looking for, and lead to more of those great results.
- Do not fix bugs before approving them in the Platform. This will make it harder for you to keep track of the status and progress of issues, and may also confuse things for the testers, especially if you attempt to run a bug fix verification cycle later on. The smoothest flow for you would be to review the triaged bug results in the Platform from the TTL, make your final bug value determination, push the results to your BTS, and then assign to your developers to fix. Once fixed, we can run a bug fix verification cycle for testers to confirm the fix was successful.
- Be careful about exporting non-approved bugs to BTS. Integration with your bug tracking system will save you tons of time and energy in sending discovered issues to your developers. Although you CAN export bugs into your BTS prior to Approving them in the Applause Platform, keep in mind that testers typically get compensated for their bug finding efforts, so it’s important to keep the community engaged and interested in your Product by going back and Approving those bugs shortly thereafter.
- Test one bug for BTS export before doing it in bulk. When you want to export your approved bugs to your BTS, before sending many over at once, try sending just one at first. In case there is a failure, you won’t have to redo anything major, and you’ll avoid getting bombarded with failure notifications. If there is a failure, you can try troubleshooting by reviewing the help topic on [ilink url=”https://help.applause.com/how-to-integrate-with-bug-tracking-systems/”]How to Integrate with Bug Tracking Systems[/ilink] , or reach out to your PM for help.
- Just because you can’t reproduce a bug doesn’t mean it’s not a bug. If a tester submits a bug and you are unable to replicate it, consider that there are several factors that could cause this, the most common of which is the environment on which it was found. If you run into this, we recommend that you lean on the detailed reproduction steps, how often it was found to be reproduced, and the screenshots & videos provided by the testers. If you see it’s clear that the bug occurred, the tester should get credit by having their bug Approved. Remember that bug Rejections hurt a tester’s rating, so don’t necessarily reject a bug because you can’t reproduce it on different environment. If you feel the bug isn’t important enough, or if you’re OK with it occurring on just one specific environment, you can always elect not to move that bug over to your BTS.