Reporting issues – and working with them – is a major function of the crowdtesting model enabled by Applause and the testing community. During the test cycle, the testers are testing your product guided by the test cycle instructions and supervised by your project team at Applause. As they encounter issues they will do their very best to provide a clear, complete and accurate documentation of it. Once results are reported, you will be able to view, triage and rate them in the Applause Platform.
Viewing Issue Reports
You may either view issue reports within a test cycle context, or from a global list containing issues reported from all cycles.
Test Cycle Context
In order to view the issue reports collected in a specific test cycle, follow the below steps:
1. Log in to the Applause Platform and navigate to Test Cycles.
The Test Cycles page will be displayed. Basic information on the issues collected in the test cycle is already presented in the list view, such as the total number of issues, and of which, number of new and pending issues.
2. Search for your test cycle in the list. You may use the search box at the top, show/hide Closed cycles, and/or re-sort the list by clicking on the column headers.
If you are logged in to an Agency account, you may also select to show/hide all agency test cycles.
3. Click on the test cycle name to view the complete test cycle information.
4. Click on the Issues tab to access the issues collected in the test cycle.
The Issues List will be displayed, containing the list of issues reported by the testers during the cycle, their status, severity & value rating, BTS identifier (if exported) and BFV status (if verified). You can filter the list by a variety of fields, show/hide issues not exported to BTS, search by issue ID (in Applause or BTS, if exported), or run a textual search.
5. Locate a specific issue, and click on the issue title to view the complete Issue Report
Global Issues List
In order to view the issue reports collected across all test cycles, follow the below steps:
1. Log in to the Applause Platform and navigate to Issues.
The global Issues List page will be displayed, offering similar functionality to the test cycle context one.
Issue Report Content
The Issue Detail page allows you to review the tester’s findings and corresponding technical data, approve or reject the bug, or request more information as needed. Issue highlights are presented at the page header, while complete report and communication options are under the Description tab. A complete change log is available under the History Tab.
The following issue information is available in the header (some fields may not be displayed depending on the current state of the issue):
- Tester: The name of the tester that submitted the bug. Clicking the testers name will bring up the testers profile. Additionally, you can click the blue badge next to their name to mark them as a Favorite Tester.
- Environment: Where the bug was found – either a mobile device, web browser, or the like.
- Build: The build of the product the issue was found on.
- Product: The name of the product being tested.
- Issue Type: What kind of issue it is – either Functional, Visual, Technical, etc.
- Source: What testing method was used to find the issue – either Exploratory or Test Case.
- Severity: An estimate provided by the tester on the impact the issue might have – either Critical, High, Medium, or Low.
- Resolution: Whether or not the issue has been resolved.
- Export: In case the issue was sent to your Bug Tracking System (BTS), the issue’s ID in your BTS will be displayed.
- Status: An indication on the triage state of the issue – either New, Approved, Rejected, Pending, etc.
- Approval: The value of the issue, assigned by the user approved the issue during triage.
- Verification: An indication on the fix verification state of the issue – either Not Requested, Fixed or Not Fixed.
The content provided by the tester consist the comprehensive details of the issue report, and includes the following information:
- Recommendation: List of outcomes for initial triage by your Testing Team Lead (TTL), ensuring the issue was in scope, it is reproducible, not a duplicate reporting. The TTL will also suggest the issue value.
- Frequency: How often the issue occurs.
- Action: The steps the tester followed to find the issue.
- Expected Result: What the tester feels should occur once the Action was taken.
- Actual Result: What actually happens when the Action was taken.
- Error Result: The error message displayed to the user, if any.
- Additional Information: This can be used to detail whether or not the tester was on a Wi-Fi connection or Mobile, the Fix Verification status of an issue, what browser was used, or any other information the tester feels is necessary to pass along.
- Attachments: Screenshots, videos or log files relevant to the reported issue. Often your TTL will describe the attachment requirements to the testers in advance.
- Community Reproductions: List of additional reproductions of the issue, commonly by other testers and environments. Note that testers are not obligated to reproduce other testers’ work, however they will gladly document their reproductions should they try to submit a bug and find out it was already submitted.
In addition, the Description tab also includes two communication channels you may use:
- Tester Messenger: Allows you to correspond directly with the tester submitted the issue. This is a great channel to ask for clarifications, for instance on credentials used, steps taken, etc.
- My Notes: Allows you to log notes for yourself, the Project Manager or the TTL, without involving the tester. This provides opportunity to clarify project- and testing-related matters, such as understand TTL recommendation, discuss issue rejection reasons, etc.
The History tab provides a track record of any and all changes made on the issue. All changes, from something as minor as a text field to as major as a change in status are stored and tracked on this page.
Triage & Rating
To ensure only relevant, accurate and complete issues are provided by Applause to your team, all results submitted by the testers need to be triaged, and eventually either approved or rejected. The triage is a critical task, impacting both you and the quality of results you’ll be getting from Applause, as well as the testers’ rating (and payout).
During triage, you may use the issue report to complete the following tasks:
- Edit the issue if needed so it is at par with your content expectations
- Ensure the issue severity is properly set
- Leave notes to the Testing Team Lead and the PM if needed
- Message the tester, in case you require further information on the issue. Testers will be given time to respond, yet the issue will be automatically rejected when the test cycle will be closed by your Testing Team Lead or PM.
- Review the issue report content, and approve or, if it is invalid, out of scope, or the tester didn’t follow instructions, reject it
Note that your TTL is able to assist you with initial triage, and per your request either recommend approval/rejection, or complete those instead of you.
Best practices for issue triage are available here.
To triage issues:
1. While in the Issue Report page, click on the Actions button.
2. Select one of the following:
- Edit Issue: Apply changes to the issue report details. Make sure to save your changes.
- Approve: Approve the issue as a valid one. You will be asked to then set the issue value: somewhat, very or exceptional.
- Approve as Won’t Fix: Approve the issue for the sake of tester rating and payout, yet there is no intention on fixing it. The issue value will be automatically set to ‘somewhat’.
- Reject: Reject the reported issue as invalid. You will be asked to set the reason for rejection: duplicate reporting, need more info, tester did not follow instructions, issue is out of defined scope, scenario works as designed, or any other reason. You are encouraged to still add this issue to your Known Issues list to prevent future reporting of it.
- Request More Information: The issue report is lacking or unclear, and you’d like the tester to update it.
Please note: After an issue is approved, the Approve, Reject, Request More Information options will be grayed out and inaccessible.
Further Managing Issues
For Issues, you may also manage the following attributes:
- Fix worthiness: Mark/unmark the issue as “Won’t Fix”. Won’t Fix issues will be tracked to prevent testers from re-reporting the issue, yet there is no intention on fixing it.
- Known Issue: Mark/unmark the issue as a “Known Issue”. Known issues will be tracked to prevent testers from re-reporting the issue, and to allow a fix verification once fixed.
- Resolution: Mark/unmark the issue as “Resolved”. Resolved issues are considered fixed and verified, thus no need to keep tracking them as known issues.
To further manage issues:
1. While in the Issue Report page, click on the Actions button.
2. Select one of the following:
- Export: Export the issue to a .csv file, or to your Bug Tracking System (if the integration was set up
- Request Verification: Once the reported issue is fixed, you may want to utilize the testing community to verify the fix, as a Bug Verification workflow. You may initiate the BFV in an existing or a standalone test cycle.
- Learn more about how to initiate the Bug Fix Verification workflow and the different BFV methods offered in the Applause Platform here.
- Mark (or Unmark) Won’t Fix: The label “won’t fix” will be displayed next to the issue status as well as to the Resolution status. Sometimes when a bug is submitted, extenuating circumstances prevent a valid issue from being fixed. When this occurs, you can mark the issue as Won’t Fix. This still triggers a payment to the tester, as it’s a valid issue, but you can go about without fixing it.
- Mark (or Unmark) as Known Issue: Known Issues are marked by a small tag next to the issue name, and are included in the list under the Known Issues tab.
- Mark (or Unmark) as Resolved: The Resolution status will be updated. Marking an issue Resolved signifies that work has been completed and verified on the bug and will remove it from the list of active issues.
Note: you can also the below at bulk from the issue list in the Issues tab. Simply select the relevant issues from the list, click on the Actions button above the list, and select the required action.
You may also apply many actions on multiple issues at a bulk, by selecting them from the grid, and selecting an action from the Actions menu above the issues list.