Applause allows you utilize our testing community not only for the initial reporting of bugs, but also to run a re-test once a new build with fix to those bugs is available, effectively verifying the bug fix. There are two methods for running a Bug Fix Verification (or BFV) to confirm that you have fixed the discovered bugs
1. Utilizing an existing test cycle, which gives you and your PM greater control and management capabilities in the BFV execution, including ability to invite any tester with matching devices/environments to verify the bugs
2. Trigger an automatic creation of a standalone test cycle in which the original tester who submitted the bug will be assigned for the re-test, as well as a second tester with matching devices/environments. As the Applause Platform will automatically de-assign those testers if re-tests were not completed in 48 hours (and assign others instead) there is almost no place for you or your PM to intervene and manage the BFV execution. This is especially important when specific testers should be selected due to access permissions, availability of IoT devices and other hardware, NDA requirements, etc.
With both methods available, Applause has fairly quick turnaround times for the fix verification,, and each can be initiated at any time via the Applause Platform.
Initiate BFV from the Current Test Cycle
Running a BFV from a current test cycle offers the following benefits:
- BFVs, test cases and exploratory testing can all be run within the same cycle using the same testers.
- Requesting BFVs as part of a standard test cycle allows for iOS builds to be resigned.
The BFV will be done using the same build as set for the current test cycle.To initiate a BFV from a current test cycle, edit an existing open test cycle or start a new one. Then:
1. While in Edit mode, Go to Step 2: Testing Scope, and scroll down to Known Issues and BFV under Testing Scope.
2. Click on Edit Known Issues & BFV
3. A list of bugs and known issues available for the product is displayed. Select the fixed bugs you’d like to verify. Note that the selected bugs are also checked to be added to your Known Issues list.
4. Remember to click Done when all of the bugs are selected. Then, remember to save the test cycle so your changes are implemented.
Note that when a bug is added to BFV, a BFV-type test case is created and is made available to testers who are already in the cycle.
Important Note About Cloned Test Cycles:
- Regular/Generic test cases are copied to the new test cycle.
- BFV test cases are NOT copied to the new test cycle.
- You can edit the new cycle and reselect (as described above)
Initiating A Standalone BFV Test Cycle
In certain cases, you may consider running the BFV as a standalone, dedicated test cycle, managed automatically by the Applause Platform. Such ases may include:
- There is a need to specify a build for the BFV, which is different than the build used in the current test cycle .
- Matching the tester originally reported the bug is significant – however there is nothing to prevent other testers from participating in the verification.
- There is no need to run BFV next to exploratory or normal test cases.
To initiate a BFV in a standalone test cycle, browse to the issue from where it is displayed, either within the test cycle it was reported in or the global Issues list. Then:
1. Click on the Actions drop down, and select Request Verification.
2. The Fix Verification dialog box will open.
Enter the following in the dialog box:
- Choose a build to conduct the re-test on.
- Type in any comment or further instruction
- Click Send
Important note: Applause’s iOS resigning tool is not supported for Standalone BFV cycles. If you need to verify a bug fix over a resigned iOS build:
- Initiate the BFV from the current cycle, or
- Upload the build in a new different cycle to auto-resign, and then select that existing build for your BFV.
Note: You may save time by selecting multiple issues for the issue list (either within a test cycle or the global one) and click on Ask for Verification as a bulk.
The verification request will be picked up by an automatic process within 15 minutes, designed to create a standalone new BFV cycle every week to verify all the fixes requested during that week. The standalone BFV test cycle will be managed automatically and will not be displayed under your Test Cycles list.
Once the cycle is created and the BFV is added to it, the original tester is invited to verify the fix, along with additional (random) tester that meets the reported bug’s devices/environments
Once a cycle is created, the original tester + one additional tester who has the matching environments will be sent an invitation to verify if the bug still exists.
Initiating BFV via Your Bug Tracking System
The majority of our customers prefer to take the results of our test cycles (i.e. bugs) and export them to their native Bug Tracking Systems (BTS), such as Jira. This enables them to manage the subsequent work with their development teams, in the system they are used to. I. See more here on how to set up your BTS.
Customers implementing the Two-Way Jira Integration may also automatically indicate to the Applause Platform on bugs ready for BFV based on the issue status in Jira, as well as automatically get updated with the verification results into their Jira issue. Read more on how to set up the Bug Verification Workflow using the 2-Way Jira Integration.
Viewing BFV Progress and Results
The testers will view the BFV as if it was a “regular” test case, and report back their findings in the form of a Pass or Fail (or Blocked). In the highly uncommon event that you wish to have the testers submitting a new bug when verification fails, make sure to note that when initiating the BFV.
To view the BFV progress and results, browse to the issue from where it is displayed, either within the test cycle it was reported in or the global Issues list. Then:
1. View the current BFV status for the bug under Verification:
2. View details of each BFV run, including date executed, test cycle number, result and device/environment under Additional Information: