As you utilize Applause services to test your products as part of your Software Development Life Cycle, test results are collected and available for you in the Applause Platform. Naturally, a significant portion of your attention will gravitate towards reacting to the collected results – triage issues, correspond with your Applause team to get more information, export issues to your Bug tracking System, and prioritize fixes into subsequent releases. At the same time, you may also like to observe the data in an aggregated, trended manner.
The Product Activity Dashboard provides interactive metrics such as the Applause Quality ScoreTM, tables and graphs, that allow you better understanding of the status and trends of your product and builds. The information – presented at both the product and build level, is geared towards assisting you to:
- Identify the overall quality of testing executed for a build.
- Track trends in quality, and promote process improvement accordingly.
- Focus on areas in your product most requiring your attention.
- Manage the collected issues across the bug life cycle, and acknowledge impact on your team’s tasks.
- Monitor the testing services provided by Applause to allow you and your project team to optimize the testing strategy for your product.
Viewing Product Activity Dashboard
To view the Product Activity Dashboard:
1. Log in to the Applause Platform and navigate to Products.
2. Locate the relevant product from the Products List. You may sort the table, filter it by Status and Application Type, or run a textual search.
3. Click on the Product Name.
The Activity Dashboard for the product will be displayed, with the Product Overview page offering information general to the product, as well as per-build “cards” containing the build’s Applause Quality Score (AQSTM), Confidence Level (CL) and issue distribution at a high level.
Learn more about the Applause Quality Score and Confidence Level here.
4. To drill into the Build Overview page, offering more details about a specific build, click on the “See more details” link at the bottom of the build card.
To optimize performance, not all information displayed is calculated in real time. At the bottom of the Activity Dashboard you’ll be able to see when was the data last refreshed.
The Product Overview aims to provide high-level information on your product, as well as the results collected on your latest builds. Up to 10 builds will be available on a page, and paging option will be available at the bottom of the page if needed.
Note that the builds are sorted based on their creation date, from the newest to the oldest, allowing you to concentrate on more recent results and trends.
Information displayed about your product might indicate on the level of maturity it achieved with utilization of Applause services, including:
- Date created.
- Number of active components. (Learn more about how to set components to your product here.)
- Number of test cycles previously activated. (Learn more about how to create test cycles here.)
- Bug Tracking System (BTS) integration. (Learn more about how to export issues to your BTS here.)
You can review and edit the complete details of your product’s settings by clicking on the “See more details” link.
For each of your product builds, the following information will be displayed to provide context to the results:
- Created Date.
- Date when last issue was submitted.
- Number of test cycles previously activated for the build (this also includes cycles where the build was tested only part of the duration).
- Number of active test cycles currently testing the build.
Applause Quality ScoreTM and Confidence Level
The Applause Quality Score (AQS) is a calculated value – ranging from 0 to 100 – which describes the quality of testing results for a product or build during one or more test cycles based on testing done and results collected.
AQS makes it easy to see how your overall quality is trending build-over-build, and to indicate progress and/or opportunity for improvement. The purpose of the AQS is to help make the decisions around when is the right time to release a build much less of a gut call, and more data driven and defensible. By providing a tangible number based on testing results, AQS empowers you to make those critical release/don’t release decisions in a much more fact-based quick and easy manner.
While based on sophisticated data science models, the end results is as simple as a single metric displayed across 3 ranges:
- High: between 90 to 100
- Medium: between 66 to 89
- Low: between 0 to 65
Next to the Quality Score, you will also find an indication of the level of confidence we have in our calculations. The Confidence Level (CL) for an individual build is based on the scale and scope of the testing conducted for the build such as duration and coverage, as well as the breadth of historical data collected on previous builds of the product or app.
Once calculated, the CL is presented as one of three available values:
Learn more about the Applause Quality Score and Confidence Level here.
The Issue Breakdown section provides an overview of the collected issues, throughout the bug life cycle. The total number of issues is displayed across the different phases:
- Submitted: a tester submitted the issue. This count includes pretty much all collected issues, except those discarded or disputed.
- Approved: a project team member (TTL, TE, TA, SDM) or a user from your team triages the submitted issues and approves or rejects them. This count includes all approved issues, except those marked as “Won’t Fix” (you probably care less about those).
- Exported: approved issues may be exported to your Bug Tracking System (BTS) either manually or through the integration. This count includes all issues set with a BTS ID, except those marked as “Won’t Fix”.
Learn more about how the test cycle results are managed, including how to export issues to your BTS here.
In addition, the issues are also broken down by their assigned Severity (set by the tester when submitting the issue, and re-confirmed during triage), Value (set by the user approving the issue during triage), and – where applicable – by Priority set in their Bug Tracking System. By default this is presented for the Approved issue – click on the other issue life cycle phases to view the internal Severity/Value/Priority distribution for them. Note that hiding some of these metrics is possible – contact your Applause team if needed.
Last, in case there are some issues still pending triage, this will be indicated. You should complete triage in a timely manner so you can treat the results displayed more reliably (and, it increases the Confidence Level for AQS).
The Build Overview page consists of several sections, providing greater insights into the test results for the build.
To access more detailed data visualizations about the build, click the “See more details” link at the bottom of the Issues Breakdown section of the relevant build.
As in the Product Overview page, high level information about the build is displayed to provide context to the results.
Issue Life Cycle Phases
As in the Product Overview page, you are able to toggle the results by issue life cycle phases, impacting all other Build Overview sections.
As in the Product Overview page, breakdown of the issues by Severity, Value and Priority provides the main context to for the testing results.
The Coverage section enables a deeper observation of the collected results based on technology and location.
- Coverage by Technology offers the distribution of results by the Operating Systems issues were reported against. To drill even deeper, hover over the Operating Systems (OS) – mobile OSs will be sliced by device and OS version, and desktop OSs will ve slices by web browser and OS version.
- Coverage by Location offers a map view highlighting the distribution of results based on the tester country issues were reported from. You may zoom the map in and out and drag it to focus on interest areas.
Examining the results by coverage allows you to better address issues as well as align the testing strategy of future builds.
Issues by Components
Issues by Components provides an in-depth view of the results by the components set for the tested product. On top of the overall distribution, you may also crosstab the results by:
- Issue Type (default)
- Priority (where applicable)
Identifying how issues are distributed across components allows you to better address issues as well as align the testing strategy of future builds.
The Testing Services section allows you to better understand the testing process. While testing is performed by the Applause Community and driven by your project team, understanding issue accumulations, testing times and triage status keeps you informed about the process, allowing you to better fine-tune the testing strategy of future builds with your Applause project team.
Under the Testing Services section you’ll be able to view:
- Triage, including the overall issue approval rate, distribution of issue approvers between the Applause and your users, and indication of issues still pending triage.
- Testing Times, including average cycle duration and time to first and last issues.
- Issue Accumulation, offering a timeline of the accumulation of submitted issues at the test cycle level.