Open Category List

Test Cycle Setup Best Practices

5 minutesread

This article will provide you with information and best practices that will help you get the most out of your test cycles. By following these guidelines, you will enable your Project Manager to set up the most thorough and comprehensive test cycles that will ultimately provide  the most relevant and actionable results with which to work.

First and foremost, it is always recommended that you Clone a cycle based on a previous cycle. Cloning a cycle will carry over all scope & instructions, as well as the testing team and TTL from the previous cycle.

Setup & Scope

Testing Coverage

The device/OS/carrier/geography combinations you would like to have bugs reported and/or test cases claimed against.

Defining In-Scope vs Out-of-Scope Issues

In order to ensure the success and proper focus of the test team, it is very important that our customers provide details about what they hope to accomplish in each test cycle. Clearly defining scope guidelines for testers to follow will enable the Project Manager and team to test apps with a more targeted approach and deliver results that are more relevant and useful to your business. For best results, be prepared to provide the following:

  • Testing Focus: A high level overview of the functional areas of the application for testers to focus new or regression testing on; and any steps they will need to take to access the site/app.
  • New Items/Changes For This Build: This is a detailed export and description of release notes of the specific fixes or features implemented in the build to provide better testing focus and results
    • Release notes or list of bugs/enhancements that have been addressed in the build, the title of which are typically sufficient as long as they are fairly descriptive
      • Gives the testers more context and details around what has changed so they can focus effort and test strategies more effectively
      • Example: If you changed the search feature it is important to add the additional detail of what has changed (i.e. added ability to do wildcard searches) – Testers will want to focus on the appropriate area(s) to make sure it has not broken and returns the correct information.
  • Bug Values & Examples: Provide a list of specific issues that you consider to be Exceptionally, Very, and Somewhat Valuable. This provides more focus to the testers and shows them what is really valuable to you.
  • Known Issues and Out of Scope: Known bugs list must include BOTH your internal bugs and all bugs submitted by Applause in previous cycles. Review Known Issue Management section for instructions.
  • Video & Tips: It is very helpful when the Test Manager working with Applause records a short video/webinar covering their expectations for the upcoming test cycle. This should include scope definition, product explanation, and any additional information that would be useful for testers. This helps us ensure that we are providing the most thorough coverage and our testers will deliver the most relevant results.
  • Out of Scope: Define areas of the app that do not need to be tested, including areas that may still be in development that you do not want the testers touching. This helps to ensure that any bugs that are filed are relevant.

Cycle Naming Convention

Give your cycle a name that has meaning, character, and personality. Testers will frequently be invited to numerous cycles at the same time, so naming your cycle accurately helps with clarity and avoids confusion. A good test cycle name typically includes the name of your company or product, the type of testing taking place, and the start date. For example:

  • Applause, Inc. – New Chat Features – 07/11/2016
  • Applause, Inc. – Regression Suite – 07/11/2016

Components

Components are a feature in the Applause Platform that allow you to set a custom list of different application/product components that testers can then select when they submit bugs. The idea is to make it easier for you and the testers to categorize bugs based on the part of the application it effects/is related to, and to be able to focus testers on testing in a certain area of the application. Components are set up in Step 5 of Product creation. If you think Components might be useful for your cycle, reach out to your PM who can get it set up for you.

Activation Timing

When setting up and activating your cycle, there are a few important factors to take into consideration:

  • Cycle Duration: The length of a cycle will vary depending on what offering you have purchased, your specific testing goals, the maturity of your product, etc. Your PM will be able to assist in determining the ideal cycle duration, but typically we recommend no more than 3-5 days in length.
  • Lead Time: It’s important to give your PM adequate lead time before kicking off a new build so they can search for and prepare the testing team. Typically 1 business day’s notice is sufficient.

[hr]

Test Cases

If you have purchased test case execution hours for your engagement, you will be able to set up and distribute test cases to your testers. The Applause Platform has a built-in test case management system for entering test cases and logging execution steps, Pass/Fail results, and screenshots. Instructions for creating Test Cases in Excel.

Setup and Distribution

If you require Applause to help create test cases, please provide appropriate documentation as a guideline (e.g. use cases, requirements docs, etc). If this documentation does not exist, provide a primary point of contact for us to work with.

Any cycles with new or updated test case coverage typically require 24 hours notice and need to be setup with the PM so we can ensure appropriate coverage by:

  • Reviewing for clarity
  • Reviewing to confirm timing estimates
  • Making appropriate tester or environment coverage updates

When building out your test cases, there are a few things to keep in mind:

  • Environments – Determine the environments you want the test case(s) executed against
  • Number of test case executions per environment – This is to help limit the amount of results you want to have to a reasonable level and prepare to slot testers into particular environments
  • Approximate time to complete test case – This gives the test team more info on whether they can commit to finishing the test case(s) within the time allotted against their schedule

[hr]

Getting the Most out of Bugs Results

Customized Bug Reports

The Applause Platform allows you to define custom field within your bug reports on a test cycle level. These custom fields allow testers to capture additional information about a bug, beyond what the standard bug report template contains, which will then be visible upon bug submissions, through the BTS export, or when exporting to CSV. Custom Fields are set up in Step 2 when creating a new test cycle.

Bug Tracking Systems

If you have an external bug tracking system, we highly recommend you integrate it with the Applause Platform. This allows for bugs found to easily flow to your BTS with little/no manual intervention, which gets the bugs into the hands of your developers faster. BTS Integration is set up when creating a Product in the Platform.

SDK Data

The Applause SDK is a mobile app quality tool that standardizes and improves mobile testing deliverables. Most of the Applause’s Functional testing packages come with the SDK, which, when instrumented into your app, collects usage analytics, tracks session information and crash reports from the testers. It helps you stay on top of serious quality issues and find and fix problems immediately without compromising you users’ security. More information on the Applause SDK can be found here, or talk to your Project Manager.

 

0
0
21
5 minutesread

Related Knowledge Base Posts