test-suite

2024.10

true

Test Suite User Guide

DELIVERY:

Automation Cloud Automation Cloud Public Sector Automation Suite Standalone

Last updated Nov 11, 2024

Test sets

Test Sets are logical groups of test cases. The purpose of test sets is to define groups of tests that should be executed together within one run. For instance, a smoke test is a group of tests that only check for top-critical capabilities. Whenever any test case from a smoke test fails, there is a critical problem. Executing a smoke test should not take too long. They are executed for instance whenever a developer commits changes to the source code. By comparison, a regression test is a comprehensive test which should provide a detailed overview over the system under test. Execution often takes hours or even days. To define the set of tests to be executed for those purposes, test sets are created. A test case can be assigned to several test sets.

Conditions

Test sets in Test Manager can include both manual and automated test cases, including automated test cases from multiple Studio projects.
Test sets linked from Orchestrator to Test Manager include only the test cases from Orchestrator.
You can run test sets from either a default or a specified execution folder. This allows you to exclusively execute test cases that are part of that particular folder. Moreover, you also have the option to specify particular packages and their versions from which the test cases will be exclusively run.

Creating test sets

To create a test set:

In the Planning section, click Test Sets. The Test Sets page is displayed.
Click Create Test Set. The Create Test Set window is displayed.
Fill in the form as follows:
- Name - Enter a name for the test set.
- Description - (Optional) Enter a description for the test case. In order to add formatted text, you can use Markdown syntax.
- Enable RPA activity coverage calculation - Select this to view the activity coverage of the test set under Execution, after each automated execution.
- Labels - Add any number of labels which makes it easy to find test set with joint labels.
Click Create. The test set is created and displayed in the Test Sets page.

Assigning test cases to a test set

You can simultaneously assign test cases to a test set in two different ways:

Using Static Assignment - select the test cases manually.
Using Dynamic Assignment - input the label of the test cases that you want to assign.

Statically assigning test cases to test set

Open the test set you want to assign test cases to by clicking it.
In the Tasks menu, select Assign Test Cases.
The Assign Test Cases window is displayed.
Select the test cases you want to add. You can search for test cases by entering the name, the key, or a label.
Select Assign. The test cases are assigned to the test set and are displayed on the Test Cases tab.
Tip:

Dynamically assigning test cases to test set

You can choose to assign all test cases to a test set, based on their labels.

Select the test set that you want to assign test cases to.
In the Tasks menu, select Assign Test Cases.
The Assign Test Cases window is displayed.
Input the labels of the test cases that you want to assign.
For the example in the below illustration, all test cases with the label ASP:9 will be assigned to the current test set.
Select Assign. The test cases are assigned to the test set and are displayed on the Test Cases tab.

On the Test Cases tab you can see and edit test cases that are assigned both statically and dynamically in separate sub-tabs.

Navigation and search

To quickly find your test sets, use the search function and the filters. Navigate within the page using and configuring the paginator. Alternatively, you can use the breadcrumb to navigate between the pages.

Filter - You can use the filter to narrow your search. For example, you can search for the test sets that have been updated by your user, by specific labels, or the source (e.g., Orchestrator). The filters are automatically saved and kept active until you clear them.
Search - Use the search bar to find test results by their key, name or labels (requires full search term match)
You can use the search bar at the top of the page to look for test results.

Enabling activity coverage

Note: You can use this feature only if your Test Manager is connected to Orchestrator 2023.10 or higher.

To enable the activity coverage of an existent test set in Test Manager, perform the steps below:

Open the test set for which you want to enable activity coverage.
Go to the Configuration tab.
Select Enable RPA activity coverage calculation.

Figure 1. Enable RPA activity coverage calculation setting enabled

What to do next: For this test set, you can now select Execute then Automated, and check under Execution to see the activity coverage.

Note: For test sets linked from Orchestrator, you can enable or disable the activity coverage only in Orchestrator.

Configuring test sets for specific execution folders

Within a test set, you can automatically execute test cases only from a default or selected Orchestrator folder (also known as the Execution folder). If you want to execute the test cases from a specific package and version within a folder, you can select the package and its corresponding version.

Selecting an execution folder

To limit test case execution to a specific Orchestrator folder, follow these steps:

Navigate to your Project Settings and set a default execution folder. This selection applies to all your test sets.
Visit Automation project configuration for more information about setting a default execution folder.
To set a unique folder for a specific test set, go to Test Sets and select the test set for configuration.
1. In the Test Cases tab, opt for Select Automation.
2. Choose a folder from the Execution folder dropdown list.
3. Select Save to ensure your test cases are executed within this specified folder.

Selecting a specific package and version

In addition to the execution folder, you can also select a specific package and version from which to run test cases. To select a specific package and version, perform the following steps:

Navigate to Test Sets and open a test set.
Go to the Test Cases tab.
Use Select Automation.
To define a package and version, the following options are available:
- Autoselect Packages: Automatically selects all packages located in the execution folder and their latest versions, based on the test cases assigned to the test set.
- Add automation: Allows you to manually select the desired packages and corresponding versions. Use the following steps:
  1. From the Package dropdown list, select a package.
  2. From the Version dropdown list, select a version for that package.
  3. Use Select to finalize the selection.
  Repeat the previous steps for each package you wish to add to the test set.
Figure 2. Display of Selected Automations menu illustrating the chosen execution folder, associated packages, and their respective versions for a specific test set

Cloning test sets

You can only clone one Test Set at a time.

Select More Options for the Test Set that you want to duplicate, and select Clone.
Rename the Test Set and choose the information you want to clone, such as:
- Documents
- Labels
  Test Cases assigned to the original Test Set (both statically and dynamically) are automatically linked to the clone. Custom field definitions, and custom field values are also cloned if they exist.
Select Confirm to start the cloning process. The clone is created in the background.

A clone of the Test Set is created with a different ID.

Exporting test sets

You can export your test sets, separately from your testing project, so you can import them back into a different project. The export process for test sets is similar to the project export process. Visit Export project to check how exporting projects works.

Note: When you export test sets, links between the test sets and their related test cases are saved and are kept when imported into a new project.

Remember that the export only includes custom field values, and not their definitions.

Perform the following steps to export test sets:

Open your project and go to Test Sets.
Depending on the number of test sets you wish to export, execute one of the following steps:
- To export a single test set, select More Options for the specific test set and then select Export.
- To export multiple test sets, select all the test sets you want to export, and then select Export.
When the export is ready, you will receive a Project export completed notification. Select the notification to download the TMH file containing the exported test sets.

You can then import the downloaded TMH file to a different project, to re-create the test objects. Visit Import project to learn how to perform the import process.

Linking Orchestrator test sets

You can link your test sets from Orchestrator and execute them in Test Manager.

Prerequisites: Before you link a test set from Orchestrator to Test Manager, ensure you first link the test cases from this Orchestrator test set to Test Manager in Studio. For detailed steps on how to link test cases from Studio to Test Manager, visit Linking test cases in Studio to Test Manager.

Open your project in Test Manager.
Navigate to Test Sets > Link.
Select a test set from the list and click Link. The Test Sets page now shows your linked test set.
Important:
- To link a test set to Test Manager, you need to create it in Orchestrator.
- You can only link individual test sets that contain automated test cases. Note that each of these automated test cases must first be linked to Test Manager from Studio.
- Test set linking is available for connected Orchestrator instances version 2020.10 or higher.
- Test sets linked from Orchestrator are read-only. This means that you cannot edit the name, description, or version. Any changes need to be made in Orchestrator.
- The displayed number of test cases might differ between Test Manager and Orchestrator. Test Manager only shows test cases that have been created on its platform.

Applying filters and views

You can apply filters and store them in views for your test objects. Depending on the scope of a view (personal or public), you can persist the view containing the filters for a user or for all users within the project.

The available filters that you can apply are similar to the ones that appear in the Columns type of that section, consisting of:

Properties native to a test object (such as Labels, Updated by, Source, and others)
Custom labels

Follow these steps to customize your filters and view:

Go to the desired test object section (i.e. Requirements) and select Filters.
Select Add filter to choose the filters that you want to apply.
When you select the filter, also select the value that you want it to have.
Select Apply to include the filters in your view.
To save these filters as a view, select Views, and then Save view.
In the Save view window, choose to create a New view or Update an existing view with the filters.
Decide whether to keep the new view private or set it to Public for other users to access.
Select Save to confirm changes.
To apply a view that you created, select Views, and then Open views.
1. Choose a personal view (My views) or a Public view.
  You can also change the visibility of your created views between private and public, according to your needs.
2. Select the desired view and select Open to apply.