Test Sets
Provides information on Test Sets and their execution
A test set contains any number of test cases and allows the user to classify test cases into user defined categories. Test sets are also used as a starting point to execute test cases.
The Test Sets overview screen lists all test tests registered for the selected project. For every test set you see
Name
Last Execution date and time
The software version that was used for the last execution
The pass rate of all included test cases
Selecting a test set on the overview brings to the Test Set screen:
Like the test case screen, the test set screen is divided into four major areas:
Test Set
The test set area shows you the identifier, the name and the description of the selected test set.
Available Operations
Image | Operation | Description |
Mark as favorite | Click on the asterisk will toggle if the test set is listed in the favorites section of the projects screen. | |
Enable notifications | Opens a menu that allows you to select the event triggers you want to receive a notification for. | |
Edit test set information | Opens a dialog to modify the identifier, name or description of the test set. | |
Archive test set | Marks the test set as archived. An archived test set can be restored from the archived test sets overview. | |
Restore archived test case | Marks the test set as active. Note: This operation is only visible on archived test sets. | |
Delete test set | Deletes the test set. This action is not reversible. If you mistakenly deleted a test case please contact support. |
Notifications
There are multiple events for a test case that get triggered either by a user action or by the system. You can register for those events to get notified.
Trigger | Description |
Execution completed | An execution run was completed and results are available |
Test Report failed or aborted | A test report is finished with the result FAILED or ABORTED |
Modifying
The Edit Test Set dialog allows you to change the name and description of a test set.
Archive
Archived test sets are not listed in the default Test Set Overview screen. You need to click on the "Archive" tab on the Test Set Overview screen to see all archived test sets.
Test Cases
The test cases lists all test cases that are currently included in this set. In addition it allows you to add or remove test cases to and from the selected test set.
The overview lists the following attributes of any included test case:
Identifier
Name
State (see test plan states for more information)
Add/Remove test cases
By clicking on the "MANAGE TEST CASES" button the "Included Test Cases in Test Set" dialog is shown.
This dialog shows you all test cases in the current project. If checked, a test case is included in the current test set, if not checked a test case is not included in the current test set.
By clicking on a column label, the column will be sorted. By clicking on the filter icon the filter dialog will be shown, which allows to search for specific test sets
Add by test set
An alternative way to included test cases in a test set is by copying existing test sets. By clicking on the "ADD BY TEST SET" button you can select an existing test set which is copied.
Click on the "SAVE" button to copy a selected test set into the current test set.
Important: The test set is copied, not referenced. Changing the original test set after you have copied it, does not impact the new test set.
Result History
The result history provides you a quick overview of the number of executed test cases included in this test set per software version. This gives you a good indication if your test coverage is increasing, decreasing or in a steady state.
In addition the history area distinguishes between passed, failed, canceled and aborted test sets. This allows you to see the quality of a specific software version.
Visualizations
Icon | Description |
Shows individual bar graphs of the number of test cases per software version and the result state. |
Filtering
You can filter the result history by test environment, date range or tags that you have used for executions.
To reset the filter, clear all fields and click on the "FILTER" button.
Executions
The executions area gives you an overview of all executions of this test set. For every test set you see
Completion time - the time the last result of the last test report was received
Software version with which the execution was run
Test environment on which the execution was run
User who requested the execution
All tags used for the execution
Overall result of all test reports created by this execution
By clicking on an individual execution result a new tab is opened. This tab lists all test cases that where executed with the specification execution.
For every included test case the following details are shown
Test case identifier
Test case name
Revision - test plan revision
Execution No - test report revision
Test report status
Test report result
Time the test report was finished
Execution time
Execute a test set
A test set can be executed by clicking on the "EXECUTE" button, which opens the "Execute Test Set" dialog.
The execute test set dialog allows you to select a test environment on which the Subject Under Test should run, as well as a software version that you want to run the tests against.
For easier identification you can define freely chosen tags for the execution.
An execution is started by clicking on the "EXECUTE" button.
Last updated
