Test Sets
Provides information on Test Sets and their execution
Last updated
Was this helpful?
Provides information on Test Sets and their execution
Last updated
Was this helpful?
A test set contains any number of test cases and allows the user to classify test cases into user defined categories. Test sets are also used as a starting point to execute test cases.
The Test Sets overview screen lists all test tests registered for the selected project. For every test set you see
Name
Last Execution date and time
The software version that was used for the last execution
The pass rate of all included test cases
Selecting a test set on the overview brings to the Test Set screen:
Like the test case screen, the test set screen is divided into four major areas:
The test set area shows you the identifier, the name and the description of the selected test set.
Image
Operation
Description
Mark as favorite
Enable notifications
Edit test set information
Archive test set
Marks the test set as archived. An archived test set can be restored from the archived test sets overview.
Restore archived test case
Marks the test set as active.
Note: This operation is only visible on archived test sets.
Delete test set
Deletes the test set. This action is not reversible. If you mistakenly deleted a test case please contact support.
There are multiple events for a test case that get triggered either by a user action or by the system. You can register for those events to get notified.
Trigger
Description
Execution completed
An execution run was completed and results are available
Test Report failed or aborted
A test report is finished with the result FAILED or ABORTED
The Edit Test Set dialog allows you to change the name and description of a test set.
Archived test sets are not listed in the default Test Set Overview screen. You need to click on the "Archive" tab on the Test Set Overview screen to see all archived test sets.
The test cases lists all test cases that are currently included in this set. In addition it allows you to add or remove test cases to and from the selected test set.
The overview lists the following attributes of any included test case:
Identifier
Name
By clicking on the "MANAGE TEST CASES" button the "Included Test Cases in Test Set" dialog is shown.
This dialog shows you all test cases in the current project. If checked, a test case is included in the current test set, if not checked a test case is not included in the current test set.
By clicking on a column label, the column will be sorted. By clicking on the filter icon the filter dialog will be shown, which allows to search for specific test sets
An alternative way to included test cases in a test set is by copying existing test sets. By clicking on the "ADD BY TEST SET" button you can select an existing test set which is copied.
Click on the "SAVE" button to copy a selected test set into the current test set.
Important: The test set is copied, not referenced. Changing the original test set after you have copied it, does not impact the new test set.
The result history provides you a quick overview of the number of executed test cases included in this test set per software version. This gives you a good indication if your test coverage is increasing, decreasing or in a steady state.
In addition the history area distinguishes between passed, failed, canceled and aborted test sets. This allows you to see the quality of a specific software version.
Icon
Description
Shows individual bar graphs of the number of test cases per software version and the result state.
You can filter the result history by test environment, date range or tags that you have used for executions.
To reset the filter, clear all fields and click on the "FILTER" button.
The executions area gives you an overview of all executions of this test set. For every test set you see
Completion time - the time the last result of the last test report was received
Software version with which the execution was run
Test environment on which the execution was run
User who requested the execution
All tags used for the execution
By clicking on an individual execution result a new tab is opened. This tab lists all test cases that where executed with the specification execution.
For every included test case the following details are shown
Test case identifier
Test case name
Revision - test plan revision
Execution No - test report revision
Time the test report was finished
Execution time
A test set can be executed by clicking on the "EXECUTE" button, which opens the "Execute Test Set" dialog.
The execute test set dialog allows you to select a test environment on which the Subject Under Test should run, as well as a software version that you want to run the tests against.
For easier identification you can define freely chosen tags for the execution.
The TestResults.io platform supports some predefined control tags. Do not use any of the following reserved tags until asked for by the TestResults.io support or your vendor:
-sim:
-debug:
-livetrack
Please contact support for additional information on these specific control tags.
An execution is started by clicking on the "EXECUTE" button.
Click on the asterisk will toggle if the test set is listed in the favorites section of the .
Opens a menu that allows you to select the event triggers you want to receive a for.
Opens a dialog to the identifier, name or description of the test set.
In case you are registered for a least one notification the notification flag () will changed to a filled flag: . To see which notifications are currently registered click on the flag. The popup will list all available notifications with the one you have registered for being checked.
To restore a test set from the archive you can either click on any archived test set to see its details and click the restore icon or you can hover over the entry in the archived test sets list and select restore from the options on the right.
State (see for more information)
To open a test case in a new window to see even more details you click on the OPEN button ().
Shows an area graph of the number of test cases per software version and the result state.
Overall of all test reports created by this execution
Test report
Test report
