TestResults.io
  • What is TestResults.io
  • System Architecture
  • Regulated Markets
    • Health Care Industry
    • Financial Services Industry
  • What's new?
    • Update Instruction
      • Update of the Designer extension in Visual Studio
      • Update the Execution Engine
      • Update the Base- and Environment Models
    • Migration to new project file
  • Designer
    • Overview
    • First Steps
    • Installation
    • TestResults.io solution
    • The development environment
      • Tool Window Overview
        • Test Environments
          • Steps to configure a SUT VM
          • How to access files on the system under test
        • Remote Viewer
          • Control Mode
          • Capture Mode
          • Screen Editor
          • Capture Single Image
        • Screen Explorer
        • Test Case Designer
        • Image Explorer
        • API Testing
        • TestResults.io options
          • Engines
          • Servers
    • Test Cases
      • Adding Test Cases
      • Test Case structure
      • Simple Test Case (Example)
      • Executing your Test Case
      • Generating Test Containers
      • Publishing Test Cases
      • Execution Variables
    • Software Models
      • Adding SW Models
      • Referencing the Model in the Test Case
      • Generating Test Containers
      • Publishing SW Models
      • Screens and Screen Elements
        • Tables
        • Visual Sense
        • Custom Screen Elements
    • Installers
      • Adding Installers
      • Generating Test Containers
      • Publishing Installers
    • Automation Framework Reference
      • Attributes
      • Enumerations
      • Filters
        • IImageFilter Interface
        • Select
        • Resize
      • GlobalEngineParameters
      • ImageDetectionParameters
      • ImageReference
      • Interfaces
        • ITester Interface
          • ITester Methods
          • ITester Properties
        • ITestee Interface
          • ITestee Methods
        • IConnections Interface
          • IConnections Methods
          • IConnections Properties
        • IDebug Interface
        • IEngineResult
        • IHelpers Interface
        • IImageHelper Interface
          • IImageHelper Methods
        • IKeyboard Interface
          • IKeyboard Methods
        • IMouse Interface
          • IMouse Methods
          • IMouse Properties
        • IReport Interface
          • IReport Methods
        • ISUTConnection Interface
      • OcrEngineParameters
        • Best practise
        • IronOcrEngineParameters
        • LEADOcrEngineParameters
        • OcrEngineRelatedCharacters
        • OcrPreprocessingParameters
      • ResultRectangle
      • SutNotReadyException
      • TestStepAbortedException
    • FAQ
      • Two-factor authentication (2FA) and One Time Passwords (OTP)
      • Receive and verify Emails
      • Using Excel as test data source
      • Verify PDF Content
  • Portal
    • Overview
    • First Steps
    • User's Guide
      • Login
      • Projects
      • Dashboard
      • Test Cases
      • Software & Environments
      • Test Sets
      • Job Queue
      • Management
      • Notifications Summary
    • Automation Engineer's Guide
      • Test Automation Workflow
      • Automating a test case
      • Enabling Software & Environments
    • REST API Reference
      • Authentication
      • Projects
      • Test Sets
      • Test Cases
      • Test Plans
      • Test Reports
      • Files
      • Software
      • Environments
      • Untitled
      • Executions
      • Favorites
      • Statistics
      • User Management
      • Customer Management
      • Notifications
      • Configurations
      • Invoicing
  • Models
    • Overview
    • Base Model
      • BaseScreen
        • BaseScreen Properties
        • BaseScreen Methods
      • OnScreenElement
        • OnScreenElement Methods
        • OnScreenElement Properties
      • Primitives
        • Button
          • Button Methods
          • Button Properties
        • Checkbox
          • Checkbox Methods
        • ContextMenu
          • ContextMenu Methods
          • ContextMenu Properties
        • Dropdown
          • Dropdown Methods
          • Dropdown Properties
        • DropdownMenu
          • DropdownMenu Methods
          • DropdownMenu Properties
        • GenericScroller
          • GenericScroller Methods
          • GenericScroller Properties
        • HorizontalScroller
        • IconButton
          • IconButton Properties
        • Label
        • LabelWithButton
          • LabelWithButton Properties
        • LabelWithCheckbox
          • LabelWithCheckbox Properties
        • LabelWithValue
          • LabelWithValue Methods
          • LabelWithValue Properties
        • PwTextBox
          • PwTextBox Methods
        • TextBox
          • TextBox Methods
          • TextBox Properties
        • Tables and lists
          • Column
            • Column Methods
            • Column Properties
          • ItemList
            • ItemList Methods
            • ItemList Properties
          • Row
            • Row Methods
            • Row Properties
          • TableBase
            • TableBase Methods
            • TableBase Properties
        • LabelWithElementBase
          • LabelWithElementBase Properties
      • Enumerations
      • Extensions
        • EngineResultExtensions Methods
        • EnumExtensions Methods
        • ReportExtensions Methods
        • StringExtensions Methods
        • TesterConnectionExtensions Methods
        • TesterDebugExtensions Methods
        • TesterHelperExtensions Methods
        • TesterMouseExtensions Methods
        • TesterOnScreenElementExtensions
        • TesterSelectExtensions Methods
      • Helper Classes
        • BaseModelHelpers
          • BaseModelHelpers Methods
          • BaseModelHelpers Properties
        • Attributes for Designer
        • ListHelpers
      • Exceptions
        • ElementNotFoundException
        • InteractionFailedException
        • ScreenNotFoundException
      • Interfaces
        • IAppBasics Interface
          • IAppBasics Methods
          • IAppBasics Properties
        • IBrowser
        • ICommandLineWindow
        • IScroller Interface
          • IScroller Methods
          • IScroller Properties
        • ISutLocale Interface
          • ISutLocale Methods
          • ISutLocale Properties
        • ISystemHelpers
        • ISystemInteractions
        • IWindow
    • Environment Model
      • Windows
        • Screens
          • CmdWindow
            • CmdWindow Methods
            • CmdWindow Properties
          • FileDialog
            • FileDialog Methods
            • FileDialog Properties
          • FileExplorer
            • FileExplorer Properties
          • RunWindow
            • RunWindow Methods
            • RunWindow Properties
          • Window
            • Window Methods
            • Window Properties
        • SutLocale
          • SutLocale Methods
          • SutLocale Properties
        • SystemHelpers
          • SystemHelpers Methods
          • SystemHelpers Properties
        • SystemInteractions
        • WebBrowser
          • Browser
            • Browser Methods
            • Browser Properties
          • BrowserScroller
            • BrowserScroller Methods
            • BrowserScroller Properties
          • AuthenticationDialog
            • AuthenticationDialog Methods
            • AuthenticationDialog Properties
          • DownloadDialog
            • DownloadDialog Methods
            • DownloadDialog Properties
        • EnvironmentModel Enumerations
        • Interfaces
          • IWebAppBasics
            • IWebAppBasics Property
Powered by GitBook
On this page
  • Test Plan Workflow
  • Test Report Workflow
  • Connections between Test Plan and Test Report Workflow
  • Execution Failure
  • Ready for Execution
  • Complete Workflow

Was this helpful?

  1. Portal
  2. Automation Engineer's Guide

Test Automation Workflow

Provides information on the process that describes the steps from an uploaded test plan document to a fully automated test case and its reporting.

PreviousAutomation Engineer's GuideNextAutomating a test case

Last updated 6 years ago

Was this helpful?

As TestResults.io is a automated testing platform it is target for individual testers, companies with multiple internal automation teams as well as external service providers who provide automation services to their clients.

In addition, all of the provide services need to be fully traceable, e.g. revisions need to be tracked, and repeatable ().

To make sure that individual testers don't get overloaded, TestResults.io provides to option to disable the Test Automation Workflow. Please refer to the section to understand the differences in your responsibilities if you disable the Test Automation Workflow.

The Test Automation Workflow consists of two connected workflows: The test plan workflow and the test report workflow.

Test Plan Workflow

This workflow starts with the user creating a new test plan revision. Once the user has confirmed that all required information is provided the test plan is submitted for design.

After submission, the automation engineer reviews the available information and makes sure that everything is understood. If the automation engineer is missing information, even after clarification, he can reject the test plan, which allows the user to provide more information. In case the provided information allows the automation engineer to start automating the test plan the test plan is accepted.

After automation of the test plan is finished and confirmed to work, the automation engineer runs an initial execution (execution 0). If execution 0 finishes as expected the automation engineer claims that the automation of the test plan is done.

The user can verify that the test report of execution 0 fulfills the requirements and accepts the test report or requests a redesign with specific comments on the requested changes.

Acceptance of the test report of execution 0 put the test plan in the state "Ready for Execution" and the user can start this test.

Test Report Workflow

This workflow start with the user requesting an execution of a test set. When the user executes a test set an execution gets queued and a test report gets generated with revision 0. Revision 0 is a placeholder for the actual revision with the actual revision being defined at the time when the test plan is running.

Once a scaler picks the execution up and schedules it for execution, the test report is in the state Preparing. As soon as the test environment is ready and the Subject Under Test is available on the target system the system starts to run the automated test plan.

At any of the states queued, preparing & running the user can cancel the execution. If the user cancels the execution after it was scheduled for its execution (State: Preparing) it results in an canceled test report revision. If the user cancels while the execution is still queued, no test report revision is created.

A running execution always results in a test report. In case the execution is terminated because of a system error (hosting problem, execution engine problem, test automation problem) the report is marked with Execution Error. In all other cases the test report can be reviewed. The user can either accept the test report or reject it. While n acceptance of the test report finishes the work flow, a rejection triggers an Investigation by the automation engineer.

If the automation engineer identifies that the problem lies within the automation itself the test report gets marked with Error in Execution. In case the problem is based because of a malfunction in the Subject Under Test or an error in the test plan itself the automation engineer adds this information to the test report and sets the test report back to ready for review.

For revision security a test report can only be rejected within 15 days. If not rejected within 15 days the system treats the test report as accepted without marking it as accepted. A test report can always be accepted by the user, even after the 15 days security margin for rejecting it.

For regulatory reasons (FDA electronic records, etc.) a test report can not be automatically accepted.

Connections between Test Plan and Test Report Workflow

The Test Plan Workflow and the Test Report Workflow interact in two points point.

Execution Failure

In case on an execution failure the TestResults.io portal's active monitoring system will automatically reset the plan status to "Automating". This automatic interaction between the two workflows blocks the user from triggering new execution until the underlying problem is solved.

Ready for Execution

The Test Report Workflow depends on the Test Plan Workflow as its initial state is connected to the last state of the Test Plan Workflow. A test report can only be generated if the test plan is in Ready for Execution state.

Complete Workflow

see regulated markets
Automating a test case