Review test run results

Learn how to review the results of test run after creating it to identify any crashes, errors, accessibility issues, and verify that all test have passed. Test run results help you detect the over app quality and detect any potential bugs.

Navigate to test run results

After creating a test run, you can view the test run results using one of the following methods:

  • After creating a test run, select View Test Run from the success notification.

Selecting the View Test Run options from the success notification popup

  • Select the test run ID from the Test Runs page.

Selecting the test run ID from the Test Runs page

Test Run Results

The following is an example of the Test Run Results page:

A view of the Test Run Results page

The Test Run Results page contains:

  • General information

  • More actions

  • Revisit execution results

General information

The following details displayed on the Test Run Results page provide an overview of the test run’s key information:

A view of the details on the Test Run Results page

  • App version: the app version used for the test run.

  • Test: the name of the test run.

  • Test Run ID: the unique identifier assigned to the test run.

  • Creator: the user who created the test run.

  • The icon representing the OS (Android or iOS) and number of devices being tested.

  • Created on: the date and time the test run was created.

  • Elapsed: the total time that has passed since the test run began.

  • Crash: number of instances where an app crash is detected. You can select this button to filter revisit executions that contains Crash instances.

  • Accessibility: number of instances where an Accessibility Validation is raised. You can select this button to filter revisit executions that contains Accessibility instances.

More actions

By selecting the 3-dot icon, you can select the following actions for the test run:

A view of action that can be performed by clicking the 3-dot icon

  • Rerun Test Run: create a new test run with the same setup.

  • Terminate Test Run: terminate all running revisit execution(s).

  • Delete Test Run: delete the test run.

Selecting the Show button will open a dialog to filter the revisit executions by status:

Filtering the revisit executions by status via the Show button

Revisit execution results

Below the general information, a grid displays the status of each test case’s revisit execution on each device, along with validation icons.

The latest status of the revisit executions and validation icons will be refreshed periodically.

Revisiting the execution resutls

Possible revisit execution statuses are:

  • Running: the revisit execution either has not started yet, or is in progress.

  • Success: the revisit execution completed without error.

  • Failed: the revisit execution failed because the app under test crashed.

  • Need Action: the AI encountered a blocker and Blocker Remediation is required.

  • Error: the revisit execution failed due to a Kobiton system issue.

Once the revisit session is created, select the status to open the Explorer page of the revisit session.

Next to each test case in the grid, there are several icons:

  • Sensitive data indicator: if the Test Case has sensitive data, a lock icon displays.

    The sensitive data indicator and the helper text associated with it

  • View Test Case Details: go to the Test Case Details screen.

    The icon to view test case details

  • View Camera Roll: see the Camera Roll section.

    The icon to view the camera roll

Test Run Overview (Camera Roll)

Select View Camera Roll to compare the steps outlined in the test case with the steps executed during each test run session.

Viewing the camera roll to compare the steps of the test case with the steps executed during each test run session

To view the actual test steps for a device, expand the individual session.

Only completed sessions can be expanded to view the test steps.

Viewing the actual test steps for a device

If a blocker is encountered, select Resolve Blocker to remediate it.