mesmer test

Start, stop, list, describe, and manage tests.

mesmer test list

List all the test cases in the current project.

# List all test cases
$ mesmer test list
Test ID                  Name                           Status     Execution
aaaaaaaaaaaaaaaaaaaaaaaa Overflow check                 Review     Available
bbbbbbbbbbbbbbbbbbbbbbbb 5x5 = 25                       Review     Available

# List the tests somewhere else, assuming you're authenticated for it
$ mesmer --project $SOME_OTHER_PROJECT test list

mesmer test describe

Show more detail about a test, like its tags. This will also give an overview of what steps make up that test case.

# Describe the "overflow check" test
$ mesmer test describe $test_id
Test
Name .. Overflow check
Tags .. Overflow

 Test ID  aaaaaaaaaaaaaaaaaaaaaaaa

Steps:
No. Type
1   Launch   -
2   Tap      (133, 1510)
3   Tap      (150, 1790)
4   Tap      (1050, 1330)
5   Tap      (896, 1606)
6   Tap      (106, 1470)
7   Tap      (136, 1540)
8   Tap      (126, 1843)
9   Tap      (560, 1823)
10  Tap      (1053, 1343)
11  Tap      (903, 1850)
12  Tap      (70, 1200)
13  Tap      (523, 1800)
14  End      -

mesmer test tags

Show all the test tags in a given project. You can pass any combination of these tags to mesmer test start to start executing some subset of all your tests.

$ mesmer test tags
Tag name
Math
Overflow

mesmer test tag

Modify the tags on a test case.

To add a new tag, use --add:

$ mesmer test tag $test_id --add some_new_tag  # adds some_new_tag to the test
$ mesmer test tag $test_id --add tag1 tag2     # adds tag1 and tag2 to the test

To replace a test's tags entirely, use --replace:

$ mesmer test tag $test_id --replace           # removes all tags
$ mesmer test tag $test_id --replace some_tag  # removes existing tags, adds `some_tag`

To remove tags, use --remove:

$ mesmer test tag $test_id --remove deleteme   # removes the deleteme tag from the test

You can use -- to pass tags verbatim, or safely from scripts:

$ mesmer test tag $test_id --add -- --remove   # tags the test with '--remove', literally

# You can pass multiple tags safely after `--`:
$ tags=("some" "--funky" "tags" "in" "a" "bash array")
$ mesmer test tag $test_id --add -- "${tags[@]}"

mesmer test start

Start executing tests for the current project.

$ mesmer test start $build_id
 ✓  Found compatible device(s)
 ✓  Execution started

You can also pick a subset of tests to run, by only running tests with specific tags:

# Only start tests tagged `Math`
$ mesmer test start $build_id --tag Math
 ✓  Found a test device
 ✓  Execution started

# Start tests tagged `Math` or `Overflow`, on 2 devices in parallel.
$ mesmer test start \
  --tags \
    Math \
    Overflow \
  -- $build_id  # note: you need the `--` so the arguments aren't ambiguous
 ✓  Found a test device
 ✓  Execution started

mesmer test stop

Stops testing for a given build. Fairly self-explanatory.

$ mesmer test stop $build_id
 ✓  Stopped tests

# And, if tests aren't already ongoing
$ mesmer test stop $build_id
Error: no in-progress test to stop

mesmer test status

Gets the status of a given test's execution: how many test cases are queued up, in progress, finished, have passed or failed.

Pass it the build id, and check in on your tests:

$ mesmer test status $build_id
Total tests  | 10
Total issues | 2
In queue     | 1

New          | 0
In Progress  | 4
Passed       | 3
Failed       | 2
Errored      | 0
To Review    | 2

mesmer test wait

Wait for testing to stop on a given build. This command will sit there and block until all the tests have finished.

$ mesmer test wait $build_id
 ⠹  Waiting for tests to complete...  (0 queued, 1 in progress, 1 done)

# The command will exit when all testing has finished:
Passed       | 8
Failed       | 2
Errored      | 0
To Review    | 2

This might be useful in some CI workflows---if you need to wait to get test results, this makes it easy.

mesmer test results

You can get a test's results like so:

$ mesmer test results $build_id
                 Test ID                  Name                           Alerts  Owner
 ✓  Passed       aaaaaaaaaaaaaaaaaaaaaaaa 5x5 = 25                       0       William Goodall
 ✓  Passed       bbbbbbbbbbbbbbbbbbbbbbbb Overflow check                 0       William Goodall
 X  Review       cccccccccccccccccccccccc 25 x 25                        0       SupportUser

mesmer test detail

You can get detailed results for the execution of a given test and build like so:

$ mesmer test detail --build=$build_id --test=$test_id
 ✓  Fetched test case
 ✓  Fetched device groups

Name .. Always fails
Tags ..

Device: SM-G950U1 (98892a333659583649)
Errors ...... 0
Defects ..... 1
For Review .. 0
Duration .... 51.15
  Step 1 - OK
  Step 2 - OK
  Step 3 - OK
  Step 4 - OK
  Step 5 - Failed
    Object Results
    Part ID                  Message
    5f5ee8e6a8b3070010216025 User Defined Assertion Error
    Screen Results
    Part ID                  Message
    no test parts reported
  Step 6 - OK

Device: pixel3 (emulator-5560)
Errors ...... 0
Defects ..... 1
For Review .. 0
Duration .... 65.42
  Step 1 - OK
  Step 2 - OK
  Step 3 - OK
  Step 4 - OK
  Step 5 - Failed
    Object Results
    Part ID                  Message
    5f5eea63a8b3070010216064 User Defined Assertion Error
    Screen Results
    Part ID                  Message
    no test parts reported
  Step 6 - OK

mesmer test browse

Open a given build's test results in a web browser.

$ mesmer test browse $build_id
https://some-tenant.mesmerhq.com/home/xxxxxxxxxxxxxxxxxxxxxxxx/testresults/bbbbbbbbbbbbbbbbbbbbbbbb