Skip to main content

mesmer test

Start, stop, list, describe, and manage tests.

mesmer test list#

List all the test cases in the current project.

# List all test cases$ mesmer test listTest ID                  Name                           Status     Executionaaaaaaaaaaaaaaaaaaaaaaaa Overflow check                 Review     Availablebbbbbbbbbbbbbbbbbbbbbbbb 5x5 = 25                       Review     Available
# List the tests somewhere else, assuming you're authenticated for it$ mesmer --project $SOME_OTHER_PROJECT test list

mesmer test describe#

Show more detail about a test, like its tags. This will also give an overview of what steps make up that test case.

# Describe the "overflow check" test$ mesmer test describe $test_idTestName .. Overflow checkTags .. Overflow
 Test ID  aaaaaaaaaaaaaaaaaaaaaaaa
Steps:No. Type1   Launch   -2   Tap      (133, 1510)3   Tap      (150, 1790)4   Tap      (1050, 1330)5   Tap      (896, 1606)6   Tap      (106, 1470)7   Tap      (136, 1540)8   Tap      (126, 1843)9   Tap      (560, 1823)10  Tap      (1053, 1343)11  Tap      (903, 1850)12  Tap      (70, 1200)13  Tap      (523, 1800)14  End      -

mesmer test tags#

Show all the test tags in a given project. You can pass any combination of these tags to mesmer test start to start executing some subset of all your tests.

$ mesmer test tagsTag nameMathOverflow

mesmer test tag#

Modify the tags on a test case.

To add a new tag, use --add:

$ mesmer test tag $test_id --add some_new_tag  # adds some_new_tag to the test$ mesmer test tag $test_id --add tag1 tag2     # adds tag1 and tag2 to the test

To replace a test's tags entirely, use --replace:

$ mesmer test tag $test_id --replace           # removes all tags$ mesmer test tag $test_id --replace some_tag  # removes existing tags, adds `some_tag`

To remove tags, use --remove:

$ mesmer test tag $test_id --remove deleteme   # removes the deleteme tag from the test

You can use -- to pass tags verbatim, or safely from scripts:

$ mesmer test tag $test_id --add -- --remove   # tags the test with '--remove', literally
# You can pass multiple tags safely after `--`:$ tags=("some" "--funky" "tags" "in" "a" "bash array")$ mesmer test tag $test_id --add -- "${tags[@]}"

mesmer test start#

Start executing tests for the current project.

$ mesmer test start $build_id โœ“  Execution started

You can also pick a subset of tests to run, by only running tests with specific tags:

# Only start tests tagged `Math`$ mesmer test start $build_id --tag Math โœ“  Execution started
# Start tests tagged `Math` or `Overflow`, on 2 devices in parallel.$ mesmer test start \  --tags \    Math \    Overflow \  -- $build_id  # note: you need the `--` so the arguments aren't ambiguous โœ“  Execution started

Finally, you can specify a specific device to run the tests on:

# Use a particular device$ mesmer test start $build_id --device emulator-1234 โœ“  Execution started

mesmer test stop#

Stops testing for a given build. Fairly self-explanatory.

$ mesmer test stop $build_id โœ“  Stopped tests
# And, if tests aren't already ongoing$ mesmer test stop $build_idError: no in-progress test to stop

mesmer test status#

Gets the status of a given test's execution: how many test cases are queued up, in progress, finished, have passed or failed.

Pass it the build id, and check in on your tests:

$ mesmer test status $build_idTotal tests  | 10Total issues | 2In queue     | 1
New          | 0In Progress  | 4Passed       | 3Failed       | 2Errored      | 0To Review    | 2

mesmer test wait#

Wait for testing to stop on a given build. This command will sit there and block until all the tests have finished.

$ mesmer test wait $build_id โ น  Waiting for tests to complete...  (0 queued, 1 in progress, 1 done)
# The command will exit when all testing has finished:Passed       | 8Failed       | 2Errored      | 0To Review    | 2

This might be useful in some CI workflows---if you need to wait to get test results, this makes it easy.

mesmer test results#

You can get a test's results like so:

$ mesmer test results $build_id                 Test ID                  Name                           Alerts  Owner โœ“  Passed       aaaaaaaaaaaaaaaaaaaaaaaa 5x5 = 25                       0       William Goodall โœ“  Passed       bbbbbbbbbbbbbbbbbbbbbbbb Overflow check                 0       William Goodall X  Review       cccccccccccccccccccccccc 25 x 25                        0       SupportUser

mesmer test detail#

You can get detailed results for the execution of a given test and build like so:

$ mesmer test detail --build=$build_id --test=$test_id โœ“  Fetched test case โœ“  Fetched device groups
Name .. Always failsTags ..
Device: SM-G950U1 (98892a333659583649)Errors ...... 0Defects ..... 1For Review .. 0Duration .... 51.15  Step 1 - OK  Step 2 - OK  Step 3 - OK  Step 4 - OK  Step 5 - Failed    Object Results    Part ID                  Message    5f5ee8e6a8b3070010216025 User Defined Assertion Error    Screen Results    Part ID                  Message    no test parts reported  Step 6 - OK
Device: pixel3 (emulator-5560)Errors ...... 0Defects ..... 1For Review .. 0Duration .... 65.42  Step 1 - OK  Step 2 - OK  Step 3 - OK  Step 4 - OK  Step 5 - Failed    Object Results    Part ID                  Message    5f5eea63a8b3070010216064 User Defined Assertion Error    Screen Results    Part ID                  Message    no test parts reported  Step 6 - OK

mesmer test browse#

Open a given build's test results in a web browser.

$ mesmer test browse $build_id