Mesmer in CI

Using Mesmer in a CI flow is a natural extension of what we do best---making testing painless and easy through automation.

Quickstart

Before we get into the details of how to configure the CLI, we'll start with a basic working example. This CI script:

Installs the Mesmer CLI in your CI worker
Configures the CLI for your Mesmer project
Uploads a build to Mesmer
Starts tests and kicks off a crawl of that build
Waits for the tests to finish
Logs the test results

# Download the CLI, and add it to $PATH for the rest of this shell session
# (also: if your CI provider doesn't package `jq` by default, install it here)
source <(curl -s https://cli.mesmerhq.com/release/install.sh)

# Configure the CLI. You can also set these environment variables in your CI provider's settings.
export MESMER_TENANT="your.tenant.mesmerhq.com"
export MESMER_PROJECT="xxxxxxxxxxxxxxxxxxxxxxxx" # found in the URL: https://your.tenant.mesmerhq.com/home/$project_id/...
export MESMER_AUTH_TOKEN="[redacted]"            # generate with `mesmer auth login`, or "Your Profile" -> "Token" on the web

# Run your build script
# Here you invoke Gradle, xcodebuild, etc.
./build-my-application.sh --output-binary-to myapp.apk

# Submit the build for testing and crawl, capturing the freshly-created build ID
build_id="$(mesmer -q .buildId submit myapp.apk)"

# Wait for tests to finish, and then log the results.
mesmer test wait $build_id
mesmer test results $build_id

Installing the CLI

You can install the latest release of the CLI with the install script:

# Download the CLI, and add it to $PATH for the rest of this shell session
$ source <(curl -s https://cli.mesmerhq.com/release/install.sh)

This guide assumes your CI provider is running Linux. For other operating systems, the procedure is similar, but not the same: you'll need to run the appropriate installer for your platform, and use the appropriate shell scripting syntax.

Configuring Authentication

You can authenticate the CLI through environment variables---set wherever your CI platform keeps its secrets. Generally, you can configure the environment of your CI runner through your platform's web UI (GitHub Actions, CircleCI, Travis CI).

# (in your CI script, or configuration UI)
$ export MESMER_AUTH_TOKEN="[your auth token]"

You can generate an auth token by running mesmer auth login locally, followed by a mesmer auth list --show-tokens.

Configuring your Project

To configure the CLI to find the right tenant and project, check your .mesmer.yml project file in to version control:

# (locally)

# If you don't have one already, create a .mesmer.yml file
# containing your tenant and project ID.
$ [[ ! -e .mesmer.yml ]] && mesmer init

# Check in .mesmer.yml
$ git add .mesmer.yml && git commit -m "Add .mesmer.yml" && git push

Alternatively, you can set environment variables to inform the CLI about your project:

# (in CI)

$ export MESMER_TENANT="your.tenant.hostname.mesmerhq.com"
$ export MESMER_PROJECT="xxxxxxxxxxxxxxxxxxxxxxxx"

Automating your tests

To automatically submit test and crawl jobs to Mesmer, build your application, then call mesmer submit on the final binary:

# (in your CI script)

# Build your app
$ ./your-build-script -o build.apk

# Submit the build for testing and crawl
# Use an output query to pull off the buildId property
# Note: make sure that `jq` is installed in your CI environment.
$ build_id="$(mesmer -q .buildId submit build.apk)"

─── Creating the build
 ✓  Uploaded build
     Build ID  aaaaaaaaaaaaaaaaaaaaaaaa

─── Queueing up jobs
 ✓  Got test configuration
 ✓  Fetched compatible devices
 ✓  Selected device for tests
 ✓  Tests in queue
 ✓  Crawl in queue

─── Waiting for jobs
 ✓  Tests in progress
 ✓  Crawl in progress
     Crawl ID  bbbbbbbbbbbbbbbbbbbbbbbb

─── Done!

    Tests
    https://some-tenant.mesmerhq.com/home/xxxxxxxxxxxxxxxxxxxxxxxx/testresults/aaaaaaaaaaaaaaaaaaaaaaaa

    Crawl
    https://some-tenant.mesmerhq.com/home/xxxxxxxxxxxxxxxxxxxxxxxx/app-map/aaaaaaaaaaaaaaaaaaaaaaaa

# Wait for tests to finish...
$ mesmer test wait $build_id

# ...And log the test results
$ mesmer test results $build_id

This will upload the build for testing and crawl, wait for all tests to complete, and then log their results.

Mesmer CLI