Skip to main content

Mesmer in CI

Using Mesmer in a CI flow is a natural extension of what we do best---making testing painless and easy through automation.

Quickstart#

Before we get into the details of how to configure the CLI, we'll start with a basic working example. This CI script:

  • Installs the Mesmer CLI in your CI worker
  • Configures the CLI for your Mesmer project
  • Uploads a build to Mesmer
  • Starts tests and kicks off a crawl of that build
  • Waits for the tests to finish
  • Logs the test results
# Download the CLI, and add it to $PATH for the rest of this shell session# (also: if your CI provider doesn't package `jq` by default, install it here)source <(curl -s https://cli.mesmerhq.com/release/install.sh)
# Configure the CLI. You can also set these environment variables in your CI provider's settings.export MESMER_TENANT="your.tenant.mesmerhq.com"export MESMER_PROJECT="xxxxxxxxxxxxxxxxxxxxxxxx" # found in the URL: https://your.tenant.mesmerhq.com/home/$project_id/...export MESMER_AUTH_TOKEN="[redacted]"            # generate with `mesmer auth login`, or "Your Profile" -> "Token" on the web
# Run your build script# Here you invoke Gradle, xcodebuild, etc../build-my-application.sh --output-binary-to myapp.apk
# Submit the build for testing and crawl, capturing the freshly-created build IDbuild_id="$(mesmer -q .buildId submit myapp.apk)"
# Wait for tests to finish, and then log the results.mesmer test wait $build_idmesmer test results $build_id

Installing the CLI#

You can install the latest release of the CLI with the install script:

# Download the CLI, and add it to $PATH for the rest of this shell session$ source <(curl -s https://cli.mesmerhq.com/release/install.sh)

This guide assumes your CI provider is running Linux. For other operating systems, the procedure is similar, but not the same: you'll need to run the appropriate installer for your platform, and use the appropriate shell scripting syntax.

Configuring Authentication#

You can authenticate the CLI through environment variables---set wherever your CI platform keeps its secrets. Generally, you can configure the environment of your CI runner through your platform's web UI (GitHub Actions, CircleCI, Travis CI).

# (in your CI script, or configuration UI)$ export MESMER_AUTH_TOKEN="[your auth token]"

You can generate an auth token by running mesmer auth login locally, followed by a mesmer auth list --show-tokens.

Configuring your Project#

To configure the CLI to find the right tenant and project, check your .mesmer.yml project file in to version control:

# (locally)
# If you don't have one already, create a .mesmer.yml file# containing your tenant and project ID.$ [[ ! -e .mesmer.yml ]] && mesmer init
# Check in .mesmer.yml$ git add .mesmer.yml && git commit -m "Add .mesmer.yml" && git push

Alternatively, you can set environment variables to inform the CLI about your project:

# (in CI)
$ export MESMER_TENANT="your.tenant.hostname.mesmerhq.com"$ export MESMER_PROJECT="xxxxxxxxxxxxxxxxxxxxxxxx"

Automating your tests#

To automatically submit test and crawl jobs to Mesmer, build your application, then call mesmer submit on the final binary:

# (in your CI script)
# Build your app$ ./your-build-script -o build.apk
# Submit the build for testing and crawl# Use an output query to pull off the buildId property# Note: make sure that `jq` is installed in your CI environment.$ build_id="$(mesmer -q .buildId submit build.apk)"
โ”€โ”€โ”€ Creating the build โœ“  Uploaded build     Build ID  aaaaaaaaaaaaaaaaaaaaaaaa
โ”€โ”€โ”€ Queueing up jobs โœ“  Got test configuration โœ“  Fetched compatible devices โœ“  Selected device for tests โœ“  Tests in queue โœ“  Crawl in queue
โ”€โ”€โ”€ Waiting for jobs โœ“  Tests in progress โœ“  Crawl in progress     Crawl ID  bbbbbbbbbbbbbbbbbbbbbbbb
โ”€โ”€โ”€ Done!
    Tests    https://some-tenant.mesmerhq.com/home/xxxxxxxxxxxxxxxxxxxxxxxx/testresults/aaaaaaaaaaaaaaaaaaaaaaaa
    Crawl    https://some-tenant.mesmerhq.com/home/xxxxxxxxxxxxxxxxxxxxxxxx/app-map/aaaaaaaaaaaaaaaaaaaaaaaa
# Wait for tests to finish...$ mesmer test wait $build_id
# ...And log the test results$ mesmer test results $build_id

This will upload the build for testing and crawl, wait for all tests to complete, and then log their results.