Skip to main content

Mesmer Sidecar

BETA SOFTWARE

The Mesmer Sidecar is still in early beta.

That means you might run into some issues: your app not installing right, or being incompatible with the test harness.

If that happens, we'd love to hear about it: if you don't mind, please email us at sidecar@mesmerhq.com

The Mesmer Sidecar is a package that you can deploy to an Android device, attach to a debuggable app, and use to check your app's accessibility.

How it Works#

The Sidecar has a few moving parts.

The first part is the Sidecar itself---an APK that is built and deployed to a test device, and runs as an Android instrumentation test. By resigning the target app, the Sidecar can inspect the app at runtime, examining its view hierarchy and accessibility metadata. Because the Sidecar is running inside the same process as the app under test, it gets the same level of access to the app's runtime environment as the app's own code. This lets us do things other testing systems can't---like software-rendering offscreen or hidden views.

The second part is a transport---tunneling data between the Sidecar and a host computer. The CLI will make requests to the Sidecar for data about the app's internals (say, for instance, the accessibility metadata behind a given view). The Sidecar will interrogate the app under test, and write back a response. This transport doesn't depend on ADB, or really on any connection other than the network. The device running the Sidecar can 'phone home' from anywhere: whether it's sitting on your desk, or in a device farm across the country.

The third part is the WCAG evaluation engine built into the Mesmer CLI---searching the data received from the Sidecar for any violations of the Web Content Accessibility Guidelines. The WCAG 2.0 guidelines, although written originally for web content, are a de-facto standard for mobile accessibility as a whole. The WCAG engine will fetch data about your app from the Sidecar, and audit it under several criteria from the WCAG 2.0 standard. The WCAG engine is also fully explainable. For every violation you see in the final report, there is a matching explanation of where, why, and how that violation was found---and steps you can take to fix it.

Finally, the Mesmer CLI ties these three parts together. The CLI automates the entire deployment process of the Sidecar: building a version of the Sidecar targeting your app, signing it, deploying it to a device, and running the audit.

The end result of all this is pleasingly simple: you can run one command, and in a minute or two, get precise information on accessibility issues in your app.

Setup#

First, you'll need to install the Mesmer CLI:

Install the Mesmer CLI

Then, make sure you have the following things ready:

  • The Android SDK
    • We need the Android SDK to build and deploy the Sidecar.
  • The Java Development Kit (any version from 8-14)
    • We need javac to build the Sidecar itself, and keytool to manage temporary signing keys.
  • An Android device or emulator, connected to ADB
    • We use ADB to deploy apps to the device, run audits, and relay results back to the CLI.
  • An APK of your app, built with debuggable="true"
    • And finally, we need a binary of your app to test.

The Android SDK#

The CLI needs a copy of the Android SDK installed to build the Sidecar, and to deploy apps to a connected device.

You can install the SDK by installing and setting up Android Studio, or by installing a distribution packaged for your OS. The CLI knows many common default installation locations for the SDK, and shouldn't require any setup. However, if you've installed the Android SDK in a custom location, make sure to point the ANDROID_SDK_ROOT environment variable to its installation directory (see the Android docs for details).

Android SDK EULA

Make sure you've accepted the Android SDK EULA. You can do this by running yes | sdkmanager --licenses, or by installing an additional component (like the SDK Command-Line Tools) through the Android Studio SDK Manager. If you haven't accepted the EULA, the CLI will let you know with an error.

If you've installed the Android SDK via the Commandline Tools package, make sure you've installed the platform and build tools. You can do this with sdkmanager platform-tools 'build-tools;31.0.0'. The CLI will download the rest of the tools it needs automatically.

The Java Development Kit#

You need a JDK version that works with the Android SDK. If possible, install JDK 11. Otherwise, any version from JDK 8 through JDK 14 should work. You can check your Java version by running java -version in a terminal.

Note that JDK 15 and later is not compatible with the Android SDK build tools. Also note that certain installations of the Platform Tools require Java 8 strictly (specifically, when installed outside of the Commandline Tools package). These odd restrictions are both due to Android SDK internals, and there's sadly not much we can do to change them.

macOS System Java

On macOS, make sure that your system Java has the right version. You can do this by running java -version in a terminal---you should see a version between jdk 8 and 14 if everything is set up right. The Sidecar will use whichever Java version is first in your $PATH.

Installing the right version with homebrew

You can grab the right version and add it to the macOS java wrapper by running these commands:

# Install JDK 11$ brew install openjdk@11
# This symlink command should be suggested by homebrew in the output of 'brew install'.# You need it to add OpenJDK 11 to the system Java wrapper.$ sudo ln -sfn /usr/local/opt/openjdk@11/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-11.jdk

Your APK#

Finally, make sure you have an .apk of the app you want to test, compiled with the debuggable flag enabled.We need this flag so that Android will allow the Sidecar to connect to your app. To set this flag, edit your AndroidManifest.xml and rebuild your app:

<!-- Find the <application> element, and add the android:debuggable attribute: --><application  your-attributes-go-here  android:debuggable="true"></application>

Known Issues#

The Sidecar has problems with apps that use Kotlin reflection. Because it's loaded into the same Java process as your app, the Kotlin reflection libraries conflict with each other, often causing crashes on launch. We're working on a fix for this, by isolating our classes and dependencies from those of your app, but it's not quite ready yet.

Running a Quick Crawl#

The quickest way to get up and running is to run a Quick Crawl. This will iterate through your app's activities, launching each one in sequence, and recording any accessibility violations it finds. A Quick Crawl is a great way to get a general survey of how your app is doing.

You can run a Quick Crawl with sidecar quick-crawl, like so:

# Build your APK with the debuggable flag$ ./your-build-process.sh --output $target_apk --debuggable$ file $target_apkpath/to/your/app.apk: Java archive data (JAR)

# Connect a test device$ adb devicesList of devices attachedR58M340LSLB device
# Run the audit# (if the target app is already installed, pass '--force' to reinstall)$ mesmer beta sidecar quick-crawl \  --output report.html \  $your_apk

โ”€โ”€โ”€ Preflight checks โœ“  Targeting app: 'com.app.mesmer' โœ“  Discovered test device โœ“  Connected to test device     Serial  R58M340LSLB     Model  SM-G975U โœ“  Checked installed packages; no conflicts found โœ“  Forwarding tcp:42423 to device
โ”€โ”€โ”€ Build Sidecar โœ“  Built Sidecar binary โœ“  Signed output APKs
โ”€โ”€โ”€ Deploy to Device โœ“  Installed target app 'com.app.mesmer' โœ“  Installed Sidecar โœ“  Sidecar is online
โ”€โ”€โ”€ Quick Crawl โœ“  Checked 'com.app.mesmer.views.a11y.GroupedImagesActivity' in 0.88s โœ“  Checked 'com.app.mesmer.views.a11y.Rule141Activity' in 0.72s โœ“  Checked 'com.app.mesmer.views.a11y.G91RuleActivity' in 0.68s โœ“  Checked 'com.app.mesmer.views.a11y.F15RuleActivity' in 0.75s โœ“  Skipped 'com.app.mesmer.views.a11y.RulesActivity'    This activity could not be launched.    Usually, this means that this Activity needs parameters to launch.    The Activity may also contain errors that prevent it from starting.    (target app crashed while launching activity)
 โœ“  Checked 'com.app.mesmer.views.a11y.A11YRulesActivity' in 1.05s โœ“  Checked 'com.app.mesmer.views.create_profile.SignInActivity' in 0.70s
[ a few dozen lines omitted for brevity ]

 โœ“  Audit complete โœ“  Wrote report to 'report.html'
โ”€โ”€โ”€ Summary    Checked 74 screens, skipped 7    Completed audit in 101.62s    Each screen checked took: min 0.65s, max 5.91s, mean 0.80s

If you open report.html, you'll see the results for every one of your screens, and they'll look something like this:

Screenshot of a screen from a Sidecar accessibility report, showing several WCAG text-contrast violations.

You can see that this report has pulled data from most of our app's activities---and on this particular screen, it's found a few text-contrast violations.

Our WCAG engine can explain each of these violations. To see this, we can expand the first violation, and the WCAG G18 technique it's using to evaluate Criterion 1.4.3:

Screenshot of a violation from a Sidecar accessibility report, showing a single violation explained in detail: a brief summary of WCAG 1.4.3, followed by the WCAG engine&#39;s rationale on the G18 technique violation it found, and including element metadata like its resource ID.

Here, we get the entire explanation of why this violation is found: the text color, background color, computed contrast ratio, and conclusion that the text doesn't have enough contrast.

We can also see some of this element's metadata: its classname, and the resource IDs of itself and its layout.

More#

Quick Crawl is only one way to use the Sidecar. You can also run an iterative report, which will repeatedly audit the current screen on your phone as you navigate around your app.

See the sidecar command docs for more details!