The Mesmer Sidecar is still in early beta.
That means you might run into some issues: your app not installing right, or being incompatible with the test harness.
If that happens, we'd love to hear about it: if you don't mind, please email us at firstname.lastname@example.org
The Mesmer Sidecar is a package that you can deploy to an Android device, attach to a debuggable app, and use to check your app's accessibility.
The Sidecar has a few moving parts.
The first part is the Sidecar itself---an APK that is built and deployed to a test device, and runs as an Android instrumentation test. By resigning the target app, the Sidecar can inspect the app at runtime, examining its view hierarchy and accessibility metadata. Because the Sidecar is running inside the same process as the app under test, it gets the same level of access to the app's runtime environment as the app's own code. This lets us do things other testing systems can't---like software-rendering offscreen or hidden views.
The second part is a transport---tunneling data between the Sidecar and a host computer. The CLI will make requests to the Sidecar for data about the app's internals (say, for instance, the accessibility metadata behind a given view). The Sidecar will interrogate the app under test, and write back a response. This transport doesn't depend on ADB, or really on any connection other than the network. The device running the Sidecar can 'phone home' from anywhere: whether it's sitting on your desk, or in a device farm across the country.
The third part is the WCAG evaluation engine built into the Mesmer CLI---searching the data received from the Sidecar for any violations of the Web Content Accessibility Guidelines. The WCAG 2.0 guidelines, although written originally for web content, are a de-facto standard for mobile accessibility as a whole. The WCAG engine will fetch data about your app from the Sidecar, and audit it under several criteria from the WCAG 2.0 standard. The WCAG engine is also fully explainable. For every violation you see in the final report, there is a matching explanation of where, why, and how that violation was found---and steps you can take to fix it.
Finally, the Mesmer CLI ties these three parts together. The CLI automates the entire deployment process of the Sidecar: building a version of the Sidecar targeting your app, signing it, deploying it to a device, and running the audit.
The end result of all this is pleasingly simple: you can run one command, and in a minute or two, get precise information on accessibility issues in your app.
First, you'll need to install the Mesmer CLI:
Then, make sure you have the following things ready:
- The Android SDK
- We need the Android SDK to build and deploy the Sidecar.
- The Java Development Kit (any version from 8-14)
- We need
javacto build the Sidecar itself, and
keytoolto manage temporary signing keys.
- We need
- An Android device or emulator, connected to ADB
- We use ADB to deploy apps to the device, run audits, and relay results back to the CLI.
- An APK of your app, built with
- And finally, we need a binary of your app to test.
The CLI needs a copy of the Android SDK installed to build the Sidecar, and to deploy apps to a connected device.
You can install the SDK by installing and setting up Android Studio, or by installing a distribution packaged for your OS. The CLI knows many common default installation locations for the SDK, and shouldn't require any setup. However, if you've installed the Android SDK in a custom location, make sure to point the
ANDROID_SDK_ROOT environment variable to its installation directory (see the Android docs for details).
Android SDK EULA
Make sure you've accepted the Android SDK EULA. You can do this by running
yes | sdkmanager --licenses, or by installing an additional component (like the SDK Command-Line Tools) through the Android Studio SDK Manager. If you haven't accepted the EULA, the CLI will let you know with an error.
If you've installed the Android SDK via the Commandline Tools package, make sure you've installed the platform and build tools. You can do this with
sdkmanager platform-tools 'build-tools;31.0.0'. The CLI will download the rest of the tools it needs automatically.
You need a JDK version that works with the Android SDK. If possible, install JDK 11. Otherwise, any version from JDK 8 through JDK 14 should work. You can check your Java version by running
java -version in a terminal.
Note that JDK 15 and later is not compatible with the Android SDK build tools. Also note that certain installations of the Platform Tools require Java 8 strictly (specifically, when installed outside of the Commandline Tools package). These odd restrictions are both due to Android SDK internals, and there's sadly not much we can do to change them.
macOS System Java
On macOS, make sure that your system Java has the right version. You can do this by running
java -version in a terminal---you should see a version between jdk 8 and 14 if everything is set up right. The Sidecar will use whichever Java version is first in your
Installing the right version with homebrew
You can grab the right version and add it to the macOS
java wrapper by running these commands:
# Install JDK 11$ brew install openjdk@11 # This symlink command should be suggested by homebrew in the output of 'brew install'.# You need it to add OpenJDK 11 to the system Java wrapper.$ sudo ln -sfn /usr/local/opt/openjdk@11/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-11.jdk
Finally, make sure you have an
.apk of the app you want to test, compiled with the
debuggable flag enabled.We need this flag so that Android will allow the Sidecar to connect to your app. To set this flag, edit your
AndroidManifest.xml and rebuild your app:
<!-- Find the <application> element, and add the android:debuggable attribute: --><application your-attributes-go-here android:debuggable="true"></application>
The Sidecar has problems with apps that use Kotlin reflection. Because it's loaded into the same Java process as your app, the Kotlin reflection libraries conflict with each other, often causing crashes on launch. We're working on a fix for this, by isolating our classes and dependencies from those of your app, but it's not quite ready yet.
The quickest way to get up and running is to run a Quick Crawl. This will iterate through your app's activities, launching each one in sequence, and recording any accessibility violations it finds. A Quick Crawl is a great way to get a general survey of how your app is doing.
You can run a Quick Crawl with
sidecar quick-crawl, like so:
# Build your APK with the debuggable flag$ ./your-build-process.sh --output $target_apk --debuggable$ file $target_apkpath/to/your/app.apk: Java archive data (JAR) # Connect a test device$ adb devicesList of devices attachedR58M340LSLB device # Run the audit# (if the target app is already installed, pass '--force' to reinstall)$ mesmer beta sidecar quick-crawl \ --output report.html \ $your_apk ─── Preflight checks ✓ Targeting app: 'com.app.mesmer' ✓ Discovered test device ✓ Connected to test device Serial R58M340LSLB Model SM-G975U ✓ Checked installed packages; no conflicts found ✓ Forwarding tcp:42423 to device ─── Build Sidecar ✓ Built Sidecar binary ✓ Signed output APKs ─── Deploy to Device ✓ Installed target app 'com.app.mesmer' ✓ Installed Sidecar ✓ Sidecar is online ─── Quick Crawl ✓ Checked 'com.app.mesmer.views.a11y.GroupedImagesActivity' in 0.88s ✓ Checked 'com.app.mesmer.views.a11y.Rule141Activity' in 0.72s ✓ Checked 'com.app.mesmer.views.a11y.G91RuleActivity' in 0.68s ✓ Checked 'com.app.mesmer.views.a11y.F15RuleActivity' in 0.75s ✓ Skipped 'com.app.mesmer.views.a11y.RulesActivity' This activity could not be launched. Usually, this means that this Activity needs parameters to launch. The Activity may also contain errors that prevent it from starting. (target app crashed while launching activity) ✓ Checked 'com.app.mesmer.views.a11y.A11YRulesActivity' in 1.05s ✓ Checked 'com.app.mesmer.views.create_profile.SignInActivity' in 0.70s [ a few dozen lines omitted for brevity ] ✓ Audit complete ✓ Wrote report to 'report.html' ─── Summary Checked 74 screens, skipped 7 Completed audit in 101.62s Each screen checked took: min 0.65s, max 5.91s, mean 0.80s
If you open
report.html, you'll see the results for every one of your screens, and they'll look something like this:
You can see that this report has pulled data from most of our app's activities---and on this particular screen, it's found a few text-contrast violations.
Our WCAG engine can explain each of these violations. To see this, we can expand the first violation, and the WCAG G18 technique it's using to evaluate Criterion 1.4.3:
Here, we get the entire explanation of why this violation is found: the text color, background color, computed contrast ratio, and conclusion that the text doesn't have enough contrast.
We can also see some of this element's metadata: its classname, and the resource IDs of itself and its layout.
Quick Crawl is only one way to use the Sidecar. You can also run an iterative report, which will repeatedly audit the current screen on your phone as you navigate around your app.
sidecar command docs for more details!