Soundcheck

Why I built it

End-to-end tests for mobile apps are slow to write and break whenever a screen changes. I wanted regression testing that reads like the test plan a QA engineer would write, and that tests the app the way a person uses it.

How it works

A spec is one Markdown file: what the user does, and the outcomes to judge.

---
title: Check items off with no connection
platforms: [android, ios]
tools: [offline]
---

## Goal

Take the app offline, check off `Eggs` and `Oat milk`, then bring it back online.

## Outcomes

- Offline, the items check off as usual.
- Back online, they stay checked, and a second device sees them checked.

When a run starts, a coordinator agent (Claude) dispatches one tester agent per spec. Each tester drives a real device through Soundcheck's tools (see, tap, fill, swipe, offline, network-drop and more). It reads the screen the way a screen reader does, so an unlabelled control shows up as a finding rather than a flaky test.

The coordinator reviews each claim as it lands. It can uphold a failure, have it reproduced on a fresh attempt, send it back, or put it down to the test harness. At the end it goes through the problems with you one at a time, and the verdict is Ready only when nothing is left open.

Under the hood

• Server (Rust: Tokio, Axum): keeps every run in an append-only journal, serves MCP over local HTTP for the agents, and hosts the live page and reports. Nothing lives only in an agent's memory, so a run carries on after a restart.
• Drivers: persistent reader apps on Android (Java) and the iOS Simulator (Swift) read the screen and inject input in milliseconds; Chrome is driven over the DevTools protocol.
• Isolation: hooks start a blank backend for each platform, and a per-platform proxy lets specs take the app offline or inject faults.
• Distribution: installed as a Claude Code plugin, with skills to set a project up, write specs and run a suite.

In use

Soundcheck has gated every Solo Scores release since v2.17.0, with the suite growing from 183 to 268 checks across iOS, Android and web. Its testers found a stuck-upload bug and missing file states, which led to reworked status icons and an upload tray that shows whether each file is waiting, uploading or failed.