We write good manual test plans. They are markdown documents: numbered sections, a table of cases with an ID and a priority and a scenario, explicit steps, an expected result. The most recent one covers every sign-up, sign-in, GDPR, and purchasing workflow in AccelaStudy — 15 sections, 155 cases, 19 tagged defects. It took hours to write and it is genuinely good.
And it was, as an instrument, nearly useless.
You cannot check off a step in a markdown file. You cannot tell who has run what, or when, or against which build. A failure has to be hand-copied into the tracker, which means it either gets copied badly or doesn't get copied at all. And a plan that costs hours to write gets executed once, in one sitting, and then rots — because nothing shows you its coverage decaying.
Keep the document. Add the instrument.
The obvious move is to rewrite the plan inside a tool. We deliberately did not do that.
The markdown is the artifact people actually read and review. It diffs. It lives next to the code. Someone can propose a change to it in a pull request. A test-management tool that swallows the document and hands back a web form has taken something away.
So Assay imports the markdown and exports it back. Round-trip is a law, not a feature: export produces a document a human would accept as the source, and re-importing it yields identical content. Edit the plan in Assay or in your editor, as suits you.
The importer is a real parser, not a model call. Import has to be reproducible — the same document must always yield the same plan, or the tool is not a source of truth about anything. It reads sections, case tables, priorities, ⚠ DEFECT-NN tags, appendices, and Mermaid diagrams, and it tells you what it found before it writes anything.
It also tells you what it had to guess. Source plans cram several steps into one prose cell, and splitting that cell is genuinely ambiguous. Assay splits only on a real author signal — explicit numbering, or semicolons — and reports anything weaker. The bias is deliberately toward under-splitting: a tester can mentally subdivide one step that is really two, but a bogus step boundary makes the checklist lie about what was verified.
That reporting immediately earned its keep. On the very first import of the real plan, it flagged one case it was unsure about — and the uncertainty was a genuine bug in our own splitter, which had read the full stop in "e.g." as the end of a sentence and cut one step into two nonsense halves. The tool caught its own defect before a human ever saw a wrong checklist.
A run is evidence, so it cannot shift
Plan content in Assay is append-only, and a run pins to the exact version it was generated from.
This is the single most important structural decision in the tool. If the plan a run was generated from can change underneath it, every historical run report becomes unfalsifiable — you can no longer say what was actually tested, only what the plan says today. Publish a new plan version mid-run, rename cases, delete some: the in-flight run does not move a byte.
The same reasoning drives a smaller decision that looks like duplication and isn't. A run report says A-07. That string is denormalised onto the run alongside the foreign key, so it survives the plan being re-versioned, the case renumbered, or the plan archived. The key is for joins; the string is for truth.
Assign a case to a person. Or to Claude.
Cases go to teammates, individually or in bulk. They also go to Claude.
Tell a Claude session to work its Assay queue and it claims cases off a leased queue, executes them against the real system, reports each step, and moves on. Agent results are attributed as agent results everywhere they appear.
Four things make that safe enough to leave running.
Claims are leases, not locks. A session that dies takes nothing with it — the lease expires and the work returns to the queue. But every step it already reported stays recorded. Losing the lease loses the claim, never the results.
Attempts are capped. Past its limit, a case is blocked with a stated reason instead of being claimed and released forever. An agent that cannot finish something must not spin on it.
Agent keys are structurally weak. An agent can claim, report, attach evidence, comment, and read. It cannot author a plan, assign work, or manage users — not by policy, but because the token never carries those capabilities. And a case is agent-runnable only when an author explicitly says so, which keeps destructive cases (deletion cascades, live billing) human-only by default.
The claim ships the plan's environment notes. Every claim hands the agent the plan's environment preamble along with the case: the throwaway-account convention, the base URLs, the warning that staging sends real email to real people. An agent executing blind against staging without that context is how you email a stranger.
The concurrency here is not decorative. The first version of the claim query locked the wrong rows: because the query joined the run table, and every case in a run shares one run row, the first agent to claim locked that shared row and every other agent skipped the entire queue. Twelve concurrent claimers produced exactly one claim. In production that would have made the whole agent lane silently single-threaded — and it would have looked like "no work available", not like an error. It was caught by a contention test written specifically to look for it.
A failure that files itself
Failing a case in Assay requires saying what actually happened. Not optionally — the result will not commit without it.
That is because the Docket card is generated from that text, and a card generated from an empty field is worse than no card: it looks like tracked work and carries nothing a fixer can use. So the failure record carries the repro steps with per-step outcomes, expected versus actual, the environment, the priority, and a link back to the case.
Recurring failures deduplicate. The same case failing the same way across three runs comments on the existing card rather than filing a third copy; a genuinely different failure of the same case still files separately.
And nothing fails silently. Every external side effect goes through a durable queue, so a Docket outage cannot lose a defect: the result still commits, the problem is visible on the case, and the card files on retry. "The tracker was down so we dropped it" is not an outcome the design permits.
Built for the keyboard
Working a 155-case plan is repetitive by nature, and repetition is where a mouse hurts. Move between cases and steps, toggle, resolve, jump to any case key — all from the keyboard. ⌘K carries verbs as well as links, so the palette can resolve the case you are already looking at rather than merely navigate to it.
One small guard there matters more than it sounds: the shortcuts ignore keystrokes aimed at a text field. Without it, typing the word "fail" into the what-actually-happened box fires f — fail the case — along with a and i. In a tool whose keys record test results, that is not a UI annoyance; it is silently writing results the operator never intended.
What it is not
Assay is not a test runner. It does not execute your Playwright suites and it does not own CI. It tracks manual and agent-driven execution — the work a machine cannot currently prove for you. It can reference automated coverage so a run skips what CI already proves; it does not replace it.
That distinction is the point. The AccelaStudy plan exists because its highest-risk area — the GDPR deletion cascade — has zero integration or end-to-end coverage anywhere. Those are exactly the cases a human, or a carefully-scoped agent, has to walk through by hand. Assay is for those.
Assay ships with a 33-tool MCP server, so a Claude session can author a plan, generate a run, assign the work, and then turn around and execute it — the same surface the web app uses, all the way down.