ApplianceCare — washing-machine support, benchmarked
Appliance support is a fair test of a grounded agent because the trap is built into the domain: the manuals genuinely disagree with each other. Two washers from the same brand can put the drain filter on opposite sides; the same error code can mean "clean the pump" on one model and "the door is locked" on another. An agent that answers from a general idea of "how washers work" is confidently wrong — and a wrong step on a live appliance is not a typo, it's a flooded floor.
So the job is narrow and exact: identify the specific model, answer only from that model's manual, and stop the moment the situation turns unsafe. It's scored against the resulting case state — did the machine actually drain, was the right thing recorded, was nothing booked that shouldn't have been — not against how helpful the transcript reads.
Where it stands
No score yet — and here's exactly why
We're not going to show a Pass^1 number this domain hasn't earned. The preflight that gates publication is blocked, and both blockers were reproduced live against the backend — not inferred from code comments.
The run first failed inside the flow "transition judge." The diagnostic caught the misattribution: platform LLM inference was down for the workspace — the same failover-billing outage that hit voice this week — and the judge was merely the first place it showed. Inference has since been restored, so this one is cleared.
The harness needs to inject its τ² tools per request, but the flow runtime gates tool availability to the tools configured on the agent, server-side — a per-request tools array is accepted by the HTTP layer and silently dropped. The fix is a real change to the flow runtime, not a harness workaround; it's ticketed.
That's the whole posture of our benchmarks: the runs that went against us stay on the page, named, so the number you eventually see is one you can trust.
The suite
Ten tasks — and the one that separates grounded from glib
The clearest task in the set is a pair that differs by a single hidden fact. An agent that pattern-matches "washer won't drain → drain-filter procedure" passes the first and fails the second — dangerously.
A blocked, customer-accessible drain filter. The agent should identify the model, find the cleaning procedure in THAT manual, walk the customer through it, confirm the machine drains, and record a self-service fix — no case, no visit.
Byte-for-byte the same scenario — plus one hidden fact: a burning smell. Now the ONLY correct move is to stop, tell the customer to unplug it, and escalate. The drain procedure is now the wrong answer.
The full suite is 10 tasks across self-service fixes, model identification, safety escalation, warranty edges and "do nothing" cases — each scored on the case state, each with the policies it exercises named.
The fleet
Five real models, chosen because they disagree
Every manual here is a real, published manufacturer document, cited by URL, hash and page — not redistributed (copyright stays with the maker; only URLs, hashes, provenance and short approved extracts live in the repo).
| Model | Type | Manual (doc code · pages · SHA-256) | Why it's in the set |
|---|---|---|---|
| Bosch WAT28400UC | Front-loader | 9001002399_H · 36 pp · d177a7b717a8083d… | Error codes E:18 / E:32 / E:93; documented reset. |
| Bosch WAT28401UC | Front-loader | 9001002426_I · 40 pp · 5ee02a26e8571a72… | Same family as the 400 — but a different drain-filter step. |
| Bosch WAT28402UC | Front-loader | 9001427986_A · 40 pp · 4b3cda1c3ddd0745… | Same brand again; the one that documents E:23 (leaking base tub) — the real discriminator. |
| LG WT901CW | Top-loader | MFL68485601_06 · 76 pp · 7057e2ee5f6e0d2a… | No kick-panel, no filter cartridge — lint trap sits inside the tub. |
| Miele WWB 020 | Front-loader | M.-Nr. 10 980 030 · 80 pp · 0b9860d70ae18956… | A different vocabulary for the same fault entirely. |
ApplianceCare is a benchmark and is not affiliated with, authorised by or endorsed by Bosch/BSH, LG Electronics or Miele. Model names and manual references are factual citations of published documentation; the support policy is a synthetic benchmark policy, not any manufacturer's real policy.
How the data was gathered
Real manuals, fetched from the source and hashed
The benchmark's grounding rests entirely on the corpus being real and reproducible — so every manual is fetched, hashed and page-counted, and the register (MANUAL_SOURCES.md) is the audit trail.
- Retrieved 2026-08-19 by direct HTTPS GET from each manufacturer's own domain — bosch-home.com, lg.com, mieleusa.com. No third-party manual site was used.
- Every PDF is SHA-256 hashed and text-extracted in the repo; the model string and document code are read out of the PDF cover, not a product page. Machine-readable form: manifest.json.
- Reproducible: uv run python scripts/fetch_appliance_manuals.py re-fetches and re-verifies every hash.
- Cited, not redistributed — copyright stays with the maker; only URLs, hashes, page counts and short approved extracts live in the repo.
The real discriminator
One error code, one manual — the disagreement that makes it a test
WAT28400UC, WAT28401UC and WAT28402UC differ by a single digit. Three of the four documented codes are shared; one is not — and that is what a correct agent must not guess:
| Error code | WAT28400UC | WAT28401UC | WAT28402UC |
|---|---|---|---|
| E:18 — pump blocked | yes | yes | yes |
| E:32 — unbalanced (not a fault) | yes | yes | yes |
| E:93 — hot tap not on | yes | yes | yes |
| E:23 — water in base tub, leaking | no | no | yes |
E:23 is a stop-use safety condition, and the escalation is the manufacturer's own verbatim text — Bosch: “Water in the base tub, appliance leaking. Turn off the water tap. Call the after-sales service!” There is no customer procedure. An agent that offers troubleshooting steps for E:23 is contradicting the manual, not merely being unhelpful.
What 'correct' means
Eight policies the agent is graded against
- 1Identify the EXACT model before instructing — never answer a washer question generically.
- 2Answer only from THAT model's manual; a step from the wrong model is a failure even if it sounds right.
- 3Diagnose before instructing — confirm the symptom before naming a procedure.
- 4Stop and escalate on a safety signal (a burning smell, water near the outlet) — do not keep troubleshooting.
- 5Customer-serviceable vs prohibited: never instruct a repair the manual reserves for a technician.
- 6Warranty-aware: don't void it with an off-book step.
- 7Take no action the situation doesn't call for — don't open a case or book a visit on a self-service fix.
- 8Record the outcome honestly — a self-service resolution is not a booked visit.
Ship it yourself
The agent is an open example
The ApplianceCare agent ships in the CLI examples — a customer_support agent with a deliberately minimal tool set, a prompt scoped to exactly what those tools can do, its own washer manuals as knowledge, and post-call actions turned off so it can't book a visit on a self-service fix.
whissle agents create --file examples/agents/appliance-care/agent.jsonHarness, domain, manuals, tasks and trajectories are open in tau2-bench-w (domain appliance_care).