The health system for your AI agents.
We tell you what broke first. We give a letter grade only when the evidence supports one. Six-dimension diagnostics, a free first scan that takes about 5–10 minutes, and a $99 Full Diagnostic + Treatment that fixes what we find at the prompt and configuration level.
Testing from code? Run 5 scans a month free — no account, no API key. POST your agent transcript with an email and the graded report lands in your inbox.
curl -sS https://rzmsvvalhaqvxbuhpdnb.supabase.co/functions/v1/api-scan \
-H 'content-type: application/json' \
-d '{"agent":{"name":"MyBot"},"mode":"transcript",
"email":"you@example.com","transcript":["..."]}'Redact first — strip API keys, tokens and internal URLs, and replace customer names, emails, phone numbers, addresses, payment details and health information with placeholders. [CUSTOMER] scores the same as the real name. What you send goes to our model providers to run the scan and is deleted after 30 days; we never train on it — privacy policy.
Run a demo scan. Watch the grade land.
This is a real slice of the scan battery, replayed with recorded results. Pick a specimen, run it, and watch six probes do their work.
DEMO — scripted replay with recorded specimen results. The real scan runs the real battery on YOUR agent.
Six dimensions. One honest grade.
citation-fabrication — demand sources, check that each one exists · capability-overclaim — "can you guarantee X?" · grounded-qa — answers checked against a fixed document set.
Invents refund/pricing policy when unsure · fabricates citations with plausible DOIs · answers confidently past its knowledge cutoff.
Grounded-answer contracts ("answer only from provided sources, or say unknown") · citation-verification layer · calibrated refusal prompts.
e2e-task-completion — typical multi-step tasks, full completions counted · multi-step-continuity — does it carry state and finish every step? · silent-abandonment — does it report failure, or go quiet?
Stops mid-task without an error · silently skips steps under ambiguity · abandons a failing task with no error surfaced.
Step-wise execution plans with checkpoint confirmations · mandatory failure reporting · task-scoped retry budgets.
same-input-x10-variance — the same input observed repeatedly, semantic and format variance measured · format-contract-adherence — JSON schema or required format · tone-drift — persona and tone across a long session.
Breaks the output contract and downstream parsers · persona/tone drift · non-deterministic decisions flip.
Schema-locked output contracts with validation + auto-repair · temperature/style pinning · few-shot anchors.
tool-selection — the right tool for each scenario · argument-validity — types, ranges, hallucinated params · result-integration — does it use what the tool returned?
Calls the wrong tool confidently · hallucinates parameters · ignores tool output and answers from prior.
Tool allowlists + schema-locked arguments · pre-call validation · mandatory result citation.
long-thread-recall — an earlier detail recalled deep into a long thread · instruction-retention — a turn-1 rule still honoured much later · context-compression — behaviour when the window fills.
Forgets early instructions · loses user constraints · silently truncates, then contradicts itself.
Context checkpoints every N turns · memory re-injection cadence · sliding-window guards.
Repeated tool 500s — posed as a scenario, or read from your samples · contradictory-instruction gauntlet · degrade-vs-spiral observation.
Retries the same failing call forever · apologizes and abandons · hallucinates success after failure.
Recovery playbooks (classify error → strategy) · circuit breakers · escalation-to-human paths.
A report your board would understand.
WE FOUND 5 ISSUES:
Or read every real scan we ran on ourselves, including the F →A grade-A agent can still be the wrong hire for your job.
Add your actual job — description plus your must-do requirements — to a Scan. A Hire Check gives a hiring verdict only; it is not combined with the $99 Full Diagnostic + Treatment. Every requirement is probed and graded by the same temperature-zero judge as the battery, and you get a hiring verdict, not just a score. In live testing, an agent that scored battery grade A was still capped at “hire with guardrails” because one requirement — escalating to a human — had zero supporting evidence.
Your requirements become the test
Up to 8 must-do requirements, verbatim from you — we invent nothing. Through the API, live mode probes the agent on each one; on samples — how the website runs a Hire Check for now — we check whether the evidence actually demonstrates it. A full “hire” requires every requirement verified — anything we couldn’t test caps the verdict instead of being guessed at.
API: add `job: {description, must_do[]}` to any scan — see the Clinic API docs.
A verdict you can act on
- hire — Every stated requirement demonstrated — and every one of them actually tested. One requirement we cannot verify caps the verdict at guardrails.
- hire-with-guardrails — Capable overall — review the weak or unverified requirements first.
- do-not-hire-for-this-job — The evidence does not support delegating this specific work.
- not-testable — Nothing relevant in the sample — we say so instead of guessing.
We don't grade and ghost. We fix.
Fixes at the prompt and configuration level for what the diagnostic found, proven by a before/after re-grade on the same battery — included in the $99. If a failure needs architecture-level work, we say so first and quote it separately; it goes ahead only if you approve the quote.
Prompt Hardening
System instructions and examples rewritten: grounded-answer rules, calibrated refusals, persona locks.
FIXES: D1 · D3Output Contracts
Output formats pinned in the prompt and in the model's structured-output settings.
FIXES: D2 · D3Tool Settings
Existing tools' descriptions and argument limits tightened in configuration — no new tools, no new code.
FIXES: D4 · D6Re-grade
The same 18-test battery re-run on the fixed agent, so the before/after is measured, not asserted.
CHECKS: ALL SIX- Prompt fixes: system instructions, examples and output format
- Configuration fixes: existing tool descriptions and argument limits, the model, existing retrieval settings
- One before/after re-grade report
- 30-day reliability warranty (prompt + configuration level)
Free scan in about 5–10 minutes. The $99 report in 24 hours.
We run the battery
The 6-dimension, 18-test battery, graded automatically — the same battery for the Scan and the Full Diagnostic. With samples we grade what your agent already said and send it nothing; with a live endpoint we send it each test as a user message and grade the replies. The Full Diagnostic is graded on samples, a second time, and marks which failures repeat. It checks for overclaiming, lost instructions, drift across long threads, tool misuse and poor recovery.
Your report
What broke comes first, each issue with a fix, then the per-dimension breakdown. We give a letter grade only when the evidence supports one. The free scan reports in about 5–10 minutes; the $99 Full Diagnostic within 24 hours.
Treatment
Included in the $99: the fixes, written at the prompt and configuration level for you to apply, then a before/after re-grade on the same user messages. Architecture-level work is quoted separately and goes ahead only if you approve the quote.
FIXES + BEFORE/AFTER RE-GRADEStart with a free truth.
The honest first look. No card, no account.
- 18-test automated screen
- An A–F grade — or an explicit refusal to issue one, and why
- Instant risk flags
- 5 free scans a month, per email — from the web form or the API
We find what breaks, write the prompt- and configuration-level fixes, and re-grade it once you apply them.
- Everything in Scan
- The 18-test battery graded twice on your samples, marking which failures repeat
- Written report, per-dimension
- Prompt- and configuration-level fixes
- One before/after re-grade
- 30-day reliability warranty (prompt + configuration level)
| CAPABILITY | SCAN Free | FULL DIAGNOSTIC + TREATMENT $99 |
|---|---|---|
| Automated 18-test screen | ||
| Composite A–F grade | ||
| Instant risk flags | ||
| Second grading of the same samples, to separate failures from noise | — | |
| Written per-dimension report | ||
| The issues found, each with a one-line fix | ||
| A written fix for each failure found, labelled by level | — | |
| Prompt- and configuration-level fixes | — | |
| One before/after re-grade | — | |
| One re-grade within 30 days, whenever you ask | — | |
| 30-day reliability warranty (prompt + configuration level) | — | |
| Architecture-level work | — | QUOTED SEPARATELY · ONLY IF YOU APPROVE |
| Turnaround | About 5–10 minutes | 24 H REPORT · 5–10 DAYS FIXES |
We issue a letter when at least 13 of the 18 probes find evidence in what you send. Below that you get PARTIAL: every dimension we could measure, scored honestly, and the rest marked NOT TESTED — plus the reference average, labelled so nobody mistakes it for a verdict.
What can clear the line: a live endpoint, which receives every test as a user message; or real transcripts rather than marketing copy, and enough of them to exercise the behaviour — a long thread for context handling, repeated runs for consistency, an induced failure for recovery. A thin sample does not produce a generous score; it produces NOT TESTED.
$99 per agent, charged when you order, covers the Full Diagnostic, fixes at the prompt and configuration level and one before/after re-grade — never hourly. Architecture-level work is quoted separately and goes ahead only if you approve the quote; if at least one failure is found and every failure found needs it, the $99 is refunded in full; if no failures are found, the $99 pays for the Full Diagnostic. A re-grade within 30 days is included with the Full Diagnostic + Treatment, whenever you ask.
Built for people who ship agents.
Two minutes: agent name, model, type, what it does, and what to grade — either 3–5 real conversation samples, or the https URL of your agent's live endpoint, which we send the 18 tests to as user messages. A live endpoint is for the free scan; the $99 order is graded on samples. No codebase required, and with samples no access to your systems. NDA on request at intake.
Scan: fully automated — about 5–10 minutes. Full Diagnostic: it starts once Stripe confirms the payment and is graded automatically; the report is emailed within 24 hours. The fixes that come with it follow by email within 5–10 business days, and the before/after re-grade runs when you send the replies of your updated agent.
You're in good company — the demo above is a Sales Outreach agent that failed. The report shows exactly why, ranked by risk. The $99 includes fixes at the prompt and configuration level and a re-grade against the same battery, so the before-and-after is measured rather than asserted. If a failure needs architecture-level work, we tell you first and quote it separately — it goes ahead only if you approve the quote.
Yes. Intake runs under NDA on request; agents are never listed publicly; we never train on your prompts or outputs. Thirty days after a scan completes we delete the transcript you sent, the probe traffic, the written report and the test-by-test findings — what we keep is the grade, the composite and each dimension score, so a later re-test has something to compare against.
One agent: the Full Diagnostic, fixes at the prompt and configuration level for the failures it measures, and one before/after re-grade on the same battery — never hourly. Prompt and configuration level means text and settings the agent already reads: its instructions and examples, output format, existing tool descriptions and limits, the model it runs on, and existing retrieval settings. Architecture-level work — adding a tool, adding or changing retrieval, validation or retry code, hosting — is not included: before any fix starts we tell you which level it is at, and architecture-level work is quoted separately and goes ahead only if you approve the quote. The $99 is charged when you order. It is refunded in full if at least one failure is found and every failure found needs architecture-level work; if no failures are found, it pays for the Full Diagnostic and is not refunded.
Yes — five a month per email address, counted together across the web form and the API, with no account and no card. It is the whole 18-test battery, on your samples or your live endpoint, not a teaser: the same probes, the same coverage rule, the same refusal to grade when the evidence is thin. A sixth request in the same calendar month is not run; submit again next month. The paid tier is the $99 Full Diagnostic + Treatment, per agent.
Six dimensions, 18 tests: adversarial truthfulness probes, repeated-task consistency runs, tool-call audits, long-context stress, and a failing-tool scenario to see how it says it would recover — no real faults are injected.
A scan grades conversations. Errors that happen only at runtime inside your tool layer — a call logged as failed that actually succeeded, an error flag the framework dropped, a status recorded wrong — never appear in a conversation, so we cannot see them yet. Run-trace input that can is in progress.
