AI Agent: Instructions Validator
in progress
W
Wilson Tan
Business problem:
AI Agents are often setup with instructions that look fine in setup but fail in production because there’s no post-setup QA: gaps, conflicts, and unsafe promises slip through, creating bad answers, compliance risk, and extra human handoffs. Teams end up debugging by visual checking instead of a repeatable checklist, for example:
- Frequent failure modes: conflicting rules (e.g., “auto-detect language” vs. hardcoded first message), undefined placeholders/links, and broken URLs.
- Operational gaps: pricing shared without delivery country, multi-currency in a single reply, vague handoff criteria, and missing refusal scripts (delivery/stock/payment).
- Resulting impact: lower trust/adoption, higher support load, slower time-to-live, and escalations that could have been prevented.
Desired outcome:
Provide a one-click “Validate instructions” action (right after Agent setup/edit) that scans the instruction set, flags issues with clear severity, and offers concrete, safe-by-default fixe — so teams can trust AI Agents confidently.
- Readiness checklist: must-haves for handoff rules, consent copy, stop/unsubscribe variants, product links, and technical notes on demand instructions.
- UX & reporting: inline validator with warnings for conflicting instructions
Jean Paul Carbajal
Why not use Claude through an MCP to audit instructions and conversations? It's all that's needed for it to be very good.
S
Shi Hui
updated the status to
in progress
S
Shi Hui
updated the status to
planned