AI Agent: Instructions Validator
in progress
W
Wilson Tan
Business problem:
AI Agents are often setup with instructions that look fine in setup but fail in production because there’s no post-setup QA: gaps, conflicts, and unsafe promises slip through, creating bad answers, compliance risk, and extra human handoffs. Teams end up debugging by visual checking instead of a repeatable checklist, for example:
- Frequent failure modes: conflicting rules (e.g., “auto-detect language” vs. hardcoded first message), undefined placeholders/links, and broken URLs.
- Operational gaps: pricing shared without delivery country, multi-currency in a single reply, vague handoff criteria, and missing refusal scripts (delivery/stock/payment).
- Resulting impact: lower trust/adoption, higher support load, slower time-to-live, and escalations that could have been prevented.
Desired outcome:
Provide a one-click “Validate instructions” action (right after Agent setup/edit) that scans the instruction set, flags issues with clear severity, and offers concrete, safe-by-default fixe — so teams can trust AI Agents confidently.
- Readiness checklist: must-haves for handoff rules, consent copy, stop/unsubscribe variants, product links, and technical notes on demand instructions.
- UX & reporting: inline validator with warnings for conflicting instructions
Jean Paul Carbajal
Why not use Claude through an MCP to audit instructions and conversations? It's all that's needed for it to be very good.
W
Wilson Tan
Jean Paul Carbajal: Thanks for the suggestion — it's a great idea.
One important nuance: even using Claude through an MCP, the model on its own doesn't have your workspace context — your tag IDs, workflow IDs, and most of all respond.io's AI Agent best practices. Without that context, auditing the instructions would be limited.
The good news: that's exactly what we're working on. We're building MCP endpoints so the AI can access respond.io features and your workspace data, so the AI Agent can be created (and validated) accordingly.
You can follow and vote on a related request here: https://respond.canny.io/feature-request/p/ai-agent-api-mcp-to-create-and-edit-ai-agents
This request is already in progress. Thanks for helping us make it better!
S
Shi Hui
updated the status to
in progress
S
Shi Hui
updated the status to
planned