Guardrails and handoff

An AI chatbot that guesses is worse than no chatbot. The guardrails below are what stop a confident wrong answer about your refund policy from becoming a customer service problem.

Confidence thresholds

The bot scores how well the question matches its training content. Below your threshold, it stops answering and escalates instead.

Setting this is a trade-off, and the right setting depends on the cost of being wrong:

• Ecommerce policy questions (refunds, delivery promises, warranty) — set it high. A wrong answer here creates an obligation you did not intend.
• General product information — a moderate threshold is fine. Slightly imperfect answers are better than a queue.
• Anything regulated — do not automate it at all. Use restricted topics.

Restricted topics

Some questions should never be answered by a bot, regardless of confidence. Configure these to route straight to a person with a clear message:

• Medical, legal or financial advice
• Anything involving an individual’s account credentials
• Complaints and escalations — a person should handle an angry customer from the first message
• Cancellations, if you want a retention conversation

For a worked example, see the healthcare solution page, where the boundary between administrative and clinical questions is the whole design.

Fallback replies

Write the “I don’t know” message yourself. The default is serviceable; a good one is specific and moves the conversation forward:

“I can’t answer that one reliably — let me get somebody who can. What’s the best email to reach you on if we miss you here?”

Two things happening there: it admits the limit without apologising three times, and it captures a contact so the conversation survives the visitor closing the tab.

Handoff

When the bot escalates, the full transcript travels with the conversation. The agent picking it up sees what was asked, what the bot said, and where it gave up. Nobody should have to ask the customer to start again — if that is happening, the handoff is misconfigured.

If no agent is online at handoff time, the widget should switch to its offline behaviour and take a message rather than leaving the visitor waiting. See customise the widget.

Testing the guardrails

Test the failure modes deliberately, not just the happy path. Ask it:

• A question your content genuinely does not cover
• A question with a false premise (“what’s your 90-day returns policy?” when yours is 30)
• A hostile or emotional message
• A question about a competitor

The false-premise test is the one that catches the most problems. A well-configured bot corrects the premise; a badly configured one agrees with it.

Next: tickets and the help desk.