Let’s talk

AI · Customer support

Customer-support bots: what to automate and how to hand over without starting over

Customers don't mind a bot. They mind typing their order number three times and then explaining the whole problem again to a person. Here is how to decide what a support bot should handle, how to pass a conversation to a person with the context intact, and how to keep the answers right after launch.

Sort the contacts before you build

Export three months of tickets and chats and tag the reason for each contact. In most help desks a short list of reasons makes up the bulk of the volume. Then sort them into three groups:

  • The bot handles it end to end: order status, delivery dates, return instructions, password resets, store hours, a change of delivery address before dispatch, a copy of an invoice. The answer is clear and the data sits in a system.
  • The bot collects, a person decides: refunds above a set amount, damaged goods, warranty claims, complaints. The bot gathers the order number, photos and a description, so the person starts with everything.
  • A person from the first message: an angry customer on a second contact, a safety issue, a large account, anything your team marks as sensitive.

One rule for the first group: the bot answers from live data or doesn't answer. Replying to "where is my order?" with a link to a tracking page is a deflection. Reading the status and saying "It left our Houston warehouse yesterday and should arrive Thursday" is an answer.

The handover is the product

Most bad bot experiences are bad handovers. A good one has six parts.

1. Clear triggers. Hand over when the customer asks for a person in any wording ("agent", "human", "representative", "real person"), when the bot fails to understand twice in a row, when the tone turns angry, when the topic sits in the third group, or when the order is above a set value.

2. Intake first. Before the handover, the bot identifies the customer and the order and asks the one question the person will need answered: "What's wrong with the item? A photo helps."

3. A summary card, not a transcript. The person sees who the customer is, which order, the issue in one line, what the bot already tried, what the customer wants and how they feel. The full transcript sits below for anyone who needs it.

4. Same channel, same thread. Don't send the customer to email or a phone line. The person picks up the same chat.

5. An honest wait time. "A person will reply in about ten minutes" beats "Please hold." After hours, say when someone will reply and open a ticket with the summary attached.

6. A first reply that proves the person read it. "Hi Dana, I see the blender from order 4417 arrived with a cracked jar. I can send a new jar today or refund the order." Not "How can I help you?"

Training on past conversations

Your chat history is the best training material you have, and it is also full of things the bot should never learn.

  • Pick the good ones. Conversations that were resolved, not reopened and rated well. Use your strongest agents' replies as examples, not all replies.
  • Policies come from documents, not chats. Chats contain last year's return window, discontinued products and one-off goodwill gestures ("we refunded this once"). Write current policies as a separate source and let it override anything in the history.
  • Mask personal details. Replace names, emails, phone numbers and order numbers with placeholders, so the bot learns the pattern, not the customer.
  • Write a tone guide. Five rules with an example each: how you greet, how you apologize, how you say no.
  • Build a test set. Two hundred real past questions with the correct answer, including thirty difficult ones. Run it before every change to the bot.

Quality checks after launch

A bot that tested well in week one drifts as products, prices and policies change. Make checking a routine with an owner.

  • Daily: a person reads 20 random bot conversations and scores each one: correct, complete, right tone, right decision to hand over or not.
  • Weekly: read every low-rated conversation and every case where the customer came back within 48 hours about the same issue.
  • On every policy change: update the bot the same day and rerun the test set.
  • One-click flags: agents mark wrong bot answers from the help desk, and the owner fixes them weekly.

Watch these numbers: share resolved by the bot without a repeat contact, handover rate by reason, repeat contacts within 48 hours, satisfaction for bot and human conversations separately, and time to the first human reply after a handover.

Be wary of false resolution. A customer who gave up and called the phone line looks "resolved" in the chat report. Count repeat contacts across all channels, not just chat.

What to do this week

  1. Export three months of contacts and tag the top 15 reasons.
  2. Sort them into the three groups.
  3. Write the handover triggers and the fields of the summary card.
  4. Pick 200 past conversations for a test set.
  5. Name one person in support who will own the bot and read 20 conversations a day.

Boson Agents, the AI lab of Boson Group, builds support bots and agents with the handover, the test set and the quality routine designed in from the start. Write to team@boson.group with your top contact reasons, and we'll suggest where to start.

More about AI agents at Boson Group

Contact

Send a brief — we reply with a time and cost estimate within 24 hours.

Brief

Tell us about the project.

Send a brief — we reply with a time and cost estimate within 24 hours.

What do you need?

* required