Could AI Customer Service Be Driving Customers Away?

Imagine you walk into a hardware store to buy a replacement faucet. You’ve got one question on your mind: will this faucet fit the sink you already have?

You find someone in a store vest and ask. They look at the faucet. They look at you. Then they tell you it’s a great faucet, very popular, which isn’t what you asked. So you try again, with the sink measurements this time. They nod and point you to the bathroom aisle (the one you just came from).

Your confidence in this employee, and the whole store, is slipping. So you ask what you probably should have asked first: “Do you usually work in plumbing?”

“Garden,” they say. “It’s my second day.”

Ahh. But that isn’t really their fault. It’s a management problem. Nobody trained them for the questions they’d get, showed them how to read a spec sheet, or told them who to send you to when they didn’t know. Still, you walk out with the same question you came in with, and probably without the faucet.

AI customer service can leave a shopper in that same spot. An agent the brand hasn’t trained, taught how to use its product information, or told when to pass a customer along will still answer. It just answers like someone on their second day in the wrong department.

So shoppers do what you did in that aisle. They rephrase the question, ask again, then type the one thing they’re sure will work: “Can I talk to a person?”

That doesn’t mean they dislike AI. In Verint’s State of Customer Experience 2026 survey of 5,000 U.S. consumers, 61 percent said they prefer talking to a human, but 69 percent of them said they’d switch to self-service if it could fully solve their problem. What they want is someone who knows.

The shopper who asks for a person is at least telling you something went wrong. The bigger risk is the one who doesn’t. They trust the agent’s answer, order the faucet that doesn’t fit, and send it back two weeks later, while that chat shows up in your reports as a success.

On Episode 3 of the DTC Revenue Rocket Podcast, recorded live at Shoptalk, our CEO, Zak, hosted a panel with leaders from Klaviyo, Shopify, PostPilot and Clear.io on how ecommerce brands are using AI to grow. When Klaviyo’s Gina Potenza asked how brands can trust AI to get it right, Shopify’s Dwayne Doshier answered: “I’d say you don’t.” Instead, “check on it, check on your agents. And this is not a set-and-forget-it type of thing.”

The good news is that an AI agent can learn the way a new hire does. Below are its three most common mistakes, how to spot each one, and how to fix them.

How to Spot AI Customer Service Mistakes 

Most AI customer service dashboards report two things: how fast the agent replied, and its resolution rate, meaning the share of conversations it finished without handing the customer to a person. Klaviyo, for example, tells brands to aim for a resolution rate of 70 percent or higher with its Customer Agent. But a high resolution rate only means a lot of conversations ended without a handoff. It doesn’t mean the customer got the right answer.

The only way to know that is to read the conversations. Each mistake below shows up in a different kind of conversation, so for each one, we’ve listed which conversations to read.

Wrong Answers That Get Marked Resolved

A shopper asks whether a replacement filter fits their 2021 air purifier. The product page says the filter works with that brand but doesn’t list which models, so the agent assumes it fits and says yes. The shopper thanks it, buys the filter and closes the chat. The filter doesn’t fit. Or the agent quotes a 60-day return window from an outdated policy page when the current window is 30 days.

Either way, the conversation ended politely, so it counts as resolved. The cost shows up later: a return you pay to ship back and restock, and a customer who now doubts the next thing your agent tells them. Nothing in your report links that return back to the chat.

To catch these, read a sample of resolved conversations and check each answer against your current product pages and policies. Start with:

• Topics where a wrong answer is expensive: compatibility, sizing, shipping times and returns.
• Customers who returned an order marked “didn’t fit” or “not as described.” In Klaviyo, a customer’s conversations are saved to their profile, so you can see whether they chatted with the agent before buying, and what it told them.

When ECD reviewed Customer Agent conversations for one client, a welding and pipeline safety gear brand, it found the agent telling shoppers a bundle didn’t exist when the store carried five.

A conversation that ended happily can still be the one that cost you a sale.

Answers Too Generic to Help

A shopper asks whether a jacket will keep them warm standing at a bus stop in 20-degree weather. The agent replies with the product description: premium insulation, water-resistant shell. Every word is true, and none of it answers the question. So the shopper rephrases it, gets the same reply, and asks for a person.

That handoff costs you twice. Someone on your team now has to answer a question the agent should have handled, and the shopper has waited longer than if they’d emailed in the first place. If one shopper asked that question, others likely have too, and some of them left instead of asking for a person.

So start with the conversations that ended in a handoff. Klaviyo recommends reviewing them to see what’s blocking the agent. For each one:

• Note the question the shopper asked right before the handoff.
• Sort those questions by topic.
• Look for the topics that come up again and again. Those are your agent’s biggest gaps.

That’s how ECD found its client’s: questions about umbrellas and shade kept coming up, and more than 40 percent of them were being handed to a person.

The questions your agent hands off most are the ones it most needs to learn.

Customers Who Never Reach a Person

A customer’s order arrives damaged two days before the birthday it was meant for. They ask the agent for a replacement that will arrive in time. The agent sends the returns policy. They ask again, and the agent sends it again. They type “human,” then “agent,” then give up.

Some problems need a person: an exception to a policy, an upset customer, an order that’s already gone wrong. When the agent keeps answering instead of passing those along, your most frustrated customers get the least help. A problem your team could have fixed in five minutes turns into a refund request, a bad review or a customer who doesn’t come back.

To find these, read conversations where the customer:

• Asked for a person and didn’t get one
• Asked the same question two or three times
• Left without saying the problem was solved

Then check the rules that decide when your agent hands off. In Klaviyo’s Customer Agent, for example, escalation rules are set in Guidance, and the Launch settings decide where those conversations go, whether that’s Klaviyo Helpdesk, Zendesk or Gorgias.

A handoff that comes too late does more damage than one that comes too soon.

How to Fix AI Customer Service Mistakes

When you find a mistake, it’s tempting to correct that one answer and move on. But if the agent got it wrong because something was missing or out of date, it will get it wrong again for the next shopper who asks a similar question. The goal is to fix whatever caused the answer, not just the answer itself.

Klaviyo’s Gina Potenza described AI tools on the panel as a way to go “zero to 0.7” quickly. Zak took the idea further: AI “gets you from zero to point seven,” so you can “shorten your time to go live, but then make sure that everything is in place and accurate.” Launching an agent gets you to 0.7. These three steps get you the rest of the way.