An AI agent talking to your customers: what goes wrong
Six failures that repeat in production, in the order they appear, and the worst of them: the one that raises no error and looks like nobody wrote in.
Putting an AI agent in front of customers is one of those things that demos beautifully and gets estimated badly. The demo answers flawlessly, the client gets excited, and the problems show up two weeks after launch, when real people start writing things nobody tested.
These are the failures that repeat. They are not theoretical: they are the pattern of what we see in real implementations, told without identifying anyone — the cases belong to each client, the pattern is knowledge.
The demo always works
Because in the demo the questions are asked by whoever built the agent. They arrive well written, in order, on covered topics, in a single channel, with nobody angry on the other end.
Production is the opposite: half-finished messages, questions asking three things at once, people who write on Monday through one channel and Tuesday through another, and a steady share of cases the design never anticipated. The difference between the two situations is not model quality: it is the design of the system around it.
Six failures, in the order they appear
| Failure | How it looks from outside | Where it gets fixed |
|---|---|---|
| Never escalates to a human | The customer insists and the agent repeats itself | Handoff rules by intent, not by keyword |
| Loses the thread across channels | Started on one channel, treated as a stranger on another | Shared contact identity across channels |
| Duplicates in the business system | Two bookings, two records, two quotes | Search before create, and update instead of insert |
| Leaves no trace for the human | Whoever picks it up does not know what was said | The whole conversation into the CRM, not just the outcome |
| Invents a commercial condition | Promises a price, a deadline or a warranty | Closed catalogue of facts and a ban on filling gaps |
| Goes down and nobody notices | Looks like nobody wrote in that week | An absence alert, not just an error alert |
1. It never escalates. Almost always because handoff was programmed by keyword: if the customer writes “talk to a human”, escalate. But angry people do not write that — they repeat the question with less patience. The useful handoff signal is intent and context: two failed attempts, a topic outside scope, or any mention of a complaint.
2. It loses the thread across channels. Conversations exist per channel, but the person is one. Without a shared contact identity, someone who already explained their case has to explain it again — and that is the interaction that makes a customer demand a human, rightly.
3. It duplicates in the business system. The most expensive of the six, because it pollutes data everyone else later uses. The pattern is always the same: the agent creates instead of updating. Rescheduling produces a new booking rather than moving the existing one; a returning contact generates a second record. The rule that prevents it is boring: search before you create.
4. It leaves no trace for the human. The agent resolves half, hands off, and whoever picks it up gets a ticket with no context. For the customer that is worse than having had no agent: they already told their story once.
5. It invents a commercial condition. The most dangerous failure, because it sounds right. A price that does not exist, a deadline nobody can meet, a warranty that is not offered. It is not fixed by asking the model not to make things up: it is fixed by giving it a closed catalogue of facts and forbidding it to fill gaps. If the fact is missing, the correct answer is to escalate, not to approximate.
The worst one: the failure that looks like silence
The sixth deserves its own section, because it is the only one that can run for weeks unnoticed.
When an agent goes down — a token expires, an API changes, someone disables a rule — no error appears anywhere. What appears is nothing. And “nothing” looks exactly like “not many people wrote this week”. By the time someone asks, unanswered enquiries have already passed that cannot be recovered.
Making it worse: several messaging integrations have time windows for replying. Past that deadline the reply can no longer be sent even once the system is back. What was lost in the silence does not come back afterwards.
What a well-built agent has
None of this is about the model. It is about what surrounds it:
- Explicit action limits. What it may read, what it may write, what it must not touch. High-impact actions are not enabled without controls.
- Human fallback with handoff rules by intent, and somebody on the other side.
- Versioned or documented prompts. If nobody knows which instructions produced an answer, nobody can audit why it answered that.
- Tested with normal and extreme cases. The extreme case defines perceived quality, because it is the one people retell.
- Criteria approved by the client. Responses, limits and escalation. The agent represents their business, not ours.
- Volume monitoring, not just error monitoring. Because of the silent failure.
The first five are the ones we already publish in our Security Policy. The sixth we added from experience.
When it should not be an agent at all
- When volume does not justify it. With few conversations a day, a person answers better and costs less.
- When every answer is a commercial decision. If price depends on the case, the agent qualifies and hands off; it does not quote.
- When the underlying information does not exist. An agent cannot answer what the company has never written down anywhere. Writing it down is the previous project.
- When what you want to fix is the delay. If the problem is that nobody replies within 48 hours, a well-placed alert solves it for far less. That is classic automation, not AI.
FAQ
- Will an agent replace my support team? In our experience it changes what the team does rather than removing it. The agent absorbs the repetitive volume; the people handle the exceptions, which are the cases where the business is actually at stake. Planning for replacement is how you end up with neither.
- Should customers be told they are talking to a bot? We aim for reasonable transparency about the automated nature of the conversation, especially where it can affect commercial decisions or expectations of service. Beyond any legal duty, people usually work it out — and finding out afterwards costs more trust than saying it upfront.
- What happens if the agent says something wrong? Commercially, the business answers for it — which is why the client has to approve the criteria, responses, limits and escalation rules before launch. Technically, that is what action limits and human fallback are for.
- How long before it works properly? Longer than the demo suggests and shorter than a rebuild. The realistic sequence is: launch narrow, watch real conversations for a few weeks, and widen only where the transcripts say it is safe.