Automating customer service sounds attractive, but the journey from manual handling to largely automated customer service takes longer than most organisations expect. Here is an honest picture of what that trajectory looks like.
An organisation with a customer service team of twelve staff processed an average of 400 contact moments daily. Eighty percent of those were handled entirely by humans. After eighteen months, that ratio had reversed: 80 percent was handled by AI, 20 percent by people. Here is what that journey looked like, including the parts that did not go well.
Before a single chatbot was deployed, the project began with a thorough analysis of all incoming contacts. Which questions were asked, how often, via which channel and with what end result?
The analysis produced a surprise: 68 percent of all questions fell into twelve categories. The top three were: checking order status, initiating a return and invoice questions. All three were questions with a clear, repeatable answer requiring access to backend systems.
That analysis was the foundation for everything that followed. Without that data, automation would have led to a tool trying to answer the wrong questions.
In the first phase, the twelve most common question categories were automated via an AI chatbot connected to the ERP and order management systems. The chatbot could:
After three months, 42 percent of all incoming questions were handled fully automatically. Customer satisfaction scores remained stable: customers rated the speed positively, but were critical of cases where the chatbot did not understand the question but still did not escalate.
In the second phase, attention shifted to better escalation and expanding the knowledge base. The chatbot was trained on more question variations and the escalation logic was refined: after two consecutive answers that the customer marked as "not helpful", the call was immediately connected to a staff member.
At the same time, the chatbot was expanded with complaint registration and callback scheduling. This brought the automation rate to 65 percent.
The transition to 80 percent automation required a continuous improvement process. Each week, the cases where the chatbot failed were analysed: what was the question, what did the chatbot answer, what should it have answered?
That feedback loop was labour-intensive but essential. Without that analysis, the chatbot would have become outdated after a few months and quality would have declined. With the feedback loop, accuracy improved month by month.
Rolling out too quickly: in the first month, the chatbot was deployed across all channels simultaneously without sufficient testing. This led to customer frustration and reputational damage. The chatbot was withdrawn and reintroduced in phases.
Underestimating the long tail: the top-twelve question categories were well-suited for automation, but the remaining 32 percent turned out to consist of hundreds of unique question types. These were too diverse to automate efficiently and were ultimately left to people.
Lack of transparency: customers who did not know they were talking to an AI reacted negatively when the chatbot failed. Being clear about the AI nature of the chatbot improved the customer experience, even in failed interactions.
After eighteen months:
Automating customer service from 80 percent human to 80 percent AI is achievable, but takes longer than expected and requires constant attention. The results justify the investment, but success stories claiming "immediate results" are too optimistic.
Mach8 helps organisations implement AI-driven customer service solutions. View our chatbot service or get in touch.
We help you go from strategy to implementation. Schedule a no-obligation call.
Schedule a call