AI agents aren’t cutting it in customer service

Three-quarters of companies have rolled back or shut down an AI agent used in customer service, according to new research.

That high rate of failure in using AI agents for customer communications comes via a survey commissioned by Sinch, which polled more than 2,500 leaders in the industry.

The survey found nearly two-thirds of respondents already had AI agents up and running in their organizations, showing that they were getting beyond pilots into real use cases in customer service. Indeed, 88% are set to have agents in full production over the next year.

But they're having issues with those deployments, Sinch found. 74% said they had shut down or rolled back AI customer communications agents due to governance failures – that echoes analysis from Gartner last year that suggested companies were backtracking on AI customer service and rehiring human workers.

Latest Videos From

According to the Sinch report, the leading factors behind roll-backs lay in concerns about customer data exposure (31%), hallucinations or brand risks (22%), and lack of auditability (16%).

“These aren’t abstract risk categories. [Personal identifiable information] exposure means a customer’s personal data surfaced in an interaction it shouldn’t have. Hallucination means an AI agent said something confidently wrong to a real customer, on a live channel, under your brand’s name,” the report noted.

The Sinch survey follows research via Pega that showed more than half of customers aren't confident that organizations are using AI responsibly in customer service, while a report from 2024 suggested people were searching online for answers to avoid using an automated chatbot.

Governance works

The rate of AI u-turns is even higher for organizations that report having fully mature guardrails in place, with 81% saying they've been forced to rollback an AI agent in customer service, perhaps because better monitoring lets them spot failures more quickly or that are otherwise going missed.

"The industry has assumed that better governance leads to better outcomes," said Daniel Morris, CPO at Sinch. "But that's not enough: If governance was the fix, the most mature teams would roll back less, not more."

So what's happening? "The most advanced organizations aren't failing less; they're seeing failures sooner," said Morris. "Higher rollback rates reflect better monitoring and control, not weaker performance."

Despite the many roll backs of AI agents, 98% of those polled said they were increasing AI investment this year, showing a few missteps isn't enough to dissuade customer service operations from the AI bandwagon.

Trust versus tech

Sinch also argued that companies were spending more time on trust, security, and compliance than they were AI itself. Three-quarters (75%) of those polled reported investment in those areas versus 63% for the technology itself.

Beyond that, 84% of AI engineering teams are spending at least half their time on safety infrastructure.

"Engineering teams are spending most of their time building and maintaining safety systems, a lot of which their communications infrastructure should be providing, instead of focusing on improving the customer experience," said Morris. "That's the guardrail tax that slows organizations down."

The report showed chatbots and email responses were the most popular use cases for AI agents in communication channels, both in use at more than six in ten organisations for those polled.

That was followed by social media, texting, and WhatsApp or similar messaging tools, with half of those polled using AI agents to manage these channels.

The main motivations are to improve customer satisfaction (36%) and boost revenue and conversions (24%), followed by reducing operational costs (16%).

FOLLOW US ON SOCIAL MEDIA

Follow ITPro on Google News and add us as a preferred source to keep tabs on all our latest news, analysis, views, and reviews.

You can also follow ITPro on LinkedIn, X, Facebook, and BlueSky.