Scale Can’t Fix a Bad Test
Starbucks learned the expensive way: field conditions, fallback rules, and frontline judgment belong in the product plan.
Your daily signal on AI and CX — minus the hype.
DCX Stat of the day: 78% of surveyed ecommerce brands generate at least one-quarter of their customer-facing content with AI. accessiBe
In this issue:
A shiny fridge fooled Starbucks’ inventory AI
Frontline workarounds became product evidence
Governance rollbacks expose the production gap
Cekura turns failures into regression tests
IHG keeps old search beside AI
🔍 DEEP DIVE
The Fridge Was Part of the Product
Starbucks rolled an AI inventory tool across all 11,300 company-operated North American cafés. It was supposed to turn an hour-long count into a 10-to-12-minute scan. Nine months later, the company pulled it.
The failures were painfully ordinary. A shiny refrigerator doubled the milk count by reflecting cartons. Weak Wi-Fi erased progress. The system mislabeled syrups and sometimes counted the trash can as food. Meanwhile, manual counts were treated as no count, so the fallback stopped being useful precisely when the AI wasn’t.
That reaches the customer faster than it sounds. Bad inventory data becomes an unavailable drink, a wrong menu promise, a longer wait, or a barista doing cleanup during the morning rush. The problem wasn’t simply model accuracy. The café itself, including reflections, connectivity, storage layouts, and human judgment, was part of the product.
Bottom Line: If the field can break the AI, the field belongs in the test plan. Scale only makes an untested assumption more expensive.
Source: Fast Company
📬 Copy-Paste Take
Before we scale a customer-impacting AI tool, we should test it where the work actually happens, with bad Wi-Fi, awkward layouts, edge cases, and the people expected to recover when it fails. A workaround isn’t resistance. It may be the clearest product evidence we have.
🧭 OPERATOR PLAYBOOK
Put the Mess in the Test Plan
Audit one customer-impacting AI workflow for four things:
The physical, digital, and data conditions the pilot avoided.
The frontline override when the model is plainly wrong.
The customer promise affected by a bad output.
The signal that pauses rollout before failure spreads.
Then test whether the fallback still works when the primary system doesn’t.
Ask your team: Which “user error” keeps appearing because the product was tested in cleaner conditions than the job?
Signal: Frontline exceptions are not noise around the deployment. They are deployment data.
📊 MARKET REALITY CHECK
Production Finds What the Pilot Missed
Sinch surveyed 2,527 enterprise decision-makers across 10 countries and six industries. Among organizations that had put an AI communications agent into production, 74% said they had been forced to roll one back or shut it down because of a governance failure.
The rate rose to 81% among organizations that described their guardrails as fully mature. That doesn’t necessarily mean mature teams are worse at AI. Sinch’s interpretation is that better monitoring helps them see failures that less mature teams miss. The reported triggers included customer-data exposure, hallucination or brand risk, and a lack of auditability.
Why it matters: Production is not proof that the system works. It is where teams need enough visibility to catch a bad customer interaction, stop the damage, explain what happened, and roll back without turning recovery into another project.
No monitoring, no honest success rate.
🧰 TOOL WORTH KNOWING
Cekura
What it does: Cekura tests and monitors voice and chat AI agents. Teams can simulate conversations across different personas, run repeatable evaluations before release, and track live-call signals such as latency, interruptions, sentiment, and failures after launch.
CX use case: Build scenarios around real customer journeys such as cancellation, rescheduling, authentication, escalation, noisy callers, or off-script requests. Replay known trouble spots after prompt or model changes so a fix becomes a regression test.
Worth watching because: Today’s issue is about conditions a demo ignores. Cekura gives teams a way to turn those conditions into release gates and alerts instead of waiting for customers to rediscover them.
Bottom line: A test tool can show you what failed. It cannot decide what customers should experience. CX still needs to define the journeys, thresholds, escalation rules, and recovery path.
The DCX AI Today - AI Tool Directory - If you lead a CX team and want a curated shortlist of tools worth evaluating, this is your starting point.
📡 90-SECOND CX RADAR
IHG keeps traditional search beside its new AI option
IHG launched conversational search in beta on its U.S. website and app, using verified property data, guest reviews, real-time availability, pricing, points, and points-plus-cash options. Traditional search remains available during the beta.
Why it matters: Customer choice is part of rollout design. Keeping both paths visible gives IHG a cleaner comparison of findability, booking completion, correction, and where AI recommendations still need help.
✅ YOUR MOVE
Pick one AI workflow that has moved beyond the demo. Then walk it under the conditions the launch deck politely ignored.
Slow connection. Messy data. An unusual customer request. A frontline employee who disagrees with the output. A customer who wants the old path back.
Write down what breaks, who notices first, and whether that person can stop or correct the system without starting a committee.
Then add those conditions to the release gate. The goal is simple: keep customers from becoming the test environment.
If reality can break the AI, reality belongs in the test.
Until tomorrow,
👥 Share This Issue
Think of one person who’s wrestling with AI in CX right now
and forward this to them.
I’m obsessed with Wispr Flow Pro! Get a Free Month on me.
If someone forwarded this to you, they thought you needed to see it before your next AI planning meeting. Get your own copy.