Not sold on AI? Good. Ask better questions.

“Show me what happens when it gets something wrong.”

That’s a perfectly reasonable thing to say to an AI vendor. So is asking who controls the follow-up, what your people have to change, and whether the CRM integration in the demo actually works with your setup. Those questions don’t make you a dinosaur. They make you the person who’ll be cleaning up after a bad customer interaction long after the demo ends.

If you’re skeptical, you don’t need to get less skeptical before you start. You need a process that turns your doubts into questions a vendor has to answer. Here’s one.

Start with one problem you can say in a sentence

“Improve the customer experience” is a fine goal and a terrible test. Pick a moment where something specific and observable goes wrong.

Maybe your agents spend too long digging for details before an outbound call. Maybe the handoff from the BDC to the floor regularly leaves the salesperson guessing. Maybe follow-up ignores the timing customers asked for. Those are three different problems, and they need three different fixes. Sometimes the first fix isn’t technology at all. It’s an ownership rule nobody ever wrote down.

Write the problem down plainly: “Before the first call, our agents need a reliable summary of what the shopper already told us and what still needs clarifying.” Now you have something to inspect. Can the tool provide that? Does it separate what the customer said from what the system guessed? Can your agent spot a mistake fast?

It also keeps the test from growing every time someone sees a shiny feature. The demo can show twenty things. Your first evaluation needs one.

Decide what it’s allowed to do before you turn it on

You should be able to explain what the system is allowed to do without the vendor in the room. Which tasks does it help with? Does anything go to a customer without a person reviewing it? Which topics always need an employee? Who’s allowed to change the rules?

It helps to think in three levels.

Level	What it means	A good first use?
Assist	It prepares information or suggests wording. A person does the talking.	Usually the safest place to start
Review before action	It drafts. A person approves before anything goes out.	Good for follow-up you want to keep on a leash
Automate within limits	It acts on its own inside rules you set, and hands off when it hits the edge.	Only for narrow, well-defined tasks

These are planning levels, not a product menu. Where you start depends on the task, what the product actually supports and how much risk you’re comfortable with. There’s no prize for starting at the top.

For anything automated, define the off-ramps in advance: the customer asks for a person, disputes something, changes their timing or asks a question outside the task. Then name who gets the handoff. “The team will see it” isn’t an owner.

Make them demo the bad day

Polished demos show the happy path. Ask for the exceptions instead. What happens when information is missing? When two records disagree? When the customer says, “That’s not what I said”?

Ask how you’d pause the workflow while you look into a problem, and whether the information you’d need to review it gets kept. None of this assumes failure is coming. It just recognizes that trustworthy operations plan for exceptions.

“Integrates with your CRM” is the start of a conversation

It isn’t an answer. What moves, in which direction, under whose permissions? What still needs an employee to do an extra step? What depends on what your CRM provider allows, which varies more than most slide decks admit?

Walk through one ordinary task end to end. Where does the information come from, and how fresh is it? What does the employee see? If they correct something, where does the correction go? Afterward, where does the next person find the outcome? That last question catches a lot. A tool can look great on its own and still leave your team maintaining two versions of the same customer.

Cox Automotive made a similar point in an October 2026 piece on scaling AI in automotive, arguing that connected data, governance and interoperability decide whether AI works at scale. One line stuck with us: “Disconnected data just automates the wrong answer faster.”

Bring someone who does the work into this walkthrough. A manager may be happy that data lands in the right place while the salesperson notices it takes four extra clicks during a live call. Both findings count. And settle the data questions with the right people at your store: what information the task needs, who can see it, how long it’s kept and what happens when the trial ends. Ask for evidence, not reassurance.

Decide what “worked” means before the pilot starts

A pilot that just keeps running isn’t a success. It’s a habit. Agree up front who reviews the work, what they look at and when you’ll talk about it.

For preparation help, check whether the information was relevant, whether people caught errors, and whether customers had to repeat themselves. For follow-up, check whether messages respected the customer’s timing and whether exceptions reached the right person. Read the actual conversations. A usage count can’t tell you whether anyone was helped.

Be honest about cause and effect, too. Inventory, staffing and lead mix all move your numbers. A small test can show a useful pattern without proving the tool caused everything that happened.

When it’s over, you have real options: keep going as is, adjust and test again, expand one use that clearly worked, or stop. Stopping a test that isn’t earning its keep is a good outcome. It beats letting a vague pilot quietly become a permanent process nobody approved.

You don’t need to bet the store on AI. You need one clear reason to test it, control over how it runs, and evidence you trust when you decide what’s next.