Every operator we talk to has seen the same movie. The demo is flawless. The pilot is promising. Six months later the tool is a line item nobody defends in the budget meeting. The problem is rarely the AI — it is that nobody asked the boring questions while the demo was busy being impressive.
These are the five we always ask.
1. What happens when it's wrong?
Not if — when. Every AI system produces wrong answers at some rate. The difference between a tool and a trap is what happens next: Is the error visible? Is there a review step where it matters? Can you trace why it decided what it decided? A vendor who gets uncomfortable with this question has already answered it.
2. Where does our data go?
Ask specifically: What is stored, where, for how long, and is any of it used to train models outside your walls? "It's secure" is not an answer — it's a slogan. You want retention windows, deletion rights, and a straight answer about training use in writing.
3. Who has to change how they work?
Tools fail socially before they fail technically. If the tool demands that your busiest people adopt a new workflow on day one, adoption is your real project — the software is secondary. The best tools meet the work where it already happens: the inbox, the calendar, the systems your team already opens every morning.
4. What does the exit look like?
Before you sign, know how you leave. Can you export your data in a usable format? What breaks the day you cancel? If the answer is "everything," the price on the contract is not the real price.
5. What is the smallest version worth shipping?
The right first deployment is embarrassingly small: one workflow, one team, one measurable outcome. If a vendor insists the value only appears after a company-wide rollout, they are asking you to fund their bet with your risk.
A good AI purchase is boring: small scope, visible errors, clean exits, and one workflow that gets measurably lighter. Excitement is for the demo — reliability is for the invoice.
None of these questions require technical depth. They require permission to be unimpressed — which is, in our experience, the most valuable AI skill an operator can have.