Last updated: October 2026
The short answer: write down the one job the tool must do, check what the vendor does with your data, divide the price by the work you will actually put through it, run a trial with a decision date and a result that would mean “no”, and price the cost of leaving before you sign. The seven steps below turn that into a process, with real examples from vendors’ own published terms of how billing units, cancellation fees and retention rules change what a tool really costs.
Most advice on choosing an AI tool is really advice on shortlisting one. It tells you which categories exist, hands you three names, and stops at the point where the actual difficulty starts. If you want the shortlist, we built one you can work through interactively: the stack matchmaker and the find your AI tool finder both do that job, and our ranked roundup is the reading version.
This page is about the harder half: how to run an evaluation that ends in a decision you can defend, rather than a two-week trial that quietly lapses into a subscription nobody reviews again. That is a procurement problem, not a shopping problem, and the questions are different.
The Failure This Is Designed to Prevent
The characteristic bad outcome is not choosing the wrong tool. It is choosing no tool, slowly. A trial starts with enthusiasm, three people use it in week one, the person who championed it gets busy, nobody ever formally decides, and eleven months later it appears on an expenses review and nobody can say what it did. The money is minor. The cost is that the problem it was meant to solve is still there, and the organisation now believes it "tried AI for that."
Everything below is built to close that loop: a decision, on a date, with a stated reason.
Step 1: Write Down the Job, in the Form of a Sentence You Could Be Wrong About
"We should use AI for customer support" is not a job. "Support agents spend the first four minutes of every ticket finding the customer's previous conversations" is a job, and it is falsifiable — you can go and check whether it is true.
Write the sentence before you look at any product. It does three things at once: it tells you what to measure, it tells you who to ask whether the problem is real, and it protects you from the demo effect, where a tool is impressive at something adjacent to your problem and you buy it anyway. If you cannot write the sentence, the honest conclusion is that you are shopping rather than solving, and the right next step is to watch someone do the work for an hour.
Step 2: Decide Whether This Needs a Product at All
A surprising share of specialist AI tools are a prompt, a file store and an interface around a general model. That is not an insult — the interface is often the valuable part, and building it yourself is rarely free. But it does mean the build-versus-buy question is live in a way it was not for previous software categories, and it is worth ten minutes rather than none.
The question to ask is not "could we build this" but "what is the interface worth to us." If the job is done once a month by one technical person, a general assistant and a saved prompt may genuinely be enough. If it is done fifty times a day by people who should not be prompt-engineering, the product is buying you consistency, permissions, an audit trail and somebody else's on-call rota, and those are worth paying for.
There is a middle path worth knowing about, where you want a tool grounded in your own documents rather than a general model's memory. CustomGPT.ai is built for exactly that shape — a chat agent over a sitemap, a set of PDFs, or a Drive or Notion workspace you already own, answering with citations back to the source page. Whether that is the right answer depends entirely on the sentence you wrote in step one.
Step 3: Ask the Data Questions, and Ask Them of the Documentation
This is the step most often skipped and the only one that can produce a genuinely serious problem. The questions have real answers and the answers are usually written down somewhere the sales conversation will not volunteer.
- Is your input used to train their models, and can that be switched off? Look for the setting, not the reassurance. If it exists it will be documented.
- How long is your data retained after you delete it, and after you cancel? "Deleted immediately" and "retained for 30 days in backups" are both defensible; not knowing which is not.
- Who are the sub-processors? Most vendors publish a list. If a tool runs on a model provider you are not allowed to send data to, the list is where you find out.
- Is there a data processing agreement, and will they sign yours? For anything touching customer data in a regulated context, this is the gate, and it is a question for your own legal or compliance people rather than for us.
- Where is the processing done? Relevant if you have residency obligations. Also frequently configurable, and frequently not the default.
Why “after you switch it off” is its own question: Google’s Gemini privacy help page says that with the Keep Activity setting off, chats are still kept with your account for 72 hours, and chats already seen by human reviewers are kept for up to three years. That is a documented, reasonable policy — and exactly the kind of detail a sales call will not raise unless you ask for the page.
Two honest caveats. First, we are not lawyers and nothing here is legal advice — the obligations that apply to you depend on your jurisdiction, your sector and your contracts, and your own counsel is the source. Second, this list is deliberately about questions rather than thresholds, because the thresholds change and a stale number is worse than no number.
The related habit worth building is knowing what is already out there about you and your business before you add another vendor to the pile. Services such as Bitdefender Digital Identity Protection exist to map that footprint.
Step 4: Cost the Whole Thing, Not the Sticker
Any price quoted here is dated and is there to show the arithmetic, not to make your decision: AI pricing changes faster than an article can be maintained, and a stale figure is the fastest way to make a bad decision look justified. Go to the vendor’s own pricing page on the day you decide.
What does not change is the shape of the cost, and the shape is where the surprises live:
- The unit. Per seat, per message, per document, per minute of audio, per credit — and credits that expire monthly are a different product from credits that roll over. Two tools with the same headline price can differ several-fold once you divide by the work you actually do.
- The floor. Minimum seats and annual-only tiers are common at the level where the useful features live.
- The overage behaviour. Does exceeding the plan cost money, throttle you, or stop the service? All three exist, and only one of them is a billing problem.
- The setup cost in your own hours. Connecting sources, writing prompts, training people. This is usually the largest number on the list and it never appears on the pricing page.
- The exit cost. Covered below, because it deserves its own step.
Two real examples, from vendors’ own pricing pages read on 16 October 2026, show how much the unit and the floor matter.
The unit: credits that expire. PixVerse sells video generation in credits, and its FAQ defines three kinds. Daily credits expire at the end of each day (UTC); membership credits expire when each 30-day cycle ends; bonus credits bought in packs never expire. So 1,200 monthly credits you cannot use before the cycle ends are worth nothing, while 1,000 pack credits keep their value until spent. Two offers with similar credit totals can be very different products depending on how evenly you work.
The floor and the exit: what “annual” really commits you to. Adobe lists Acrobat Pro in the US at US$29.99 month to month, or US$19.99 a month on an annual plan billed monthly. The annual plan looks a third cheaper, but Adobe states that cancelling it after 14 days carries a fee of up to half your remaining commitment. Suppose the pilot fails and you cancel after three months. Month to month, you have paid 3 × 29.99 = US$89.97. On the annual plan you have paid 3 × 19.99 = US$59.97, plus a fee of up to half of the nine months left (9 × 19.99 = US$179.91), so up to about US$89.96 more — a total of up to about US$149.93. For a tool on trial, the flexible plan is the cheaper bet; the annual discount only pays once you know you are keeping it.
Do the same division with the vendor’s real current numbers and your own real volume, and do it before the trial starts.
Step 5: Design a Trial That Can Fail
A trial that cannot produce a "no" is not a trial, it is an onboarding. Before you start, write down four things:
- The tasks. Three to five real ones, from your actual backlog, chosen before you see the tool. Include one you already know the right answer to — a known-good case is the only way to calibrate whether the output is right or merely fluent.
- The people. The people who will actually use it, not the person who found it. Enthusiast results do not generalise.
- The date. A day on which someone decides. Put it in a calendar.
- The kill criteria. What result would make this a no. If you cannot name one, you have already decided and the trial is theatre.
Run the same tasks through the shortlisted alternatives, in the same week, with the same people. Comparative evaluation is far more informative than sequential evaluation, because "is this good" is a question nobody can answer and "is this better than that" is a question anyone can.
Step 6: Cost the Exit Before You Commit to the Entrance
The question that separates a reversible decision from an expensive one: if this vendor doubled its price or shut down tomorrow, what would we have to do?
Check, concretely: can you export your data, in a format something else can read, without contacting support? Does the tool hold anything that exists nowhere else — prompts, tuned configurations, a knowledge base, conversation history someone relies on? How deeply is it wired into other systems, and who would have to unpick that?
A tool that produces artefacts you keep is low-risk almost regardless of quality. A tool that becomes the only place a body of knowledge lives is a high-risk dependency even if it is excellent, and it should be judged on a different standard.
Step 7: Set a Review Date and Actually Keep It
Every adopted tool gets a date, six or twelve months out, on which someone asks three questions: is the job from step one still being done, is this still the best way to do it, and is anyone still using it? In a category moving this fast, the answer changes more often than in any software market most organisations have dealt with before.
This is also the step that fixes the failure described at the top. A subscription with a review date cannot quietly become permanent.
Red Flags Worth Walking Away From
- Accuracy or performance claims with no stated methodology. A percentage without a test set is a marketing number.
- No published pricing at all for a self-serve product. Sometimes legitimate for enterprise sales; frequently a sign the price is whatever you look like you can pay.
- A demo that only ever uses the vendor's own example data. Ask to run yours, in the demo, live.
- Vague answers to the data questions in step three. The answer being complicated is fine. The answer being unavailable is not.
- Testimonials with no attributable person or organisation behind them.
Where This Framework Is Weakest
Honesty about the method itself: this is overkill for a cheap individual tool. If you are one person deciding whether to pay for a writing assistant, steps two, three and six are largely irrelevant and the whole thing collapses into "try it on three real tasks and see." The framework earns its cost when other people's data, other people's workflows, or a renewal someone else will inherit are involved.
Its second weakness is that it rewards the measurable. Some genuinely valuable tools improve the quality of thinking rather than the throughput of a task, and a trial built around five backlog items will not detect that. Where you suspect this is the case, say so explicitly and decide on judgement — but say it out loud, because "hard to measure" is also the standard excuse for a decision made on enthusiasm.
Disclosure: this article contains affiliate links. If you sign up through them we may earn a commission at no extra cost to you. It does not change what this framework recommends, and no vendor has paid for a mention.
Frequently Asked Questions
How long should an AI tool trial last?
Long enough to run three to five real tasks with the people who will use the tool, and no longer than the date you set for the decision. Many vendors’ trials or cancellation windows are short — Adobe’s annual Acrobat plans, for example, change their cancellation terms after 14 days — so fix the decision date before you start.
Should I pay monthly or annually for an AI tool?
Monthly while you are still deciding, annually once you know you are keeping it. Annual plans are cheaper per month but often carry cancellation fees or no refunds after a short window, which can make a failed pilot cost more than the monthly plan would have.
What should I ask an AI vendor about my data?
Whether your input trains their models and how to switch that off, how long data is kept after deletion and after you switch settings off, who their sub-processors are, whether they will sign a data processing agreement, and where processing happens. Ask for the documentation, not a verbal answer.
How do I compare two AI tools with different pricing units?
Convert both to the cost of one finished piece of your work: divide the monthly price by the number of usable outputs you would actually get, and check whether unused credits expire. A cheaper plan that caps or expires your allowance can be the more expensive tool.
Related Articles
