Running a small controlled test
A repeatable protocol for a first paid run: what to fix, what to vary, which records to keep, and the pass conditions to write down before you start.
01Buying guide: from shortlist to first run
Everything in this section happens after you have a shortlist and before you have a working campaign. It is the part buyers skip, and it is the part that determines whether a disappointing result can be diagnosed or only regretted. Three questions carry the weight: how much a run actually needs, how to structure a first run so it produces evidence, and which warning signs justify walking away while it still costs nothing.
The order matters. Size the budget first, because an underfunded run cannot be interpreted. Write the pass conditions second, before any money moves, because conditions written afterwards are conclusions. Check the warning signs throughout, since most of them appear in the conversation rather than in the product.
3 pieces in this section
What to do between shortlisting and paying: size a first budget, run a controlled test, and check the warning signs that predate most complaints.
Red flags before you pay
Fourteen warning signs grouped by severity, from marketing language that promises outcomes to support behaviour that predicts how a dispute will go.
02What budget a run needs
The cost stack of a volume run line by line, an illustrative worksheet you can copy, and the floor below which a run cannot produce a readable result.
03The sequence, and what stops it
Each stage produces something you keep. If a stage cannot produce its output, that is the signal to stop rather than to push on and hope the next stage clarifies it.
| Stage | What you produce | What stops the process here |
|---|---|---|
| Shortlist | Two or three candidates that pass the nine criteria on paper | No candidate will describe its custody model in writing |
| Questions | Written answers you can quote back later | Answers arrive only by voice, or change between messages |
| Price expression | A single total for a defined run, in writing, including failures | A fee exists that was never mentioned until you asked twice |
| Contract | Terms covering refunds, cancellation, records and failure | There are no written terms, or they contradict the sales conversation |
| Test run | A record sheet, an export, and a reconciliation against the chain | Records cannot be exported, or do not match the chain |
| Scale or stop | A decision with a reason attached to it | The pass conditions were not met and are being rewritten |
Where this section sits
The criteria section defines what each stage is checking for, and the comparisons section explains the structural choices you will have made by the time you reach a contract.
Criteria
The measurable properties of a volume tool: custody, venue coverage, control surface, failure reporting and the questions that expose each one.
CRIComparisons
Class-level comparisons rather than product rankings. Pricing structures, interface models and hosting models, each with the trade-off it hides.
COM