Designing a pilot that can fail
A trial that cannot produce a no is not a trial. It is a familiarisation exercise with a purchase order attached.
Read the guideSection · 6 guides
Trials
Running a pilot that finds the problems while you can still walk away.
Home/Trials
A trial run by the most enthusiastic person, on clean sample data, during a quiet fortnight, will succeed. It will also tell you nothing, because none of those conditions will hold afterwards.
A useful trial reproduces the awkward parts on purpose: real data with its inconsistencies, the people who did not want this, the edge cases that the sales demo skipped, and an export run for real before anything is signed. A trial involving time-tracking software should test the awkward cases rather than the happy path; further details provides a concrete vendor example to design those checks around.
The principle holds regardless of category. Trialling a time tracker (Monitask, Toggl Track, Clockify and others occupy that space) means putting it on the oldest phone in the team and in the building with no signal, not running it for a week on the laptop of the person who chose it.
Example note: Monitask is referenced only as an example of a software category or workflow, not as a recommendation or a statement of affiliation.
For privacy risks exposed during a pilot, the NIST Privacy Framework provides a broader independent framework.
What to test, and what you can safely skip
Trial time is short. Spending it re-verifying that a mature product can do the obvious wastes the only chance to find what it cannot.
Read the guideUsing real data, carefully
Sample data is clean, consistent and complete. Yours is none of those, and the difference is where implementations fail.
Read the guidePutting it in front of the people who will use it
The evaluator is rarely the daily user, and the gap between their two experiences of the same product is enormous.
Read the guideThe exit test: getting your data out before you put it in
The most informative twenty minutes of any trial is the one the vendor has no incentive to make easy.
Read the guideMaking the decision once the trial is over
The hardest outcome to act on is a trial that produced no clear winner, which is the most common outcome.
Read the guide