How to Run an EdTech Pilot That Generates Real Evidence

An EdTech pilot generates real evidence when it is designed like a study, not a free trial: a defined learner group, a pre-agreed outcome measure, a comparison point, and a fixed timeframe. Run this way, a single term in one school can produce findings credible enough to support procurement decisions and investor conversations.

Why most pilots produce nothing usable

The typical pilot is an open-ended free trial: a school agrees to "try" the product, some teachers use it, and three months later the startup has anecdotes and a testimonial but no evidence. The pilot fails at the design stage — nobody agreed what would be measured, against what baseline, or what result would count as success. Everything below is about fixing that before the pilot starts.

Step 1: Agree the question before the pilot starts

Write a one-page pilot plan with the school covering: which classes or learners take part, what outcome will be measured (see measuring learning outcomes), how long the pilot runs, and — critically — what happens at the end. A pilot with a pre-agreed decision point ("if X improves, we discuss a paid rollout") converts; a pilot without one drifts.

Step 2: Build in a comparison

Ask the school to identify a comparable class or year group not using the product, or use a staggered start where half the classes begin later. This is usually easy for schools to accommodate and transforms the credibility of your findings — you can now answer "compared to what?".

Step 3: Support the teachers, then get out of the way

• Train teachers before the start, not during week three
• Check in weekly to fix blockers — usage failures are product findings too
• Resist the urge to intervene beyond what a paying customer would receive; otherwise you're measuring your support effort, not your product

Step 4: Collect data ethically

Pilots involve real learners, so research ethics apply: informed consent (via parents/guardians for minors), data minimisation, secure storage, and transparency with the school about how data will be used. Our research methods guide covers this in detail — and for AI products there are additional obligations covered in responsible AI in education.

Step 5: Analyse honestly and share everything

Compare outcomes against the baseline you agreed, report gains and gaps, and check whether results hold across learner groups rather than just on average. Share the full findings with the school — including what didn't work. Schools talk to each other; a reputation for honest evidence is one of the strongest sales assets an EdTech startup can build.

Turning pilot evidence into growth

A well-run pilot produces three assets: outcome data for your next procurement conversation, impact metrics for investors, and a prioritised list of product improvements. Turning that evidence into a story that lands with partners and investors is Stage 4 of the UCL EdTech Labs accelerator — and it's how alumni have collectively raised over £37M.