The participant who taught me to re-write my screener

A few years ago I was recruiting for a study on how people manage their home energy. The screener looked good on paper: the right age bands, the right housing types, the right boiler questions. And then I met Peter, who sailed through every filter and turned out to be the most useful participant of the whole study.

The catch? Peter would never have been recruited by the screener I'd written a month earlier. He didn't think of himself as someone who "manages energy" — he thought of himself as someone who was cold in January and angry about his bills. My screener asked about behaviour in the abstract. Peter answered in terms of what he actually did on a Tuesday.

The lesson

Screening questions should recruit people, not validate your assumptions. That means:

• Ask about last time, not usually. "The last time you looked at your energy bill…" gets a real memory. "How often do you check your bills?" gets a guess about social acceptability.
• Recruit for the situation, not the label. You don't need "solar-curious homeowners". You need people who have thought about their roof, their bills, or their comfort in the last month.
• Leave one open question. A single "tell me about why you're interested" at the end of a screener surfaces the Peter you didn't design for.

The awkward truth

Recruiting is where research quietly goes wrong. A tight, sensible screener produces a tight, sensible sample — and sometimes that sample is exactly the people who already agree with you. The fix is boring and unglamorous: test the screener on real humans before you test the product on them, and budget for the participants who don't fit the mould you expected.