In a recent post about getting more out of a tester swap, I gave developers a piece of advice that was half right. I suggested sending every tester the same short question, "What did you expect to happen after the first tap?", and asking it once, around the middle of the 14-day test.

On Indie Hackers, a reader pointed out the flaw. By the middle of the test, the first tap is a week old. And the middle of the test is when you can learn something completely different. The fix: ask two questions, at two different times. Here is how that works for Google Play's 12 testers, 14 days requirement.

Why one question at the midpoint misses

A first impression has a short shelf life. After a week of opening your app every day, a tester has learned where everything is. Ask them on day 7 what they expected after the first tap, and you get a reconstruction, not an impression.…