Effective User Testing
Why User Test?
To confirm your hypotheses
To find out where you're wrong
Testing is the core of lean startup's
User Testing Challenge
Watch, Don't Ask
User Testing DON'T's
Don't give them any buttons or links that don't work or matter
Don't give them an ugly or flashy design
Don't give a tour of the app to them
Don't ask them to play with it (unless it's a game)
Don't help them when they look confused
Don't ask if they liked it
User Testing DOs
Do task-based testing
Do record and/or take notes
Do look at these tips from the Center for Plain Language
BEFORE: Write everything out in advance
Product box, pre / post surveys, task scenarios, waivers
DURING: Shut up, watch, record and take notes
Don't tell them what to do
Answer questions with questions
Let them make mistakes. Can they recover?
Look for points of friction and unexpected attractors
categorize and tally mistakes, confusions, hesitation, ...
focus on most critical and common problems
Discover who might want and use your app
Short! 4 questions at most
Characteristics relevant to your target audience
E.g., if a recipe app, how often they cook, how adventurous they are, how do they get recipes currently
Not just age and gender -- way too broad
Set up the story: context, problem, realistic local details
"It's 4:55. You're just finishing a math class in Lunt. Your 394 team is meeting in Tech at 5:30. You're hungry. You'd like to see if any place on campus open right now is serving something vegan."
User Testing Scenarios and Tasks
No pretending to be someone else, e.g., "imagine you're a freshman..."
No instructions on how to use the app
Discover how users
A page of questions at most
Get past polite no-cost answers.
What do you think the point of this app is? Who is it intended for?
What were the 3 best and worst things about this app?
Would you be willing to test next version? If not, why not?
Things users clicked on, asked about, even though not obviously relevant to task
Could be a sign of what you should really be building
an evil attractor
Who you test matters at least as much what you test
Don't just find some random friends
Testers have to align your value proposition
Don't test a recipe app on people who never cook...
... unless your app is meant for people who don't cook
Look in the places where your target users are more likely to be
Use a very short survey to measure how well your testers match your target
After You Test...
Thanks to Hakim El Hattab for RevealJS: