Posted in Scrum Add-ons

User Interviews

Who to Interview

How many customers should I test with? (2, pg 144)

  • Focus groups lead to participants not being fully honest 
  • Interview between 5 and 8 is a good balance

How to recruit in target market? (2, pg 148)

  • Refer to personas created to help with writing screener questions 
  • Ensure to cover all of the segmentations you used forming your target market
  • Use conferences/ networking events

Starbucks User Testing (2, pg 151) 

  • Low cost and immediate
  • Harder to screen for the customers that you actually want (demographic)
  • “Hi sir, do you have 10 minutes to share your feed back on a new website in exchange for a £20 Starbucks card?
  • More success found in shopping centres than Starbucks as people tend to have more time to kill

How to Interview

In-person, remote, and unmoderated user testing (2, pg 145)

  • Moderated tests have the customer perform the test on their own, but recorded for the moderator to view afterwards
  • In-person allows richer data collection (body language)
  • Remote – be aware of technology issues of screen sharing 
  • Unmoderated can product a quick turn around as providers already have users lined up ready 

Question Guidance

Open vs Closed Questions (2, pg 158)

  • Open usually begin with why, how and what 
  • Closed questions limit the customer’s possible responses (e.g. to yes or no)
  • Tip to write the questions down in advance to check they are open

Non-leading Questions (2, pg 158 – 160)

  • Be careful not to tell the user how you would like them to respond, e.g. ‘how would you prefer it to be sorted? By date?’  
  • Be ok with long pauses and don’t help them as it will invalidate the test. Real users won’t have you to help them.
  • Don’t predict what the customer will say out loud and try to recede into the background to get the most honest feedback

Measuring Importance and Satisfaction (2, pg 59)

  • A bipolar scale goes from negative to positive (i.e. satisfaction)
  • A unipolar scale is a matter of degree (i.e. importance)
  • More than 10 choices is overwhelming
  • Fewer that 5 will lack granularity
Unipolar Example
  1. Not at all important 
  2. Slightly important 
  3. Moderately important 
  4. Very important 
  5. Extremely important 

Bipolar example 
  1. Completely dissatisfied 
  2. Mostly dissatisfied 
  3. Somewhat dissatisfied 
  4. Neither satisfied nor dissatisfied
  5. Somewhat satisfied 
  6. Mostly satisfied
  7. Completely satisfied

Product Market Fit Question (Sean Ellis) (2, pg 231)

  • Do not invest in trying to grow a business until after you have achieved product-market fit
  • This questions helps you assess if you have achieved product-market fit
  • Survey users and ask:
    • “How would you feel if you could no longer use product X?
      • Very disappointed
      • Somewhat disappointed 
      • Not disappointed (it isn’t really that useful)
      • N/A – I no longer user (product X)”

Follow-up Questions (2, pg 157)

  • ‘Echoing back’ is a useful technique – e.g. ‘I see you clicked that button. Could you tell me why?’
  • If user asks question, ask another question rather than replying with yes or no

The Data

Make sense of the data (1, pg 103) 

  • Look for patterns 
  • Place outliers in a parking lot (outliers could form their own pattern with more data collection so don’t bin them) 
  • Verify with other sources (do people inside the office agree/ customer support staff seeing the same trend) If not your sources could have been skewed 

Feedback Table (2, pg 162)

Feedback Structure (2, pg 162)
  • Group into feature set, UX design, and messaging
  • Ask valuable and easy to use questions
  • Get overall scores to compare to next wave when improvements have been made

Regular Cadence

TIP – randomly schedule users on a routine basis so that you always have some ready to test with, rather than panicking (2, pg 150)

Continuous Learning in the Lab: Three, Twelve, One (1, pg 99)

  • Three users, by Twelve noon, once a week
  • The regular cadence ensures that you ‘test what you’ve got’ so that there is no deadline pressure
MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAY
Start with the recruiting process
Decide what will be tested
Refine what will be testedRefine what will be tested
Write the test script
Finalise recruiting
Testing day
Review findings with the entire team
Plan the next steps based on findings
Three, Twelve, One Sequence (1, pg 99)

References

  1. Lean UX by Jeff Gothelf
  2. Lean Product Playbook by Dan Olsen

One thought on “User Interviews

  1. Pingback: Product Backlog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s