Posted in Product

Product Kata

A scientific, systematic way to build better products inspired by ‘The Four Steps to the Improvement Kata’ from Toyota Kata Practice Guide by Mike Rother


Product Kata Overview

Slide1

Product Kata at two levels: Company and Product. For the Product level the example relates to the imaginary case study in the book of an online course company where teachers upload course content and students pay to access them.

Product Kata Walkthrough

1) Understand the direction

Overview

Create or establish the direction metrics

  • Ensure they aren’t vanity metrics and that when they increase/ decrease it actually means something
  • See the metrics page for more on creating good metrics
company

Example: To double revenue growth

Product

Example: To increase course content

2) Analyse the Current State

overview
  • Where do you stand in relation to your vision?
  • Reflect the current state of outcomes
  • Collect a baseline for your Product Metrics
company

Example: Current revenue growth is 15% Year on Year (YoY)

product

Example: Publish course rate is 25% and second course is 10%

3) Set the Next Goal

overview

Name outcomes that you need to achieve to make progress toward the direction. They can be learning outcomes.

company

Example: Retain existing users

product

Example: Learn what the problems are with users creating the courses

4) Experiment Toward the Target Condition

Experiment with the following techniques to systematically reach the goal:

Problem exploration
  • Understand the problem; what are we trying to solve?
  • Validating the problem
  • See more in User Interviews
Solution exploration
Solution optimization

References

  1. Escaping the Build Trap by Melissa Perri

Explore more


Metrics

Which direction

Prioritising
Posted in Scrum Add-ons

Creating a New Product Backlog

Gathering Requirements

Trawling for Requirements (1, page 43)

Imagine gathering requirements as like fishing. You don’t want to catch every single fish in one haul as the processing of that would mean that some fish would go off. Focus on catching the big fish first by using a wide net, then later you can use a finer net to catch the smaller fish. This is the same with requirements as with the big requirements you can use them to understand the product direction better.

Building the Initial Backlog Techniques

Idea Generating Workshop (2, page 50)

  • Use the liberating structure, 25/10 Crowd Sourcing
  • Ask the room to use the ‘What’s the most important feature this product needs to be successful?’ question
  • The output of the session can be used by the Product Owner to form the initial Product Backlog

User Story Mapping (4)

Impact Mapping (3, page 86)

Technique for connecting PBIs back to the goals they are intended to meet.

Impact Mapping

References

  1. User Stories Applied by Mike Cohn
  2. Fixing Your Scrum by Ryan Ripley and Todd Miller
  3. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl
  4. User Story Mapping by Jeff Patton

Explore more


Refinement

User Stories
Posted in Product

User Interviews

Who to Interview

How many customers should I test with? (2, page 144)

  • Focus groups lead to participants not being fully honest 
  • Interview between 5 and 8 is a good balance

How to recruit in target market? (2, page 148)

  • Refer to personas created to help with writing screener questions 
  • Ensure to cover all of the segmentations you used forming your target market
  • Use conferences/ networking events

Starbucks User Testing (2, page 151) 

  • Low cost and immediate
  • Harder to screen for the customers that you actually want (demographic)
  • “Hi sir, do you have 10 minutes to share your feed back on a new website in exchange for a £20 Starbucks card?
  • More success found in shopping centres than Starbucks as people tend to have more time to kill

How to Interview

In-person, remote, and unmoderated user testing (2, page 145)

  • Moderated tests have the customer perform the test on their own, but recorded for the moderator to view afterwards
  • In-person allows richer data collection (body language)
  • Remote – be aware of technology issues of screen sharing 
  • Unmoderated can product a quick turn around as providers already have users lined up ready 

Question Guidance

Open vs Closed Questions (2, page 158)

  • Open usually begin with why, how and what 
  • Closed questions limit the customer’s possible responses (e.g. to yes or no)
  • Tip to write the questions down in advance to check they are open

Non-leading Questions (2, page 158 – 160)

  • Be careful not to tell the user how you would like them to respond, e.g. ‘how would you prefer it to be sorted? By date?’  
  • Be ok with long pauses and don’t help them as it will invalidate the test. Real users won’t have you to help them.
  • Don’t predict what the customer will say out loud and try to recede into the background to get the most honest feedback

Measuring Importance and Satisfaction (2, page 59)

  • A bipolar scale goes from negative to positive (i.e. satisfaction)
  • A unipolar scale is a matter of degree (i.e. importance)
  • More than 10 choices is overwhelming
  • Fewer that 5 will lack granularity
Unipolar Example
  1. Not at all important 
  2. Slightly important 
  3. Moderately important 
  4. Very important 
  5. Extremely important 

Bipolar example 
  1. Completely dissatisfied 
  2. Mostly dissatisfied 
  3. Somewhat dissatisfied 
  4. Neither satisfied nor dissatisfied
  5. Somewhat satisfied 
  6. Mostly satisfied
  7. Completely satisfied

Product Market Fit Question (Sean Ellis) (2, page 231)

  • Do not invest in trying to grow a business until after you have achieved product-market fit
  • This questions helps you assess if you have achieved product-market fit
  • Survey users and ask:
    • “How would you feel if you could no longer use product X?
      • Very disappointed
      • Somewhat disappointed 
      • Not disappointed (it isn’t really that useful)
      • N/A – I no longer user (product X)”

Follow-up Questions (2, page 157)

  • ‘Echoing back’ is a useful technique – e.g. ‘I see you clicked that button. Could you tell me why?’
  • If user asks question, ask another question rather than replying with yes or no

The Data

Make sense of the data (1, page 103) 

  • Look for patterns 
  • Place outliers in a parking lot (outliers could form their own pattern with more data collection so don’t bin them) 
  • Verify with other sources (do people inside the office agree/ customer support staff seeing the same trend) If not your sources could have been skewed 

Feedback Table (2, page 162)

Feedback Structure (2, page 162)
  • Group into feature set, UX design, and messaging
  • Ask valuable and easy to use questions
  • Get overall scores to compare to next wave when improvements have been made

Regular Cadence

TIP – randomly schedule users on a routine basis so that you always have some ready to test with, rather than panicking (2, page 150)

Continuous Learning in the Lab: Three, Twelve, One (1, page 99)

  • Three users, by Twelve noon, once a week
  • The regular cadence ensures that you ‘test what you’ve got’ so that there is no deadline pressure
MONDAYTUESDAYWEDNESDAYTHURSDAYFRIDAY
Start with the recruiting process
Decide what will be tested
Refine what will be testedRefine what will be tested
Write the test script
Finalise recruiting
Testing day
Review findings with the entire team
Plan the next steps based on findings
Three, Twelve, One Sequence (1, page 99)

References

  1. Lean UX by Jeff Gothelf
  2. Lean Product Playbook by Dan Olsen

Explore more


Personas

User Testing

User Stories
Posted in Scrum Add-ons

Product Backlog Refinement

Overview

Definition: Product Backlog refinement is the act of adding detail, estimates, and order to items in the Product Backlog.

Length: “Refinement usually consumes no more than 10% of the capacity of the Development Team” (1)

People: Product Owner, Development Team

Outcome: Product Backlog items become more transparent through refinement and the top items are deemed ‘Ready’ for a Sprint (i.e. can be ‘Done’ by the team within one Sprint)

Benefits

Product Backlog Refinement Benefits (2, page 103, Mastering Professional Scrum)

  1. Increased Transparency – adding details for what you plan to deliver, and your progress
  2. Clarification of Value – outcomes of what you are trying to achieve become clearer, and helps the team to build the right thing
  3. Breaking things into consumable pieces – increases flexibility for the team to meet ‘Done’ in a Sprint
  4. Reduction of dependencies
  5. Forecasting
  6. Incorporation of learning – gained learning is incorporated into the product

Facilitation Techniques

Fishbowl collaboration (3)

  • Three – five people collaborate around a whiteboard (the fish in the bowl)
  • Others in the room may observe but are not allowed to speak
  • If someone from outside the bowl wishes to speak they can dive into the bowl which bounces out one of the fish already in the bowl
  • People can leave anytime if they are not finding it valuable
  • This technique keeps the conversation small and productive

Three Amigos and BDD

  • Backlog Refinement is attended by someone to represent each discipline in rotation (i.e. Three Amigos of Product, Development, Testing)
  • Behaviour Driven Development is used to focus the conversation on the behaviour of the User Story
  • See my talk notes

References

  1. The Scrum Guide
  2. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl
  3. User Story Mapping by Jeff Patton

Explore more

Posted in Product

Business Value/ Vision

Business Value Levels

Strategy Deployment Levels (1, page 75)

VisionWhat do we want to be in 5-10 years? Value for customers, position in market, what our business looks likeCEO/ Leadership
Strategic IntentWhat business challenges are standing in the way of reaching our visionSenior leadership
Business leads
Product InitiativeWhat problems can we address to tackle the challenge from a product perspective?Product leadership team
OptionsWhat are the different ways I can address those problems to reach my goals?Product Dev teams
Strategy Deployment Levels

Business Value Types

Framework for thinking about value by Joshua Arnold (1, page 82)

Increase Revenue – Increasing sales to new or existing customers. Delighting or Disrupting to increase market share and size

Protect Revenue – Improvements and incremental innovation to sustain current market share and revenue figures

Reduce Costs – Costs that we are currently incurring, that can be reduced. More efficient, improved margin or contribution

Avoid Costs – Improvements to sustain current cost base. Costs we are not currently incurring but may do in the future

Type of Value (3)

Commercial Value: How much revenue or profit does this work result in?

Market Value: How many new customers will we be able to serve?

Efficiency Value: How much time or money will this save us?

Customer Value: To what extent will this decrease the likelihood that a customer leaves?

Future Value: How much will this save us in time or money in the future?

Project Rather than Product Management Smell

Build Trap (1, page 1)

Build Trap = organizations become stuck measuring their successes by outputs rather than outcomes. […] When companies stop producing real value for the users, they begin to lose market share, allowing them to be disrupted.

Progress Focus Powerful Questions (2, page 80)

For when a company is focusing on percent complete, status, or points rather than value

  • Could we achieve all of these measure and still be unsuccessful?
  • How are we validating assumptions about the user needs of the market demand?
  • What are we learning about value? How I this guiding our product decisions?
  • What has changed with our users or our competitive environment since we began this initiative?

Vision

‘Product Vision Box’ Exercise (4) (5)

  • Give your product team cardboard boxes and pens
  • Ask them to design the box for the product they would buy

Product Vision Venn Diagram

Balance multiple product attributes (6, page 172)

References

  1. Escaping the Build Trap by Melissa Perri
  2. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl
  3. What is This Thing Called ‘Value’? by Christiaan Verwijs (blog post)
  4. Fixing Your Scrum by Ryan Ripley and Todd Miller
  5. Product Box by Innovation Games
  6. Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland

Explore more


Which direction?

Prioritising

Hypotheses
Posted in Scrum Add-ons

Prioritising Features/ Backlog

How to Prioritise the Backlog

Priority Process (1)

  • Firstly on business value
  • Secondly with the team to include impediments
  • This produces the most important thing to the business
  • Business value over schedule driven decisions

Sequencing Risk Reduction (3, page 48)

  • Sequence first the activities that remove the most risk for the least expense
  • Look at it like buying information 
    • Example if you entered a lottery with a 1 in 1000 chance of winning your ticket would be worth $1
    • If someone told you one of the numbers out of the 10 number draw, that would increase your chances of winning and make the ticket worth $10 
    • It would therefore be worth spending $9 to get that information to increase your chance of success

Exercises for Backlog Prioritisation

Exercise to help prioritise the Product Backlog (2, page 55)

  • Use the liberating structure, Min-Spec
  • Display the Product Backlog in the room
  • Use the question of ‘Which features in our product backlog are needed in order to have a successful product release?’
  • Test the list generated against the question ‘If we delivered all of out must-do PBIs except this one, would we still have a successful product release?’

References

  1. Coaching Agile Teams by Lyssa Adkins
  2. Fixing Your Scrum by Ryan Ripley and Todd Miller
  3. The Principles of Product Development Flow by Donald G. Reinertsen

Explore more


User Stories

Estimation

Done
Posted in Product

Value Statements

Communication Forms

Microdefinition (1)

  • Microdefinition = next critical business objective on the way to creating the whole product.
  • Everything must be checked against this

North Star Document (2, page 136)

  • Explains the product in a way that can be visualised by the whole team and company
  • Problem it is solving
  • Proposed solution
  • Solution factors that matter for success
  • Outcomes the product will result in
  • Should evolve over time

References

  1. Coaching Agile Teams by Lyssa Adkins
  2. Escaping the Build Trap by Melissa Perri

Explore more


Which direction?

Personas

Hypotheses
Posted in Product

Prioritising Problems

Guidance on Prioritising Problems

By Risk (1, page 45)

  • Highest risk: value ratio gets top priority
  • Work on the riskiest ones first
  • Keep all on a the backlog 

Prioritisation Techniques

Importance vs Satisfaction Framework (2, page 47)

This technique is used for evaluating potential product opportunities

Importance vs Satisfaction (2, page 47)
  • Competitive example is Excel. High important to users and it is the leading example. So product in this space to compete needs to be something like the online sheet tools for collaboration
  • Opportunity example is Uber where user satisfaction was low with taxi service but their need for a solution is very important to them
Measuring Customer Value
  • When importance and satisfaction are both percentages:
Customer value delivered = importance x satisfaction

Opportunity to add value = importance x (1-satisfaction)
Customer Value Example
  • For orange it is:
    • Customer value delivered: 0.9 x (0.35-0.2) = 0.135 
  • For blue it is:
    • Customer value delivered: 0.6 x (0.9-0.6) = 0.18 
  • So blue created marginally more value than orange. 

The Kano Model (2, page 64)

  • Performance needs – adding more will increase satisfaction. For example fuel efficiency on a car. Increasing this will always increase satisfaction.
  • Must-Have needs – they never increase satisfaction. They only decrease it if they are missing 
  • Delighter needs – the customer is not dissatisfied at all if they are missing as they are unexpected and result in very high satisfaction if they are included
Using the kano model to compare with competitors
FeatureCompetitor ACompetitor BYour Product
Must Have Feature 1Yes/ NoYes/ NoYes/ No
Must Have Feature 2Yes/ NoYes/ NoYes/ No
Must Have Feature 3Yes/ NoYes/ NoYes/ No
Performance Feature 1High/ Med/ LowHigh/ Med/ LowHigh/ Med/ Low
Performance Feature 2High/ Med/ LowHigh/ Med/ LowHigh/ Med/ Low
Performance Feature 3High/ Med/ LowHigh/ Med/ LowHigh/ Med/ Low
Delighter Feature 1Yes/ NoYes/ NoYes/ No
Delighter Feature 2Yes/ NoYes/ NoYes/ No
Kano Table

Return on Investment

Approximating ROI (2, pg 84)

References

  1. Lean UX by Jeff Gothelf
  2. Lean Product Playbook by Dan Olsen

Explore more


Which direction?

Metrics

Hypotheses
Posted in Product

What is the Problem?

Problem Statement Formats

Existing product problem statement elements (1, page 25)

  • The current goals of the product or system 
  • The problem the business wants addressed (i.e. where the goals aren’t being met) 
  • An explicit request for improvement that doesn’t dictate a specific solution 
[Our service/ product] is intended to achieve [these goals].
We have observed that the product/ service isn't meeting [these goals] which is causing [this adverse effect] to our business. 
How might we improve [service/ product] so that our customer are more successful based on [these measurable criteria]?

Template (1, page 26) 

New product problem statement elements (1, page 27) 

  • The current state of the market
  • The opportunity the business wants to exploit  (i.e. where the current solutions are failing)
  • A strategic vision for a product or service to address the market gap
The current state of the [domain] has focused primarily on [customer segments, pain points, etc] 
Our product/ service will address this gap by [vision/ strategy] 
Our initial focus will be [this segment]

Template (1, page 27)

References

  1. Lean UX by Jeff Gothelf

Explore more


Metrics

Prioritising

Hypotheses
Posted in Scrum Add-ons

User Stories

Stories express something that will be valuable to a user (2)

Benefits of User Stories

  • They act as a placeholder to remind us to have conversations rather than follow written requirements contracts
  • They encourage interaction through the conversations
  • They are comprehensible by all, from devs and throughout the business, and have no technical jargon
  • They are the right size to be able to plan and estimate
  • When working iteratively we don’t have to write all stories upfront and can refine as we go along the project
  • To make clear the purpose
    • Example from Star Trek if he wants the date to be auto-logged rather than saying it out loud – is it to save him time or for audit logging? One is a lot stricter needs than the other! (4)
  • Focus on who it benefits
    • Example as a commuter, I want a car to drive to work. Very different if the commuter is a surburban commuter vs a farmer in South Dakota (4)

Splitting Stories

Vertical Slicing

  • Always slice stories from the point of view of our customer
  • All stories must have value, which can be to our customer directly or indirectly
  • The technical layers that make up the value are to be kept together (like the layers of a birthday cake) as one layer on its own does not provide value
  • Splitting stories into the technical layers can lead to defining the solution and restricting abilities to creatively iterate

Using Scenarios to Split Stories

  • See BDD and Three Amigos for writing stories with scenarios
  • Use these scenarios to break the story down into smaller pieces of value

Cake Metaphor (3)

Cupcake_parts
Cake Metaphor Doodle
  • To split the story down you could break it down horizontally and take each ingredient in turn
  • But if you slice horizontally you will just get egg and not know if the cake ingredients all work together to provide value (it could taste awful all together!)
  • Instead, bake a cupcake!
  • You get all the ingredients but a smaller size to check the recipe works
Cupcake
A Cupcake Doodle!

User Story Construction

The Three C’s

CARD
  • Description that defers the details
As a [user type]
I want [functionality]
So that [benefit]
CONVERSATION
  • Verbal conversations are best
  • Highlights that we don’t know all the detail
  • Reminder of conversations we have had, work that has been done, any wider context
Confirmation
  • Acceptance Criteria
  • Must be matched for a story to be considered done
  • User point of view only
  • No design specifications unless the user wants them

Good User Story Guidance

INVEST Criteria

INDEPENDENT
  • Stories shouldn’t have to be completed in a specific order and should not rely on another story to be started
  • This is not always achievable
  • Common examples are when you need something to exist before you can build upon it or when one story reduces the complexity of another making it seem logical to do them in a specific order
  • How do we respond to this?
    • Don’t put dependent stories in the same sprint to keep the flow
    • Join together dependent stories
NEGOTIABLE
  • Stories are not contracts – they are short descriptions of functionality
  • Open to discussion with the team
  • Simplify, alter, add to in whatever way is best for the goal and the product
VALUABLE
  • Valued to user/ customer
  • Technology assumptions have no value to the customer!
ESTIMABLE
  • Good size
  • Just enough information, but not too much to become confusing
  • If an investigation is needed before estimation, use a spike
SMALL
  • Lower Limit is coffee break size, i.e. big enough that you deserve a coffee break for finishing
  • Scrum team will highlight size issues in refinement
TESTABLE
  • Language is important
  • It must be specific, unlike “A user never has to wait for it to load”
  • “It takes two seconds max to load in 95% of cases” is much better

Acceptance Test Writing Guidance (2)

  • Always add tests as long as they add value
  • Capture assumptions
  • Provide basic criteria for when story is Done
  • Questions to ask
    • What else do devs want to know?
    • What am I assuming?
    • What can go wrong?
    • Circumstances where story might behave differently
    • Usability
    • Stress
    • Performance

Stories should be closed, not open ended

  • Use language to ensure that the stories have a definite closure
  • Continuous jobs are bad (e.g. managing an ad)
    • Instead use closed actions like “review responses” and “edit response”

User Story Card Example (3)

Story_Card
User Story Card Doodle
  1. Story reference number
  2. Author of the story
  3. Value/ importance
  4. Status/ release
  5. Size/ estimate
  6. Date created
  7. Dependencies
  8. Metrics (if relevant)
  9. Description
  10. Story title (this is the only mandatory field)

User Story Smells

Too Small

EVIDENCE: Frequent need to revise estimates depending on order

FIX: Combine the stories together

Not Independent

EVIDENCE: Difficulty in planning the iteration and lots of swapping stories in and out

FIX: Re-split the stories

Gold Plating

EVIDENCE: Extra wow factor being added that isn’t in the story

FIX: Increase the visibility in the Daily Scrum

Too Much Detail

EVIDENCE: Too long discussing the story

FIX: Strip the story back to its basics and focus on the user point of view

Trouble Prioritising

EVIDENCE: Stories cannot be prioritised logically

FIX: Stories may need to be broken down smaller or re-written from a value point of view


References

  1. The Scrum Guide
  2. User Stories Applied by Mike Cohn
  3. User Story Mapping by Jeff Patton
  4. Scrum: The Art of Doing Twice the Work in Half the Time by Jeff Sutherland

Explore more


Refinement

Prioritising

Estimation