Posted in Scrum Add-ons

Product Testing

“More than half of our ideas deliver no value, we just don’t know which half” – John Wanamaker (3, pg 78)

Product Testing Statistics

Experiment Statistics (3, pg 79)

  • 65% of feature are rarely or never used
  • At Google and Bings, only 10% to 20% of experiments generate positive results (Harvard Business Review)
  • At Microsoft 1/3 have positive results, 1/3 have neutral results, 1/3 have negative results 

Product Testing Grids

Qualitative Tests

Quantitative Tests

Marketing Tests

Marketing Materials Landing page/ smoke test

Explainer video

Ad campaign

Marketing a/b tests


Product Tests



Interactive prototype

Wizard of Oz & Concierge

Live Product

Fake door/ 404 page

Product analytics & A/B tests

MVP Test Grid (1, pg 93)

Qualitative Tests

Quantitative Tests


Usability testing A/B testing


User Interviews Surveys
Research Methods Framework (1, pg 230)

Qualitative Marketing Tests

Marketing Materials (1, pg 93)

  • To understand which benefits resonate with customers
  • To understand how they react to different ways of showing the benefits
  • Aim is to understand how they find the marketing material and why
  • Marketing material can be landing page, video, advert, email

Quantitative Marketing Tests

Landing page/ smoke test (1, pg 94)

  • Traffic is directed to a landing page
  • On this page they are asked to show interest (e.g. a sign up button, or a plans and pricing page)
  • There is no product yet
  • A ‘coming soon’ message is often displayed to those who show interest

Explainer video (1, pg 94)

  • Same as landing page
  • For products that are harder to explain on a landing page (e.g. dropbox)

Ad campaign (1, pg 94)

  • As adverts don’t allow you to display a lot, this is more appropriate for optimising customer acquisition and not product-market fit
  • Can advertise to different demographics to check hypothesis about target market
  • Measure clickthrough rate to measure which ads (and from which demographics) prove more successful

Marketing A/B testing (1, pg 94)

  • Test two alternative designs to compare how they perform
  • Run the tests in parallel with 50% of the traffic to each for simplicity

Crowdfunding (1, pg 94)

  • Advertising your product on a site like Kickstarter and asking people to pay for the product in advance of it being made
  • Set a minimum threshold for funding where you do not build your product before you have raised £X of funding
  • The donators to your product will then get your product once it has been built (i.e. pre-order the product with a discount)

Qualitative Product Tests

Wireframes/ Mockups/ Interactive Prototypes (1, pg 100) (2, pg 124)

  • Demonstrate or show concepts to user to gauge their feedback (e.g. wireframes)
  • Have an ‘ask’ as a definitive pass-or-fail criteria
    • Commitment, monetary value, time, or another investment to show that they are interested
  • E.g. dropbox did a video of their concept (advert as if they had built it) to convince investors
  • Variations in interactivity and fidelity
  • (fidelity refers to how closely the artifact looks like the final product)
Example of a Wireframe Sketch

Low fidelity prototype (4, page 49)

  • Start with the a persona
  • Draw the homepage and ask what actions the user wants to do from there
  • For each action draw a box (each box is a story)
  • Continue until the persona has completed their actions (including exploring edge cases) and then start with another persona
Example of Low Fidelity Prototype

Concierge (2, pg 122)

  • Deliver the end result to your customer manually
  • Customer understands that it is being done manually and there is no appearance of a final solution to them
  • Conduct with just enough users as this is labour intensive

Wizard of Oz (2, pg 123)

  • Deliver the end result to your customer manually
  • Customer is not aware that it is manual behind the scenes and thinks they are using the end product
  • Tempting to leave them up as if successful you will get value from it, but it is expensive to run
  • Can be combined with A/B testing

Quantitative Product Tests

Fake Door/ 404 page (1, page 100)

  • Good to test demand for a new feature
  • Include a link or button on the product to direct customers to a new feature
  • The link leads to a page saying it hasn’t been built yet and asking for why they would find this feature valuable
  • Overuse will make customers unhappy

Product A/B tests (1, page 100)

  • Used to compare performance of two alternative user experiences in your product

Qualitative Behavioural Tests

Usability testing

  • Online tools can be used to give a user a task and record them completing the task
  • Users are asked to talk through how easy it is to complete a task

Quantitative Behavioural Tests

A/B Testing

  • Two different versions of the product are shown to the user
  • Differences in behaviour are tracked (e.g. conversion percentage)


  • Tracking on the product of the users behaviour
  • Data can then be analysed to see if hypothesis was acheived

Qualitative Attitudinal Tests

User Interviews

  • One-on-one interview with a user
  • Coming soon: Tips on User interviews

Quantitative Attitudinal Tests

User Surveys

  • Coming soon: Tips of User Surveys


  1. Lean Product Playbook by Dan Olsen
  2. Escaping the Build Trap by Melissa Perri
  3. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl
  4. User Stories Applied by Mike Cohn
Posted in Scrum Add-ons

Product Metrics

Criteria for Good metrics 

Actionable (2, 143)

  • Demonstrate clear cause and effect
  • Understand how value was achieved (eg was it engineering or marketing)
  • Blame culture when metrics go down is avoided

Accessible (2, 143)

  • Everyone can get them
  • Allows metrics to guide as they are single source of truth
  • “Metrics are people too”
    • E.g. website hit is not as accessible as customer visiting site

Auditable (2, 143)

  • Data is credible to employees
  • Can pot check the data with real people to verify

Iterating Metrics

Analytics Metrics Loop
The Lean Product Analytics Process (1, page 260)

Product Metrics Structures

Pirate or AARRR Metrics

Originally by David McClure

Pirate Metrics
AARRR Metrics Framework (1, page 239)


  1. Acquisition (prospects visit from various channels/ users find your product)
  2. Activation (prospects convert to customers/ users have their first great experience)
  3. Retention (customers remain active/ user return to your product)
  4. Referral (Customers refer prospects/ users recommend)
  5. Revenue (customers make your business money/ users pay for your product)


  • Can calculate conversion through each step of the funnel (3, page 106)


  • Does not consider user satisfaction (3, page 106)

HEART Framework

  • This framework is for a specific product or feature
  • Happiness (how satisfied the user is with the product)
  • Engagement (how the user interacts with the product)
  • Adoption (same as activation in Pirate Metrics)
  • Retention (same as Pirate Metrics)
  • Task Success (how easy is it for the user to complete the task)

Specific Metric Details

Retention Parameters

Retention Curves (1, page 243)

  • Days since first use does not start at 0 usually as this would be 100% and would alter the scale of the graph
  • Can use cohort analysis (i.e. plotting the retention rates of different user cohorts (groups) onto the same axis to see the difference in the retention parameters for the separate groups
Retention Curve
Retention Curve (1, page 243)
  • Parameter 1 to notice: The percentage where the graph starts on Day 1 shows the initial drop off rate
  • Parameter 2: Rate that the retention curve decreases from Day 1 value
  • Parameter 3: Terminal value for retention curves is where the retention flattens out. If it is 0% then your product will ultimately lose all of its customers


  1. Lean Product Playbook by Dan Olsen
  2. Lean Startup by Eric Ries
  3. Escaping the Build Trap by Melissa Perri
Posted in Scrum Roles

Scrum Master

“The Scrum Master is responsible for promoting and supporting Scrum as defined in the Scrum Guide”(1)


A Scrum Master…

  • helps everyone understand Scrum
  • is a servant-leader for the Scrum Team
  • helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which aren’t (1)
  • The Scrum Master helps everyone change these interactions to maximise the value created by the Scrum Team (1)

Scrum Master service to the PO

Maximising Value 

  • Supports understanding product planning in an empirical environment (1)
  • Ensures the Product Owner knows how to arrange the Product Backlog to maximize value (1)
  • Coaching question:
    • How is the product going? (2)

Product Backlog Management 

  • Finds techniques for effective Product Backlog management (1)
  • Helps the Scrum Team understand the need for clear and concise Product Backlog items (1)

Development Team Interactions

  • Ensures that goals, scope, and product domain are understood by everyone on the Scrum Team as well as possible (1)
  • Coaching Questions (2) 
    • How is the team doing?
    • How each of you is fulfilling your roles?
    • How can you help one another?

Agility and Personal Development

  • Supports understanding and practicing agility (1)
  • Teaching Questions (2)
    • What must you believe about the team and the organisation to be a good PO
    • What parts of the role feel like a stretch for you?
    • What parts of the role do you feel you have mastered?
    • Which parts will you have to make yourself do?
    • What should I, as the coach, watch for to keep these beliefs?

Product Owner Support Exercise (3)

  1. Brainstorm with the Product Owner all the duties that their role should perform (or both Scrum Master and Product Owner if one person is doing both roles)
  2. Talk through them and highlight the ones that they are not able to do because of time-constraint/ training/ empowerment/ conflict of interest between roles/ etc
  3. Present the outcome to the Product Owner’s manager/ ‘trace the money’ if the Product Owner isn’t the decision maker

Scrum Master service to the Devs

  • Coaching in self-organisation and cross-functionality (1)
  • Removing impediments to the Development Team’s progress (1)
  • Facilitating Scrum events as requested or needed (1)
  • Coaching the Development Team in organisational environments in which Scrum is not yet fully adopted and understood (1)

Scrum Master service to the organisation

  • Leading and coaching the organisation in its Scrum adoption (1)
  • Planning Scrum implementations within the organisation (1)
  • Helping employees and stakeholders understand and enact Scrum and empirical product development (1)
  • Causing change that increases the productivity of the Scrum Team (1)
  • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organisation (1)


Scrum Master Qualities (4, pg 128)

  • Leads by example
    • Scrum values
    • Trust in empiricism
    • Positive mindset
    • Adaptive approach
  • Enables and empowers others
    • Doesn’t solve people’s problems, but makes opportunities transparent
    • Knows he/she doesn’t have the best answers
  • Creates and environment of safety and is comfortable with failure
    • Safety in conflict
    • Trying new things
  • Cares deeply for people and is also willing to challenge when they are capable of more
    • Assumes positive intent and doesn’t judge people
    • Meets people where they are and helps them find their next step
    • Inspires to hold themselves to even higher standards
  • Opertates with integrity and stays calm under pressure
    • His/her leadership provides consistency and stability
  • Shows low tolerance for organisational impediments
    • Willing to challenge and speak the truth
    • Advocate for the team


  1. The Scrum Guide 
  2. Coaching Agile Teams by Lyssa Adkins
  3. Fixing Your Scrum by Ryan Ripley and Todd Miller
  4. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl

Explore more Scrum

Scrum Theory Scrum Values Development Team Product Owner Definition of Done
Posted in Scrum Roles

Product Owner

‘Responsible for maximising the value of the product resulting from work of the Development Team’ (1)

Be one mind with the sponsor (2)

Vision keeper (2)

Product Owner Responsibilities

Maximising Value

  • how this is done may vary widely across organisations, Scrum Teams, and individuals (1)

Managing the Product Backlog

  • expressing Product Backlog items and ensuring the Development Team understands them sufficiently
  • ordering the items in the Product Backlog to best achieve goals and missions and so it is clear what is to be worked on next
  • transparency of the Product Backlog is ensured and it is visible

Product Owner in the Business

Product Owner is One Person 

  • the Product Owner is one person
  • the Product Owner may represent a committee
  • those wanting to change a Product Backlog item’s priority must address the Product Owner (1)

Stakeholder Mapping (3, pg 178)

  1. With the PO (and identified stakeholders) try and think of every person in the organisation who has a vested interest in the outcome.
    • Where is the money coming from?
    • Who’s job is going to change because of this product?
    • Who might interact with a customer differently as a result of what we are building?
    • Who might be angry if they don’t know what is going on with the product?
  2. Put the following categories on the wall and put each stakeholder inside a category
    • Required for the Sprint Review: these people must inspect and provide feedback on the product increment to enable the Scrum Team to make informed decisions
    • Keep informed of progress: they don’t need to inspect every increment but need to know what progress has been made
    • Monitor: should receive updates periodically (check with them how often they would like these)

Product Owner’s Decisions (1)

  • the entire organisation must respect the Product Owner’s decisions
  • the content and ordering of the Product Backlog makes the decisions visible to all
  • no one can force the Development Team to work from a different set of requirements


Product Owner Behaviours


  • Committed
  • Responsible (outcome)
  • Authorised (to make decisions)
  • Collaborative
  • Knowledgable (about the business)

Product Owner’s Relationship with the Development Team

Tips from Lyssa Adkins (2)

  • No micromanaging
  • Hold the team to account
  • Show genuine disappointment
  • Be present


  1. The Scrum Guide
  2. Coaching Agile Teams by Lyssa Adkins
  3. Fixing Your Scrum by Ryan Ripley and Todd Miller

Explore more Scrum

Scrum Theory Scrum Values Development Team Scrum Master Definition of Done
Posted in Scrum Events

Sprint Planning


“The work to be performed in the Sprint is planned at the Sprint Planning” (1)


Definition: Sprint Planning is a meeting involving the entire Scrum Team to collaboratively decide what work will be taken up for the Sprint and how the work will be done (2, pg 27)

Length: Maximum of 8 hours for a one-month Sprint

Input: Product Backlog, Definition of Done, Retrospective Improvements, Team Capacity Planning, Impediments, Product Increment

Outcome: Sprint Goal (The What) and Sprint Backlog (The Why)

People: Product Owner, Scrum Master, Development Team

By the end of the Sprint Planning, the Development Team should be able to explain to the Product Owner and Scrum Master how it intends to work as a self-organizing team to accomplish the Sprint Goal and create the anticipated Increment. (1)

What can be done this Sprint?

Product Backlog Items

  • The Development Team forecasts the functionality that will be developed during the Sprint for the Product Owner’s objective
  • The number of items selected from the Product Backlog for the Sprint is solely up to the Development Team. 

The Sprint Goal

Definition: The Sprint Goal is an objective that will be met within the Sprint through the implementation of the Product Backlog, and it provides guidance to the Development Team on why it is building the Increment (1)

  • During Sprint Planning the Scrum Team also crafts a Sprint Goal
  • The Sprint Goal is one coherent function
  • The Sprint Goal means all the Development Team work together rather than on separates objectives
  • The Sprint Backlog items can be negotiated with the Product Owner whilst the Sprint Goal

How will the chosen work get done?

  • The Development team decides how it will deliver the selected Product Backlog Items for the Sprint
  • The Development Team designs the system and the work needed to convert the Product Backlog Items into an Increment
    • Just enough to forecast what it can do in the Sprint
    • Work for the first few days of the Sprint is broken down more
    • Work may be of varying size, or estimated effort
  • The Product Owner helps to clarify and negotiate the selected Product Backlog items
  • The Development Team may also invite people with specific expertise to support them

Sprint Backlog

Product Backlog Items + Plan = Sprint Backlog


Sprint Planning Facilitation Technique (3, pg 128)



  1. The Scrum Guide
  2. Scrum Insights for Practitioners by Hiren Doshi
  3. Fixing Your Scrum by Ryan Ripley and Todd Miller

Explore Other Scrum Events

Sprint Daily Scrum Sprint Review Sprint Retrospective


Posted in Scrum Events

Daily Scrum

Are we on track to achieve our forecasted Sprint Goal?


Definition:  […] for the Development Team to synchronise Development Team activities, create a plan for the next twenty-four hours, and ensure everyone in the team is aligned toward the Sprint Goal (3, pg 31)

Length: 15 minutes

Inputs: Sprint Goal, Sprint Backlog, open impediments, (optional) information radiator

Outcome: Update Sprint Goal, updated Sprint Backlog, updated impediments, (optional) updated information radiator. Optimises the chances of achieving the Sprint Goal.

People: Development Team, (optional) Product Owner, (optional) Scrum Master

Timing and Location: Same place, same time every day to set a routine


The structure is set by the development team.

Common 3 questions (1)

  • What did I do yesterday that helped the Development Team meet the Sprint Goal?
  • What will I do today to help the Development Team meet the Sprint Goal?
  • Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?

Daily Scrum Questions (2, pg 152)

  • Is anything stuck?
  • If something is stuck, how can we all work together to get this work unstuck?
  • Who needs help?
  • What’s the most important thing we need to accomplish today?
  • How do we increase the odds that the most important things get to done?

Daily Scrum 1-2-4-All Liberating Structure (2, pg 155)

  1. Devs spend one minute silently answering ‘What opportunities fo you see for making progress on the team’s sprint goal?’
  2. Devs spend 2 minutes in pairs generating ideas and insights on what they thought about in step 1
  3. Each pair joins with another for 4 minutes to develop the ideas further noticing how their ideas compare to each other
  4. Ask the group ‘What stood out during your conversations?’ and have each group share their idea with the team

Fist of Five voting (2, pg 156)

  • Ask the team to express their level of confidence in a statement by voting with the fingers on their hands (1 = Not at all, 3 = need to discuss it more before I agree or disagree, 5 = Totally agree)
  • Example question: ‘How confident are you that we’re on track to meet the sprint goal?’


Standing Up (3, pg 33)

  • It is a myth that everyone has to stand up
  • It can be used as a technique to help the meeting not break the 15 minute time-box

Daily Scrum Smells

  • Not a status meeting
  • Not used for detailed discussion and problem resolution


  1. The Scrum Guide
  2. Fixing Your Scrum by Ryan Ripley and Todd Miller
  3. Scrum Insights for Practitioners by Hiren Doshi

Explore Other Scrum Events

Sprint Sprint Planning Sprint Review Sprint Retrospective
Posted in Scrum Events

Sprint Review

“held at the end of the Sprint to inspect the Increment and adapt the Product Backlog if needed” (1)


Definition: a formal event to inspect the increment and collaborate on the next things to optimise value

Length: 4 hours for a month long Sprint. For shorter Sprints the event is usually shorter.

Inputs: Product Increment (which must adhere to the Definition of Done), Product Backlog, Impediments that Stakeholders can help resolve

Outcome: Product Increment, Feedback on Product Increment adapted in the Product Backlog, updated impediments

People: Product Owner, Scrum Master, Development Team, Stakeholders (by invitation from the Product Owner)

Sprint Review Contents

The Sprint Review includes the following elements (1)

  • The Product Owner explains what Product Backlog items have been “Done” and what has not been “Done”
  • The Development Team discusses what went well during the Sprint, what problems it ran into, and how those problems were solved
  • The Development Team demonstrates the work that it has “Done” and answers questions about the Increment
  • The Product Owner discusses the Product Backlog as it stands. He or she projects likely target and delivery dates based on progress to date (if needed)
  • The entire group collaborates on what to do next, so that the Sprint Review provides valuable input to subsequent Sprint Planning
  • Review of how the marketplace or potential use of the product might have changed what is the most valuable thing to do next
  • Review of the timeline, budget, potential capabilities, and marketplace for the next anticipated releases of functionality or capability of the product

Sprint Review Agenda (2, pg 37)

  1. Product Owner reviews the planned Sprint Goal and what has been accomplished
  2. The Scrum Team inspects the Product Increment with Stakeholders by discussing the approach they took, reviewing the challenges and their resolutions, and inspecting the Product Increment
  3. The Product Owner discusses the Product Backlog and likely completion date
  4. The Scrum Team and Stakeholders collaborate by determining what to build in the upcoming Sprint

Sprint Review Agenda (3, pg 182)

  • Explanation of the PO’s vision for the product
  • Review of the release plan and budget
  • Description of the Sprint Goal for this sprint
  • Review of the work and the product increment that the team built
  • Assessment of current customer analytics and data
  • Discussion of impediments the team is currently facing
  • Collaborating with stakeholders to refine the Product Backlog
  • Summary of where we’re going next and what we discovered in this event


Sprint Review Smells

Sprint Review Smells (2, pg 38)

  • There isn’t anything to show at the Sprint Review so it is cancelled
    • It is a mandatory event so cannot be cancelled
    • Opportunity for inspection and adaption should not be missed
    • Feedback from Stakeholders helps to determine the direction for the next Sprint


  1. The Scrum Guide
  2. Scrum Insights for Practitionersby Hiren Doshi
  3. Fixing Your Scrum by Ryan Ripley and Todd Miller

Explore Other Scrum Events

Sprint Sprint Planning Daily Scrum Sprint Retrospective