MVP Test Grid (1, pg 93)

Research Methods Framework (1, pg 230)

Marketing Materials (1, pg 93)

• To understand which benefits resonate with customers
• To understand how they react to different ways of showing the benefits
• Aim is to understand how they find the marketing material and why
• Marketing material can be landing page, video, advert, email

Quantitative Marketing Tests

Landing page/ smoke test (1, pg 94)

• Traffic is directed to a landing page
• On this page they are asked to show interest (e.g. a sign up button, or a plans and pricing page)
• There is no product yet
• A ‘coming soon’ message is often displayed to those who show interest

Explainer video (1, pg 94)

• Same as landing page
• For products that are harder to explain on a landing page (e.g. dropbox)

Ad campaign (1, pg 94)

• As adverts don’t allow you to display a lot, this is more appropriate for optimising customer acquisition and not product-market fit
• Can advertise to different demographics to check hypothesis about target market
• Measure clickthrough rate to measure which ads (and from which demographics) prove more successful

Marketing A/B testing (1, pg 94)

• Test two alternative designs to compare how they perform
• Run the tests in parallel with 50% of the traffic to each for simplicity

Crowdfunding (1, pg 94)

• Advertising your product on a site like Kickstarter and asking people to pay for the product in advance of it being made
• Set a minimum threshold for funding where you do not build your product before you have raised £X of funding
• The donators to your product will then get your product once it has been built (i.e. pre-order the product with a discount)

Qualitative Product Tests

Wireframes/ Mockups/ Interactive Prototypes (1, pg 100) (2, pg 124)

• Demonstrate or show concepts to user to gauge their feedback (e.g. wireframes)
• Have an ‘ask’ as a definitive pass-or-fail criteria
  • Commitment, monetary value, time, or another investment to show that they are interested
• E.g. dropbox did a video of their concept (advert as if they had built it) to convince investors
• Variations in interactivity and fidelity
• (fidelity refers to how closely the artifact looks like the final product)

Low fidelity prototype (4, page 49)

• Start with the a persona
• Draw the homepage and ask what actions the user wants to do from there
• For each action draw a box (each box is a story)
• Continue until the persona has completed their actions (including exploring edge cases) and then start with another persona

Concierge (2, pg 122)

• Deliver the end result to your customer manually
• Customer understands that it is being done manually and there is no appearance of a final solution to them
• Conduct with just enough users as this is labour intensive

Wizard of Oz (2, pg 123)

• Deliver the end result to your customer manually
• Customer is not aware that it is manual behind the scenes and thinks they are using the end product
• Tempting to leave them up as if successful you will get value from it, but it is expensive to run
• Can be combined with A/B testing

Quantitative Product Tests

Fake Door/ 404 page (1, page 100)

• Good to test demand for a new feature
• Include a link or button on the product to direct customers to a new feature
• The link leads to a page saying it hasn’t been built yet and asking for why they would find this feature valuable
• Overuse will make customers unhappy

Product A/B tests (1, page 100)

• Used to compare performance of two alternative user experiences in your product

Qualitative Behavioural Tests

Usability testing

• Online tools can be used to give a user a task and record them completing the task
• Users are asked to talk through how easy it is to complete a task

Quantitative Behavioural Tests

A/B Testing

• Two different versions of the product are shown to the user
• Differences in behaviour are tracked (e.g. conversion percentage)

Analytics

• Tracking on the product of the users behaviour
• Data can then be analysed to see if hypothesis was acheived

Qualitative Attitudinal Tests

User Interviews

• One-on-one interview with a user
• Coming soon: Tips on User interviews

Quantitative Attitudinal Tests

User Surveys

• Coming soon: Tips of User Surveys

References

1. Lean Product Playbook by Dan Olsen
2. Escaping the Build Trap by Melissa Perri
3. Mastering Professional Scrum by Stephanie Ockerman and Simon Reindl
4. User Stories Applied by Mike Cohn