Summary

The law of market failure says most new products fail even when competently executed because teams build it right before proving they are building the right it. Escape thought land by turning a market engagement hypothesis into a clear XYZ hypothesis, then use pretotyping to collect fresh, trustworthy, relevant, and statistically significant data with real skin in the game. Minimize distance, dollars, and hours to data; test locally, cheaply, and repeatedly; treat disproven ideas as saved time and effort.

“Make sure you are building the right it before you build it right.”

“Test a little before you invest a lot.”

Chappy’s Review

A really good practical book on rapid experimentation and prototyping to land on the right product based on user feedback.

Key Takeaways

• Build the right it before you build it right: validate the premise before investing in execution.

• Turn the market engagement hypothesis into an XYZ hypothesis that is clear, testable, numeric, and focused on a sample of 100–1000.

• Run multiple experiments (3-5) because one result leaves wide uncertainty.

Notes

1: Hard Facts

1: The Law of Market Failure

• Hard fact: failure is not an option
• Law of market failure: most new products will fail on the market, even if competently executed
• 70-90% of new products fail
• Fear of failure is why most big companies fail to innovate
• Fails due to…
  1. Launch: GTM
  2. Operations: technical risk
  3. Premise: market risk
• “Make sure you are building the right it before you build it right”

2: The Right It

• Don’t spend too much time in “thought land”
• Problems with focus group market research:
  1. Lost in translation problem
  2. Prediction problem
  3. No skin in the game problem
  4. Confirmation bias problem
• Needs to be concrete, not in head
• Eg. the Webvan problem
• How to know? You need data

3: Data Beats Opinions

• Must be fresh, trustworthy, relevant, and statistically significant

2: Sharp Tools

4: Thinking Tools

• Market engagement hypothesis: how will the market want to engage with your idea?
• Clear, testable, and expressible by numbers hypothesis
• XYZ hypothesis: ↑
  • Eg. 10%+ of coin laundry users will pay $5 extra to have their laundry picked up and returned within 24h
• Zoom to a sample size of 100-1000

5: Pretotyping Tools

• Eg. IBM speed-to-text mechanical Turk
• “Test a little before you invest a lot”
• Pretotyping: tests before the prototype
• Mechanical Turk pretotype: do things that don’t scale
• Pinocchio pretotype: fake product to test how much you would use it if it were functional
• Fake door pretotype: eg. landing page
  • But make sure there’s some win for hitting trap door (eg. discount)
• Facade pretotype: order → hop on a call (one step above fake door)
• YouTube pretotype: video + a way to collect ‘skin in the game’
  • Even just a sophisticated dynamic mockup
• One-night stand pretotype: try it once (eg. Airbnb)
• Infiltrator pretotype: eg. put product in IKEA and test checkout rate, even if fake
• Relabel pretotype: relabel another product with diff value prop

6: Analysis Tools

• Skin in the game points:
  • 0 for interest interview, social media engagement
  • 1 for email
  • 10 for phone #
  • 30 for 30-min call about product
• 10-90% chance of product success
• Run multiple experiments
  • Min 3-5
• Cognitive reframe: you were wrong → you saved so much time / effort

3: Plastic Tactics

7: Tactics Toolkit

• Think globally, test locally (niche)
• Test now - overcome subconscious fear or rejection
• Think cheap, cheaper, cheapest
  • TTD down to hours(!)
• Tweak and flip it before you quit
  • Imagine the opposite → take inspo
  • ↑ pre-pivot
• Minimize distance + dollars + hours to data

8: Complete Example: BusU

• Write idea
• Write XYZ hypothesis
• Devise experiments, order by TTV
• Make sure there is a business (via data) before investing more into it

9: Final Words

• Process:
  1. Start with an idea
  2. Identify the MEH
  3. Turn MEH into XYZ hypothesis
  4. Zoom into a smaller set of smaller XYZ hypotheses
  5. Use pretotyping experiments to collect your own data
  6. Use TRI meter + skin in game calc to analyze data
  7. Decide on the next step
  8. Go for it
• Iterate + try again and you will eventually succeed
• Make sure you really care about what you’re building
• Can apply to non-profits etc. as well
• Go for the right “right it”