If I had an hour to solve a problem, I’d spend 55 minutes thinking about the problem and 5 minutes thinking about solutions

Albert Einstein

Review

Teresa Torres is a household name in the Product community, and this might be the most influential product book written for a decade. It became an instant classic with Product Managers for good reason.

I suspect it resonated with PMs because so many teams are struggling to do product discovery well. Teresa made it seem easy.

Once you’ve mapped your opportunity space as a tree, prioritization and research become easier. You’ll be able to systematically identify the most important thing to do next (research, experiment or build).

Meeting customers regularly increases your both the quality and quantity of insight, and helps you navigate your opportunity space.

There’s a ton of value in this book, so I recommend reading it in full.

Key Takeaways

The 20% that gave me 80% of the value.

• Agile helped teams build more software – but agile doesn’t help us build the right thing. The fruit of discovery work is the time saved not building something that won’t work
• Historically, teams viewed discovery as a one-time activity at the beginning of projects and focused too much on delivery. Teresa advocates for continuous discovery → having weekly touch-points with customers to conduct small research activities with a desired outcome in mind
• Successful products provide value to both the business and the customer. How we frame a problem influences how we solve it – therefore teams should frame problems in a customer-centric way
• Mapping the opportunity space and selecting which opportunities to pursue are high leverage points in a product team’s system, they are crucial to a team’s success
• The Opportunity Solution Tree helps teams select the most impactful thing to do next – even if that activity is more discovery/exploration
• What should we build next? -> what should we do next?

• Prioritizing opportunities instead of solutions will save time. Product strategy doesn’t happen in the solution space, it happens in the opportunity space.
• Teams should be assigned outcomes, and given freedom to explore and select opportunities. It’s unfair to assign an outcome to a team they can’t influence. It’s important to help teams find manageable metrics, that will ultimately impact the business outcome.
• Visualize what you know in a customer’s experience map – and use it to guide your interviews. Interviews help discover and explore opportunities. Meet with customers regularly and you don’t have to cover so much ground. Instead you can get insight into the opportunity area that you’re working on now – just in time to influence what potential solutions could look like
• You and your team can take a step towards this way of working today. Pick your moment to advocate for more discovery. To maintain autonomy teams must show their work to stakeholders – opportunities explored, customers spoken to, metrics identified etc.

In-depth Summary

Continuous Discovery Habits

• The book describes a continuous discovery framework that’s equally applicable to new products and iterating on existing ones.
• The Two Types of product work:
  • Discovery: work that helps you decide what to build
  • Delivery: work you do to build and ship

• Discovery:
  • How do you know you’re making a product or service your customers want?
  • How do you guarantee you are creating value for customers in a way that creates value for the business?
• Delivery:
  • many product teams focus too much on delivery and treat discovery as something that happens as a one-time activity at the beginning of projects.

Interesting aside: How software development methodologies evolve

Project Management Approach: Business owners decided what to build Discovery happened infrequently (with the yearly planning cycle.) Projects with fixed timelines were assigned to engineering teams Project managers managed the work, budget and schedule Challenges: Software development is unpredictable Often over time and over budget Needs of the business trumped needs of the customer Teams learn late that customers don’t like their product Lots of waste

Agile and UX Approach: – projects were too big and teams were building the wrong stuff – agile advocated for shorter cycles, sustainable pace, maximum flexibility, simplicity – rise in the adoption of Scrum and Kanban – growth of user-experience design and user research → to collect customer feedback Benefits: – shorter development cycles – better product metrics – better at usability – better at iteration and starting small Challenges: – leaders didn’t want to give up discovery – stakeholders cling to their ideas still – teams are bad at estimating – typically the whole organization didn’t become agile – stuck in old cycles – teams weren’t allowed to fail – had to deliver on time and on budget even if they know their thing wasn’t valuable – usability testing done late, user research outsourced, teams measured by what they delivered not value created – still too much time was wasted – teams learned after shipping code they’d built the wrong stuff Instrumentation showed us that our new stuff didn’t work

Continuous discovery: – product teams adopting continuous discovery habits – engaging with customers to understand their wants and needs – co-creating products with them

Continuous Discovery Mindset (Required before you start)

Working definition of Continuous Discovery

Product teams adopting discovery habits in a structured, and sustainable way, enabling them to continuously infuse their product decisions with customer input. Fresh customer input helps with decision making.

A Common Framework for Continuous Discovery

• Balance the needs of the business with the needs of the customer. Profit shouldn’t come at the cost of serving the customer. Serving customers is how you generate profit.
• Begin with the end in mind. Shifting from output mindset to an outcome mindset. Lay the foundation for success – by starting with outcomes not outputs.
• Task product trio with delivering an outcome. Give them space to explore the best outputs. Give the team the latitude thy need to create value. They have a choice – choose to engage with customers – do the work to understand the customer context.
• Organizational context has a big impact on success. Will the team choose to put the customer first? Or take shortcuts and do things the business wants without regard for the customer.

The Challenge of driving outcomes

• The purpose of the customer touch-point is to conduct research in pursuit of a desired outcome. Looking for ways to serve the customers that also create business value.
• Finding the best way to achieve that desired outcome is an ill-structured (or wicked) problem. There are many potential solutions, no right or wrong answers only better or worse ones.
• Much of the work needs to be about framing the problem. Frame the problem in a customer-centric way! This avoids creating features that work only for the business.
  • Opportunities are customer pain-points and desires
  • Opportunity Space = Problem Space = Desire Space
• To reach a desired outcome – a product trio must discover and explore the opportunity space. The opportunity space is infinite. Problem framing is therefore important – how we frame a problem influences how we solve it. Try out many framings – explore how each impacts the solution space
• Two of the most important steps for reaching a desired outcome are first:
  1. How we map out and structure the opportunity space
  2. How we select which opportunities to pursue
• Don’t skip these steps. Don’t start with an outcome and generate ideas.

The Underlying Structure of Discovery The Opportunity Solution Tree

The Opportunity Solution Tree is a visual representation of the paths you might take to reach a desired outcome.

Benefits of an Opportunity Solution Tree

• Resolves the tension between business and customer needs
  • Starting with the business need means everything is in service of the business. Filtering by customer opportunities ensures everything is in service of the customer.
• Builds and maintains a shared understanding of how you might reach the outcome
  • Stops the team settling on the first or most obvious solution. Helps the team generate more options (How else might we solve this problem?). Opinion battles don’t encourage collaboration, the tree does. Builds a shared understanding of the options.
• Helps adopt a continuous mindset
  • Helps break down a project (outcome or opportunity) into smaller chunks. Teams can focus on solving one thing at a time. Which helps deliver value every sprint
• Unlocks Better Decision Making
  • Stops you bouncing from tactic to tactic, you can take a more strategic approach to selection
  • Helps with the 4 villains of decision making
    • looking at a problem too narrowly
    • looking for evidence that confirms our beliefs
    • short term emotions affect our decisions
    • overconfidence
  • Inspires a compare and contrast mindset – not a whether or not mindset. Which of these customer needs is more important for us to address right now? What else could we build? How else might we address this opportunity?
  • Good discovery only reduces the chance of failure – be prepared to be wrong – balance having confidence with doubting what you know.
  • Visualizes the decision making process
• Unlocks Faster Learning Cycles
  • The problem space and solution space evolve together – as you explore solutions you learn more about the problem. If something doesn’t work…
    • Based on my understanding of the customer I thought this would work. What did I misunderstand about my customer? How might we revise the opportunity space now
• Builds confidence in knowing what to do next
  • Helps us know if we’ve gone wide enough, if we are doing enough assumption testing, if we have enough solutions mapped.
• Unlocks simpler stakeholder management
  • Helps justify decisions to leadership. Show your work. Share the right amount of detail with leadership.

How to focus on Outcomes not Outputs

• Product teams will need to do discovery work to identify connections between product outcomes (metrics they want to influence) and business outcomes (metrics that drive the business)
• Translate business outcomes into product outcomes that you can deliver
• Negotiate appropriate product outcomes with leadership
• Determine when to set learning goals versus performance goals
• Focusing on outcomes is hard
  • Don’t know how to measure the outcome
  • Don’t know how to impact it – make it manageable
  • Or if it’s the right outcome to pursue the first place. Managing by outcomes is only effective if you choose the right one
  • ‘Tails.com’ Retention Example:
    • 90 day retention was a sensible yardstick. But it took too long to iterate against. The team kept shortening (90→ 30 → 5). They were able to experiment faster, but they weren’t sure if they were driving the business outcome
    • Customer interviewers uncovered 2 churn factors
      • Not all customers understood the value of tailor-made dog food
      • Some dogs didn’t like the food
    • The team now had two things to action and measure – that they knew would drive business value
      • They needed to increase the perceived value of tailor-made dog food
      • They needed to increase the number of dogs that liked the food
• Business Outcomes are often a lagging indicator – Identify a leading product outcome as a proxy
  • It’s hard to use lagging indicators to drive work – you can’t the impact of your work quickly
  • Explore leading indicators that predict the direction of lagging indicators
  • Assigning a leading indicator is better than assigning a lagging indicator

Business outcome	Measure business value	Retention
Product outcome	Measure how the product drives business value	% Dogs who like the food
Traction metric	Tracks usage of specific features	Owners who use the transition calendar

• Assign a team a business outcome – they might not be able to influence it without others ☹️
• Assign a team a product outcome – great – flexibility and autonomy to the team 🙂
• Assign a team a traction metric – it implies you know the right metric to move ☹️
  • Might be OK for a junior team
  • Might be OK for mature products with proven traction metrics (previously done discovery)

🚧 It’s unfair to assign a business outcome to a team that can’t directly do everything they need to do. Instead assign product outcomes to each team – that are in service of the business outcomes

• Benefits of Focusing on Product Outcomes
  1. Autonomy – Gives the team autonomy to find the best solution (they have the expertise)
  2. Anti-Fragility – you don’t need to know the solution, or you can be wrong
  3. Measurement – becomes an important part of what you do
  4. Teams make faster progress – don’t have to wait for lagging metric to work through
  5. Control – teams are in control of Product Outcomes
  6. Reduces communication overheads
• Outcomes are the result of a two-way negotiation between the team and the product leader
  • Product leader brings the business view – what’s most important for the business
  • Product trio bring the customer and technology knowledge
  • Negotiation typically includes discussion on:
    • Expected gains, risk, resources, approach, removing competing priorities, time to expected results, discovery time.
      • If the business want more impact. That normally results in bigger bets and a bigger chance of failure.
  • During the negotiation – we shouldn’t be talking about solutions – this should emerge from discovery
• Those that participate in the setting of their own outcomes take more ownership
• On SMART Goals and OKRS
  • Specific, Measurable, Achievable, Relevant and Time-bound goals
  • Teams that set specific, challenging goals outperform those that don’t
  • They create focus, inspire effort and persistence and help surface relevant organizational knowledge
  • Caveats
    • They have to believe they can achieve it
    • They need to be committed to the goal
    • Teams need to be involved in their own outcomes
    • If the team doesn’t have the strategies for how to achieve them, they can be demotivating
  • Internal learning goals are more effective than performance goals
  • Getting teams to do their best is more effective than setting specific goals
  • Unless you can give them the strategies they need
  • Start with a learning goal – discover opportunities and drive engagement
    • Then task with a performance goal
    • E.g → Learn what leads to churn → Start to tackle it
  • OKRs are one way of managing based on outcomes
• Shift your product leader toward an outcome mindset.

Product Leader asks the team to deliver outputs (not outcomes)	Ask for more business context. – Who is the target customer? – What business outcomes are we trying to drive with this initiative? – Why do we thing this initiative will drive that outcome? Check if the product outcome is really a leading indicator of the desired business outcome
Product Leader sets outcomes BUT with little input from team	Map out the product outcomes that might drive the business outcome Give feedback to your leaders If given a product outcome – ask for the business context What business outcomes are we trying to drive with this product outcome? Clearly communicate how far you think you can get in the allotted time
Team set outcomes BUT with little input from product leader	Try and shift towards a two-way negotiation Before setting your outcome – ask for more business context – What’s the most important thing to business right now? – Try to frame this conversation in terms of business outcomes – Is there a customer segment that is more important than other customer segments? – Are there strategic initiatives we should know about? Use the information you gain to map out the most important business outcomes and what product outcomes might drive those business outcomes – Get feedback from your leader – Choose a product outcome that your team has the most influence over
Team negotiates outcomes with product leader	Congratulations 😀 Check you’re being tasked with a product outcome, not a business outcome or a traction metric If you are being tasked with a traction metric – is it proven? If this is the first time you’re working on a new metric, do you have a learning goal?

Discovering Opportunities – Opportunity Space

The Experience Map – Visualizing What You Know

• Before mapping the opportunity space – you need to take inventory of what you already know
  • Map out your own perspective first
  • Explore everyone else’s perspective – together you’ll have a richer understanding
  • Merge them into a shared experience map
  • Make explicit what you know, what you don’t, what your hunches are
  • Note where you have open questions for customer interviews
• The Experience Map reflects what you know about a customers experience
  • The map will guide your customer interviews
  • Help you structure the opportunity space
  • Set the scope of the experience map – think about how broad or narrow to go
  • Start with the desired outcome – focus on a question
    • E.g: what’s preventing our customers from completing their application today?
    • The outcome constrains what you capture
  • Think about how broad or narrow to go
    • If focused on optimization – narrow is OK
    • If focused on open-ended outcome – expand the scope
      • E.g for Netflix: increase hours watched – set broad scope
        • Not as broad as ‘entertainment in general’
        • But as broad as through satellite, twitch, YouTube and competitors
    • Constrain scope, but not too much
• Steps to building your experience map
  • Start individually to avoid group thinking – develop your own perspective:
    • Visual not verbal – externalize your thinking – so you can examine it
      • Draw from your customers perspective
      • Not everything – just what’s relevant
      • Think about acquisition, retention, work of mouth etc.
      • Language is vague – drawing makes your ideas more concrete
    • Explore the diverse perspective of your team
      • Take turns to share drawings
      • Ask questions – explore your team mates perspective
      • Pay attention to the differences – be curious
      • Share your point of view – don’t advocate
        • Answer questions to clarify your thinking
    • Co-Create a shared experience map
      • Synthesize your work – don’t pick the best
      • Start by turning each of your individual maps into a collection of nodes and links
        • Node – distinct moment in time, action or event
        • Links – what connect notes
        • Links – might be a loop
      • Create a new map that includes all of your individual notes
        • Collapse similar notes together
        • Determine the links between each note
        • Add context (what are they thinking, feeling, doing)

• Avoid these common anti-patterns
  • Getting bogged down in debate
  • Using words instead of visuals
  • Moving forward as if your map is true
  • Forgetting to refine and evolve your map as you learn more

Continuous Interviewing

💡 The purpose of interviewing is NOT to ask what to build. It’s to discover and explore opportunities

Don’t ask people what they need – People’s identity impacts their answers:

• People don’t have a good understanding of their own behavior. Example:
  • What factors do you consider when buying jeans? → Fit
  • Tell me about the last time you bought beans? → Online
    • How did you know they would fit? → I didn’t, they were a brand I liked and they were on sale
• People struggle to answer direct factual questions accurately
• People give answers that align to their identity not behavior
• We have a tendency to rationalize our behavior – your brain summons a compelling reason and it feels like the truth
• Be wary of a coherent story, told by both the thought leaders in our space and by our customers themselves

Distinguish research questions from interview questions – ask about behavior:

• If you speak to your customers occasionally, you’ll have to ask them about everything every time. If you speak to them weekly, you can ask about what you need to learn about in that moment. I call this concept just in time learning and it’s great – I do it with books
• Balance each interview:
  • Broad exploration into needs, pain points and desires
  • Diving deep on the specific opportunities that are most relevant to you
• Don’t explore anything that isn’t valuable to your customer!
  • Primary Research Question: What needs, pain points and desires matter most to this customer?
• Memories about recent instances are more reliable than our generalizations about our own behavior or our answers to direct questions.
  • Tell me about the last time you did x. Asking about a specific instance better reflects actual behavior, not perceived behavior
• Think about how broad your question should be. Do you care about your product or also your competitors? Or tangential products and experiences
  • Narrow is good for optimizing your product
  • Broad is good for new opportunities

Excavate the Story – use temporal prompts:

• If you ask a short question, people will give short answers. They’ll match you.
• Use Temporal prompts – Start at the beginning, What happened first?
  • Where were you? What happened next? What happened before that? Who was with you? What problems did you face? How did you overcome that challenge?
• Bring them back to this instance, instead of letting them generalize. In this specific example, did you face that challenge?

You won’t always get what you want – its OK:

• Let the participant talk about what they care about most
• Don’t force it. The stakes are low. It’s OK if you don’t get what you need. Especially if you’re interviewing regularly

Synthesize as you go – make 1 pagers:

• Interviewer Snapshots turn detailed notes into insights. A one page synthesis of what you learnt in each interview
• Interview Snapshot
  • Name
  • Photo
  • Quick facts – Customer Segment etc.
  • Insights – weaker signal opportunities, turn into opportunities over time
  • Opportunities – need, pain point or desire
  • Memorable quotes

Draw the stories you collect:

• Experience map will help guide each interview
• Is their story the same or different to your experience map?
• Draw the underlying structure of the stories that you hear – the nodes and the links

Interview Every Week:

• Helps explore an ever-evolving opportunity space
  • Needs change
  • Markets change – competitors, products etc.
  • A digital product is never done – the opportunities space is never finite or complete
• Just because you chose a target opportunity to build, doesn’t mean that you can stop interviewing
  • As it assumes you chose the right one – and you’ll be able to address it
  • Interviewing is not a step in a linear process – instead our goal is to interview continuously
• Enables you to revisit the opportunity solution tree
• It’s mush easier to maintain a habit than to start and stop it

Automate the process:

• The goal is to wake up with a weekly interview scheduled without you having to do anything
• The goal is to make it easier to interview, than to not, by the use of automation

How to get interview participants

• Recruit participants while they are using your product or service
  • Do you have 20 minutes to talk to us for 20 EUR?
  • Or put ads to drive traffic to an interview landing page
• Ask your customer-facing colleagues to interview
  • Customer calls to cancel the sub – ask them why etc.
  • Customer has a question about feature x
  • Customer requests a customization schedule an interview
• Make it clear who you want to talk to, make it easy to schedule
  • I’d love for you to share your feedback with our product team, can we schedule 20 minutes for you to talk with them?
• Interview your customer advisory board
  • If hard to reach, or you have a small market, or their time is too valuable…
  • Set up a customer advisory board
  • Use them as participants
  • Offer an ongoing incentive
  • Scale it to suit the the number of interviews you need

Interview together, act together – The Trio:

• Get all the trio to do it – don’t have a single person be the voice of the customer
• You’ll each hear different things – different perspective

Avoid these common anti-patterns:

• Relying on one person to recruit interview participants
• Asking who, what, why, how and when questions. Use story based instead
• Interviewing only when you think you need it
• Sharing what you learned by sending out pages of notes or a recording – use snapshots instead
• Stopping to synthesize a set of interviews – just keep interviewing at a sustainable cadence

Mapping the Opportunity Space

With each interview the structure of the opportunity space continues to evolve. Keep interviewing until patterns emerge

The Power of Opportunity Mapping and Focus:

• Listening to users will help you list countless needs, pain point and desires
• Looks for gaps between expectations and how the world works
• How do you know which ones to action? Take inventory of the opportunity space
  • helps you find the best path to the desired outcome
  • gives you structure to evaluate things
• Most product teams bounce from one opportunity to the next. Our job is not to address every opportunity
  • It’s to find and address those that drive the desired outcome
  • This is how we create value for for customer
  • Focusing and limiting work to opportunities that might drive our desired outcome is what ensures that our products are viable
• The aim is to address the customer opportunities that will have the biggest impact on our outcome first
• Start by taking an inventory of the possibilities – explore options
• Compare and contrast impact of addressing one opportunity against the impact of addressing another
  • Rephrase opportunities that aren’t specific enough
  • Group similar opportunities together
• Requires critical thinking – help to ensure that we are always addressing the most impactful opportunity.

The Opportunity Solution Tree

• Trees can tame backlogs.
  • A flat list of opportunities is hard to prioritize. Some are interrelated, some are a subset of others. Some don’t come with clear solutions – some do. Much of that nuance is lost in a flat list
• The tree helps us expose and understand the complexity of the opportunity space
• A tree shows ‘Parent-child’ and ‘Sibling’ relationships

• Helps us group customer needs – deconstruct the problem
• Breaks big opportunities into a series of smaller ones
  • Allows us to tackle problems that seem unsolvable
  • Allows us to deliver value over time
• Start by choosing easy things with large impact. Focus on shipping value over time.
• Solve enough smaller opportunities and you’ll have solved the larger one
• The tree helps asses and prioritize opportunities and make fast decisions

Identify Distinct Branches:

• The idea of distinctness is key – each opportunity has to be unique and self contained
• What customers do to address their needs today helps inform user research and helps give structure to the opportunity space
• Your user research can help you
• You can analyze by distinct moments in time
• Or identify key moments from research
• Map frequently occurring nodes to your top-level opportunity space
• The key is to make sure there’s no overlap between the moments in time – overlap prevents you from working on one at a time
• Streaming moments example:
  • Deciding to watch something
  • Choosing something to watch
  • Watching something
  • The end of the watching experience

Take an Inventory of the Opportunity Space:

• Review interview snapshots for opportunities to add
• Frame opportunities as a customer need or pain-point (not a solution)
• Make sure it’s common across customers – not unique to the one you spoke to
• It must feasibly affect the outcome if you address it
• If it feels like it can live under one or more branches re-frame it to be more specific OR split it into multiple distinct opportunities

Example: “the interface is hard to use” ❌  This is too vague!

• I can’t figure out how to find a show I have in mind ✅
• I don’t like typing out the show name ✅
• thumbnails are too big ✅
• …

Add Structure to Each Branch:

• Group similar opportunities together – and nest them under a new parent opportunity (don’t fret if your parent label wasn’t mentioned in an interview)

• Or combine them into one – if they are actually the same thing
• As you group siblings together, you end up with mini trees

Just Enough Structure:

• Opportunity mapping looks deceptively simple – but it requires critical thinking
  • Examine each opportunity to check its framed well, it’s clear, and can move the desired outcome
  • Expect to spend more than 30 minutes, but don’t spend more than a day
  • Get enough structure to the picture – but not so much that you’re overwhelmed by detail
• Structure is done, undone and redone
  • revise your tree often
  • re-frame opportunities as you learn more
  • do just enough to capture what you already know – trust that it will continue to grow and evolve overtime

Avoid these common anti-patterns:

• BAD: framing opportunities from your companies perspective (frame as the customer instead)
• BAD: vertical opportunities = a parent with one child, who has one child in a single stack – collapse or add siblings
• BAD: opportunities have multiple parent opportunities – if top level opportunities represent distinct moments in time – then no opportunity should have two parents
• BAD: opportunities are not specific
• BAD: opportunities are solutions in disguise – distinguish an opportunity, by asking is there more than one way to solve this problem? if not → it’s an opportunity
• BAD: capturing feelings as opportunities – look for the cause of the feeling

Prioritizing Opportunities, Not Solutions

You are never one feature away from success – and you never will be.

• We focus too much on the solution space – on shipping the next release. We assume that success comes from launching features – this is the build trap.
• Product strategy doesn’t happen in the solution space – it happens in the opportunity space. A solution-first mindset is good at producing outputs not outcomes.
  • Customers don’t care about the majority of our releases. They care about their needs, pain points and desires
• Product strategy emerges from the decisions we make about which outcomes to pursue, which customers to serve and which opportunities to address.
• Sadly many teams jump past this, and start prioritizing features

Product Strategy	Opportunity Space – customer pain points, needs and desires
Product Tactics	Solution Space – experiments, solutions, features

Focus on One Target Opportunity at a Time:

• Deconstruct large opportunities into smaller solvable ones. Adopt an agile mindset. Work iteratively and deliver value over time. Limit work in progress and explore multiple solutions.

Use the Tree to assess & prioritize:

• Focus on assessing opportunities at the top level (or highest level possible) saves time comparing everything. Compare and contrast each set of parent opportunities against each other. Once you’ve picked the highest impact opportunity, you can ignore the others. Move down the levels until you have an opportunity with no children (A leaf node).

Assessing a set of opportunities:

• Opportunity Sizing
  • How many (customers)?
  • How often?
  • Which of these sibling opportunities affects the most customers?
• Market Factors
  • How might addressing the opportunity influence your place in the market? or your position vs competitors?
  • What external trends might (opportunities and threats) might impact which opportunity you choose?
  • Table stakes – or differentiators?
    • A missing table stake – could be bad news
    • A differentiator – could open up new customer segments
• Company factors
  • Consider organizational context – assessing and prioritizing opportunities
  • Support company vision, mission and strategic objectives
  • Deprioritize what doesn’t fit with company values
  • Political capital required for more controversial things
  • Original thought – also control and influence of the team to get something done
• Customer Factors
  • Evaluate how important each opportunity is for our customers
  • How important are these (needs, pain points desires) to customers?
  • How satisfied are they with the current solution?

Embrace the messiness:

• Don’t quantify everything (on a scale 1-5 etc.). That’s treating an ill-structured problem as if it were well structured – and will lead you to believe there’s one right answer. There isn’t.
  • Balance having confidence in what you do – and leaving room for doubt in case you are wrong
  • Make data-informed, subjective comparison for each set of features

Doing is the best form of learning:

• Balance data-informed vs analysis paralysis. You learn more from doing and observing that from thinking your way to perfect decisions

Reversibility:

• Jeff Bezos on reversibility
  • Level 1: Hard to Reverse: Be cautious
  • Level 2: Easy to Reverse: Move fast, try and learn
• Also known as one-way door or two-way door decisions
• Prioritizing an opportunity is reversible. You’re making a commitment of the next few days or weeks. You’re not committing to addressing that opportunity.
• Discovery decisions are reversible decisions. Thinking of them as such frees us from confirmation bias.

Avoid these common anti-patterns:

• Delaying decisions until there is more data
• Over-relying on one set of factors as the cost of others
• Working backwards from your expected conclusion

Discovering Solutions – Solution Space

Idea Generation and Selection

Individually brainstorm ideas for your opportunity – get to 15-20:

• Once you know your opportunity (pain point, need or desire) where do you start?
  • Brainstorming
    • Quantity leads to quality
    • Our first idea is rarely our best
    • Creativity measured by
      • fluency – the number of ideas we generate
      • flexibility – how diverse the ideas are
      • originality – how novel an idea is
    • originality and flexibility are correlated with fluency – volume of ideas helps everything
    • the most original ideas tend to be generated toward the end of the session
  • Get beyond ‘first ideas’ – go deeper into high priority opportunity areas
    • Product teams have backlogs full of ‘first ideas’. AND even though we have a huge number ideas, we don’t have many for each opportunity. We aren’t going deep enough where it matters.
  • Innovate where there’s a strategic opportunity (generate several ideas to ensure you uncover the best ones) don’t reinvent the forgotten password flow
  • Brainstorming Tips (inc.Individuals > groups due to the illusion of group productivity )
    • Four rules of brainstorming (from the father of brainstorming Alex Faickney Osborn)
      1. Focus on quantity
      2. Defer judgement
      3. Welcome unusual ideas
      4. Combine and improve ideas
    • Individuals generating ideas alone outperformed groups
      • more ideas, more diverse, and more original
    • Why individuals do better
      1. people work harder when working individually
      2. group conformity challenges
      3. production blocking – interrupting each others thinking
      4. downward norm setting – group performance tends to be limited to the lowest-performing member
    • Illusion of group productivity makes us thing that brain storming is effective
  • How to become unstuck – When you run out of ideas
    • Cast aside limiting beliefs – you aren’t bad at generating ideas
    • Take frequent breaks – spread it out throughout the day
    • Change scenery
    • Experiment – find what works best for you
    • Take advantage of incubation – taking a break or sleeping on it
    • Look for analogous products for inspiration – competitive research
      • Start with competition then look wider – what industries have solved similar problems? look for similarities in the target opportunity
    • What might extreme users need?
      • First time users vs power users vs disabilities vs young vs old
    • Consider wild ideas – wild ideas can improve more feasible ideas
• Putting it all into practice
  • Review your target opportunity – context – make it distinct
  • Generate ideas alone
  • Share ideas across your team
  • Repeat steps 2 and 3 until you have 15-20 ideas for the target opportunity

Evaluate the ideas as a team – get down to 3:

• Remove ideas that don’t solve the target opportunity (this happens!)
• Dot-vote as a team to get down to three ideas (how well does the idea address the target opportunity?)
  • Everyone gets 3 votes – can spend them how they like
  • Do multiple rounds if needed:
    • Remove the winner
    • Get people to go again
    • Go until you have 3 left

Avoid these common anti-patterns:

• Not including diverse perspectives
• Generating too many variations of the same idea
• Limiting ideation to one session
• Selecting ideas that don’t address the target opportunity

Identifying Hidden Assumptions

Confirmation bias grows with commitment:

• Confirmation Bias: Seeking out evidence that confirms our beliefs
• Confirmation bias escalates with investment. The more we invest, the more committed we become

Be prepared to be wrong – don’t fall in love with ideas:

• explore why your ideas won’t work
• working with a set of ideas!

Quickly testing assumptions (instead of ideas) is the key to innovating quickly:

• There isn’t enough time to A/B test or prototype everything
• INSPIRED – the best product teams complete a dozen or more discovery iterations per week
• This is only possible if you test assumptions – NOT ideas
• Assumption testing is quicker than idea testing –
  • pace helps us with ‘escalation of commitment bias
• Knowing what assumptions you’re making is the biggest barrier to testing them

Types of Assumptions

Desirability	Does anyone want it? Will anyone get value from it?
Viability	Should we build it? Many ideas work for customers but not the business. Ideas need to generate more revenue than they cost to build, service and maintain
Feasibility	Can we build it? Technical, Legal, Security, Culture, Regulations
Usability	Is it usable? Is it discoverable? Will they find it? Will they understand how to use it or what they need to do? Are they able to do what we need them to do? Is it accessible?
Ethical	Is there any potential harm in building this idea> What data are we collecting? How are we storing it? How are we using it? If our customers had full transparency, would they be OK with it?

Story Map to Get Clarity:

• Start by assuming the solution already exists (and that you’re mapping out what end-users will do to get their value from the solution)
• Identify the key actors – who needs to interact with who for it to work?
• Map out the steps each actor has to take for anyone can get value from the solution
• Sequence the steps horizontally over time (if steps are optional list them as such)
• Put actors on rows

• Every time you assume that an end-user will do something, you are making assumptions of different types (desirable, viable, usable, feasible, ethical) – you may have a few for each
• Example – Our subscriber comes to our platform to watch sports
  • Desirability: Our subscriber wants to watch sports.
  • Desirability: Our subscriber wants to watch sports on our platform.
  • Usability: Our subscriber knows they can watch sports on our platform.
  • Usability: Our subscriber thinks of our platform when it’s time to watch sports.
  • Feasibility: Our platform is available when our subscriber wants to watch sports.

Conduct a pre-mortem:

• Imagine its six months into the future, you launched, and it failed. What went wrong?
• Phrase question as if the outcome is certain.

Walk the lines of your opportunity solution tree:

• You likely made assumptions on your tree – walk back up to the top and capture assumptions
  • Generate assumptions by asking the following
    • this solution will address the target opportunity because …
    • addressing the target opportunity will drive the desired outcome because …
• You might need to test, that your product outcome is going to drive your business outcome

Explore potential harm:

• Data, addiction, accessibility, inequality, privacy, trolls, user-user harm, brand, customer expectations, opportunity cost
• Front Page Test

Lastly – Prioritize your assumptions:

• Use a 2×2
  • How much do we know? (about this assumption)
  • How important? (is this assumption)
• Test Leap of faith assumptions
• Make the mapping relative to other assumptions
  • do we have stronger or weaker evidence?
  • more or less important than x assumption?

Avoid these anti-patterns:

• Not generating enough assumptions
• Phrasing assumptions such that you need them to be false (positive framing is easier to test. Customers will remember their passwords)
• Not being specific enough
• Favoring one category at the cost of others

Testing Assumptions, Not Ideas

Gather evidence and test assumptions for your top 3 ideas at the same time – this helps us not fall in love with one idea

Simulate an experience and evaluate behavior

• Our goal is to collect more evidence on the assumption:
  • Collect data about what people do in context (not what they think or say they do)
  • Simulating an experience is a strong test – we hope the participant will behave as in the situation we emulate
  • Choose the moment to simulate carefully
  • Account for shortcomings of the simulation when evaluating results
  • Set specific with your evaluation criteria (or hurdle rates in advance)
    • If our assumption is true, what would we expect the participant to do?
      • Log how many do, and how many don’t
    • helps you align as a team
    • helps you guard against confirmation bias
  • You can’t prove it works – the burden of truth is too high. If your assumption holds true – you’ve reduced risk though.
  • Key outcome is to agree as a team on the smallest assumption test you can design that still gets you results that the team feel comfortable acting on
• Early signals vs Large-scale experiments
  • We don’t want to invest time, energy and effort into an experiment if we don’t have an early signal we’re on the right track
  • Rather than starting with a large scale experiment or A/B test – you’ll be surprised how much you can learn from getting feedback from a few customers
  • Each round of testing helps you inform what’s the next most important assumption to test
  • You stop testing when you feel comfortable
• False positives and negatives
  • False negatives
    • you want a representative sample – the smaller the sample, the more chance you’ll get a false signal by selecting one type of person too much
    • false negatives shouldn’t matter, if your testing your assumption in a few ways
    • use a mix of research methods to better understand the assumption your testing
    • False negatives are often not that bad, there are always more ideas and we can always try again
  • False Positives
    • Either move to test another assumption OR move to a bigger experiment

It’s not science, our goal is to mitigate risk – not to seek truth. Our bar can be lower

Running assumption tests – How can you conduct 15-20 discovery iterations a week?

• You need to get better at quickly executing assumption tests
• Tools to help:

Unmoderated user testing	Post a stimulus (prototype) – Define tasks to complete and questions to answer. Participants come in their own time. You get a video of their work. – careful with screening – you want your own users – you can even upload your own audience
One question surveys	When was the last time you watched sport? Please select all the sports you watched in the last month?
Analytics	How many people have searched for sports. Or related to sports

Avoid these common anti-patterns:

• overly complex simulations
• using %s instead of numbers when defining evaluation criterial
• not defining enough evaluation criteria
• testing with the wrong audience
• designing for less than the best-case scenario
  • design your assumption tests so they are likely to pass
  • if they still fail – then that’s a sign /red

Measuring the impact

Don’t get distracted by your assumption tests – stay true to solving your outcome.

Discovery and delivery are iterative – you’re never done with discovery.

Iteratively invest in experiments. Start small and grow your investment over time.

• as experiments grow, you’re going to need to test them with a real audience

Instrument your evaluation criteria to evaluate assumption tests on live prototypes

• Instrument what you need to evaluate your assumption tests
  • Define evaluation criteria for each assumption – measure it
    • Count #people who do an action (not #actions) if one person doing many actions isn’t as valuable as many people doing actions
    • On search results – measure the position in results – measure the ratio of views to conversions

Measure the impact of releases against your desired outcome (or North Star)

• Measure what you need to evaluate progress toward your desired outcome
• If your product doesn’t go across the whole user journey – you might not know the end outcome you’re trying to influence
  • incentivize people to tell you what happened – survey them
  • just because something doesn’t happen on your platform doesn’t mean you shouldn’t measure it
  • don’t be afraid to measure hard things

Don’t try to measure everything at the start – learn as you go

Distinguished between Business Outcomes, Product Outcomes, Traction Metrics:

• You don’t want to wait to have the final answer before iterating – find leading measures
• Run a small test on one day – see if you move your metric that day – for those that engage

Avoid these common anti-patterns:

• Getting stuck trying to measure everything
• Hyper-focusing on your assumption tests and forgetting to walk the lines of your opportunity solution tree
• Forgetting to test the connection between your product outcome and your business outcome

Managing the cycles

Continuous discovery isn’t sequential. It’s often circular and more messy.

The fruit of discovery work is often the time we save when we decide not to build something

Sometimes customers want something – but don’t expect your brand to solve that problem for them

The Importance of Now, Next, Later

• Sometimes you’ve got a need that can be solved, but now Is not a good time, so you can push it back on the roadmap
• Balance short with long term – low hanging fruit, with harder long term problems
• You can use your discovery work and quick win tests – to make the case to go after harder solutions

Iterate through small opportunities for big impact.

Avoid these common anti-patterns:

• Over-committing to an opportunity
• Avoiding hard opportunities – don’t confuse quick testing and iterative delivery with avoiding hard solutions – the longer we spend building – the more we should de-risk
• Drawing conclusions from shallow learnings
• Giving up before small changes have time to add up

Show Your Work

The more leaders can understand where teams are, the more they will step back and let teams execute

Get your stakeholders on board – else everything else is wasted

Don’t anchor your conversation on the solution space

• stakeholders have thoughts on solutions, not grounded in discovery
• the hippo is going to win

Slow down and show your work – go back to the tree:

• Don’t start with conclusions
• Show your opportunity solution tree
  • start at the top
  • remind stakeholders what the desired outcome it
• Highlight the top-level opportunities (you can invite them to add more opportunities)
• Share how you assessed the priority of the space
• Share the context about your target opportunity – customer pain point
  • use the snapshots to back that up
  • share the solutions you generated
  • share the set of 3 solutions you plan to move forward with (ask them if they’d have chosen different)
  • share the assumptions and how you prioritized them
  • share assumption tests – and data – repeat

Continuously manage your stakeholders

Showing your visual decision making progress takes your stakeholders along for the journey

Don’t present a conclusion, show your process, which leads to more buy-in and long-term success

Avoid these common anti patterns:

• Telling instead of showing
• Overwhelming stakeholders with all the messy details
• Arguing with stakeholders about why their ideas won’t work
• Trying to win the ideological battle instead of focusing on the decision at hand

Developing Your Continuous Discovery Habits

Start Small and Iterate

I rarely had the support of senior leadership to do product discovery well.

Find a way to get closer to the customer – find a way to advocate for human-centered design.

I had a strong sense of agency – you choose how you work – you might not be able to change the way companies work.

Instead of asking for permission – start small and iterate from there

• Build your trio – don’t work alone – build the relationships yourself – start small with them
• Start talking to customers – it’s the keystone habit – weekly changes everything – continuous interviewing

Work Backward – As you work on requirements for the solutions you were asked to build, story map your ideas – use them to identify hidden assumptions – be aware of your assumptions

• Identify assumptions with stakeholders – the idea will improve right then and there

Document what impact people think a feature will have

• Do post-release impact reviews (expected impact vs actual impact)
• Are we trying to solve a customer problem with this feature?

The best time to advocate for discovery is when a feature falls short of expectations

• Never say ‘I told you so’ – be a collaborative problem solver

Use retrospectives to reflect and improve

• What did we lean in this period that surprised us
• How could we have learned sooner?

Avoid these common anti-patterns:

• Focusing on why a strategy won’t work – instead of focusing on what is within your control
• Being the annoying champion for the right way of working
• Waiting for permission instead of starting with what is within your control