[Music] [Applause] Wow we're getting really good at discovery if I've learned anything today I feel like I have a lot of power because I'm standing between you and your breathe and your beer but I promise I'm gonna do the best I can to deliver a lot of value in the next 30 minutes or so I know it's late in the day but I want you to play along with me a way to think back to when you were in elementary school learning multiplication tables don't worry I'm not gonna make you do any math I was
in third grade we used to do these full-page multiplication tests they were one minute you had to go as fast as you can and you'd get all the right answers day after day I competed with the same boy he'd win some I'd win some but we always finished one - I loved the thrill of putting my hand up hoping I was the first one here's the thing these tests were not just about going fast he had to get all the right answers and as a third grader this really motivated me I went home and I
practiced I wanted to learn all the multiplication tables because I found comfort and getting it right okay now I want you to fast forward in time to remember learning algebra algebra also again I'm not gonna make you do math I promise algebra also was about getting the right answer in fact I loved flipping to the back of the book as fast as I could to see if I got the right answer and in algebra I was with what my middle school teacher called a half lesson learner I was rushing to do my homework before he
was done teaching the lesson but you might remember in algebra there was more than just getting the right answer we had to meticulously work work step by step we had to show our work I wasn't very good at showing my work my brain moved faster than my pencil and I really did not want to slow down wanted to get the right answer and move to the next problem all right we're gonna fast forward in time again this time I'm in a computer science class in college and the days of getting the right answers are long
gone does anybody recognize the Traveling Salesman problem yeah okay the problems we worked on in college were too complex they don't always have a right answer instead they had better and worse ones some of them were unsolvable they were designed to test our thinking not to lead us to a right answer and I remember the first time I tackled one of these problems I was devastated and it's not I didn't realize that there was no right answer I just knew that I couldn't find one so hoping for partial credit I did the best I could
you've probably all been there and it turns out I did just fine but I didn't know what to make of this I really yearned for a right answer I found comfort in right answers so if I look back on this journey starting in third grade rushing to find the right answer later in eighth grade learning that I had to show my work it just wasn't just about finding the right answer and then finally in college being introduced to problems that were so complex there wasn't always a right answer I realize all of us every single
one of you that's working on a software product we're going through a similar journey in our own organizations so when I illustrate this today using Netflix I have to start with the few disclaimers because I've never worked at Netflix as a product coach I've never coached a team at Netflix I have no idea what it's like to work there other than I read there hunter and succeed slide deck about their culture I'm using Netflix as my example only because I suspect most of you are familiar with their product in fact if I asked you I
bet every single one of you could come up with a list of prioritized improvements to the Netflix product right you could fill a backlog full of user stories you could create a twelvemonth roadmap right after all this is what we do as product people we create plans we plan for the future we find comfort in being prepared now for those of us that are early in our product careers or young in our product practice we find certainty and comfort in these right answers we obsess about them we try to get there as fast as possible
right we've jumped to our first solution we fill that backlog we look at the empty roadmap and we want to fill it as quickly as possible and in meetings with stakeholders we advocate for a point of view yeah I know we've all been there and when we get suggestions for change we resist them when we get pushback we yearn for autonomy and when our boss asks us to build something other than what we suggested we complain to whomever will listen we curse the dreaded hippo after all we already found the right answer and it's perfect
but nobody will listen so we get frustrated we start to daydream about our next promotion hoping that when we're the hippo our perfect answer will finally see the light of day we're like the third graders rushing to the right answer what we don't realize is that we're creating the stakeholder problem ourselves we only present our final answers our wrote our backlogs full of user stories and our roadmaps full of features and release dates we're asking our stakeholders to give their opinions about our answers but we haven't taken the time to show our work to defend
our point of view so of course they're not aligned with our to right answers we all know that when a hippo disagrees with us the hippo always wins so as our product practice starts to amateur we realize it's not just about the right answer we realize like the algebra student we have to defend our answers so we start to use data to bolster our right answer we talk about total addressable markets returns on investment we learn to write business cases we learn to speak the language of the hippo we draw a straight line from our
business need to our perfect solution the more sophisticated among us draw a straight line from our business need to a customer need to a perfect solution and it's beautiful just like an algebra problem we can't wait to flip to the back of the book to see if we got the answer right now in this scenario we're giving our stakeholders more to respond to they can give us feedback on our numbers they can question our data maybe they can bolster our argument or find flaws in our logic for example one of our stakeholders might ask why
do we think we're gonna see a 5% of new subscribers create reviews and if you've listened to anything I've said you to you over the years you maybe even have an evidence-based answer from your discovery work our stakeholders also have knowledge and expertise that we may not have maybe your boss learned this morning that the lifetime value of a retained user went up from one hundred and eighty dollars to two hundred and twenty dollars great that means you're it your idea just got better on the other hand maybe a general manager just highlighted that in
the history of Netflix no single feature has had more than a 2% lift and retention rates suggesting that your 25% estimate might be a little too high right I've been there here's what's happening when we start to show our work we're giving our stakeholders more to engage with we're letting them question our numbers we're letting them question our thinking not only are we gonna get more engagement from our stakeholders but our ideas gonna get better but we still have a problem if our stakeholders disagree with our final answer we haven't given them any alternatives we're
still gonna find us arguing our right answer versus their right answer now every stakeholder that's ever existed can create an equally good if not better business case for their own idea their stakeholders that's what they do and there's no way for us to prove that our idea is the one best idea because there is no one best idea so again if our stakeholders like their own ideas better than our ideas we fall under this mind versus yours trap and once again the hippo always wins now again as our product practice continues to mature we realize
it's not just about defending our right answers maybe there are no right answers maybe there's only better or worse ones maybe our jobs as product people is to generate and evaluate options so instead of going to our stakeholders and saying here's the one next magical thing we should be doing we're sharing with them all the options that were generating and how we're evaluating them instead of over committing to our favorite idea we invite our stakeholders to id8 with us we learn to leverage of their expertise we co-create with them wouldn't the world be a lot
better if we could co-create with our stakeholders instead of the hippo always wins we might actually win together now I know what you're thinking because you like all audience members at any conference talks it's the beginning of time I got this I already do this of course you do right or some of you are thinking this would never work with my stakeholders they're too unreasonable they don't know enough about technology there's a million reasons to think that the problem is out there my stakeholders are too stubborn they won't change they don't come to my meetings
I want to walk through a really clear example of what it looks like to co-create with your stakeholders because in my experience working with teams this is a giant gap we're getting really good at discovery but we still are struggling to bring people along with what we're learning and I know this from personal experience as a product manager I struggled so much to show my work all right so let's talk about this what does it look like it starts with visualizing our thinking any Martin's introduction he talked about the opportunity solution tree of course we're
going to talk about that but I am going to tell you that if this structure doesn't work for you it's not about doing it my way it's about finding your way that allows you to visualize your thinking in a way that invites others to participate with you now if you're not familiar with the opportunity solution tree we're gonna review how it works it starts with a focus on outputs not I'm sorry it starts with a focus on outcomes not outputs it was awesome here here's why when we focus on outputs like new features and functionality
we're focused on right answers in a world without right answers so instead we need to work with our stakeholders and say what are the business outcomes that if we hit them would create value for our customers and for our businesses this is a much easier conversation to have and agree on that arguing about what we should build next so that outcomes are the top of the tree that blue oval here's the thing if we're truly human-centered we can't just focus on our business needs we have to do the work to understand our customers needs I've
talked about this a lot I talked a lot about discovering opportunities that's all the green ovals what's an opportunity an opportunity is a customer need a pain point a desire a want it's an opportunity to intervene in our customers lives in a positive way it's what we do when we create products now I'm actually hearing this language a lot more in the product world we talk about how many times today we heard what problem are you solving but I often hear product teams jump to the first problem they identify they talk to a customer and
they hear about a need they immediately want to solve it I love the sentiment they're right we really want to help our customers but if our job is to generate and evaluate options we can't work with one opportunity at a time we need to survey the landscape we need to map out the opportunity space we need to be able to compare and contrast where can we have the most impact there's one quote that I was introduced to several years ago that has had a giant impact on the way that I think about product work it
comes from John Dewey John Dewey was an American educational philosopher from the turn of the 20th century he wrote 150 page books that I spent 15 hours reading because it's really dense and it's magical I'm gonna share with you that quote today John Dewey said to maintain the state of death and to carry on systematic and protracted inquiry these are the essentials of thinking now I'm not gonna lie I had to Google protracted it means for longer than you're comfortable ok that's awesome I get chills thinking about this so we can't what Dewey is telling
us is we can't jump to our first solution we have to maintain a stay to death so that we can carry on a systematic in search for longer than we feel comfortable we don't do that very often but this is what Dewey is saying the essentials of good thinking so this is what I like to see when people explore the opportunity space so again those are all the green ovals can we conduct a systematic and protracted search of our customers needs so we're truly human centered now the opportunity solution tree also allows us to communicate
these are the solutions that we're working on those are the yellow ovals and these are the experiments that we're running to evaluate those solutions those are the orange ovals so what we're doing here is we're showing we're generating and evaluating options when we share a traditional roadmap with our stakeholders one that includes a long list of features and release dates we're showing one road the one road we're gonna take to create value when we share an opportunity solution tree with our stakeholders we're showing all the roads we might take and the conversation changes it changes
from what outputs we care about to what's the best path to reach our desired outcome now I want to illustrate how this works using our Netflix example so I want you to imagine that you've just worked with your stakeholders and you've agreed that how come we need to we need to work on is improving the minutes watched by your average subscriber it's for those of you that live here in the Bay Area we know that we're addicted to engagement and so now we need to map out the opportunity space because we're good human centered product
managers so how do we do that where do opportunities come from Steve earlier today gave us a pretty good hint they come from collecting customer stories specific customer stories so Steve talked a lot about going from the specific to the general so again if I worked at Netflix I might I don't want to ask you why do you value flex that's general how do you watch Netflix that's too general how about tell me about the last time you watched Netflix now as I started to collect these specific stories opportunities are gonna start to emerge suppose
I ask someone to tell me about the last time that I watched Netflix and I hear I really wanted to watch Netflix last night after dinner but I couldn't find anything to watch maybe I saw a commercial for Good Omens and I wondered if it was any good so I logged onto Netflix and I just couldn't tell right by the way of course it's good I haven't washed it but it's Neil Gaiman and Terry Pratchett of course it's good okay um or maybe I interviewed somebody else and they said all of my friends are talking
about Avengers endgame and so I wanted to go watch it so log on to Netflix but I couldn't find it right few years ago I got addicted to Breaking Bad and I love to binge watch episode after episode or maybe I just finished stranger things because it's summer and the third season just came out and I hate it when I finish watching a show because it's so hard to figure out what to watch next I wish I knew what my friends are watching maybe you heard this story that Sunday's pajama day in my house and
I like to catch up on Game of Thrones and I'm only on season two so I like to watch episode after episode and it's so annoying that I have to watch the intro over and over again see how by just collecting stories I'm starting to see customer needs customer pain points and when you interview your stories are gonna be much longer than this we only have a little bit of time together um so now you've done a whole bunch of discovery work you've learned a lot about your customers needs it's time to bring your stakeholders
along what do you do I see for common things and they're all on mistakes let's get rid of those right now the first we record our interviews and we email out the recordings and we say listen to what I learned and nobody listens to the interview of course not it's your job to do the research your stakeholders have their own job your research is way more important to you is to them they're not gonna listen to your recordings second you write up these beautiful interview notes and you send them out again nobody reads them I
can't keep up with my email I'm not gonna read your pages and pages of interview notes so the third thing we do is we spend hours on the perfect research deck we communicate our themes and our guiding principles and how we're gonna make product decisions till the end of time nobody comes to your meeting nobody really pays attention they're all on email right like this is just not a good way to communicate research the fourth one is the worst and whenever I see someone do it it just causes me so much pain and that's that
the only time they share their research is when it supports whatever argument they're making in the moment yeah and nobody believes you right because everybody has an anecdotal story to the contrary here's what's wrong with all of these methods so earlier today we heard about telling stories and getting exposed to your customer firsthand we need to do all of those things but what's really missing is we're struggling to communicate the big picture to our stakeholders when we talk about one customer need or two or three customer needs your stakeholders always have a counter example so
we need to work to show the landscape of customer needs and this is what I like about the opportunity solution tree it allows us to quickly communicate across all of our interviews here's what we're hearing now again this isn't about the right answer or even the right visual if you have another way of doing this by all means use your way of doing this the key here is when we could share the landscape with our stakeholders we can say across all of our interviews this is what we're learning what else would you add you also
have conversations with customers is there something missing have we missed something and here's the thing remember our stakeholders have knowledge and expertise that we don't have so if you work on the consumer experience at Netflix odds are someone in your organization works in the content the content partner experience for those of you that work on marketplaces we know that the two sides of our marketplace never come into conflict right so maybe that stakeholder can tell you a little bit about their world and help you map out the opportunity space so instead of battling about is
my solution right or is your solution right before we've even agreed on what problem we're solving we can work together to map out what we know about our customers and how we might address those needs now I know that not all of you have weeks and weeks and weeks to map out the opportunity space and nor would I recommend that you spend that much time doing it this is why I teach continuous discovery we only create value for our customers when we ship code so we have to build something so eventually we're gonna have to
pick some of these opportunities to address one of the things I like about the opportunity solution tree is it does help us prioritize we can use the tree structure to make better decisions so instead of going to our stakeholders and saying here's all the things that we learned what should we do we can say we found these big categories let's look at just the top three groupings where do we think we should play now remember we're not presenting the final answer we're giving them options we're evaluating options together and if we pick this middle opportunity
we can look at its sub 2 news we can ignore the rest of the tree so earlier today we saw Marty Kagan's long list of how to assess opportunities if you were to assess every opportunity that you found you would be answering questions till the end of time right so this is helping us have a good conversation about options with our stakeholders without evaluating every single thing that we encounter it's helping us to align with our stakeholder roots around what problem we might solve great so now it's time to talk about solutions which is everybody's
favorite place to have big strong opinions so again if we go back to that Dewey quote we're gonna look at systematic and protracted in we need to generate a lot of ideas now every product team I've ever worked with tells me they're inundated with ideas their backlogs represent the next five years of work but if you were to take that backlog and map it out against your opportunity space you would have a lot of first ideas every idea meets a different need the problem with this is we're not taking advantage of creativity we know that
creativity loves constraints we know that our first idea is not our best idea we know we often have to give that seventh eighth twenty idea to really find something that's desirable viable feasible I'm gonna argue usable and ethical so I really encourage teams to conduct a systematic and protracted inquiry generate more ideas than makes you comfortable not only is this what makes you better a better decision maker it also allows you to bring options to your stakeholders it's not about the final answer it's about generating and evaluating options together and when you have to decide
which of these to pursue because you obviously can't pursue twenty ideas at once when you're whittling it down to just a few to experiment with and fight your stakeholders to be part of that conversation notice how we're not arguing about my right answer versus the right answer but we're taught word together discussing how can we get best get to our desired outcome together I also like to encourage teams as they run experiments we learned a lot about experiments from David today to go ahead and put that on the tree share your experiment plans with your
stakeholders capture your results on the tree ask them to interpret results with you so again you're not coming to them with the final answer but you're saying look here's what I'm learning what do you think what decisions would you make based off of this data now earlier I said a lot of your stakeholders have their own jobs they don't have time for this so you're going to invite them to ideation sessions and they're not going to show up you're gonna try to send them your opportunity solution tree and they're not going to look at it
that's not how we manage stakeholders we need to sit down with them face-to-face or over zoom like the whole world and walks them through what we're learning the key is when we have an opportunity solution tree supporting our conversation or whatever your favorite way of externalizing your thinking it allows you to sit down with them and say remember we agreed that would create value for the business is if we could increase the minutes watched do you remember us making that decision okay good then we went out and talked to a bunch of our customers we
mapped out the opportunity space here's what we learned do you want to add anything would you change how we grouped things notice how you're exposing your thinking so that your stakeholders can question it that you can leverage their expertise you can co-create together you can share these are the solutions we're considering these are the experiments that we've ran again it's not about the right answer it's about generating and evaluating options I would love it if we could get here so there's a few things I want to summarize as we wrap up it's almost beer time
first stop fixating on the right answer now as I told this story I told it as if it was something that happened early in your product career that's not true I've been doing this for 20 years and I still catch myself every single day fixating on the right answer it happened today at lunch literally every day I catch myself needing to be right defending something too strongly you probably hear it in my presentation I speak with a lot of conviction right this is a hard thing to do because it requires that we separate our ego
here's what I want to share with you and this is going to be tough to hear because it's not what we want to believe you are not one feature away from success it gets worse you never will be never it's not about there's a write feature it's not about we're gonna do one more thing and then magic happens it's not about we'll hit this release date and will suddenly have product market fit product market fit is a moving target we live and work in a complex world so we have to let go of there being
a right answer we need to learn to show our work and I will tell you this plague to me my entire product career if you talk to anybody that I worked with anybody I was a bull in a china shop I wanted it my way I still want it my way from being honest I could see the potential of what we were doing and I wanted to get there as fast as possible I was still that eighth grader flip into the back of the book and moving on to the next problem and I was frustrated
that people couldn't come along with me and on my worst coaching days I still do it I still argue with people about why my way is better and that's dumb there's so many ways to do this which is why even today using my opportunity solution tree which I have totally fallen in love with my own idea I I don't need you to use it I don't I'd love for you to try it but I don't need you to use it I want you to think about how can you externalise your thinking how can you show
your work so we learn not just to work with our cross-functional team we're getting good at that but with all of our business stakeholders so we're leveraging all of the expertise in our building here's why the problems we work on are so complex we need everybody to help us discern the better options from the worst ones everybody your salespeople yes all the ones that request their favorite feature every day your CEO shows up to work talking about whatever the competitor did over the weekend right we need all of these people to work together to find
the best path to our desired outcome so here's what I'm gonna ask of you when you go back to work tomorrow or maybe Thursday try some of this out try to show your thinking invite your stakeholders to co-create with us here's what I think you're gonna find those hippos that drive you nuts they're suddenly going to turn into reasonable people all right I usually talk about continuous discovery today I want to talk about managing stakeholders because everybody before me on this stage did their job and it was magical to experience please keep this conversation you
going you can find me a product org and on twitter at tea toys thank you very much [Applause] [Music] Oh
