Hello insiders this week I have a very Special guest I got todd from the search and discovery team with me today welcome Todd thanks for having me. Todd People have been asking for this interview ever since we started the channel so I'm super excited to kind of dig in a little bit to help people understand a little bit around how we Recommend videos and and how we think about that? Maybe we could just start by going over like what is the goal of the search of discovery team like why do you guys exist?
yeah, so there's a You know as you know there's over 400 hours of video being uploaded to youtube every minute So that's a I think like 72 days worth of content per Day and Given that each person probably only has an hour to to watch Youtube every day It's our job to help them find what they want to watch and not just what they're going to watch what they're going to want to enjoy and keep them coming back to YouTube and Every day to watch more videos. So really Understand each viewer what they like and give them the videos that they're going to find engaging So how do you how do you solve that problem? That sounds like a very?
Kind of ambitious goal like how do you break that down into a Mental model? Yeah so the way We think about it is our job is to follow the audience because by listening to the audience We will do a better job of giving them what they want so the way, we think about the systems that we build is that they're really a Gigantic Feedback loop for the audience in any given day we collect over eighty billion different pieces of feedback from the audience We look at things like well. What did they watch?
What did what did we recommend that they decided not to watch? How long did they watch for but increasingly? We're moving Beyond.
Just watch time as one of our key goals we look at satisfaction signals like did they like it that they dislike it when We recommended it on the home page as they say that we're not interested. We'd give viewers the ability to tell us at so we can stop recommending certain types of videos that they didn't like and Recently even started using surveys to try to even better understand the audience and what they like and don't like so we send out Tens of thousands of surveys every day to people asking them whether the recommendations were giving them are good or not as well as Following up with people after they've watched a video To find out how enjoyable that video was was one of the best videos they've ever seen or was it kind of an average video Because we believe that not every minute spent on YouTube. It's probably equally valued And we want to be able to to really recognize when viewers have a maybe even a life-changing moment with a video So that we can recommend more videos like that cool now you've been on the team for three years.
Yeah I'm guessing you've seen a lot of interesting experiments and the work that the team does has evolved over time could you share some? Examples of changes that have been made and maybe why you guys thought about making those changes. Yes So you know changes are happening all the time?
Because we're always trying to get better about giving people the right Content that they're going to find satisfying at any given time We probably have several dozen different experiments running with different changes that we think might help improve engagement and satisfaction But we're also very careful about how many of those experiments make it into the final product Each experiment is tested with millions of viewers so we really want to See how it does before we roll it out to everyone and only a minority of those changes Do we do you put into the final product and we go through a review process for every change? And we look at dozens of metrics. I know that a lot of creators could have kind of asked me well What's the one thing that I should focus on to being successful on YouTube is it watch time as it likes is in comments And what we found over time.
I've even asked you that one And as you probably remember I've said that You know while I wish I could give you a simple answer when we learned to truly understand the audience of what they want is Not just about just looking at one thing like watch time or likes because you know for example with watch time You know we've come across some changes in the past that we've considered like We have this one change recently that significantly increase watch time across YouTube it actually If we would have rolled out this change people would have watched Millions more hours every day on YouTube So you might think well obviously that's one of the good ones Let's let's roll that one out, but when we looked at the dozens of metrics we looked at For each change we found that There were some interesting things happening that weren't the same across all viewers What was happening is one we were recommending longer videos on average Mm-hmm So you might see more videos that are like an hour long or more? But our hit rate in terms of like how often a viewer would find a video they want to watch was lower So you'd be less likely to find a video But when you did find one, you liked you might watch for longer, so overall watch time is up But what was ultimately happening when we dug in was? We were increasing engagement with a small heavy segment of the audience and actually Reducing engagement from a casual user segment because we look not just at like the total volume of engagement we also Break it out and see how it's affecting different groups of viewers and We care about growing the appeal of Youtube to more and more viewers over time We felt that this was a representative of the type of change.
We wanted to make because it was actually making the experience worse for You know a significant segment of viewers that we cared about and so rather than make them pay the price for for another segment we said let's go back and see if we can find a way to get get the Improvements here without hurting it for a different segment, so we look at these things very closely and you know spend hours a week digging into the details But ultimately it does change a lot Year, you know 200 to 300 changes do end up happening. So it's a lot to keep up with but You know each change that we do roll out. We can be confident that we're making the experience better cool it's it's easy to think about people often talk about the algorithm yeah!
um, I think in reality it's a little more new answer on that um, maybe what we could do is talk about Specific spots within Youtube where recommendations are made like there's homepage. There's watch next there is a trending Search as well Maybe can we step through like starting with homepage How do you think about that problem solving that problem of recommendation, and then we can move on to the others sounds great? So yeah a lot of people talk about it as if it's just like one algorithm And we do have different systems and each surface or feature that the user sees kind of has its own set of systems behind it that are trying to optimize for Whatever that features trying to achieve so for example on the home page We are really trying to be the one-stop destination So it's the first thing you see when you open it up on your mobile app or type youtube.
com into your browser We really want to be as efficient as possible In helping you find something whether it's a trending video or whether it's something by your favorite creator We're going to get you there as quickly as possible So you don't have to navigate around the app to find what you're looking for so We're going to pull in videos from all sorts of different sources and try to find the ones that are going to resonate with you So we'll pull in recent videos from The Channels You subscribe to We'll pull in Videos that have been watched by people who tend to watch the same sorts of videos you do and we'll also pull in a Few videos from channels you may have never watched before It's actually a great place for channels to get discovered by people who aren't already subscribed to the channel we found that You know the home page actually used to be only your subscriptions and often times we'll get the question from creators or people Internally asking well, why don't we just you know since people subscribe to those channels? Why don't we just you know give them what they subscribe to We used to do that and as we've Introduced more and not constrained the homepage to videos you've you know manually opted in to added more recommendations the engagement has grown dramatically, and so we we try to include both familiar content as well as Discovery content and Every once in a while We'll kind of retest to make sure that We've made the right decision there And we'll run another one of those experiments and millions of users where we actually give them just the subscriptions when they start up the app and We've done that several times since we made that change and every time we see dramatic drops in Engagement on home when let me limit it just a subscription So that's if you had that question about why isn't it only subscriptions? It's because when we show that people actually visit YouTube less often well Which is not really what we're going for so home is really that personalized One-stop shop that's supposed to bring the most relevant of YouTube to you without you having to Type a query or navigate around that great, and I guess if for those who?
viewers who are Super interested in just the people that are subscribed to there is always the subscriptions tab. That's right. It's just one tap away there's a subscription tab that filters down to the top that you've subscribed to and it's a We still offer that and other people who want to just jump into that as well cool What about what's next so watch next?
That's what we call it in our team externally sometimes people call. It suggested videos The recommendations you see on the sidebar while watching a video on desktop This is one of our most powerful recommendation areas drive a huge amount of engagement on YouTube Kind of similar to how we call it watch next We're trying to find the next thing you want to watch so unlike the home page. Which is really centered around You only um we have some additional context of what you're currently watching now and so That's going to sometimes influence what you want to watch next so we're going to we're going to pull in some different videos We're gonna pull in more videos from the channel that you're watching I'm going to pull in videos that People who watch the video you're watching what else did they watch definitely?
That's a key part of it We're going to pull in similar videos that have like similar titles in keywords and descriptions that might be similar and finally, we're not just going to limit it to Videos that are related to the video you're just watching this is another change that's happened over the years It used to be called related video Because we thought that that was kind of like the thing that would work the best and what we found through our experiments is that? In addition to showing related videos if we also show personalized recommendations that might be interesting to you But not directly related to what you're watching people will watch more And so sometimes you want might want to go down the rabbit hole on a video and watch more related, but other times It's like hey I was just looking up how to tie a bow tie Mm-hmm And I got that information and now what I want to watch next is actually I want to catch up on my favorite Vlogger over some other channels so we don't we don't limit suggested videos so it's a mix of stuff that might be related and And and what's relevant to you and ultimately what Determines what shows up? There is what performs the best so the videos that you see there are the ones that we think are most likely to drive viewers to watch and engage in the 80 billion Data points that I talked about earlier of Those data points come from there, so we just you know try to keep doing.
What's working and test various videos and different placements and the ones that work the best tend to stick around and the ones that don't tend to All right, so now we've talked about home Wats next let's talk about search right so search Similar to watch next we have some more context there because you've given us a query so In general we're going to you know start with what you'd imagine. Which is we're going to find videos that match that query Either in the title or the descriptions or maybe we've seen that People who search for this query tend to tend to watch these sorts of videos and so we'll start with a set of Relevant videos, and then similar to what I was referring to earlier. There's this feedback loop so you might say hey I have a thumbnail of an iPhone and I have the text iphone in my thumbnail and my title is all about iphone review and in my description.
I say iphone iphone iphone Why am I not number one in the search for iphone well? It's because we don't only look at What we call the Metadata that you provide about the video, but we also look at the performance and so We're going to see when people search for iphone is this type of video that they tend to watch or not? And so the ones that you're going to see at the top aren't just the ones that match the query But also match what the feedback has been from viewers so it's a combination of Relevance and then that feedback loop of what's was actually engaging audience cool and then Trendy Trending this one we get so many questions ago So trending is You know one of the things that we built trending around was to be kind of like the water cooler of YouTube Mm-hmm and so It's not just a popular list It's not the most popular Videos that are getting the most views or that sort of thing a lot of people get confused and think that's what it is We really wanted to be represent the videos that people are having a conversation about And when you think about that?
What are the types of videos that people are talking about as a water cooler? They tend to be broadly appealing videos like so there may be you know a video That's getting a lot of views, but it's within sort of like a really Narrow audience like people may not be talking about that broadly So we are we do tend to focus? Trending on videos that are broadly appealing for example to different types of people with different preferences And we're also looking at videos that aren't just don't just have a lot of use in the past, but actually Have some velocity and are increasing and on the rise So that's why you might see videos and trending that you know may not have millions of views yet But we're trying to actually identify those things as they're taking off and we also look at things like Are people talking about this video on the web?
Did we see that? Hey? This was actually embedded in you know a lot of different news articles, and so there are Conversations happening outside of Youtube about these videos those are some of the things we looked at it's to try to Tap into that water cooler Element cool, and you know one thing I've heard people ask about trending is whether the suggestions are paid Yeah, I've heard that question as well So we we feel really strongly that any Advertisements that we have on YouTube are clearly indicated as ads.
You'll see like a little yellow badge that says add Same goes for trending there aren't any of those in trending and That's because we don't have those ads in trending. We don't have any programs. We're Partners or advertisers and pass to place the videos in trending.
It's not a It's not a pay for placement thing so all the videos you see in trending there are because of the ranking that we've come up with And there's no way for people to pay to influence that so it's it's definitely not Ads or or paid placements cool hey insiders? Hope you enjoyed the first half of that interview with Todd Part 2 is going to be uploaded sometime next week if you haven't already make sure you subscribe and hit the bell notification, so you will be the first to know when Part 2 is available and in the meantime.
