I'm Renee I am the YouTube Creator liaison and I'm Todd I lead the product team for growth and Discovery at YouTube so how does Discovery how does the recommendation system actually works so the first thing that creators should understand is that the recommendation systems are really centered around each individual viewer so often times creators will say hey uh the recommendation system is pushing out my video to people or why isn't it pushing out my video yes they they may ask that and the way the work it works is it isn't so much about pushing it
out as much as it's pulling um for each viewer so when you open the homepage uh YouTube is going to say hey Rene is here we need to give Renee the best content that is going to make Renee happy today it's very much centered around each individual person and ranking the videos for them as opposed to looking at each individual video and trying to figure out who might like it so there could be videos like a video could have these absolute metrics like it's click-through raid it's um view duration all of that but you might
like it and I might not and you would consider that when you recommend it yeah so a lot of creators they look at their analytics and they're very focused on their aggregate performance because that's the easiest thing that we can kind of present but the way the system is working is actually it's not just using that average view duration or the average click-through rate it's using the performance data as input But ultimately trying to understand and predict for each viewer is kind of similar to how recommendations work out side of recommendation systems um when you
want to figure out a movie to watch one thing that that I do is I I talk to my friends and I say hey what have you seen lately what do you want to see and the recommendation system sometimes we describe it as automating word of mouth I love that and by that we mean the system is able to understand what other viewers like you have watched and then based on understanding what they enjoyed and what they didn't it can help uh when it's time to give recommendations to you to pull in um sort of
like those recommendations uh from other people the nice thing about it is that can come in multiple waves so maybe like right now there's a video that that reaches a certain audience but then like in six months there's something else that's related that pops up that makes this video relevant again fortunately you know our system is enabled to be able to recommend that again again if it's if it's relevant and maybe to a different audience than than enjoyed it the first time you've given me examples of that in the past where somebody will search for
V like a certain kind of video and find one and you'll be able to identify more audience or something will pop up in the news and suddenly people are more interested in it or a big Creator will make a video on it and then people will see that and said like all these new ways where videos can find new audiences yeah sometimes it's just a matter of like Nostalgia as well where there's just been enough time since people have uh thought about something and then uh somebody kind of rediscovers some some Trend or concept and
they want to go back and kind of relive it through watching videos about that thing and you see a Resurgence of uh of something that hasn't been popular for a while how much do you think about factors like time of day or device type like if I'm on my phone in the morning versus whether I'm on the television at night the recommendation system uses time of day and device um as some of the signals that we learn from to understand if there's different content that is appealing uh in those different contexts and so as a
viewer you can see different results at the same time on your television versus uh on your mobile phone um as well as different results in the morning versus the day we try to identify if you tend to have a preference for like watching news in the morning and comedy at night um we'll try to lean into that we'll try to learn from other viewers like you if they have that pattern so that that we could see if that works well for you and all of it comes together to really just give the right content to
the right viewer at the right time we often hear from creators like what's the one number like is it click-through rate is it watch time how do creators like optimize for all of these factors one thing to understand is that there's no single answer to that question as much as creators would love to have one the the reality is is that we've enabled the system to learn that different factors can be in have different importance in different contexts watch time may be more important in television versus mobile or it may be more important in certain
types of content like uh podcasts as opposed to music like does it really matter whether a music video is 4 minutes and 30 seconds versus four minutes in terms of a viewer satisfaction maybe watch time isn't really as important as something like a aike yeah um in that context sometimes when people hear about algorithms they conflate that with just like engagement at all costs but one of the things I love about YouTube is that it doesn't seem laser focused on just getting the maximum amount of Engagement possible yeah so you know for quite a few
years now we've uh learned that not all time spent with video is equally valued by our viewers uh we get feedback that viewers want some videos to be a lot more efficient in getting to the point in the value that they they get from them and so uh while we do look at how long people watch videos it's only one of the factors that we consider when understanding the value people are getting we introduced this concept of satisfaction uh quite a few years ago where we're trying to understand not just about the viewers behavior and
what they do but H how do they feel about the time they're spending what would they say about their experience watching a video we we ask them directly with surveys built into the product we collect millions of responses and feed that directly into the recommendation system so that we can recognize when creators are delivering kind of more value per minute or or more total value uh than maybe the the time spent would indicate otherwise and so uh we look at things like likes dislikes these survey responses um people can tell us they're not interested in
certain recommendations we have a variety of of different signals to to get at this satisfaction and we've seen that when we add those signals into the ranking that it actually leads to people coming back to YouTube more in the long run and that's really what our goal is is not just to get suck a lot of time from you today that's not that's not the point the point is like we want to build a relationship with our audience just as creators want to do with their fans yeah uh so that you know we can have
viewers keep coming back um because we're delivering a lot of value to them in the long run so for creators do you think it's less important to look at Absolute like what's your CTR what's your watch time and more like how they're currently performing and how they can improve based on their metrics and not like a universal sense of metrics yeah I think creators seek out benchmarks because they want to know like is this good am I am I um I totally understand that unfortunately like the reality with the way the metrics work and you
know the fact that that they're kind of dependent on who saw the video and how broadly it it was distributed it's really um you know the ultimate CTR you get or the ultimate um absolute uh watch time you get it's not really comparable like across channels and it's even you know sometimes challenging to interpret when you're comparing across videos within the same channel you know increasingly while we do provide those metrics um I would take a step back and also think about your higher level goal as a Creator yeah and oftentimes creators talk to us
about views they talk to us about building up subscribers or maybe they have goals that are about you know selling merchandise or or or something else and you know maybe for those goals the number of views you're getting is ultimately what you care about uh or the quality of the views or the conversion rate of the views or yeah right and so if you struggling a bit with the depth of you know all the different types of data maybe just take a step back and say okay well you know just as one example would you
as a Creator rather have a video that has a 20% CTR and 10,000 views yeah or a 5% CTR and 100,000 views yeah absolutely the second right which video do you think was better uh the video with 5% and and and well I'd have to look at a lot more metrics to give you a real answer to that yeah I mean as a as a Creator I would be satisfied with the video that that reached a l larger audience I might be looking at you know is there an opportunity to reach an even larger Audience
by making the thumbnail you know more appealing yeah but given that it's achieved my goal um that's probably the most important part yeah in in terms of like comparing those two videos and so you know don't forget to take a step back and look at the big picture because those view numbers and those impression numbers they encapsulate you know all the factors that go into the recommendation system so if you're if it's performing relatively well in terms of CTR versus other videos that your audience um or the new audience might be interested in it's going
to end up getting more Impressions because it's going to rank higher in the feed yeah what about multilanguage audio we have creators now for example are uploading the main language the first language and then a week later uh uploading a dub track how are you thinking about recommendations in a multi language sometimes asynchronous world when we first started uh enabling creators to upload different audio tracks or dubs as some people call them um we needed to add some uh new capabilities to our system in the in the recommendation side to make them aware that this
video actually is available in multiple uh languages so that was one thing that we did to to to start enabling those to reach different audiences and then the next thing that we did for multitrack audio was to set up a feedback loop so that we could learn uh different signals for each track independently rather than just understanding the video across languages we want to know oh well this is working especially well in this particular language but not so well in this other language and so we set up those feedback loops to learn from that performance
so if you're a Creator who's interested in extending your reach through dubs what I'd recommend you do is a couple things first make sure that your titles and descriptions that you're also uploading translated titles and descriptions so that when we recommend or return and search these videos in different languages users uh can can see those and and know what they would be clicking on the second thing is to recognize that there's benefits from uh having more of your catalog available in a particular language so when a viewer does discover your channel they can watch more
than just like one video the more catalog that they have to choose from the more likely it is that they'll find another video that they'll enjoy watching which is a great signal back to us that you provided Great Value so we've seen in particular uh creators who uh dub at least 80% of the uh viewership ship of their Channel I think it's watch time yeah uh tend to have more success than those who uh dub less recognize that there's kind of a critical mass in terms of offering a catalog in a language so you might
want to focus on you know getting to that like 80% of your catalog within a few languages rather than doing like 20% yeah uh with more languages yeah and that's like current catalog based on watch time doesn't really matter you don't have to go back eight years but like the things people are currently watching watching that's right I think that's great because You' talked before about the power of series and giving viewers a journey and the ability to have things that they're going to they love this video what's the next thing they're going to want
to watch for you and that just seems to translate into the multil language as well we talked previously about like the recommendation system and sometimes views can go down for videos we of times hear creators talk about views going down Channel wide as well what are some of the things you look to when creators are talking about es and flows in in views yeah I mean the first thing is that that is natural um and just because your views on your channel are going down in a particular period doesn't mean that like your channel is
going to die and go to zero we see many waves of uh interest in channels where you know audiences might you know kind of binge for a while on a particular Channel and then move on to other channels for a while and many channels that that go down quite a bit will then come back up quite a bit unfortunately we we don't hear as much from creators when they're their their Channel is like anly blowing up no one complains but then when it goes back to uh another you know goes back down you know creators
get concerned but so the first thing is to just recognize this is natural um it's not particularly reasonable to expect that you're going to always be at your highest level of views from all time that doesn't mean that it's the end of the career um it could just mean that that wasn't as good as as your as your alltime best um so it's normal um I would encourage you not to worry about it too much and just focus on um what can you do to you know continue to evolve respond to uh the feedback you're
getting from your audience and from the analytics um sometimes you may need to move on from a particular format or topic that that has worked well for you in the past um because the the audience has been saturated on that particular area a couple of things you've told me have been very helpful one is don't just look at the current month and analytics but go out to the year to see if maybe they are spikes and returns to normal or a couple years to look at seasonality yes we do see seasonality can can play a
role um encourage you to look Beyond you know 90 days or or more to kind of see the full context often times you know we saw some some creators that you know saw a nice boost in demand around the the election uh in the US earlier this year and then you know people moved on to watch other things after the election happened and so think about some of those external Dynamics I encourage folks to use Google Trends to look at you know you can see uh Trends and how often people are searching on YouTube for
particular topics and while that isn't all doesn't always tell the whole story it can give you one interesting signal about just like how much demand there is and particular area the other thing to keep in mind is the other side of the ecosystem is Supply supply and demand on the supply side we do see you know dynamics that happen where a particular Trend might or or topic that that has you know increased interest in the early parts of the trend there may be only a few channels or a few videos about something and so there's
there might be high demand and low Supply so a lot of views per video and then later there might be a lot more creators that are producing content in that area and um you might see more Supply than demand and so those those things are uh going to interact and will ultimately influence how much uh views there will be per video another bit of advice that I really loved is you said look at the subscription tab because that's a way to see what your how your core audience is reacting without any involvement from the algorithm
right so this the way that works the subscription tab is just a chronological feed of that viewers channels that they've subscribed to so it's a pretty consistent um you know audience that's going into that tab to catch up on their subscriptions that um doesn't have any sort of like dynamics of the recommendation system in it and so why I encourage creators to look at that is because it's kind of a controlled little uh area and then you can look at like well what is the click-through rate and what is the average view duration of subscribers
who are looking for for those videos in their feed and it can give you a little bit more of a benchmark to kind of on the content in an Apples to Apples yeah is it me or is it the algorithm yeah I would just look at it uh compare your videos it can help you understand oh well people aren't clicking on this in the subscriptions feed as much as they did my last video what can I learn from that maybe it's maybe it's the thumbnail or the intro to the video or the Topic's not as
broadly appealing because I think sometimes people forget that like you can only like you'll recommend a video to an audience but some videos naturally have a smaller audience than other videos or they have a much like more narrow Focus or just the broadness of appeal is different that's right earlier this year some of the talks that we did at like vidsummit and Vidcon you mentioned bringing more large language models into recommendations how is that working yeah so for those of you who have uh played around with Gemini or Chachi BT or some of these other
chatbots the technology behind them is a large language model we've applied the large language model technology to recommendations at YouTube to M attempt to make them more relevant to viewers how does this work uh in a couple different ways one because the the models are larger and what that enables uh them to do is develop a deeper more Nuance understanding of content um of of the and of each viewer and so rather than just kind of like understanding like oh this video is about you know uh Indian cooking it might actually be able to understand
the ingredients of the dish better and uh maybe some more elements of the video style and and how it's presented or the emotions that the Creator might be conveying um these are all things that um a larger model could enable smaller models uh tend to rely a lot on what we call memorization okay so it might memorize that that this video tends to be um good with this type of viewer for example um whereas a larger language model uh can uh learn a more generalized um pattern of rather than just like this single video with
this audience it's like oh this type of video and this and and the patterns could be much more nuanced so to use an analogy here taking the cooking example um a small model that's memorizing is kind of like uh a cook that is reading from a recipe it's like a fixed set of instructions and I'm going to make this uh this dish and you know the fact that I can exec those instructions doesn't necessarily mean I'm an expert Chef so the more generalized um you know larger model would be more like an expert Chef that
can not only follow recipes but understands the fundamentals behind cooking and they can more dynamically respond to different conditions so like rather than just following the recipe well what happens if somebody comes in and they say oh I'm I'm allergic to that ingredient or I don't eat meat um and now you still have to serve them up uh a dish the chef who has more nuanced understanding uh will know what ingredients to substitute that are still going to make a delicious dish so that's that's how I kind of think about this new technology enabling our
better recommendations we want to be more like the expert chef and less like the you know just following the memorized recipe Todd thank you so much for your time I really enjoy talking with you it's great to be here and keep the feedback coming so uh if you have feedback for me or the team please put it in the comments and I'll be I promise I'll read them and keep it real keep it keep it real
