today we'll be answering the question uh we are seeing a ten percent decline in tick-tock viewership and we'd like to figure out why hey we're here today for a mock pm interview i'm here with stephen stephen could you tell us a little bit about yourself your background hey everyone uh y'all probably know both me and jacob we're the co-founders of exponent today i'll be in the hot seat and i'll be answering a pm interview question um as you know uh i was a former product manager at google um and this question today will be an
execution style question which are more commonly asked by companies like facebook all right so we'll jump into it today we'll be answering the question uh we are seeing a 10 decline in tick-tock viewership and we'd like to figure out why and in this question i'll be sort of role-playing a tick-tock data scientist or data analyst and you can ask me any questions you have and i'll try to work through the problem with you awesome cool i'm going to take just a second to write some notes down okay cool so the first place that i'd like
to start out here um as i approach this problem so obviously 10 decline seems like a really big deal so it seems like something might be going wrong here that we should definitely look into uh the thing that i'd like to check is first of all is this actually an issue or is this something that we're misinterpreting or getting wrong here potentially so the first sanity check that i'd like to see is if we've has we have we changed anything with the logs recently or is it possible that there's a logging error here um the
way i determine that is first of all uh have we have we updated the logs or the way we log things at uh tick tock recently that's a great question um we as far as i know we haven't updated anything with the with the logging other than just sort of usual updates and things like that um but yeah it does seem to be an actual issue or as best as we can tell it's a real decline okay cool just a sanity check um and we can keep that in the back of our head in case
we really can't understand what's going wrong the second thing that i'd like to investigate is whether or not there's some pattern to this decline so um it sounds very unusual to me that a 10 decline would be occurring across viewership on tick-tock but perhaps this is not so unusual so i guess in the past six months have we seen a decline of 10 uh that looks similar to this in terms of the duration and the like deflation of users um we do see like sort of seasonality and day-to-day differences um but this decline seems to
be sort of cutting across those um and seems to be not sort of seasonally related to anything we've seen before at this time of year or um or this this week or anything like that okay great so what i'm hearing then is this sounds like it probably is an issue so i've um eliminated the option that this is maybe seasonal or maybe there's a logging issue and i'd like to jump into actually diagnosing what could be going wrong here and so the framework that i'll be using to kind of walk through this or talk through
this is basically what it would take for the bite of info to get across from the client to the server and so i want to investigate i'll start with like if there's any human errors but i'd also like to investigate if there are hardware or operating system errors if there are app errors um and if there are maybe like network or server errors so i'll be walking through those but first i'd just like to start with potentially a human uh change or like something with going on so um i'm not sure if you can answer
this but is there any uh like world event going on or something that might be affecting to talk to viewership in any way um i'm not super aware of any any major events that are going on right now do you have any ways we could maybe detect that or figure that out yeah um so one way that i might investigate this is actually checking out tick tock hashtags or like trending topics in hashtag and uh to talk to see if there's any pattern there of something that comes up so i maybe look at like the
top 20 hashtags in different areas and see if there's something going on there and i might slice it by region to see if there's like a certain region that's having a decline so i guess my question first of all is is there a regional pattern to this decline and second of all um regardless of the regional pattern could you tell me whether or not the there are like new hashtags that are like spikes recently um in tiktok um we are seeing yeah sort of like i mean there are trends all the time in these hashtags
so we're seeing a lot of different things we're not seeing any sort of regional variation that seems to indicate any problem though um doesn't it doesn't seem like we're seeing anything there okay um so there could be something like uh covet or something like that that i was trying to slice for or figure out um if i had more time i might investigate even more hashtags and kind of look at the pattern there both within the app experience and then i might also uh just check like general site tools to see if site traffic has
been increasing or decreasing both on mobile and uh general but i um it's sounding like this might not be a human error or this might be a tough one to figure out so i'd actually like to move on to talk about hardware before uh digging into this one so we can kind of leave a flag about whether or not there might be human errors but it sounds like there might not be so i'd like to move on to the hardware part so the hardware case i guess uh what i'd like to check is can we
slice the viewership decline by the hardware version so are we seeing for example an iphone 20 that just came out that is no longer operable with the tictac experience for some reason um so do we see any pattern across the hardware versions yeah it's a really good question um yeah unfortunately we're cutting across all the different platforms and devices and we're seeing a pretty similar decline across all of them um so it seems like it might be something else got it okay um and then can we do the same thing for operating systems so like
maybe ios 20 just came out is there some pattern of ios versus android um we're not seeing any major pattern there i mean in general there's more usage on the sort of newer or standard operating systems than other ones but the decline seems to be hitting sort of across them again yeah okay um so we've sort of eliminated the hardware and the operating system again we're following that byte as it goes up from the client to the server um i'd like to investigate the app maybe to see if something's going wrong there so um if
we slice by app version do we see any pattern of that viewership decline across app version again we're actually seeing a pretty similar decline across app versions as well okay um then i'd like to investigate the network um to see if there's any network issues so can we slice by carrier um to see if there's any like relationship with the carrier and the declined uh viewership yeah we can look at that um yeah we look at that and we're not really seeing any differences there either sort of an overall decline okay cool um so we've
while we haven't gotten to the right answer we've definitely eliminated a lot of stuff so we've eliminated the hardware the operating system the app and the network and we've sort of said that the human error might not be the case so my hypothesis now is that it must have something to do with the server or the data center so first of all can i get a check on the data centers like everything green is there any sort of like outage in our data center or report of anything like that going on um yeah all systems
are green and seem to be fine uh we haven't noticed any outages or anything like that we're seeing uh like space utilization is not growing quite as fast um so we're seeing more sort of available space than would be normal yeah okay cool that's super helpful so space utilization would indicate to me that like uh tick tock is not storing as much information or data or the growth of that might be slowing down um so i'd actually like to investigate um so you know there could still be a server-side bug that i i'd want to
investigate so we could come back to that later but for now it sounds like there might be something here with the space utilization and so i'd like to investigate um uh well has there been any change in the number of video uploads per day for tick tock like uh the number of like uploading maybe like people the funnel into tick tock is slower than normal yeah there does seem to be a pattern here where uh video uploads have also been slowing down um and it's sort of at the same time as the viewership has been
declining huh okay so it sounds like we have a we've identified some component of the issue it's unclear whether or not this is the cause of the effect of the issue um so i'm trying to think of like what could cause uh the people to stop uploading videos and so there's two things i can think of one is external causes like maybe people have decided to not upload videos maybe people are stuck at home and have nothing to upload which is probably not the case because if anything tick-tock has exploded over um the covet period
or maybe there's something some uh internal issues with tick-tock like tick-tock has created some bugs or problems so actually the thing that i'd like to check next is have there been any increase in feedback reports to tick-tock like in terms of the total number of tickets filed for uh support issues for tick tock yeah that's a good question we are seeing an increase in support tickets okay cool and uh i don't know what level of granularity we have around those tickets but like i guess does it seem to be um an error state or like
a bug or like is there any sort of sense of like what type of information we're getting there yeah i can't slice too granularly by the type of ticket but it does seem to be um we are seeing reports of bugs yeah got it okay so this leads me to believe that there's some bug on the platform um however one thing that i'm a little stuck by is that we've already investigated all the different aspects um but it's possible that some of these aspects may have been affecting video uploads and not been affecting viewership so
i'd like to investigate these same kind of components the same like bite and flow but instead of viewing it on the side of like the viewer or like coming to the viewer i'm going to try to view it on the see it on the uploader perspective so can we slice by hardware or operating system to see if the video upload uh quantity i guess like has been deviated at all across hardware versions or operating system versions yeah that's a great great point um yeah we can look at that and when we slice by hardware and
operating versions operating system versions we're not seeing any major differences in view uploads um is there anything else we can check out um can we check the app version next and if there's any deviation there yeah and so we checked the app version and it does seem to be that there's a different line in the latest app version for video uploads all right okay that's super helpful so it seems like there is a deviation for app version for the platform um and so it just to clarify like is it the latest app version that has
the issue yeah it seems to be the latest one has um has the decline in previous app versions no longer in use but the the rate of upload seems to be decreasing yeah got it okay um so it seems like uh the app version is the culprit here that's causing the videos to not be uploaded as much um and that might be causing the viewership decline and the one thing that i maybe want to check just to make sure that this is all correct or my hypothesis is correct is looking at the app versions i
guess it's it's gonna be hard to slice so i think this is maybe what's causing the issues it's really hard to slice by the previous app versions videos and to see if the viewership of those videos are the same because the videos all go into one platform so it'd be hard to kind of identify this but then instead what i might do is i might dig deeper into those support tickets to see if there are um this specific issue i might go through the video upload flow myself to see if something is going wrong here
um and then i might also check you know if we haven't up if we haven't launched that app version in all regions or maybe we did a staged rollout or we rolled it out um yeah like first in a different country i might check to see that stage roll out to see if we're seeing that decline for the smaller subset of users that still might not have that updated app version so that's the other thing that i would check um but i i yeah so to whatever extent you can verify any of that information that'd
be helpful but otherwise it seems like we're kind of close to the right answer here yeah i think we are close um let's look at we can look at our staging sort of deploy of this app version and we can drill into the data that we have there from our set of data users and when we look at that we do see that this decline seemed to precede the official launch of this app version so it does seem to be related to this particular app version so i think we've confirmed that got it cool um
so it sounds like we've sort of solved the issue just to recap what i kind of went through um i was first investigating like the logging errors things like that and then i went through to see if there's any issues on the viewership side but the key component here sounds like it was the um investigating the video upload side and figuring out then uh what was affecting that video upload speed that then affected the viewership um so yeah i think that's that's the final answer i think of the of the question yeah i think that
was great um so yeah just to recap yeah it seemed like we've confirmed that the issue was on the video upload side which was app specific that you said seemed to be sort of globally affecting viewership due to the lack of content that was basically being uploaded um so a little bit of a trick question because it seems like the problem was on the viewership side but it turns out it's actually on certainly content uploading side so you did a great job of like drilling into that and really systematically analyzing what the potential causes of
the problem could be and i really like the way you structured your answer around sort of methodically exploring those and walking through the thought process of how you were doing that uh so i thought you did a great job cool thanks yeah um this is definitely a fun interview because it's more like a puzzle and um i think having a structure and clear path and articulating to the interviewer what is effective about or how you're thinking about the problem is really effective um and i you know if i were to go back i would i
might you know bubble up at the top and even call out there are different user segments that we're investigating and that i was only investing in the viewership one um and that might help capture that full picture as i'm going through it with the interviewer um but otherwise these questions are really common and they're asked to you know companies like facebook and lyft and etc and uh they can seem really tough but uh i i also think uh it was definitely discouraging to keep getting a no from you every single time i was answering those
asking those questions but um it was helpful to eliminate options and actually get closer to the final answer awesome yeah i think you did a great job dealing with it cool um well thanks uh for interviewing me jacob and uh thanks for those of you watching um this is a great example of a pm execution style interview
