Curriculum - Flipkart cart additions are down by 15%. What would you do?

so in today's scenario i want you to imagine that um we're working at flipkart and we're noticing that there's a decline of about 15 of cart additions to the additions to the shopping cart in the last three days um can you help us diagnose the issue [Music] hey everyone we're here today to do another product management mock interview we're going to be focusing on a root cause analysis type question which are similar to execution questions where you have to figure out what's going wrong with a particular issue i'm joined here today by kanjin um kanji

go ahead and introduce yourself for folks watching hi everyone i'm kanjin i am currently a product manager at razor pay which is a fintech company in india and uh prior to this i completed my postgraduate program for the from the indian school of business and before that i was working at for a couple of months at bain and company and idea cellular which is one of the leading leading telecoms of india so that's about me and i'm super excited to be here today awesome well we're excited to have you too um and so kanjin as

mentioned we're going to be going through a root cause analysis question and so i have a certain scenario that i'll propose to you and then we're going to talk about how to actually go through that issue does that make sense awesome okay so in today's scenario i want you to imagine that um we're working at flipkart and we're noticing that there's a decline of about 15 of cart additions to the additions to the shopping cart in the last three days um can you help us diagnose the issue uh sure so stephen before i jump into

uh sort of finding out what the problem is uh just want to understand what do we define as cart additions here so that we are on the same page got it so cart additions are let's imagine a user that's on the flipkart experience the flipkart app um they add an item to their cart so they may click and then they're an item appears in their shopping cart sure also uh flipkart has both website and mobile app so are we looking at a generic 15 decline or has it been observed only on the mobile or app

or the website we're noticing it mostly on the mobile app experience okay sure uh so i will just take two minutes to sort of make my framework and think about flipkart in general and uh will then share the framework same framework awesome take your time okay yeah okay i'm ready uh so stephen now our first question is uh the card additions that we are measuring so it's sort of like a metric uh which might be measured using some analytics tool uh so have we checked whether this analytics tool was working well over the last three

days yeah that's a great yeah that's a great question so um we talked to the analytics team we want to make sure that this data is correct or right and it does seem like the data coming through is um good like there's no issues with logging or analytics that have happened in the last few days okay so now that we have all of that answered i will just share briefly my outline of the framework that i'm going to follow to arrive at the root cause uh so first i will look at the external factors which

will involve uh note which will involve uh getting information on uh what competitors uh have done in the last three days has there been any product launch or has there been any new announcement which might have taken away a certain uh customer base or certain uh product uh so which might have taken away certain customer base from flipkart second we look at the demographic feature so has it affected any particular customer segment or any particular demographic feature of that particular customer segment third we look at the macro economic feature macro economic changes so has there

been any news or any seasonality impact or any kind of uh pattern in the external factors which which might impact the overall purchasing behavior um fourth would be what are the internal changes that we have done so have we made any updates in the mobile app have we made any design changes etc then will be the overall journey of the user uh on the mobile app because we are closing it down on the mobile app so we will see all the steps of the user journey and we'll see if any particular point before the cart

auditions has experienced any glitch or bugs so we'll basically look at any incidents that have come up uh and if none of this is the reason then we might look at certain uh certain factors which are out of control but which still might impact the overall purchasing behavior uh so does that look good to you yeah that sounds great thanks for laying it all out sure so as i mentioned so the first question is uh uh in india especially flipkart uh has a couple of e-commerce giants as competition and uh so have has there been

any new announcement or entry of any competitor or any kind of a product launch or a big billion sale or uh sorry big billion sales for flipkart so any sale that has been announced yeah that's a great question so we want to see if there's a competitor um we have limited knowledge here at flipkart so we don't fully know um what the competitors are doing from a cursory perspective or looking at social media profiles things like that we're not seeing anything in particular but there it's not entirely clear there could be some detailed targeting or

some you know customized um campaigns that we're not visible to or not visible to us i see okay uh so uh also have we observed if this has impacted customers from a certain city or a region uh so is there any impact of location on this particular decline we're not seeing an impact of location on the decline so when we look at the location we're seeing a spread across many different places okay great also has any particular segment been impacted so let's say females of age 20 to 30 who were earlier purchasing a lot of

cosmetics have suddenly not started so suddenly stop purchasing that or is it again a uniform distribution yeah so um we can look at some basic uh factors like gender age location and we're not seeing a deviation for any of those sort of big demographic categories that we typically look at okay thanks for clarifying that uh so now we will look at what flipkart has been doing um so has a flipkart announced any campaign or any uh other let's say prop product or any other um uh sort of a subsidiary which might have taken attention away

from the mobile app um we definitely launch campaigns all the time um it's not clear that any one of these campaigns would be taking users away from the mobile app or there'd be any detriment to the mobile app but we do have campaigns running um is there something specific that you're hoping to get with with that or right so how i would look at this is uh considering that we don't see any pattern in the demographics and the geography we would first look at the products uh if there is any pattern in the kind of

products which have seen a decline in card editions and if we can identify that if there is a pattern or if there is even a pattern among the categories then we can look at the campaigns which were running for those particular categories and products and then we can sort of attribute the decline to that so do you have any information on that right um so we when we look at the campaign data and things like that we're not it doesn't seem like there's a correlation to a decline in the mobile app experience sure okay uh

so then i'll move ahead with the technical parts of it uh so uh as uh so just to summarize for everyone who's doing this uh when the customer will open flipkart or launch the flipkart app on their mobiles uh if they have already signed in to the app then they will directly see their home page uh so the first thing that they will see is the home page versus if they are going to sign in for the first time the first screen that they will see is the login screen versus if they are a new

user they would see a sign up screen so this will be the first touch point for the customer so the next thing that i would want to know is have there been any upgrades to the mobile app in the last three days on both ios and android yes there actually has been we've done a major update in the last few days um to the mobile apps okay and uh how are these updates uh related to anything uh in the cart edition uh touch cart audition milestone or are they not at all related to that we

definitely we've made some changes that affect the whole shopping cart flow so there definitely is a relationship between the changes that we pushed and the experience that we might be serving to our users okay so uh like can you be a little more specific as in have we made some design changes which would maybe increase the discovery of our products or have we offered an easier checkout journey or have we enhanced the maybe or have we changed the look and feel of the add to cart buttons yeah so we have improved the checkout flow um

to make it more seamless in an effort to improve the conversion rate on the checkout experience right uh also uh have we observed if this decline is only to ios customers or only to android customers uh we are seeing that a client only on android customers oh okay right and uh have we received any incidents or bugs report uh on the uh app changes that have been done so uh from any kind of customers or it's i mean we haven't received any uh reports but we still think that the upgrade might have done something yeah

we're not seeing so many more reports especially since we launched the experience maybe a few days ago but we are seeing an increase in some bug reports um and it seems like there's some bug reports that people are having issues paying or checking out their products right uh so now uh to go a little more deeper into the issue uh i feel that it is something to do with uh the process from right from uh seeing the product and uh finalizing the product to the checkout process uh but because we are observing a decline in

the cart editions uh the first and the directly attributable uh feature to that would be the add to cart uh option so first point where the customer would see that button is on the product listing page so it does not necessarily have to be uh specifically the page of that product but it will be like a generic listing uh so is the button visible there for all the products um the button so you're asking if the button is visible for all the products for the set of users um we can spin up a dev version

and kind of take a look at it and it does look like the button is visible to us we're not totally sure if there are other versions where the button might not be visible but as far as we can tell it seems like the button is visible on the platforms that we're looking at okay and uh once the again so now it comes to the journey from seeing a a list of products to selecting the particular product that the customer wants to add to cart and then clicking on the button and then uh that particular

action or that particular click to action translating that particular product to show in the customer's cart so uh what we need to do is we need to observe if this particular flow is getting completed so uh have we logged in the event data or have we observed the events data wherein the add to cart event is uh transitioning into the actual product adding being added in the card do we have that information yeah so when we take a look at the data um we actually see that there's been no cart additions on android um in

the past few days so we're actually seeing a huge decline in card editions on that specific version of android that we've launched uh version of the app on android right so uh then stephen i feel uh that because we are not noticing or we don't have any attributable data to the uh installs of app as such um also the product discovery or the product visibility doesn't seem to be an issue as well uh my bet would be on the add to cart button so we should probably uh do a complete analysis on the event flow

for every customer on the android app to check how many customers are actually clicking on the add to cart button and not seeing or not uh or their products are not being added to the cart so that's where the issue seems to be lying yeah you totally got it um and i mean we can go a little bit further into this and talk about where exactly the issue is or how we debug it um but i think you sort of nailed it on the head um maybe if you could just summarize one last time for

the viewers before we go to some feedback and things like that um of sort of your your journey through this flow and how you came to the answer that you came to uh so how uh one should see any problem coming in any metric for a product or a particular product feature is that first we look at the external reasons so more often than not especially for e-commerce websites a lot of external factors such as campaigns or competitions announcements or the company's own announcements they lead to a shift in trend for the customers and that

may lead to a certain segment of customers not actually adding things to the cart or not actually looking at purchasing at that particular point in time so considering that we were looking at a three-day window i had to sort of understand if and it's a big window so i had to understand if in those three days uh any such announcement from the competition or any such new product launch from the competition was affecting the cart editions on flipkart once that was out of the way uh we look at which particular location or segment of customers

have been impacted if at all there is uh there is an identifiable pattern in that then we can directly attribute that to the shift in maybe the purchasing power or the uh the trends or the various factors that may be contributing to the way they uh come on flipkart and add products to the card once that is out of the way uh we look at the residence of that particular problem on our ecosystem so is it on the website or on the mobile app or we really don't have any data on that which is a

very unlikely scenario we usually can attribute that very well from the data so here because we observed that it is residing in the mobile app and uh after a little more probing uh we also found out that it is on the android play store itself so the android version is facing the problems uh we surely know that it is not uh attributable to any change in the external factors and it is rather something that is internal to flipkart so once we know that it is an internal problem we start looking at the funnel and the

conversion from each step in the funnel so right from the installs to the sign ups to the product discovery to the visibility of the various buttons on the product listing page to the clickability of the particular buttons and then we identify how many clicks have actually resulted in the action to get completed so in this case uh that was our best bet because before that we didn't find any problem so that's how i arrived at the final uh answer that this might be the actual root cause of the card edition so declining by 15 awesome

um well kanji that was awesome to see you go through that i wrote a bunch of notes about what i thought you did really well um but before we jump into it we'd love to hear a self-reflection what do you think went well about that interview what do you think didn't or would you change if you can go back and then i can jump into some feedback and we can talk generally yeah so uh first so first i'll talk about the areas of improvement that i feel i should have done better or i could have

done better uh the first one being when i asked you about the website and the mobile app issue so where exactly the cart editions were observed and you did tell me that it was in the mobile app i should have asked you a bit more about where or which versions of mobile app or which play store uh was this particular issue happening on or which particular device type uh this issue was happening on uh that answer would have given me a much better visibility into uh the funnel so i could have understood that because it's

on android and not apple i could have understood that it is something which is a bug issue and not exactly a customer side of an issue or a or an external issue so that's one and second is uh because it's a root cause analysis uh or it might have been the case that uh i should have probably asked for a little more data so in terms of the numbers so in the real world i would have probably just gone on my analytics board and i would have asked for a lot more data uh so that

is another thing that i could have improved on um in terms of what i did good i think uh the framework uh that i made however it might have not been uh it might not have been required to look at so many factors but if you develop a habit of looking at everything from a 360 degree view then you don't tend to miss out on things and it becomes like a general way of solving things which as a pm i feel helps a lot in the longer run um so that is what i think i

did good totally yeah um definitely agree with you i mean overall i thought this was a really smooth excellent answer of a root cause analysis question um they can get very scary and hairy because you don't know where to go with it and it feels very intimidating um like oh is this the right direction or is that the right direction etcetera etcetera and i thought you handled that very well and um what i really loved about your answer is in the beginning you sort of started off with a full framework of okay these are the

ways that i'm going to go about this problem um i'm going to think about it like um through this and then this and then this and then this um and and kind of go through that and i definitely did wish we came back to that in the in the middle or if there was a white board or something like that it could be really helpful because then um i could really see like okay hey like this is where um this is where that issue was or something and like therefore we've eliminated this hypothesis or something

like that um i also um i i thought you don't know the the going back to the hypothesis was really you did this a couple times or you summarized or you were like okay this is now where i think i am or this is now what we're going to do um i thought that was especially great when you did that with the cart analysis we're like okay now let's just really dive into the flow and look at the flow and see where the issue is in the flow i thought that was really effective too um

i i one thing that um i wanted to call out that um you know i hope you don't mind sharing but or that i share with the audience is um there's a thing that a lot of interviewees do um and it's called up speaking um i don't know if you're familiar but it's it's sort of when at the end of your sentence you go up like answering it like a question um kind of at the end of things and i think um that was one piece of feedback i did notice is that you did that

a couple times throughout the interview um and it took away from the confidence and the gravitas of what you were saying um and it's a hard habit sometimes i do it too actually when i answer interview questions but i just thought it'd be helpful to share to the audience as well that it's something just to watch for when you watch recordings of yourself doing interviews or um especially at exponent we always really recommend people record themselves and watch themselves because then you see all the flaws the sort of tough things but that's just one common

behavior interviews um that always can can make you sound a little bit more confident a little bit more um sort of like uh steady in your answer um any reactions thoughts comments to any of the feedback i just shared uh no i think yeah very fair point so i'd like to add on that uh the way uh a lot of uh your pm interview also depends on the confidence and the smoothness with which you approach the question because it tends to get very intimidating in the real world scenario for example i remember when i was

interviewing for the companies last year uh in 30 minutes we were expected to solve all at least two interview at least two cases and uh if you don't have a control on how you are speaking and what you are speaking and not writing it at the same time and showing it to the interviewer the interviewer might just lose track of what you're doing and that can cost you a lot so a very fair point totally um yeah and actually i mean to sort of close or wrap toward the end of this conversation um if folks

want to learn more you can always check out exponent at tryexponent.com we help people land their dream tech job with our courses our coaching and our vibrant community we believe that people shouldn't interview alone the reason i bring this up now is that part of exponent's mission is to help people practice better and there have been studies to show that practicing for interviews just getting those reps in can go a long way towards building that confidence and getting that type of interview that you want to see in the interview to succeed um and so um

i did want to just mention that about exponent um but before we close conjunct do you have any ask last thoughts tips um anything else you want to share with folks watching yeah so firstly i would like to top your closing comment that i have in fact prepared a lot of my product cases uh watching exponent mock interviews uh so definitely uh that's one tip that i can give that any any mock interviews will help you get a sense of what kind of questions can come up and how should you hold the conversation during a

product management interview uh again continuing on that point uh what happens is that a lot of people feel that because it's a product case you in order to show that you know everything and that you're an expert people don't make conversation and they forget to ask questions or they forget to course correct themselves during the process and hence they get so lost in their own uh points that it just leads to a very jumbled up solution so yeah that would be my final uh comment and lastly for any help on product cases uh one should

always refer to uh or one should always fall back on actual practice so you will have your peers or your family members or your cousins friends whoever you can find just speaking to them and just uh telling them to be your interviewer will give you a lot of confidence which will help you in the actual scenario awesome and if you're looking to learn more about kanjin we'll put a link to her youtube channel in the description below um we really appreciate kanji taking the time and for those of you watching if you have any comments

or questions now is the time we'll try to answer every single one of them um just comment below in the video and hit that like and subscribe button as well if you thought this video was valuable um again thank you so much for joining and good luck on your upcoming interview