let's say you're a pm at nike and you've noticed site-wide conversion has declined year-over-year what do you think are some of the causes and how would you evaluate this [Music] hey everyone i'm kevin way i'm a product manager at ibm data and ai and today we have a mock pm interview with jeff lee and before we jump into today's question jeff can you tell us a little bit about yourself yeah hi guys so um i'm currently a product manager in silicon valley at an early stage startup uh helping to launch a product from idea to
initial launch prior to joining the startup i was at autodesk for about five years working on 3d design software awesome well we're super excited to have you and today we'll be doing a root cause analysis type question for this question let's say you're a pm at nike and you've noticed site-wide conversion has declined year-over-year what do you think are some of the causes and how would you evaluate this okay great so give me a second to jot down some notes if that's all right okay so first um some clarifying questions so i guess when we
mean when we say conversion do we have a specific definition of conversion let's say that customers are coming they're adding items to the cart but we're seeing that more customers are not checking out okay got it so we'll just say conversion is basically uh full checkout great and then when we say site-wide um are we implying everything on like because um are we are we implying the e-commerce side of nike's business yeah okay great okay so i got that um and then so i guess in this i'll think i'm gonna think about some ways that
i might evaluate why this metric is declining and then i'll mention how i might actually evaluate or dig into those hypotheses does that sound good that sounds good okay awesome okay cool so i've got a couple ideas here um on why uh conversion rates have maybe dropped for nike on their e-commerce side um year over year so i've got a couple questions i um i'll have some recommendations on how we can evaluate them so the first one is i might want to evaluate on whether or not customers are purchasing in other streams so that might
mean are they uh buying how are they buying are they buying via mobile versus web um you already kind of mentioned that this is specifically for e-commerce but i'm kind of also curious to see if they're purchasing or purchasing trends have changed in retail and brick and mortar stores so that's the first question that i would probably ask okay yeah um i think that we can say that because of covid it's safe to assume retail is down and mobile and mainly desktop web is up okay so retail down and let's say mobile is down you
said and web is up and web is up got it okay cool okay got it so that's one thing that we can take a look at um so for example uh you know because we're specifically focusing on site-wide you might assume that retail being down might be a leading indicator for your mobile versus web um you know purchasing trends or conversion rates um it sounds like because mobile is down as well but web is up uh maybe mobile is outweighing the the percentage of web purchases going up so overall site conversion might be going down
but let's move on to the other questions and maybe we can circle back on it so um i'm curious also to know if we feel like e-commerce as a whole has gone down so obviously it sounds like year over year right now nike conversion rates have gone down but do we see industry-wide trends that are similar or is it just nike that's experiencing something like this the reason why i'm asking is i'd love to know if we can segment it down to maybe it's the shoe industry or maybe it's just e-commerce as a whole people
aren't buying on web because of covid or or maybe they're buying more and we're the only ones not reaping the benefits just kind of curious to see um what we might figure out from that analysis no we we don't notice any significant changes here okay so no significant changes okay and then um so let's dig into mobile versus web a little bit more so you mentioned that web is up and mobile is down do we know do we notice i guess are those trends measured year over year as well so we're noticing mobile is down
year over year and web is up year every year is that what you're saying so year over year we can say that web is up and mobile is the changes are not significant okay got it interesting so we can say changes aren't that significant for mobile and the web is up so there must be something else that's causing conversion rates to go down overall okay so let me um i guess the next thing i might ask is are we um have we offered or changed our product offerings at all so obviously i'm assuming from nike
there's things like shoes and apparel um and gear um and even equipment um but i'm kind of curious have we kind of changed what our offerings look like um what personas we're going after um that might give us some clues as to um you know what what variables we've been changing and how they might be impacting our our overall metrics yeah this is an interesting thing so we have recently added a new section okay the more high-end boutique section on the e-commerce website got it so high-end section let's call it a boutique section like you
mentioned okay and i can probably safely assume that this is the cost of these items or on average is probably a lot higher than the items before this was added yeah the cost is on average higher okay got it so average cost is higher okay um are there any ui or ux changes in terms of being able to search for items uh select and compare them add them to your cart and even the checkout process itself any any major changes no there are no significant changes to the interface or the user experience okay no significant
changes um are we noticing that customers are um returning and purchasing again um after they've purchased once or do we notice that there's any any significant changes on you know recurring or returning customers so we can say that in general customers are coming back and there's no significant churn here that we have to look out for okay got it and then um are we noticing any um new competitors by any chance do we notice anybody in this space um that we're trying to compete with or go after um considering that we're nike obviously there's a
lot of apparel and sports companies but just kind of curious if there's any particular brand that we're trying to tackle um with maybe with our new product offerings or anything along those lines yeah so we actually do consider some luxury apparel as competitors now so you can think maybe a concrete example would be adidas yeezys okay got it so we can we can compare ourselves with adidas yeezys okay cool so um let's see here um i guess to follow up are we noticing if um customers are do we notice any changes in um i guess
the time on platform or in cart so what i mean by that is our customers adding products to the cart and then they sit in the cart for some time do we notice that there's significant changes of that over time um or is that generally the same as before just kind of curious so when we look at customers um well i i'll turn this back to you so if given the the two options right uh items in the car being left for a while or no beings and there's no significant changes there what would you
do differently as a pm or what would you look at yeah that's a great question so if items were left in the cart more or less i would try to take a deeper dive and understand what types of items are is that happening with um so to see if we can again segment the problem even smaller um the reason being is you've mentioned before that we've added a new product line um we're trying to tackle and address uh you know a competitor like adidas yeezys and maybe because it's new people might be having some confusion
or they might be having some i won't call it buyer's remorse but whatever comes before that so maybe some paralysis uh when it comes to purchasing so uh i'd love to see if that's possibly a cause for um for people not buying and not actually converting over to uh to end purchases um if that wasn't the case and people weren't paralyzed we might look at other other streams and other reasons um to maybe identify the problem a bit more okay yeah we can say that there might be some analysis paralysis or there might be some
uh items being left in the cart for extended periods of time that are above the normal trend lines okay great and do we notice that that's specifically for our new product lines do we notice that it's happening for for that product line by any chance yes we do see that it is significantly happening more to those these new boutique items cool so great so just as a recap um it sounds like overall most things are the same except for a couple key changes um it sounds like obviously there's some purchasing trends that are different because
of covid so we're noticing um retail is down mobile is a little bit down but sounds like not significantly in web is up but we are noticing that um oh we have noted that there is a new high-end boutique product line that we're offering at this time uh so that's a major change in our product offering and we're actually doing that so that we can compete with you know a competitor like adidas yeezys um we're also noticing that some people are getting hung up on uh when they add those those types of items those boutique
items to the cart and they're getting some analysis process and purchasing so that might be one one reason why they're not actually converting is because um they're seeing a new item and they're having trouble making a final decision on whether or not they can purchase so um let's see i think overall um there's a lot of leading indicators to point towards that as a potential hypothesis so that's the hypothesis that i'll suggest forward is that because we are now offering a new product line that hasn't been fully explored yet people are having some trouble buying
or completing the purchasing process when they're adding these luxury items to the cart it might be because they are still comparing other items that are high-end boutique maybe they're comparing it across to other competitors or maybe um because it's just pricer overall and people tend to uh be a lot more conservative when with their spending uh they might be a little bit more reluctant to actually hit that purchase button and commit to spending the money so um i think right now it sounds like from our initial exploration of the potential causes of our um dip
and conversion rate it sounds like you know the high-end boutique items are probably what's causing the issue yeah that is a pretty accurate insight so why don't we take this one step further and think um you know now that we have this insight uncovered what might be something that you would do to remediate this problem okay got it give me one second let me think about that some more okay great so um some of the pain points that i've identified from these people that are getting hung up on eventually purchasing these high-end boutique products um
the first one is just confidence in their purchase so knowing that they're getting a quality item uh and and you know having that kind of um cold feet when it comes to committing is a big issue for them so thinking about how we could um instill some confidence in in their purchasing and make it okay for them to actually purchase um the second thing is kind of this idea behind um i call it like social validation so because it's a new product line people don't know if it's actually any good um versus you know the
other like for example air jordans that have been around for such a long time people know it's a trustworthy brand but because this is a new product line there may not be enough um you know social validation or really just reviews on the product itself for people to have confidence and be comfortable purchasing and then the last thing is um they might be spending a lot of time comparing this brand new product to either existing products or even other high-end boutique products so those are kind of the three pain points that i'm looking to solve
a potential fourth one is price but i think in this case we're probably not going to price these products differently it seems to be in a different price band and tier obviously if you price it cheaper people would tend to buy more but i'm gonna be eliminating that uh from you know the pain points that i'm considering so some suggestions to fix these three major problems uh in terms of confidence and purchase it's really just making sure that uh the buyer is okay with buying this item especially because it's online and they don't get a
chance to try it on so maybe we can just re-ensure that you know our return policy or trade-in policy is up to snuff or is really visible when they're purchasing uh or we give them a chance to exchange it for anything any other item in the store hassle free no questions asked you know really hammer in or hammer home that customer service aspect of it just so that people are a lot more comfortable when they purchase um you see companies like amazon doing this pretty successfully where they kind of basically have this and even costco
they have this kind of no questions asked return and exchange policy and it makes people more confident in buying because they know that they can take it home and try it before before they commit the second thing is something as simple as adding a review system for for those new items obviously there's already a review system for the the previous nike items that people have already purchased and used um this is even more so important because it's a brand new item and you know you don't have a lot of confidence when you go to a
page and there's been no reviews and nobody's purchased it and you're basically the first guinea pig trying to buy that boutique item so you know either i'm thinking of some ways that we might incentivize uh getting the reviews or maybe offering a discount for the first person to review or something like that might be something we could consider um basically as many ways as we can to to gather reviews as soon as possible so that other um other buyers in the future have a bit more confidence and understand what it's like from a buyer's perspective
to actually purchase and use this boutique item and the last thing is just simply a comparison tool i think of for example apple when you purchase uh you know the iphones and you're trying to compare specs between all the three i think just something as simple as that to compare even within the boutique items or from a boutique item to a legacy nike item uh might be really helpful so for example if the boutique i under my shoes we might want to compare all the colors all the prices materials and things like that just so
people can get a snapshot view of what they're looking at before they purchase it's a little bit harder to do than something like a device because device have specs and they're measurable and this is more along the lens of fashion which is subjective but still i think some people might want to know um whether or not i'm purchasing a shoe that is meant for let's say cross training or maybe another shoe that's meant for like walking or tennis for example um so that just might be really helpful for you know a snapshot of information that
might be really helpful for the purchaser so i would say those three things so again that's um you know really robust um return and exchange policy that's more upfront and prevalent from a ui perspective so they know that they're confident in purchasing and returning a more robust um or i guess a more uh friendly review system that encourages first-time buyers to uh to review as soon as possible and then lastly a comparison tool that just makes sure that these new items that are now added to the catalog are compared to other comparable items and maybe
even competitors as well okay so i'm curious about the comparison tool in the sense that what would you be comparing specifically yeah um so i think um initially i would be comparing um all the um i guess the the customizable parts of the shoe so what i mean by that is um you know choosing knowing what sizes are available knowing um if they're meant for uh men or women or if they're unisex knowing what style they are in general if there's a style category within nike um knowing um you know colors knowing mature main materials
knowing what brand line they're a part of i mean because you mentioned it's a boutique item i'm assuming it's going to be its own brand um so i would like to maybe compare something like this with something like like a jordan and because i might know jordan more i might lean towards buying jordan or maybe because i didn't like jordan's i might lean towards buying this new boutique item so those are some things that might help me um and then overall just a rating so kind of pairing that with the with the review system um
if i know that these boutique items are getting really rave reviews um maybe having like a top testimonial or even just a five star review would be really helpful for me to just get a snapshot and understand if i'm buying the right item versus something else it's probably a little bit more difficult to compare very similar items but i think if you're purchasing one shoe and another shoe that are similar in price but once she is like really poorly rated and obviously you know there's a lot a lot higher likelihood that you'll buy the shoe
with a better rating even though the other shoe might have the color that you like or whatever for example okay yeah because i i get the iphone comparison in the sense that you're comparing specs right i guess with shoes they're you might just be comparing colorways and available sizes okay um so given these three potential ways to rectify the solution would you say that there's one that you would recommend um let's see so i think of the three um the one that i would try to i would probably recommend would be kind of a review
system or a social you know kind of a social validation um feature and the reason why i say that is because something like returns and exchanges probably already exists on nike somewhere even if it's not as prevalent on specifically the boutique items and then a comparison product may or may not already exist as well and there's a way that you can do that currently you can already go and you can always just pull up multiple tabs and compare it that way so it's it's doable it's just a little bit harder um reviews are a little
bit tougher because you either have them or you don't so i think maybe adding in a future to incentivize people to review them for the first time whether that's a cost incentive or maybe be maybe making them a part of some nike community kind of how yelp elite has their community um but really incentivizing the community community to come together and give their own reviews and insight into a product would be really helpful because then you don't feel like you're the first one first guinea pig or first penguin dump diving into like the icy cold
waters right so i think that's really really important when it comes to buying especially when uh it's something in terms of fashion because there's such so much cultural significance to it um that i think you know having people that are saying yeah this is a cool item you know you're more likely to buy it cool uh thanks for that jeff and so before i give my feedback looking back would love to hear your self-reflection how you think this mock interview went cool um so i think that uh we'll start with the first half um in
terms of the root cause analysis so um i think what went really well was um going through and exploring all a bunch of different reasons that might um pertain to why conversion is happening or declining you over a year um i think what i liked about my answer in particular was i tried to go with macro and micro so macro meaning things like the economy and external factors to nike um like covet and things like that and then micro in terms of like what are we as nike doing differently and how can we segment our
problem uh to really compartmentalize and figure out what's going wrong um so i like that part of my answer in general and i also like that it seemed like there were some recurring themes uh in terms of um the answers that i was getting it sounded like the boutique items happened to be hitting on a lot of the hypotheses that i had so it gave me a lot of indicators strong indicators that this is probably causing some change in our conversion year over year um in terms of i guess the product design portion of it
uh where i was promoting features that one was a little bit harder for me only because it seems like a lot of the solutions that i suggested were probably already existing in some way shape or form and i was maybe recommending some slight improvements on them um so i wish that i had um thought of a bit more novel uh features that i could promote um and and suggest so that we can get more people to you know buy um you know buy these boutique items that maybe aren't buying before yeah i think the idea
of macro to micro is a good approach another one is thinking about internal and external factors and maybe starting with internal factors since they're easier to predict and understand right but macromicro definitely works too and you did do a pretty good job of identifying that it was a new product line that was causing this so in this case the luxury apparel section and i think that um when it comes to the product design it is you might want to think about assuming uh some of these things that don't exist so for example like assuming a
review system doesn't exist um that might be something that would be good to clarify first to yeah ask it because you know i think it's pretty standard for uh e-commerce already have review systems but i think um also you know your the the pain points you pinpointed were pretty spot on um you know giving confidence in case they have cold feet social validation and allowing customers to have a way to compare i think um you know if you know given more time i think that one thing that would be good to keep in mind as
we're brainstorming solutions is to think how the user persona is different how folks who purchase high-end apparel may be different than folks who purchase regular nike apparel um there might be some differences in needs there and how they behave but um overall it i i really like the approach and thanks again for being on today's show super valuable um for the viewers if you have a different approach to this question leave a comment and we would love to hear what you have to say thanks for tuning in and good luck with your upcoming pm interview
