are you interviewing for senior product roles well you're in luck because in this video i'm going to cover a more advanced way to answer the goal setting success metric questions like what goals what success metrics would you set for product x so we are going to cover five things today number one we always start off with product understand if you've been following this channel you know this sets the foundation for everything the second which is the most differentiated part is we're going to unpack the strategy of that product the third we're going to talk about
metrics that reflect that strategy number four we're going to talk about the risks and counter metrics and number five we're going to talk about the ecosystem impact so the example we're going to go through today is instagram shops so let's dive into the first part of product understand [Music] so the first thing of product understand you want to start off with one sentence on what the product does and again strong senior level product leaders are concise about this and very focused so that one sentence here for instagram shops is instagram shops is a marketplace business
that allows businesses to set up a online shop with their brand allowing buyers to discover their items listed either directly on the shops tab through an ad or discover items through creators that are working with the business okay it was one sentence but it was a long run-on sentence i will admit keep it tight and focused the second thing of product understand that senior leaders will do is they call out the complexity in the ecosystem which means a lot of these ecosystems yes they are supply and demand but a senior leader will differentiate themselves by
calling out that in this instagram shops ecosystem sure there are the businesses there are the people buying we'll call them shoppers but there's also the advertisers which the businesses are but also there are the creators who are basically advertising on the businesses behalf and then there's instagram which is the stakeholder here because they're the ones designing this product and you have to make sure they're happy to want to build this product the next step as you call out the complexity of the ecosystem players you'll want to call out what they're there to accomplish so basically
what is the value that they get in this ecosystem so businesses are here to list their items and get sales which equals revenue for their business shoppers are here to find interesting items that they would love to own creators are here to produce content and engage with their audience and promote products that they find valuable and what about instagram here is the third part that i think senior leaders are really good at calling out which is the business model behind the main stakeholders instagram here so instagram makes money when businesses sell products why because they
take a percentage of the gross merchandise volume which is basically the number of items sold times the price so they might not be taking that cut right now but the long-term strategy will be that that's how businesses like amazon work by allowing certain brands to be on their platform they will charge a percentage of the sales for using the platform that's important because that's going to be what helps you drive some of the metrics that you'll want to measure so with this in mind senior leaders for these success metric questions will early on call out
the nordstar metric because it shows focus so i'm going to call for instagram shops my northstar metric is a number of items purchased why is this good for the ecosystem well it's good for the businesses because the more items purchased the more revenue that businesses make okay so that one's obvious for the shoppers it's good for them to purchase the majority of the items from one platform because it makes it easy for them think about it if a shopper has to go to multiple brands websites like let's say i go to macy.com and then i
go to what's another e-commerce store bonobo bose what is that site but no groupos anyways they have different uis and me just figuring out the user interface and how to manage the website and the checkout flow and have to put in my credit card information again and again will add friction to that buying process versus if you reduce that fragmentation and put it into one user experience where the user interface is consistent whatever you buy that's going to reduce a whole heck of a load for users buying things so it is also good for them
to maximize the number of items they buy on a platform that will make it easier for them to buy and we mentioned it's good for instagram because they take a cut from all the things that are purchased so the more items that are being purchased the more revenue that the company is going to be making so this next part senior leaders will call out trade-offs between certain metrics so here let me call out that i could have said my north star was the number of orders the number of transactions which a lot of people answering
this question will call out how i strategically think why it is not that i value more the number of items sold versus the number of orders so what is the difference the number of orders i can have one order with five items me counting just one order is going to hide the volume that is flowing through in our platform so for me what matters to me here is seeing activity like 10 items in one order versus five orders with one item each because the former is going to lead me to have more revenue as a
company and help businesses more and tell me that users have a preference to buy just more volume on our platform versus just measuring orders so again this reflects senior product thinking because it shows the trade-off and like strategic of what i really care about when i'm building this product and that's then going to flow into how i design the product or how we market the product well first like why do i care about 10 items one order versus five orders and one item each because i want people on instagram shops to be developing habit around
buying items on the platform i wanna see that they're buying everything from their daily knickknacks to their larger purchases of say furniture like this couch right here which is new by the way um on the platform so calling at the north star metrics is already getting into metrics so i know that sort of transgresses into the second section so another area of opportunity to show senior leader thinking is to proactively address just measurement so i have the number of items listed i want to call out the risk that a lot of the items who don't
have necessarily a buy button on instagram shops might be hard to track whether an item's actually bought so there's that risk that i'm gonna proactively call out versus waiting for the interviewer to ask me i think a good thing to say is one it's gonna be really important for these brands to get let's say the facebook pixel or yeah it is the facebook pixel because facebook is the umbrella company in order to follow the flow from a discovery on instagram so that even if it's taking them to the brand's website that we're tracking the conversion
bridge between the two so measurement we're gonna make sure that each of them have a pixel in order for us to do the measurement if that's not possible that's where i have to explore a couple of things like well one we might not figure out attribution because that gets lost when they go onto the next website so what is the last thing that we can capture before someone let's say goes to the other site or purchases on the other site and there might be a click so [Music] now we're going to move on to our
second part and this i told you is one of the key things that are different from my past execution success metric frameworks that are really going to help you stand out as a product or senior product leader which is unpacking the strategy to define your metrics so here let's use the instagram shops example a couple parts of the strategy i'm going to call out first is one i think instagram shops distinguishes itself or what it wants to see is to integrate shopping into everyday content why because that increases discoverability of products which increases into things
like shopping time and obviously things purchased so what does this mean basically we talked that for instagram shops a lot of creators are basically promoting the products in the content they produce and creators have an affinity with their audience so this part of the strategy we're leveraging creators to do the marketing for businesses might increase things that we're going to be talking about so the first metric i want to measure is the number of clicks on shoppable posts and even the click-through rate per week i'm going to want to compare this to the number of
let's say visits to the shops tab and here what am i trying to understand so again senior leaders will talk about what they're trying to understand with the metric i basically want to see are more people getting into the shopping flow through organic content i mean creator advertised whatever content versus people all of a sudden saying oh i feel like shopping so let me go into the shops tab or even people that are finding products through ads which of these channels are we seeing getting people to spend more time shopping and i'm a conjecture that
this is a really effective channel for discovery because a lot of the times maybe i'm like scrolling through instagram and at that point i'm not i'm not like in the market to do shopping but if i see on a post in my feed that oh this crater that i care about like oh she has on a cute dress even though i wasn't intentionally going to go shop now it's bringing me in to the shopping mode because i have the ability to and doing this all while people didn't have the intentionality to shop which leads me
to my second metric i want to measure is basically the time spent on instagram shops and also the number of shopping sessions per week are we basically increasing the amount of time people spend shopping on the platform or on e-commerce in general because we're helping them discover it in this different way like through the more organic way rather than when people feel like going to shop like that has to have that motivation for them to go to like an amazon to buy things versus in this new way they're discovering products through their friends through content
creators and that's going to seep into the time that they spend as entertainment and to now shopping so i would conjecture we'll see people spending more time shopping and more sessions people spend ordering things that they might not have and the last thing because of this content featuring shopping things i would imagine the number of interactions on posts with posted by creators is also going to increase why because on a current post today you have comment share like etc and now this is giving them a new avenue to interact with the post in some way
or even seeing the post in a different way so for example instagram is all visual based and let's say someone posts content and it's not like often i will comment on that person's let's say dress and like that's a beautiful dress instead i'm going to be like oh cool place wherever you're going to but now if there's this extra dimension where i can comment on something they're wearing and potentially buy it i would then imagine people interacting with that more whether it's through writing comments or actually going through the purchase funnel or doing that shopping
off of the post the next part of the strategy to help instagram shops be successful is this is an easy way we're going to make it easier for buyers to purchase items because they can shop several brands from one place which means we might imagine they're going to buy more so what are some metrics i want to measure if this part of the strategy is working or not the key thing i want to measure besides the north star we talked about is the average number of stores visited per user per week again the hypothesis is
that if we create a one-stop shop for people to shop various brands it's going to reduce the friction it takes for people to go to multiple websites and figure out the ui and add in all their payments but also that people will be shown multiple items across different brands in one place to sort of mix and match purchases and i would use this number to then benchmark against the number of visits to brands on other e-commerce sites to see are we increasing the number of visits to different brands or the number of brands that people
would shop in a week versus outside of shops because if so that again is great for the businesses who now are getting extra volume of people visiting the stores that they might not have if they had to go on another website the next part of the strategy is by helping businesses understand the full funnel view from advertisements or discovery to sales this is going to get businesses to spend more ads dollars on the instagram shops platform versus maybe some other channels in this business with ads one of the struggle that has pervaded like since forever
and especially on tv is that brands have a hard time measuring return on investment so the ones that give them absolute clarity allows them to be able to compare apples to apples and feel more certain about the benefit that they're getting so how i want to measure if brands are actually buying into this story strategy is measuring the number of active shops per month and we're going to define active shops as brands that have sold at least one item this tells us how many of these brands are interested in facebook as a channel and see
it as a potential channel to bring large volumes of sales to these businesses and within this metric i probably also want to do some segmentation or a lot of these businesses large businesses or smbs the next metric i want to measure here is the total ads dollar spent and here i would want to do a pre versus post analysis and see our businesses brand buying into this story that we're talking about like basically this metric is meant to validate this hypothesis that the more clarity businesses have on the end-to-end funnel between discovery to purchase the
more they're willing to spend their ads dollars so are we seeing this to be the case before instagram shops became a thing versus after and the last part of the strategy is that my hypothesis is for instagram shops one key part of the strategy is that by understanding what people will spend money on versus click on we're going to do a better job of understanding what people actually want to buy and hence better personalized ads which leads to higher click-through rates and higher conversions which again benefits the business benefits facebook because they take a cut
so what are the ways i want to measure if this strategy is working or not is the click-through rate on ads again pre versus post so after we've financed that algorithm to incorporate some of the signals and data we're getting from buyers instagram shops activity are we seeing the ads that we're delivering to them to be stronger in personalization and above the click-through rate i want to measure just the total number of conversions that happen and are we increasing conversions more so this part of coming up with the strategy of breaking it out and then
figuring out metrics to then analyze to help you validate or invalidate the strategy is really going to help you stand out to show senior product leadership thinking [Music] so we'll cover the last two parts so part number four is calling out risks and counter metrics and here you want to start off with thinking about like what are the things that could go wrong in this ecosystem and first with instagram shops as with anything related to payments or buying you're going to have the risk of fraud so the things i want to measure here are just
the number of shops flagged for fraud and the number of tickets that are complaints from our buyers another risk we'll see is that maybe the shops are legit but they're selling unacceptable pg rated r items and here i would want to measure maybe the number of shops reported or the number of items that are reported and lastly another big risk is just having items on the platform that are low quality and here we'll be able to assess if this is a problem by measuring the return rate and the cancellation rate because if items are of
low quality people have a higher tendency to return them or cancel the order after reading reviews and number four ecosystem impact basically if shops succeed what impact is this going to have on the entire instagram ecosystem one we talked about it's going to increase the interactions with posts and creators and the thing i want to measure here i think i already talked about above is just the number of interactions with posts so basically are we seeing more activity on each individual post and here i want to measure this metric for posts that have a shoppable
post versus not shoppable to see if those shopable posts are garning more activity and then i want to measure the engagement on the platform as a whole so if people increase their time shopping on instagram shops that's going to mean people just spending more overall time on instagram so what is the i would want to measure the time spent on instagram for those in maybe the control group who don't have instagram shops or don't use it and then for the test group of people who are using instagram shops and i might also want to measure
the lifetime value per user so now the lifetime value per user is highly determined on ads revenue but now we're adding this additional component where people are buying directly on the platform and instagram getting a cut from each purchase are we increasing the lifetime value for users that historically just didn't click on ads so this is the way to answer the success metrics goal setting question in a way that's really going to show that your strategic thinker and that you have product leadership capabilities for product roles you're also going to be asked trade-off questions which
are also going to help you show your strategic thinking skills so here are two videos to check out if you want to ace trade-off questions which are most likely going to be asked in the same 40 45 minutes that you get and i will catch you guys later
