Curriculum - LinkedIn Product Manager Mock Interview: Success for LinkedIn Events

for this question andrew how would you measure success for linkedin events hey everyone i'm kevin way i'm a product manager at ibm data and ai and today we have a mock pm interview with andrew yu and before we jump into today's question andrew can you tell us a little bit about yourself hey yeah thanks kevin i'm andrew i'm a product manager on linkedin at linkedin i'm on the growth team and specifically on the my network team on the growth team awesome well we're super excited to have you and today we're doing an execution type question

for this question andrew how would you measure success for linkedin events yeah thanks kevin uh so first before we even get into it i'd like to just lay down uh lay out the structure of how i would approach this so uh really going to be approaching this from three different um three three steps so first would love to talk about what the mission of linkedin is and of this product that we're talking about linkedin events and that will guide our uh the entire conversation here then i want to move into a brainstorming phase and talk

through uh what metrics might be good to measure success for linkedin events uh and why um and then doing some trade-offs there and then finally we'd like to prioritize based on the brainstorm that we had and arrive at the success metric that um i believe would be the best for defining success uh of linkedin events um so yeah i'll just get right into it so the mission of linkedin is to help provide economic opportunity for every member of the global workforce and so when we think about linkedin events um i think when we talk about

measuring success let's think about how linkedin events ties into that overall mission so as a member i might attend events to stay informed stay up to date in my industry i might attend events to uh to connect with other people so the networking angle so really i would say the mission of linkedin events is to help people connect to each other so that they can be connected to economic opportunity so let's use that as a working definition for our mission statement for linkedin events so next i'd like to talk through who are the actual users

of this product and so events is a two-sided marketplace we have the actual events creators and then we have the actual event attendees so we'll need to start defining metrics for for each of those groups so let's go ahead and start on the supply side first where we're actually creating events so as an event creator i would like to be able to just create an event event from scratch i'd like to be able to see the number of times my event has been viewed i'd love to be able to see the number of people who

are attending my events so already there we have a list of metrics i'm just going to lay them out so number of total events created on the creator side a number of times the event has been viewed on the creator side um and then from the creator side as well seeing how many attendees i are attending my event so that's that's on the that's on the creator side now if we look on the participant side let's walk through a couple metrics as well um so as an attendee event attendee i need a way to even

view them to even know that this product exists so from a discovery ability standpoint we can call this something like number of uh events uh viewed so this is more like an impression event and then from there uh if we do a funnel analysis let's talk through what happens next number of events viewed and the number events uh number of events that have gone to that page and started looking at some details so we could call that something like event scroll page or some kind of events view page interactions and then finally at the bottom

of the funnel we're really trying to get these members to attend these events so that we can help drive and deliver the value of connecting them to economic opportunity through the events so let's call that confirmed event attendees so up to this point we've talked through the mission of linkedin the mission of the linkedin events product which is to help connect people to each other into economic opportunity through an event and we've listed out all the different metrics for both the supply side and the demand side um and so now what i'd like to do

is start prioritizing them based on how closely each of those metrics fits with our mission and the value that we're trying to deliver so um i think one thing else that is important to call out here is that we should think through some guardrail metrics so counter metrics as to how do we know if um if we're actually going in the opposite direction so we have success metrics but what metrics would help us um identify if we're if user actually using this product in a bad way and so one couch or metric could be something

like uh number of people who say they're gonna sign up but then don't actually show up so we could call that something like an event delinquency rate or something like that so now we have all these different metrics uh at our disposal to talk through and so let's start talking about prioritization so i would say that if we're defining a core north star success metric for linkedin events it should be the metric that closely the most closely uh measures the core value that we're trying to deliver for our members and so going back to the

list on the supply side we have something like raw number of events created that's a little too high up in the funnel on the attendee side we have number events viewed in total by the attendees i think that's also little to have in the funnel so i think really at the end of day the confirmed attendees number is going to be what really captures the value of linkedin events um and would be a great success metric uh and then of course we can optimize it a little bit we can do something like adding a time

scale on it so for example number of confirmed attendees within seven days of posting the event or something like that depending on what exactly uh the goal is of the event and how how soon we want members to actually start attending the event but those are more optimizations and i think uh the real success metric here is number of confirmed attendees cool so how would you evaluate the trade-offs for this one yeah it's a great question i think with any metric there's going to be pros and cons there's going to be ways to so if

we say very simply put how closely does this deliver the value or measure the value that's one way to put it but if we're going to be a little bit more precise we should talk about how easily could this metric be gamified right and so what do we mean by gamified if a metric can be manipulated um or taken advantage of with uh with actions that do not reflect the true value that we're trying to deliver we can call that gamification so for example um if we could just simply uh increase the number of uh

the number of events shown by changing the borders or the size of the cards that are being shown that because that now lends itself to being a little bit gamified so um i think the pros and cons of this metric the confirmed event attendees uh are the following so i think the pros are that it's very it's a it's it's low in the funnel and there's a lot of intent uh captured so the intent is that hey there was the event that was created there are the attendees who saw it and then they went all

the way down the funnel and then they've shown their intent that they want to attend by saying that they're going to be a confirmed attendee so i think that's one pro i think one con is that um it's really difficult to uh to confirm whether an a person actually showed up to this event especially i think now given that a lot of events are virtual um it's hard to have that kind of confirmation of like did they actually attend the event so i could see this potentially being gamified so for example you might have uh

you know people creating fake accounts to say that hey use these fake accounts to uh say that you're attending this event so that we can bolster our attendance numbers so maybe one thing we might want to do is add a follow-up engagement metrics or something like confirmed attendees who have you know posted something in the comments section of the event so i think that would add an additional layer of engagement um and uh not not falsifiable action somebody that might be something we could do um but yeah i think overall the pros are that it

really captures the uh the value and the end goal that we want for this metric for this for this product which is to get the right and right events to the right people and have those people be interested in these events enough that they actually attend those events okay so this is a good metric let's say 20 of event attendees are down for one day how would you troubleshoot this yeah so first of all that would be a pretty big cause for alarm um so you would start by uh opening up uh an investigation and

looking at a couple of different uh dimensions of this drop so there's there's a couple questions that i'd like to ask first off so uh can we rule out some of the low-hanging fruit so uh seasonality is one thing uh did we ramp anything recently that might have caused this drop so i would first ask kate is it are we on a holiday like are we is it is it during the christmas break or something um and so can we rule out seasonality if the answer is yes there um that we can rule it out

hey this is just a normal week then the next thing i do is look at those recent ramps it's like hey did we push something out yesterday that broke something right so again those two are like just very high level uh the first things to kind of get out the way and then the next uh dimensions that start looking at which is more of the meat of the troubleshooting is that let's actually look at the user properties so which segments are being affected is it 20 of all event attendees is it twenty percent of attendees

in certain geos is it twenty percent of attendees who are uh a certain um uh activity level so is it like our is it all coming from our daus is all coming from uh our monthly user or something like that so that would be the next question and we could go really deep into the the segmentation here and look at a bunch of different cuts but i do want to move on to a couple other dimensions that i'd look at so the other dimension i look at is the feature dimension so are there specific features

that are broken that would lead to this drop so one example here could be that uh maybe the maybe the event attendance button is broken right like maybe there was uh there's just something that broke uh overnight for a specific uh for a specific experiment that we're running that is leading to this 20 drop uh perhaps it's uh some performance issue that now uh we can't uh we can't actually call the api endpoint to get all the events uh within uh our preferred time and so that's actually leading to uh a drop in event attendance

because there's not enough attend events even being shown and so that actually leads me to the last uh dementia that i'm like looking at which is the overall funnel analysis so if we're trying to troubleshoot this 20 drop let's look at each step so is there a drop on the number of total events just even being seen in the first place if no let's keep going down the funnel uh is there a drop on the number of events that are being looked at more deeply like on the detailed level is already there we might find

the drop-off so say for example the description field of the event isn't loading then a hypothesis there is that hey uh people uh who are viewing this event don't have a good sense of what this event even is so they're not even going further down the funnel so if we keep going down uh maybe at the end of the day we identify that the button itself is broken but yeah i'd like to do a funnel analysis uh going through each of the steps to troubleshoot uh so just to summarize uh the way i troubleshoot this

20 event attendee drop is by looking at the uh low hanging fruit of seasonality just recent ramps anything that we pushed uh would look at user segmentation and then look at a feature segmentation and then do a full-on funnel analysis cool okay um i like that approach would you say that there might be a different way to approach this answer if it were for in-person versus a virtual event yeah absolutely that's a great point so i think uh that the short answer is yes it would be a different way to approach us and the first

the main reason i would think so is because we there's less of a capture mechanism in real life attendance so for example on virtual attendance you can at least see a participant's list and there's a way to show confirmed attendees on your participants list but that's the end of it whereas uh in the physical environment what you could do is do follow-ups in person you could do things like uh reaching out to attendees in person um right after the event and saying hey we want you to do some follow-up action um you know please do

that while we're hanging out by the you know by the snacks and debriefing right so um i think the main way i would approach this differently if it was physical is i would do an uh an additional audit of what the steps are for physical events uh kind of like we did at the very beginning and either add or remove any steps and uh think through what the success metric would be from that standpoint i think overall it'd be pretty similar like i think we would still be confirmed attendees but i think the um the

potential the feasibility to do some follow-up action is actually a lot easier in person um i miss going to those meetups i miss going to real life you know like you can do things like hand out swag you could do things like uh have people actually sign up uh on ipads there's like all sorts of these things you can do that you just don't get um on virtual right right thanks um so that's all i have before i give my feedback looking back would love to hear your self-reflection on how you thought this mock interview

went yeah so i think that with each of these steps uh that that we we went through um i could have deep dope into uh individual ones and provided a little bit more specifics so for example when i was talking about uh the event like whether or not the metro can be unified i think i could have gone more in depth there i think when we talk about something like uh when when i was doing the metrics brainstorm uh talking through uh like number of attend events viewed or things like that like there's a lot

more detail that can be added there and i was i was trying to go through as much uh as broadly as possible so i think depending on the interviewer they may want to see more of that they may want to see more details but some interviewers definitely want to just see how much you can cover in a short amount of time so uh yeah i think there's a potential to do both ways yeah i really like the funnel analysis approach that you did and how for your metrics you started abroad you didn't immediately jump in

and say the metrics i want to capture are number of views et cetera you you kind of spoke at a high level about the importance of the things we want to capture and then you talked about how to quantify that that was good i think it would have been better if in the beginning we clarified together if we are looking for virtual attend virtual events versus in-person linkedin events and um just any other clarifications that you wanted to ask i think that would be uh good to make sure that we're on the same page before

we jump into a question yeah it's a great point and um one thing i did i really did like was talking about gamification so i i see where you're going with this this is kind of like if you are the pm and you want to increase the number of likes for example on something you know you could just make the like button really really big and you would be able to increase that number but you wouldn't really be adding any value yeah i think there's a rule that's like um just always be thinking of the

bad actor like that worst actor like what they might be doing uh and try to fight against that kind of gamification but yeah that's a that's a great point yeah yeah for sure and um i think one thing that especially with these social networking companies these days one thing that might be important to call out is any anything related to political or spam sort of um actions so especially with the elections coming up these are always in the news and as the pm i think it might be good to keep keep an eye out on

um anything related to that yeah that's a great point just calling out trust and safety privacy things like that uh really good and and i think the great thing about that is like that can be it can be called out up front um to just kind of set the tone as well be like hey so we're on top of this like we're thinking about this um yeah right yeah especially with the increased scrutiny with uh tech companies but yeah in in general i i really do like your approach and i i thought this was great

thanks again for being on today's show super valuable and for the viewers if you have a different approach to this question leave a note in the comments would love to hear what you have to say thanks for tuning in and good luck with your upcoming pm interview