An Introduction to Product Analytics

great hi hi everyone thanks for uh tuning in my name is sharu gazali and i'm here to talk about product analytics so this is going to be a basic introduction to what product analytics is how it works and how it can help you and your business take things to the next level so a little bit about myself um my name shark as i previously mentioned uh i work for a company called invigo it's the car subscription startup and i'm the growth marketing manager there so product analytics is part of my day-to-day uh and it's really

helped us build a better product today and hopefully we can show you how to do the same in the past i've worked in e-commerce at a company called raz in pakistan and before this i was at kareem in the uae so let's get started so today we're going to be talking about product analytics we're going to start with what it is how it works how it can help you build a better business and then we're going to do a high-level overview of a couple of tools that you might want to learn about and last we're

going to finish with what an implementation looks like for your product analytics infrastructure so what is product analytics we could use a very boring definition which would be quantitative data about product usage but it's more useful to think about it as a way to tell a story about the people using your product who they are how they're using it in order to help you find ways to improve their experience and eventually positively impact your business so where does product analytics sit along your funnel what we're looking at here is a quote unquote pirate funnel where

awareness is a phase where a customer is learning about your product maybe they're looking at ads for the first time on social media they're searching on google and it's before they actually start interacting with your product acquisition is the phase where they finally start to get into your product um activation is when they take the first important step for some companies that might be buying something for another one it might be playing a song or a video for others it could be friending someone like it would be on facebook or linkedin revenue is when they

first become paid customers in a lot of cases you might think of activation revenue as the same but it depends on your business and how you look at things retention uh i think the most important step here is when a customer becomes a repeat customer it shows you when they start to really like your product and want to keep coming back and the last step is referral which is when they start bringing other users into your product product analytics sits along the lower part of the funnel from when users first enter your product until the

point when hopefully they never leave um and it will help you develop a detailed understanding of how they behave once they first start using your product so how is it different from marketing analytics um i think people tend to confuse the two terms so um a few distinctions that are important to focus on here is that product analytics is user focused so you're looking at a specific person and what they're doing with your app whereas marketing analytics is more channel focused so you might look at how many users facebook is bringing you versus uh paid

google or linkedin ads product analytics is a little more granular so you're literally looking at every step a user is taking within your product rather than marketing analytics where you're kind of observing things from a higher level product analytics is focused on the lower part of the funnel as we discussed previously whereas marketing analytics is focused more on the upper part of the funnel and the goal is to get more users into your app and at the end of the day with product analytics you're hoping to improve your product to improve retention conversion get more

referrals while with marketing analytics your goal is to improve your roi from specific initiatives you might be carrying out so how can i help you the main question it's going to help you answer is how can i change my product experience to reach my business goals and here are some examples of how you might start doing that one could be finding bottlenecks in your product funnel meaning where users are stopping before going towards the stuff that you want them to carry out understanding how users are using your product to just observing behavior uh to uncover

things you might not have known about your users analyzing a b test performance so seeing how one feature performs against another or a state where there is no additional feature and identifying what makes some users your best customers and how does it help you get the answers so here's some examples of the kinds of analysis you can get from a typical product analytics tool you have product funnels event reports user flows and retention analysis we'll go into this a little bit later but this is kind of showing you what you want to get to once

you have the right tracking in place so before we move on does anyone have any questions three two one okay great so how does it work it works by implementing tracking to help you tell a story about your users so to illustrate this and find out what the building blocks are to your analytics setup we're going to imagine you're hosting a party you've invited a few friends arranged snacks drinks and some games so let's see what someone's night at that party would look like you have a guest named shahrukh he arrived at 9pm wearing a

blue shirt he brought two friends with him had a coke ate some doritos laid chariots for 30 minutes left at midnight and said he had a great time and loved the food in particular um this isn't the most amazing part of you apparently but uh i think it'll help you understand how we're going to look at tracking so let's say you want to tell your friend about this party and this guest's experience at the party what are the details you need to tell them a story that actually gives them insights into what actually happened you

could say i had a guest who showed up and did some things that would be a pretty bad story uh you could say i had a guest who had a drink a snack play the game and left slightly better but not really telling your friend anything useful about the party so the details is where you really start to get the story my guest's name was sharok he wore a blue shirt he brought two friends arrived at 9pm left at 12 a.m he had a drink it was a soda he's a coke and he had one

cup and you can see you can basically pay attention to different details about each of the things that were done um it would be a little creepy if you did this in real life so i wouldn't recommend it but the purpose of this example is to illustrate how we look at tracking when we talk about product analytics so you can think of these as a user who is your guest user properties which are details about your guest events which are the things that they did and event properties which tell you a little bit more about

those things that they were doing so these bring us the building blocks of product analytics which are events and users events can be thought of as anything that happens within your product so button tabs screen views and users can be thought of anyone who's interacting with your products uh when it comes to events it's important not to think of it too vaguely or too specific for example if you're running an e-commerce app page viewed would be a little too vague whereas 55-inch samsung tv view would be a little too specific ideally you'd say product page

viewed and we'll talk about this a little bit more so properties provide context about both users and events and are key to collecting actionable insights so with the properties from our party i can figure out the following since i'm paying attention to what category of drink everyone was having i know how much soda i need to buy next time since i know how long each person is playing a game on average i can plan the amount of games accordingly to keep people there for even longer how many friends each guest brought will tell me how

much additional food drink whatever i need to prepare and most importantly it can help you find the difference between the experience a person who enjoyed the party had versus one who didn't so if most users or most guests are like shahrukh and they really enjoyed the food and they had food and the guests that didn't really like the party did not eat anything you might say that food has a huge impact on how people feel about my party so next time i'm going to make the food a lot more accessible so coming to event properties

the example i gave you earlier of not keeping events too vague or too specific comes down to the fact that you might want to get those specifics from tracking properties so rather than counting a samsung tv viewed lamp viewed shoe viewed event you just want to see that people are viewing products and you can find out more details about that through the product name product category and product price and what this would allow us to do is aggregate the information so i can go into my analytics tool and look at how many people viewed a

product where the category was tv which might be a lot more useful for me than just seeing how many people beat the samsung 55-inch tv another example of an event is the checkout startup so your user funnel will determine what kinds of events you want to track and the nature of each event will determine what details you want to know about the event so when someone starts a checkout meaning they've added their items to their cart in an e-commerce app you might want to know how much their cart is worth at that point you might

want to get a list of items that they have in their cart and you might want to know what kind of promo code they applied select event properties you can also track properties against users and those help you understand who your users are how they interact with your product and what kind of customers they are so attributes like name email address age date of birth tell you a little bit about who they are operating system app version any other i guess more technical details will tell you how they're accessing and interacting with your product and

number of purchases average order value sign update acquisition channel these are all the examples of the historical information about your customers so that might help you tailor your experiences towards them and kind of understand where people are coming from in their app so before we move on to a more practical demonstration of how this can be put together does anyone have any questions about users events events property the really creepy party where we were paying attention to everything people were doing okay let's move on so we're not going to look at how product analytics can

help you build a better business let's say we're an ecommerce app and our goal is to increase revenue to increase revenue we can look at doing one of three things one is to increase the number of users accessing the app oh sorry i think i just saw some questions there about the recordings never mind um sorry so one of the things you can do to increase revenue as an e-commerce app is to grow the number of users who are accessing your app in the first place as a result you would likely have more people converting

so that means more revenue another thing you can look at is increasing the conversion rate so for every 10 people accessing your app if two of them convert today you can get more revenue by converting four from those ten in the future and the third way to increase revenue would be to increase the average basket size meaning you have the same number of users starting same number of users going through but each of them is spending a little more money let's say we decide to focus on increasing the conversion rate so how do we think

of conversion conversion is the number of people checking out from the number of people installing and we want to find out how we can increase this percentage and we might want to develop some hypotheses why this happens but if we just look at it as an install in a checkout it's a bit too simple because the user journey actually looks a lot more like this someone will install the app go through your onboarding hit the home screen maybe they perform a search then they might filter they might go look at a product they might go

back to search and you basically get the point there's no there's no set journey that they have to go through to get to your checkout but what you can do is develop a hypothesis that is focus around the fact that people might not be converting through a crucial step in the app so if you know what journeys in your app tend to look like you can say i want to track the main events that lead to a checkout we're going to do with that is build a funnel so what you're looking at is what a

funnel typically looks like on the left you have the first event that someone would carry out followed by the next key event that they have to carry out the next the next and next you might see that from the journey we saw earlier we're not including search and filter that's because not everyone has to search and filter but these are the steps you have to go through to get from installing the app to checking out and those are the steps that we want to influence so as you can see you've got eighty percent of people

who view your onboarding screen accessing the home screen and from those 25 are viewing products from those 50 are adding an item to a cart and from those 50 percent are finishing the checkout process so what we can see here is that you've got a big drop-off between viewing the home screen and viewing products so we might want to find out what's happening when they get to the home screen why aren't they getting to a product so at this point we might want to look at what's happening at that point what events are being carried

out there so here i might choose to look at the percentage distribution of what users are doing when they're on the home screen some users are searching some users are tapping specific product categories and some are just ending their session right there and that's a bit alarming because half my users who get the home page are leaving before they even try to look for a product so here's where you might want to develop a hypothesis as to why that might be happening so i might say my users aren't trying to look for products because they

don't know how to use my app to find what they're looking for or i might say that they think it's too much work to actually go from the home screen to a product based on these hypotheses i might plan a couple of experiments to see how i can improve the conversion at this step so if we assume that it's too much work maybe we want to make it easier for them by showing them our most popular products by default if we think that they don't understand how to use the product we might want to develop

an onboarding flow to walk users through it meaning a more detailed onboarding flow than the one they've already been through let's say we want to add the popular products to the screen now we can see how we can use our product analytics and infrastructure to actually deploy an a b test and analyze its impact on conversion which is the goal that we're aiming to increase so we built a variant variant b which shows the five most popular products to users on the home screen variant a will just be the same product that you have today

and since we are tracking user properties we can add variant a or variant b as a user property for people who are seeing each of those versions so if i'm one half of the user base and i'm seeing control you can then look at data about me separately from someone else who is seeing the variant and if we look at the same funnel again it might look something like this so in the blue you have the conversion funnel for users who are seeing the old version of the app and in green you have the conversion

funnel for users who are seeing the variant where we're showing popular products and we can clearly see there's an impact we've doubled our conversion from the home screen to the product viewed screen so what have we done here in this example we had a hypothesis that our conversion is being impacted because users aren't performing a critical step then we looked at a funnel of our key steps between install and checkout to identify where the biggest drop-off was we then looked at the behavior before that point to understand where users were going instead of going to

the destination you want them to go to and we developed a hypothesis as to why that was happening using that hypothesis we planned an experiment tracked each experiment and saw that it doubled our conversion so let's say we have sorry before i move on to a different way of approaching improving conversion do you have any questions at this point there's one in chat how do you know which experiments to run or what's a sign of an issue so i don't think you know specifically what experiments to run it's more about identifying what you think is

impacting things negatively to begin with so we didn't just go in saying we want to run an experiment of popular items we saw that people were not moving beyond the home screen and we thought it could be due to having to perform too many steps to get to a product it could be due to the fact that they don't understand how to use the product it could equally be due to the fact that we're acquiring the wrong customers our product is in the wrong language or a number of reasons but because we decided to bet

on the hypothesis that it's um a problem with the difficulty of getting to a product that's when we decided we wanted to run the specific experiment to show popular products what's the sign of an issue on the product i think that depends on your business so for different kinds of industries you can research what your benchmarks are for different conversions you can understand how you expect users to behave at certain points so if someone is adding a card and all they have to do is press the confirm button in checkout it's pretty concerning to you

that someone who put in that much work already is not just pressing one button so it kind of depends on how you want to look at things which is based on what you see your business says great so another way we can look at conversion is or another another hypothesis we might have to our conversion being bad is that we aren't encouraging users to perform actions that lead to checkout so we might want to test this hypotheses by looking at different users so we can split users based on their behavior and traits just to analyze

how your good users behave compared to your bad users so using the user property of number of purchases we can create two groups or segments of users non-purchasers who have number purchases equal to zero and purchasers whose number purchases is greater than zero we could also use events and say we want to define non-purchasers as people who have never performed the checkout success event while our purchasers are those who who have then you decide what you want to learn about these two different groups and you can look at them side by side so on the

left here's an example of looking at events and their frequency for each of these groups i'm looking at the average products viewed for someone who bought something versus someone who did it people who bought something viewed an average of ten and a half products while people who didn't buy something viewed an average of only four and a half um i can also look at the number of products they tend to favorite which is still lower for non-purchasers than purchasers alternatively i could also look at attributes about the users so here i'm choosing to look at

their acquisition channel to see where users come from and how those users are split between my segment of purchasers versus non-purchasers so as you can see paid social mostly results in people who don't buy anything paid search is a little bit better but organic search which is people who are just landing on your product through google or i guess in rare cases bing they perform much better so using these insights might choose to do specific things by looking at the events i'd want to maybe try to encourage users to view more products or favorite more

products and by looking at acquisition this might indicate that it's time to double down on our seo efforts because those bring the highest quality users and in this case we've done something which is not just limited to changing our product but we've taken our insights and we've fed them back to our marketing team so that they can target better quality users because we have been able to identify patterns in the data that show us where the best users are coming from so do we have any questions before we move on okay great so there's a

couple of other forms of analysis that i'm just going to mention that we haven't talked about yet but it's good to know about because they come kind of out of the box with most of the product analytics tools you might see online one of these is user flows which basically shows you a user journey from a specific starting point or up to a specific endpoint so it helps you answer a couple of questions where are users going from event x so if event x is my home screen i might find fifty percent of them are

performing a search twenty percent are going through filters 10 are going to different categories and the rest are dropping off this is useful for observing user behavior in a less structured way than funnels which as we mentioned earlier rely on you defining specific steps that usually have to be carried out in a specific order and it also helps you understand what your key events are to reach your goal so if you find that almost everyone who gets to the checkout has to go through certain steps and there's no other way for them to get there

that's a very good way to build your first funnel for install to check out another form of analysis that excuse me that product analytics tools help you do is retention analysis so this helps you answer questions like if a user performs one event how long does it take for them to come back and perform the second event or how long did users who join in january save versus users who joined in march and this is really useful specifically for finding which groups of users stay with your product longer which is basically the goal of any

company is to keep users happy and engaged for as long as possible it also helps you understand if you're getting better with time because you can see how different groups of users have stayed engaged so you'd want to see users you've acquired more recently staying engaged for longer than users you got a long time ago because it shows that you're improving your product with time from what you've learned if there's no questions i'm going to go on to popular tools so there are a ton of product analytics tools out there um i've only focused on

a few of them three main product analytics tools here and google analytics just because most people have heard of it mixpanel is again free to start it's based on events allows you to view user data at a granular level like we saw earlier it also comes with messaging and a b testing modules if that's something that your business needs amplitude is also free to start based on events gives you granular user data you can export the segments that you create or the the cohorts of users based on when they've joined or certain actions they've performed

to other tools like your crm software or anything else and it also comes with a sql interface so if you're someone who's comfortable with sql and likes to crunch their own data that may be a plus for you a heap is another free to start tool it's also event based um and i think the benefit here is that it can be mostly implemented without technical knowledge because you don't have to pre-define events and start tracking them so by default it'll track things that your users are doing out of the app google analytics is not something

i'd consider for the kind of insights we looked at earlier mostly because it's built specifically for websites and it's page based so you don't get the same kind of user level information that you would with these other tools it's super important to have as part of your marketing stack because it tells you a lot of information about where people are coming from how long they're staying on specific pages and i think it really helps you with content but as far as understanding granular user behavior it's not really the best tool so what is an implementation

uh sorry i have a question here this tool is for mobile apps only no so uh mixpanel for sure at least you can use it on a website as well uh google analytics also can be used on both websites and mobile apps and i believe amplitude and heap can also be used on the web so it's not restricted to mobile apps i'm talking about mobile apps just because i'm familiar of the four which would you say is the best so i've worked the most with mixpanel um just because it happened to be there when i

joined the last place where i was using one of these tools however i've looked at amplitude amplitude seems really interesting because they provide a lot more insights that are generated by the tool about your users than mixpanel however if you're looking for one tool that you can use to also send push notifications or emails a mixpanel would probably be better to start with it also depends on what your business is like so each of these has different pricing models mixpanel is based on the number of users you're tracking each month and they don't really have

an event cap as far as i know amplitude is based on the number of events you send so if you have fewer users but you have a very detailed product uh where they're performing lots of events mixpanel is a better solution but if you have a lot of users and your product is pretty basic amplitude might be the better bet because you'd pay only for the number of events rather than the huge user base you're looking at i think heap is based on sessions so if you're how frequently people use your product would probably be

the the metric to judge whether or not it's the best tool for you so what does an implementation look like um there's not going to be technical but just it'll give you a bit of a framework for how to approach setting up product analytics for your company so we'd start by defining your business goals be clear on what your objectives are because that'll determine what kind of questions you ask and what kind of tracking you're going to implement a social network a music streaming app an e-commerce app they're all going to have very different experiences

and very different information that you want to know about the users within them so those will kind of determine not just the tool that you pick but the kinds of events you track the level of detail you go to the next step is to develop a tracking plan so you'd start by defining the key events that happen in your app between a user first starting to use it and when they get to the point that you really want them to get to decide what context you need from these events so again what we spoke about

with a product viewed event do you need product names do you need the time of day someone did something if you're a taxi booking app um of the information that you feel is important at each of these places is really going to be key to write down and record at this point and the last step is determine what you need to know about each user so again you start somewhere but you can keep building as you find more questions that you can't find the answers to with the existing tracking that you have so once you've

figured out what your tracking plan looks like you want to choose the right tool for you uh you could look at the tool first and then figure out your tracking plan from there but uh i think this developing the tracking plan first will give you a feel for whether you're gonna be events heavy users heavy sessions heavy and then you can pick your tool accordingly as we discussed earlier the nature of your business will determine what pricing model works best and hence which tool you should go with if you do you could test multiple tools

at the same time especially because most of them are free to start and generally with most b2b sas platforms they give you a free trial so as long as you're set up for trialling multiple tools you should be good and one easy way to do that is to use a tool called segment or something similar because it lets you set up one set of events event properties and user properties and you can then use that platform to send the events to other tools so rather than setting up tracking for mix panel separately and then amplitude

separately and then heap separately you can do it once and decide to use it with one or more tools and that's not just limited to product analytics if anyone's interested we can talk a little bit more about the other areas you can use the tracking you're going to set up but i think that brings us to the next and most important step is to follow a consistent process and framework um so think about what is the key information you need to look at and how frequently you need to do it um think about how you're

going to ensure that you don't lose track of your data as you build more features and that you're constantly implementing tracking for every new thing that you're doing to actually learn if it's effective um i think the habit that you should really be getting into once you have this in place is to ask the right questions and then find a hypothesis and keep asking the right questions till you land at a few different options that you want to test and then you analyze and find your winning variants so that's all i have for today thanks

for listening let me know if you have any other questions specifically on tools or implementation or anything we covered today uh chaz i don't think we have any questions uh hello mm-hmm one more question in chat you're muted hello can you hear me now yes go ahead okay sorry about that uh the question is apart from product analytics what other areas are critical and growing product-based businesses uh so the answer to that will be heavily dependent on the kind of business you're in and who you ask so for me as someone who is coming from

a marketing heavy background i think a crm tool is really important especially if you're in the customer facing product because a lot of times you can use these events that you're setting up for product analytics to set up very timely communications with them and personalized communications as well so if you have the attribute about me that my favorite color is blue you might contact me when you get a bunch of blue shirts in stock and tell me that you have i have blue shirts available so come back and buy something so for me the crm

tool is super important um what else would be important here um i think that's really key so product analytics crm i think attribution is really key so if you are using paid advertising a lot to acquire customers digitally you should be using an attribution tool like adjust or apps flyer to tell you where people are coming from and help you understand the roi of each of your channels types and specific ads so bringing that together with a product like google analytics which will help you understand how people are getting to your website and your app

and then product analytics understand how people are interacting with the app once they're there will give you a very good picture of the end-to-end user journey and where you really need to invest your time and effort please repeat information about crm tool so i didn't recommend any specific crm tool i was just saying i recommend that as a tool specifically if you are a customer-focused app as far as crm tools are concerned i think braze and urban airship are like the industry standards but they're kind of expensive now i'm using a tool called customer i

o which is super lightweight and really flexible and it works well with systems like mix final especially segments so if you have segment setup you can export all your properties and events to both mixed panel amplitude and customer io and it lets you do most of what the larger crm tools let you do as well but it's super light on the pocket so maybe check that out i haven't used hubspot and zoho personally but i've heard they're good tools as far as i understand they're more on the crm side for kind of the sales side

of things but it might have been a long time since i've looked at it myself so if there are any questions please go ahead so i'm guessing it's how many customer segments do you usually track and when do you focus on one when you focus on when do you ditch the others um not how many so i think it depends on how many segments are of interest to you so i think any the one anyone would want to start with are people who are paying you versus people who aren't paying you and then once you

actually have enough information to start influencing that split you'd want to go a little bit deeper to find out maybe from your paying customers how many of them are referring more customers or how many of them are buying items that gives me the highest margins again it's super dependent on your business and it's really important to develop the mindset of asking these questions so i if you want to understand what's different between what's the difference between customer x and customer why that's where you'll start but then you might ask even more in-depth questions and that'll

lead you to finding more segments i wouldn't say ditch others figure out what your key ones are focus on those and then identify more in order to actually influence your top line do i have an email you can contact me through uh it's just reaching out on linkedin i'll just post my linkedin in the comments could you give us outlines or resources on how these tools integrate with websites so if we're talking about the tools that that i was showing you earlier they all have really great resources for free on their website so obviously every

software as a service tool wants you to buy their product and they will teach you how the entire field works so i suggest looking into those i haven't implemented them on websites myself i mean other than google analytics but these are mostly seen used with apps and my my experience is deepest with mixpanel so i don't want to comment too much on the other tools that i haven't used as far as being involved in the site architecture i'm not a developer so i can't really speak to whether or not you would consider it embedded in

the site's architecture or not it's it's usually just implemented through uh a tag like like google tag manager you can use that to implement mixed panel and amplitude but you i believe you just have to define what your events are on a specific page so if you're looking at a form you define an event as a form submission event and you might say that you want to track the properties that people are submitting to perform so user's first name last name email address etc should all get tagged against that user once they submit the forms

which is a one-size-fits-all solution if you're running a business as a one-man army i think if it's a customer-facing business i would suggest you start with mixpanel but again that's kind of based on my own comfort level with the tool also it lets you send emails and develop a b tests relatively easily i don't think it's better in those areas than tools that are specifically designed for those purposes but uh i think that's a good place to start if i were you but you should definitely have ga implemented what does it take to be a

a product analyst does it require you to code uh i don't think being a product analyst i'm not a product analyst uh i i just use project analytics very heavily for my day-to-day job i'm not sure if i would actually qualify to be a specialist product analyst but we'd have to see what does it take uh you don't need to know how to code it would be good if you know sql and at least understand how to retrieve information but i think with most jobs that aren't extremely technical it's more about the mindset than anything

else are you able to ask the right questions can you think about how to mix the data that you are getting from orders with the data you're getting from users and find insights that actually matter you can always learn the technical skills but it's actually the mindset and the approach that i think is harder to teach what other methods are used in product analytics apart from eb tests when these are non-informative so if an a b test is non-informative meaning that you didn't see a significant change in either variant um i'm guessing that's what you're

talking about so we actually had this with a feature we built once and it wasn't a new feature it was just a different way of visualizing the products that were being shown on our app and we thought it was super important to do that because it made the app less messy and easier to use but that's anecdotal so we built something that kind of giving too much away it it basically just made it a lot simpler but we didn't see it actually have an impact on conversion so what we decided to do was go back

to the drawing board and see if we actually developed the right experiment we found that there were some things about the new way we had done things that were a little bit confusing potentially to some customers so we built another variant of that and the second time we ran that test with the slightly altered way of showing things we actually saw positive results so sometimes your a b test might not show you anything which means the hypothesis you had such as showing in our case showing five popular products on the home page will encourage more

people to see to actually view products might not be the right hypothesis maybe it was the fact that users don't know how to use your app and you need to put up training videos or something else to target that hypothesis no problem thanks for the question great so uh if there are any other questions i am i think scheduled to be on for another 15 minutes at most but if not i'm not sure if charles is going to be taking over or not okay great i think that concludes our meetup thanks it was a pleasure

i really enjoyed the questions feel free to connect with me on linkedin and reach out if you have any other questions or if you think i got something wrong that i need to learn more about i would love to do that but thanks everyone have a great day sorry were you saying something thank you very much for your time tonight um all right thanks jazz have a great day everyone bye you