what's pretty cool is a few guys who have been living the centre of building up these growth teams kind of for the past you know seven or eight years Edie joined uber to start the growth team when it was five people and then over the three and half years became how the VP of of both product and growth for kind of Olive uber and Gustav joined and the Airbnb growth team was him and two other people when he got started a few years ago and that is scaled over a hundred so it's fun at a
scale up off site to talk about scaling we most have been talking about scaling organizations there's a chance to talk about how do you scale the organization that continues to to scale your users you know what's what's fun for me is and I've known ed and Gustav now for most of a decade kind of in this growth world and you know we were all started back when sort of this idea of a growth team this idea of a team that's focused on this sort of intersection product and marketing to to build stuff was pretty novel
and almost didn't even really exist and we were all kind of doing it in a social world and I remember ed started a company in 2008 called you know friendly Jimmy go and then friendly you know Gustav started company in 2007 called haste on and then went over to boxer and these were like how do we get to like millions of users in the early days so I'm curious if you just kind of talk about for this heyday of social viral growth explosion and sort of what are some of the key kind of crazy things
you learn then that have really helped you as you think about growth today I don't want to start sure so yeah I guess 2007 summer of 2007 when Facebook launched the platform was I think part of that explosion of just crazy viral growth spammy apps on the Facebook platform that was actually the first that those were the first apps I built that reached tens of millions of users you know within months time and I think it was a great platform on which to just experiment with a lot of different ideas because it was so easy
to build these simple Facebook apps and just run lots of a/b tests and so much of I think figuring out how to grow something is it's a combination of an art and a science and by running lots of experiments you start to build an intuition for the types of things that work and don't okay what were some of the early tricks you found that like made you go super viral you were pretty famous back in those days yeah I mean you were at Facebook back then right so I think Facebook created new rules after I
found some loopholes almost oh yes yeah there was one I remember I think the most viral Facebook app I'm hesitant to even admit this but it was called zodiac photo album and the way it worked was literally with one click it would create a photo album with 12 photos in it one for each zodiac sign and then show all of your friends who had that zodiac sign in each of the 12 photos photos and tag all of them and I think we were allowed to tag like 20 people per photo so it would show up
to 20 of your friends per zodiac sign times 12 so with one click you tagged 240 friends and all 240 people would get an email saying I'd just tagged you in a photo so that has a very high response rate people kind of want to see you when they're tagged in the photo they would go they'd see oh it's a Zodiac photo album create your own at you know the zodiac app within I think within 24 hours we had tagged tens of millions of people in photos income I think including Zuck and then I heard
the platform team shove it down shortly now the wonder of those days are over of kind of those crazy viral pops well and then gue stop you were really central to a lot of the early experimentation on mobile I remember we first met when boxer was blowing up and you guys had figured out some really unique ways to blow up using notifications and and inviting can you kind of remind us like well how did you get to this world of growth and how did you kind of figure out stuff in those early days especially on
mobile which people think is a lot harder yeah and social was so before boxer I was sad come to call hey Sam with co-founded and it was a product like a messaging product on mobile we couldn't grow through spend a lot of time trying to learn how to grow when I joined boxer a couple years later the first kind of approach and first thing I kept in mind is like we need to think about growth and at the time I'm so back to the earlier questions I think there's a couple of like growth kind of
tape becomes important and one to have new platforms so you have Facebook platform you have the mobile platform you have Google SEO there's a bunch of larger platforms that makes people actually care about investing in them at box or the iOS platform which key to those things were the address book and push notifications those are things that kind of like warrant there and very beginning once they start happening people started exploring where you can do with those things with boxer when I joined the most important things about the walkie talkie app on iOS and Android
you push a button you start speaking in the other in the other end we found that the amount of friends that you have when you open the app that was a very first time on the app and how you display them was absolutely key to whether they were going to use the app or not so we spent basically the first year optimizing that those those things like basically how many people do you have when you open up the app - the first time and how do you get to a list like that right when you
open the app well you match phone numbers with address books spend a lot of time investing in matching phone numbers letters books not that different from Facebook platform and we kind of try to optimize how many address books that many phone numbers for different people and we're doing this in the best interests of the users like you wouldn't end up using the app unless you actually saw friends on the app so that was basically the goal this there this was all in the best interest of the users and sometimes you forget like why you do
things so and I remember your company got acquired by Facebook and you kind of jumped into a little bit more mature growth team at Facebook you've been kind of virally hacking and building these things on the platform what was sort of the adjustment to go from kind of being in a sort of experimentation mode to something that was a little bit more mature but what did you see when you observed kind of that mature Facebook growth team yeah so well one thing I saw was I was finally working at a company that had an amazing
product that people loved and I just never actually built that myself before so that was pretty cool and with that came amazing retention amazing engagement and Facebook was already at a size where retention and engagement were bigger drivers of the Northstar metric and they use than new user acquisition so kind of everything I had worked on in the past was all about how do you get those first million users at Facebook it was how do we keep the users we've already acquired keep them engaged minimize churn and to me that was one of the biggest
differences and most of the growth team at Facebook at the time was actually focused on friending because we discovered at Facebook that friending was the number one driver of retention and engagement so then there were teams on the Facebook growth team that were basically focused on things like pym Piquet people you may know and and other other ways of causing people to form more connections so I think that was one big takeaway the other one we could talk about it later is just the way the team was structured I know got into that a little
bit across like their different functions as the team grows actually can you walk through that I was actually good time and kind of thing about how would that structured then I want to get into this transition to these sort of different real world companies sure sure the real world companies yes well so one thing I learned at Facebook was I really liked how the growth team there was structured with all functions so product engineering analytics design and and even marketing so those five functions were all critical to work together and again all within that growth
org and when I was at Facebook I think the growth team was about around 500 people or so oh wow and but across all those different functions in fact all of all of analytics back then was part of the growth team and I I kind of took a lot of those learnings when I went over to Eber and started our growth team there was just initially a five person team kind of picked one person for each function but I I think that was one of the big takeaways from Facebook well and you know that's a
perfect transition because when I've been talking a lot of people about sort of what the real success of Buber and Airbnb I mean Joe gave a great talk about sort of why Airbnb is become so important the world but a lot of people have said it's kind of we took these hotels and transportation which were sort of old world real businesses and applied a lot of these digital growth technology and learning to them and so kind of talked a little bit about what it was like going into Airbnb and starting the growth team with two
people and uber and starting the little growth team and like what did you see there and how did you kind of start that this sort of practice from what you learn before I'm sure so I think the kids beef red blood okay I think the initial ID to have a growth team is that there is some function of the company that really cares biometrics they're really optimized for the goals and some time ago like this was unusual in startups as most people did not look at metrics when they made presentations that did not experiment so
instead of making everybody responsible that you made one team responsible for that and that ended up being the go team I think really great companies today they have everyone kind of caramel matrix using experimentation and doing all those things that go teams do to make decisions so I think things have changed in a little bit and the role of the growth team ask kind of the team that and evangelize those things have changed a little bit when I joined Airbnb a lot of people join at the same time that experience from from from all those
things so pretty quickly I got a lot better set of tools than I had ever had a boxer around experimentation around support from data science and that just I would say in some sense made like all the infrastructure of doing a growth team that it would be like really really easy compared to my previous experience and I learned a ton from all those people it's been pretty pretty incredible but yeah at the very beginning we like everyone else had everybody had always been an incredible product and as a company we along those years have learned
to kind of identify and optimize every single step of that experience a lot of its off work of course offline but of the stuff that we do online a lot of things that we can do to just like measure and and and make the user experience better that's what we've been doing but I think yet the role of the girl team have changed from being that evangelizing team that that do all of it to kind of analyze him to wrestle the company so everyone to drop a new product in a very similar way I'd say
at over there were kind of three big differences from Facebook when I went over there in terms of forming our initial growth team so the first was it's a two-sided marketplace and when we came up with our Northstar metric I remember having conversations with that small team of five one of whom Mike Tao is sitting in the audience can you define Northstar metric you've used that a couple times I that's actually really important for sort of the growth discussion definitely so at Facebook it was monthly active users it's basically the one metric that if you
talk to the way I like to think about it is if I talk to anyone on the growth team and ask them what number are we trying to grow they would be able to say it's that number and if they're not working on something that ultimately is growing that number they're probably working on the wrong thing so that's kind of how I like to think about it and it's helpful to just set goals you know we've said every every half we kind of have this is our six-month goal but then we track our progress every
week and see are we on track and there's of course seasonality and stuff it's not just a straight line but you could kind of see like how are we doing versus our goal so at uber when we came up with our Northstar metric we had to consider their two sides of the marketplace so it's not just monthly active riders ultimately we decided on trips weekly trips as the Northstar metric because for every trip to take place you need riders and you need drivers and so by having trips of that Northstar metric you could then break
that down into the supply side and the demand side and you need more riders more drivers and you can do growth accounting on each side of the marketplace to say well to get more riders you need more new riders reengage turned rider and and you know minimize churn of your existing riders and same thing on the driver's side so that was one thing was kind of agreeing on this is our Northstar metric in a two sided marketplace so that was a bit different from Facebook I'd say a second thing was because it was goober as
a real-world company there were several parts of that funnel that you can't just measure online or is on Facebook you can track every every click every pixel all of that stuff with uber there are steps of that funnel where it involves you know a driver going in and getting their vehicle inspected or uploading their documents and ultimately downloading their iPhone app going into their car and doing that first trip and you know for me it was super eye-opening when I did my first trip as a driver I walked out to my car I done everything
I was activated I walked out to my car I walked back to my house because I got so nervous about doing that first trip and that's when it suddenly hit me this is why you know so many of our first our drivers who are activated never do that first trip so that that real-world component added kind of some extra challenges in terms of one how do you measure it and two what are some things you can do to influence the way people behave in the real world and then the C I said three things and
now I mentioned two so two sided market place the the real world component to it and I'll come back to the third one I remember did your different orbit our metrics well and so as these teams have scales now them you know way up you know you talk about data analytics design product sort of internet marketing oh that was the third by the way paid acquisition that's what I was going to actually ask so is Facebook we didn't really do very much paid acquisition but at uber we had an entire performance marketing team well initially
it was a one-person team but ultimately became a team where we would actually spend money to acquire both riders and drivers one x-acto in our social world we're all about these organic how many people can we organically get into our products and if we spend a dollar on user acquisition we always said a dollar too much and you know the boards are like how do you get more free virality but I think you know both uber and airbnb because your businesses people actually pay for how do you think about paid acquisition differently than this sort
of organic exhibition how do you train a team to sort of understand that and think about that differently do you want to start going oh sure so let's see I would say a lot of those platforms that you are doing growth on Google Facebook they are turning kind of the opportunity of doing free growth there is getting smaller and opportunity of doing paid growth better is getting larger so most good growth teams should invest in paid marketing I think that and Mike's plans are seeing looking at other companies what they've been doing large companies to
do paid marketing that tend to integrate that with the product team and engineering teams so there are engineers and product managers supporting those teams they called a tech person like that and I'm sure what the uber name bees is at that stage right now where basically most of the leverage you'll have at that scale is going to be through engineering data science and product management that's a different world than when you start off and start off you have a smaller smaller team and yeah it's probably yeah I need somewhere I'd say the way I think
about paid acquisition is it can just further accelerate growth so uber would have kept growing even if we had never done any paid acquisition but especially in a competitive market it kind of makes sense to pay up to what the writer that driver is ultimately going to be worth when you're in a situation where the faster you grow the better because as you get bigger and bigger there are a lot of other efficiencies that kick in so the whole network becomes more efficient once you have more liquidity so there's value to getting there as quickly
as possible it's kind of tricky to know exactly how to calculate the LTV of a rider or a driver and a two sided marketplace because you know the riders pay the drivers and some of that money goes to burrs so how much is the rider worth versus a driver worth and we had a team actually focused on that exact question team of data scientists and it was more complicated than I understood but basically there's certain markets that are more supply constrained or a driver the incremental drivers worth a lot more other markets that a certain
time here might be more demand constrained where you might be willing to pay more for the rider and that varies over time and by geography and the quick comment color on that I would say most companies that grow really really large they do it on figuring out one platform so they are really good at online marketing they're really good at SEO a little bit of a rally or the Facebook platform or something like that it's rare that you have like I figure out all the platforms so I would think about the proxy someone has the
product that you have and which of these platforms does is it the most suited to most suited to search is the most two to display is the smoke Masuda tube rally it really depends on the park I would say interesting and where do you guys see any upside right now in sort of the the marketing or growth world I mean it seems like a lot of these channel to gotten pretty saturated Facebook's gotten relatively expensive to be buying on Google's various platforms and video yeah I've heard a lot of people talk about video ads or
sort of the new place even people are actually going and advertising on TV you know where do you guys just think about sort of where those those growth opportunities sit out there now I mean I come back to build a great product because if you have that product market fit and the people using your product love it but all of the channels are out there today that will allow people who love a product to share it with their friends so I think that's actually the most important thing and then everything else will follow yeah I
mean I would I don't think there's anything short-term that will remain like successful for a long time so the best advice is invest for the long-term whether it's SEO whether it's paid marketing with a store ality take like a couple of years look out and see and I want to be a name for uh like like look for channels that actually do have to scale like channels that have like hundreds of millions of people discovering products through those channels and then invest for the long term and engineering data science is critical to be successful in
my opinion I'm surprised neither of you mentioned Instagram the tip I got yesterday at f/8 talking about to Facebook focuses I think Instagram app install ads are significantly under priced relative to Facebook so you guys go back and start buying on Instagram you might be so surprised how does so one of the unique things again in the social viral world we kind of grew scaled on the internet you guys also have to grow city by city what are some of the things that you did to kind of set up teams for growth that could actually
do growth in a city that sort of that could sort of customize it for the city or how much did you try to make it sort of a centralized playbook that you'd control from the mothership so I mean every city would have its own City team beginning with a launcher again I mentioned Mike in the audience he was a launcher than was our general manager in Boston and then started basically headed up product for the growth team so it was really valuable to have the person who was running product for growth be one of these
X operations people and working with all the city teams was really important in terms just making sure that product and operations were integrated and uber we now have a team at uber called product ops and it's basically a team that kind of is that bridge between the product team and the operations team but as much as possible we tried to just have a playbook and you know one one thing that's helpful with uber being a global business and I'm sure you guys see it as you as much if not more so at Airbnb is when
we would launch in the new city there'd also be a lot of there would already be a lot of pent-up demand we'd see a lot of people trying to use uber in a city before we launched in that city so there were certain things we could do to kick off that flywheel but I'll share one example of something where we actually did some something very different to the product in another city and this was back when we were in China we were we found that it was hard for drivers to sign up to become a
bird rivers in China because of the firewall there were like latency issues and just like signing up online and a lot of drivers had phones but didn't have computers and like it was just hard for them to sign up so one of the engineers on our China growth team built a WeChat thought WeChat you know like a message the messaging app or the Facebook of sorry the Facebook and also the Internet of China really and so we built this WeChat 5 where as a driver you could take a picture of a QR code and then
it would just start asking you questions what's your name what's your birth date what's your driver's license number and people would quickly be able to sign up to become a driver and we'd have these onboarding sessions with hundreds of drivers if not even more than that showing up in one room all on WeChat signing up which was amazing until we Chad started blocking us because they were owned by Tencent which had invested in our competitor so that's one example of something we had to do very differently in a different market I would say the two
things that would call out one is translation so absolutely key to have a way that you can translate content across languages over no profit over over looks and it's like huge growth driver like a lot of people don't speak English and need the experience that you're offering in different language the second thing is weed what I call good product gaps so if you look at your product globally there are some parts of the product that won't work in some countries whether it's authentication like Facebook connect with its payments mechanisms search SEO and just like identify
those gaps they're not that many countries in my experience that are dramatically different there may be five or ten that are quite different than us but most countries are not that different cool and just lastly because growth is all about experimentation and often experimentation against instincts what is one great case where you and your growth team like really either believed something was going to make a huge impact and then we're disappointed when it didn't or where someone just did an experiment and even you you were like oh this is never going to work and then
it like blew you away so we have actually instituted institutionalized this thing I would call it the experiment review every few weeks the gurth team gets into a room and we show experiments that we've shipped and gotten significant results on in the last last month say and the engineer will go onstage shield experiments show the control and the experiment before they show the answer they'll ask the audience what do you guys think I'm gonna hear things control how many think experience experiment and it turns out that we often disagree most of the times we disagree
which is exactly the point you're trying to make by doing experimentation that predicting product decisions are really it's really tricky so I would say this happens all the time and if it doesn't happen that you're not taking big enough risks you should be seeing there things are counterintuitive sometimes when they're very kind of intuitive having experience of seeing likes permutations makes you ask the right questions oh maybe the metrics is wrong this time that happens as well but yeah things should be really counterintuitive give one to bid and want to advise if you have a
website and you had people come to your website and you have accounts don't lug them out like keep people log in for as long as you can there's really no benefit in automatically logging people out after after a couple months I have a couple days in terms of growth so because I would I would look for Amazon for some advice and what have they done things so I mean one thing I found interesting is often it's the smallest tweaks that can make the biggest impact and there's not necessarily correlation between how much work you put
in and how much of a step change it makes or sometimes an inverse correlation so I'll share one interesting example from Facebook we were looking at the the sign up basically the funnel and the viral metrics by region and we noticed that Japan was our least viral country and when we dug in and tried to understand why we saw that very few new users were actually sending invites to any of their friends and we had like a small team in Japan I spoke with a country manager there and he said you know people just think
it's rude to send so no one's going to it's like a cultural thing but people don't want to intrude on their friends and send invites so we brainstormed a bit and we decided to literally make a copy tweak and instead of calling them invites we called them announcements and we would say as part of the you know signup flow you'd import your address book to find your friends and then instead of saying invite all your friends we said let's Facebook now announce to all of your friends that you're on Facebook and this is where they
can find you and Japan went from like our least viral country to our most viral laundry but it was literally just a copy tweaked to to make people more comfortable with sharing what the product they were using with their friends that's great I've been talking to some mobile developers that even the difference of invite versus add somebody which is sort of vague or if you press if you change your language from invite to add people will send the SMS is out to their friends to invited them to a service at a much higher rate than
the word invite so these really you know subtle things they don't expect they have a time for like one or two questions from the audience if anybody has be okay and you talked about the North Star and you said it over it was about the number of rides but you could have easily been optimizing for crazy retention as zero growth and new user growth or new rider growth or all new riders growth and to news last comment like horrible retention so how did you balance out knowing what you're almost subs Northstars were so that you
could have the best north start that mean totally yeah that makes a lot of sense and it actually changed over time so early on we were kind of in land-grab mode where it was all about new user acquisition new riders new drivers but we would look at growth accounting so you'd say well the total number of active riders or active drivers equals like last week's active number plus any new users you get - an evil users that turned plus any users that re-engaged and you can look at each of those three things kind of the
magnitude of each and early on and the growth curve when you're kind of at the bottom of that s-curve the new user acquisition number is probably going to be much larger than like the churn or the reengagement and so it makes sense to focus the most on growing that number but as you get bigger and bigger your as your active user base gets larger the churn as an absolute number goes up because it tends to remain like a you know constant percentage or roughly like constant percentage of the absolute number and then at the same
time new user acquisition gets more difficult because you've already reached out to so many new people so if you just monitor look at the growth accounting and kind of see like those are the levers which ones can have the biggest impact early on it might be new user acquisition but later on it's probably going to be retention and reengagement but that said you kind of need that retention from the beginning because if you're just like I've run plenty of things in the past that haven't had retention and it's fun while it's going like this but
if that retention is not there ultimately it's not really going to help that like zodia calendars exactly it's that kind of stuff you earlier you told a story of like the challenge of getting people to go from registering as a driver to actually like overcoming the nervousness to go out and do the first trip as a driver interested in what are some of the experiments you ran there what you learned about how to actually get people to do that yeah Wally definitely so you know I'll just mention one one example is we saw that sometimes
like a driver a new driver would log in and might not get a trip request right away and sometimes that driver might just be waiting for a really long time because they're a brand-new driver and maybe they're waiting somewhere maybe they are waiting in front of their house and there just isn't them but bunch of stuff happening right there or there's another driver who's more experienced and who knows if you wait a block away from here they're more likely to get a request so like one of the experiments we've we ran was what if we
make it like make sure that that first experience is better are there things we can do to kind of get that get that driver hooked so that that's one example you know we also experimented with are there ways we can get potentially existing drivers or other people to like get on the phone with a driver and talk them through that that first that first trip and so the team's experimented with a bunch of different things like that trick just to kind of probe on something new stuff alluded to regarding growth and organizationally do you guys
think that the right study state outcome is for all teams to become growth oriented and growth people inside of them or to have a separate standalone team that's kind of fully vertically integrated and if the latter how what conflicts have you seen between say marketing you know who's focused on owning the homepage and the language and the brand kind of goals or life cycle who thinks that they under tension versus growth and likewise with product groups that feel like they own a product and a customer experience that they're trying to get with you guys kind
of hacking away at that so I would say that the ultimately the company becomes a growth game in the sense that you use data to make your decisions like you don't make you basically look at the data because it gets so hard to predict what are the right decisions to make like once you make an optimization so ultimately at every be all the larger teams are using data to validate experiments and that's kind of the idea of a growth team for us means we're using the funnel yura on the top of the funnel you're you're
finding users on the they're not already using ever be in other companies growth teams is a smaller organization that kind of evangelizes that to their sort of product team but everybody really the entire crack team is now a growth team and the goal of the data team that had to build these tools is to get everybody to use them like they are literally measured by how many people using experimentation frameworks we actually merged the product team and the growth team about a year ago at uber for largely for those reasons especially as we started focusing
more on engagement and retention within the growth team and that's kind of what the core product team was working on as well it just made more sense to bring it all together now we just added to talk about your lifecycle and your homepage copy teams and everything and brand they actually when they kind of merge together and they sort of have the orientation we're really growth and marketing or sort of one thing as opposed to as opposed to two you understand sort of everything we do is in the spirit of growth even if it is
driving our brand or other stuff you know rather than trying to keep them separate and contentious is for them with that merging a good time for one more question what questions for a super aggressive in China and growing where any really crazy things you guys discovered our hacks there that maybe people could discuss people could use back here listen make sure there were but I probably can't I mean that's actually I don't have any like super crazy ones that are coming to mind right now but I probably shouldn't talk about any of the stuff that
worked really well about this given that we're probably still doing it if it did okay sorry the awesome well thank you hopefully you'll be scaling your orgs and your your user bases together think okay ed Andrew stop a lot [Applause] you
