Keynote: How AI-Powered Personalization is Redefining Lifecycle Marketing

it is now with great pleasure that I get to introduce George katrian co-founder and CEO of offerit George is one of the Brilliant Minds in our community he has a PHD in mathematics from Cornell he was at McKenzie before co-founding offerit his presentation last year on chat GPT and automated experimentation was a huge hit and we're thrilled to have him back to talk about how AI powered personaliz ation is redefining life cycle marketing please join me in welcoming George to the stage So today we're going to discuss how AI powered personalization is redefining life cycle

marketing and in particular we're going to talk about the traditional approach which is called next best action and what are some of the problems and limitations with that and how given recent advances in AI That's getting replaced with a new approach called AI testing so so what qualifies me to talk on this subject I'm the co-founder and CEO of offerit we're a tech company that provides AI testing software for marketers um we have a lot of amazing customers Brands like Yelp MetLife and other great companies we have a team of about a 100 folks and

growing more than two times a year so I find that for a lot of marketers Ai and machine learning is is like a black box even if you're using it currently in your companies maybe with custombuilt models or through vendors that you work with um it's hard to get information as a marketer of what's actually inside these models and how do they work so my goal today is to demystify that a little bit and open up the black box for you as relates to AI machine learning for making decisions for identified individuals in the context

of life cycle Marketing in particular we're going to cover four things so first just to level set what is machine learning anyways and what are the four main types of machine learning then we'll talk about next best action which is the traditional way to use machine learning in order to make personalized decisions for for customers and then finally we'll talk about what's what's starting to replace it which is AI testing and how that works this all goes back to classical statistics which was mostly developed in the first half of the 20th century and so here's

an example of something you would you would do in classical statistics you have a bunch of historical data imagine each point is a group of customers like a cohort that all started with your company at the same time and you know for them how long have they been your customer and you want to predict of those people What proportion are going to repurchase your product within a time frame so it's like a propensity to repurchase and so In classical statistics you're going to draw a line you'll fit a line to this STA and so you

can think of that line as learning you've taken this historical data that line that you fit is learning from that data and it allows you to make new predictions so for a new group of customer that maybe wasn't in your historical data or an individual you can now look up on the xais how long have they been a customer and then you can connect the dots and see okay what's the propensity to repurchase for that individual and so that's classical statistics but of course in real life data doesn't always look beautifully linear sometimes it does

squiggly things and so in those cases you can still fit a line but you can tell it's maybe not not the ideal fit right in this case ideally you'd want to do something that maybe looks more like this classical statistics didn't really focus on this and the reason for that is because to fit this type of shape to data you need to do a lot of computations way more than you can do with pencil and paper or a slide rule and so forth and so as a result classical statistics was focused on linear situations that

all started to change in the 1980s and 1990s a lot more compute became available and so it started to make sense to actually do research and figure out how to do this type of fitting and that became known as the field of machine learning and so why is it called machine learning just like our line learned from the data and then made predictions here we're doing the exact same thing we're learning from data but we're using the power of computers machines in order to do all of those computations that are needed in order to do

you know more complicated things like this now of course in real life you don't always have one variable on your x-axis that you're predicting on you might have multiple ones like you might know how long has someone been a customer how many purchases did they make in the last 90 days are they in the Loyalty program maybe 100 other things and then you're going to put that into a linear model in your classic statistical situation and you're going to Output a propensity to repurchase with machine learning it's the exact same idea you're taking that same

data about people but now you're putting it into a model that's nonlinear so it can find more complicated relationships between your input variables and what it outputs and then it's going to output something and if done right that's going to allow you to be much more accurate so that's really the benefit of machine learning is it allows you to get much more precise and so you've probably heard about some of the particular ways that people build models that do this they have fancy names like neural networks or gradient boosting machines or support Vector machines but

essentially it's a very simple idea it's just taking input data and then in some nonlinear way generating an output in order to fit certain certain data that you may have so that's really all that machine learning is it's just a generalization of good oldfashioned classical statistics from linear situations to doing more complicated things in the search of better Precision now um within machine learning there are four sub branches that correspond to different uses of machine learning and these are prediction pattern identification generation and decisioning they have more fancy names that are used by Machine learning

practitioners and all of them have applications in marketing and probably things you're familiar with so like in prediction you have things like propensity to repurchase propensity to turn category Affinity these are all predictive models in pattern identification of course lookalikes many of you have done that you take a group of customers you pass it to Facebook you ask them to find similar customers so you can Market to those people that's an example of pattern identifying machine learning generation we've all played with chat GPT uh or generating images that's that's you know generative models and then

finally decisioning so these are machine learning agents whose whole purpose is to make decisions they're not making a prediction about a customer they're actually deciding what action to take in order to maximize some kpi like revenue or conversions and an example of that is is AI testing and this this fourth area is actually offer fits Focus so let's talk about the traditional next best action and what I think I'll um hopefully show you is first of all just how does that work many of you maybe have either used next best action at your companies or

you've heard of companies that use next best action so when you hear that what does that actually mean and then what are some of the limitations or problems with this with this traditional approach so in next best action you start with your customer base and then you build a collection of predictive models so you're using that first bucket of machine learning so for example things like propensity to churn propensity to repurchase category affinity and so forth and so you end up with a score on a scale of zero to 100 for each customer among your

customer basee and so already you might see a problem which is that prediction is not the same as decisioning you have a score for each person but that doesn't necessarily tell you what you need to do in order to maximize the performance of your campaigns with that person so my favorite example here is that I'm I'm the father of a 2 and a half-year-old and so my wife and I buy a lot of diapers I have a very high category affinity for diapers but that does not mean that you should Market diapers to me because

how many diapers we buy is going to depend on my son's digestion that month much more than on which which emails I get from from a diaper company and so there needs to be something a bit more complicated than just saying you know high on high on this so do this to this customer and so how do how do people solve this in the next best action framework well you take your customer base in these scores and you divide people into segments so in this case you have propensity to churn and you group people into

five buckets based on their score and then same on category affinity and you end up with 25 groups of customers in that grid and so each group is a segment and then you're going to have some kind of rule of how you Market to that segment so for each of those 25 groups you're going to do something different so there's a problem with that which some of you may already see which is that what it means is that if two people end up in that same segment we're going to Market to them in the exact

same way and so initially we might feel like we're using all of our first-party data you've taken this very rich data that you have you've passed it into these machine learning models it's great we're doing AI we're doing ml we're using all of our first- party data but there's a problem we've taken it and now we've boiled people down to these segments and then after that we're not looking at that data anymore so let's look at an example Benjamin and Michael are both in the same segment they both have high propensity to churn and kind

of mid-range category affinity for baby products but it might be for very different reasons Benjamin is a grandfather he's a bit of a doting grandparent he buys a lot of toys for his grandkids so hence the category affinity and he recently had a bad customer service experience and so he's frustrated with with the company and he's thinking of taking his business elsewhere because he's been a loyal customer and he's dissatisfied that he wasn't treated accordingly Michael is a college senior uh he's likely to churn because he's going to graduate and move to a different city

um and in his case he he did some babysitting in college to supplement his income and so as a result he has some some category affinity for baby products so the action that you need to take for these two people is actually not identical ideally what Benjamin needs is a phone call and an apology for the bad experience so that he feels valued as a customer what Michael needs is maybe wait a few months and then send him a coupon so we can get him in the habit of shopping with us after he moves to

a new city but in this next best action approach we don't have the ability to do that because we've thrown away all of that rich data that we have but we do have that data we know maybe you know how long have they been a customer responses to satisfaction surveys so we maybe know that Benjamin had a bad service experience because maybe he told us himself basket sizes channels purchase frequency email interactions web browsing Behavior but in this traditional next bestest action approach we're really not using any of that apart from putting people in these

buckets based on their their scores the second problem you can probably tell here is some of the ways that you can segment people are informative for things like product or incentive but there's other decisions that as a marketer you need to make channel frequency send time time of day and so something like your term propensity isn't really informative on those things and so again when you do this next best action you're sort of stuck not being able to decision on those things unless you have separate grips GS for each thing that you're decisioning on and

it just becomes enormously complex so that's all the first problem with the traditional next best action it's just not very personalized but there's a second problem and so we kind of took everybody we divided them up into these segments and then we're going to have rules for each segment well where do those rules come from and so in many cases it's kind of a conjecture right well we think maybe for the segment this is the right thing to do and I like to call that it's not next best action it's kind of next best guess

and and so for marketers when they're more sophisticated about this you can actually do a bunch of AB testing and this is what companies that have very large marketing teams and that are very advanced in doing this will do is for each of these segments they'll do a bunch of experimentation to figure out what is the performance maximizing thing to do for that segment of people so here's an AI generated image of someone waiting for one of those AB tests to reach statistical significance and as you can tell the the more fancy you try to

get with this in the smaller your segments the longer you're going to have to wait for each of these tests the harder it is to do this experim exp mentation work and every time you want to change something you're going to have to redo an inordinate amount of testing so that's why these these models when they're used become very very cumbersome to maintain so from an AI machine learning practitioner's point of view this is all kind of crazy and the reason for that is you've basically taken in next best action predictive models and you've shoehorned

them to do decisioning through this very cumbersome system of segments rules and ab tests there actually exists a branch of machine learning whose whole purpose is decisioning these are models that are natively built to do decisioning um but of course this is a more recently developed field it's more complex in certain ways and so that's why certain companies do the traditional approach to next best action still today it's because it's more accessible in some ways but it's quickly becoming becoming obsolete so let's talk about decisioning and in particular AI testing and how that works so

for these models their output is not a prediction their out is actually decision but their inputs are the exact same so you pass all of the first-party data and then for each customer AI testing agents will output decisions of what incentive to give them um what channel to communicate with them through uh what time should you do that and they're doing this with the aim of maximizing your performance and so this this process in marketing can be described as AI testing because you're using these AI agents to essentially replace the whole testing and decisioning process

how does that work in practice well either with offerit or with any other AI testing tool that you use it becomes a new point in your Tech stack between your data systems and your marketing automation tools and so your customer data gets fed into your AI testing agent and the AI testing agent then makes decisions at the individual customer level so for Veronica send this email at this time with this subject line in this incentive for George uh send this text message with no incentive at this time for somebody body else do nothing today then

that all gets executed executed through your marketing automation tools and it's a continuous loop so several days later the AI the AI agent checks back and it looks what happened with these decisions I made these decisions which of them led to good outcomes in the form of Revenue or conversions or whatever you're seeking to maximize which of them did not and that allows them to get smarter at finding out for each customer based on their characteristics what are the actions and tactics to be using in order to maximize the outcomes of your campaigns and so

let me just give you a couple of examples to bring this to life uh one of them this is a customer of offerit is a leading credit card company and one reason I like this example is because they're known to have one of the most sophisticated marketing teams out there and so the journey that they started using AI testing for was already very thoroughly AB tested and optimized and it was the credit card refer a friend for small business cards and so they already knew if you're going to be emailing everybody the same thing what's

the best time of day day of the week subject line and so forth and so the reason they deployed AI testing is to use their first-party data to personalize much more strongly to where for each person you're now making not just you know whatever is best for everybody on average generically you're actually sending what's best for that individual and so in their case they ended up nearly doubling the performance of that Campaign which was already very well optimized and that amounted to 16 million of annualized bottom line value which as you can imagine is a

big deal for their business so they're doing a big roll out right now where they're going to be deploying AI testing across multiple points in our customer Journey another example is a leading review app which many of you may have on your phone and so in that case they already had a very sophisticated data science team they were already optimizing push notifications how do you maximize people's engagement with uh with their with their reviews and so they deployed AI testing to further strengthen the personalization so that for each person you know who do you send

a push notification about plumbers versus nightlife what time of day what day of the week how frequently so you avoid people uninstalling the app or silencing the push notification and got over 50% uplift and pushed session conversions so this can be quite quite impactful when uh when when used correctly so the way that I like to think about this is it's it's kind of going with AI testing from next best action to next best everything and the reason I like to describe it that way is it's next best everything in two ways first of all

you're now actually using all of your first-party data with traditional next best action you sort of have the illusion that you're using all of your first party data and in a way it's true you're feting it into these predictive models but then when these models output scores you end up boiling people down to segments and you're you're not really differentiating between people that fall in the same segment and so as a result with next best action you have the sense that you're using all your first- party data you're not really capturing the richness whereas with

AI testing because these models are built to be natively decisioning you are and the second reason relates to the output with next best action you construct some segmentation maybe it'll help you choose something like your discount level or which product to talk about but there are all these other things that you want to be decisioning on frequency time of day Channel and so forth and with AI testing it lends itself much more naturally to this than trying to construct you know multiple next best action structures to solve this so what are some of the things

that we that we cover today just to recap first of all machine learning it sounds very mysterious it's actually just a generalization of classical statistics it's just good oldfashioned statistics that we all love generalized from linear relationships to be able to do more complex not necessarily linear relationships all in the name of better better accuracy uh there are four main types of machine learning they all share a common toolkit but they also have very distinct tools and capabilities that have been developed for each of these four types uh which is prediction pattern identification generation and

decisioning and the traditional next best action approach even though you often hear about it in in concepts of like oh this is very Advanced this is very sophisticated well that was probably true you know 10 20 years ago but in fact it has a lot of issues because it's taking um predictive machine learning and shoehorning that to do decisioning which it wasn't meant to do through this intricate system of segments rules and ab tests and in fact so as a result it ends up being very slow very cumbersome to maintain as a marketer and also

just not very personalized and then finally this is being replaced with recent advancements in AI with AI testing which is about using decisioning Machin machine learning so these are AI models AI agents which are built to be making decisions not just to be making predictions that's what they're natively designed to do and so that has two main benefits the first is you end up making much more personalized high quality decisions because you're truly using all of your first-party data and then second of all it's it's a much more elegant setup it's not so cumbersome to

maintain with all of these segments and rules which even for very large companies is hard instead with AI testing it's a much simpler setup because you have these Ma learning models and they make the decisions and if once you set things up it's uh it's it's a it's a simple robust configuration so uh thank you very much if you're interested in learning more about this feel free to drop by our booth we're we're here on the third floor in Booth 201 and also for qualifying companies we offer something called AI Academy if you're interested in

learning more or if you think some of your colleagues want to go go in depth and so at no cost we have a team of two instructors that will come to your office and run a one-day Hands-On Workshop to discuss um recent advancements in AI including AI testing and so if that's something that potentially is of interest to you and uh and your colleagues feel free to talk to us about this or go to go.of fit. aademy thank you very much