[Music] quest hello i'm josh starmer and welcome to stat quest today we're going to talk about gaussian naive bayes and it's going to be clearly explained note this stack quest assumes that you are already familiar with the main ideas behind multinomial naive bayes if not check out the quest the link is in the description below this stack quest also assumes that you are familiar with the log function the normal or gaussian distribution and the difference between probability and likelihood if not check out the quests the links are in the description below imagine we wanted to predict if someone would love the 1990 movie troll 2 or not so we collected data from people that love troll 2 and from people that do not love troll 2 we measured the amount of popcorn they ate each day how much soda pop they drank and how much candy they ate the mean for popcorn for the people who love troll 2 is 24 and the standard deviation is 4 and a gaussian or normal distribution with mean equals 24 and standard deviation equals 4 looks like this likewise the average amount of popcorn for people who do not love troll 2 is 4. and the standard deviation is 2 and that corresponds to this gaussian or normal distribution now we calculate the mean and standard deviation for soda pop for people that love troll 2 and draw the corresponding gaussian distribution then we do the same thing for the people that do not love troll 2. lastly we draw the gaussian distributions for candy gaussian naive bayes is named after the gaussian distributions that represent the data in the training data set now someone new shows up and says they eat 20 grams of popcorn and drink 500 milliliters of soda pop and eat 25 grams of candy every day let's use gaussian naive bays to decide if they love troll 2 or not the first thing we do is make an initial guess that they love troll 2.
this guess can be any probability that we want but a common guess is estimated from the training data for example since 8 of the 16 people in the training data loved troll 2 the initial guess will be 0. 5 so we'll put that up here so we don't forget likewise the initial guess for does not love troll 2 is 0. 5 so let's put that here so we don't forget oh no it's the dreaded terminology alert the initial guesses are called prior probabilities now the score for love's troll 2 is the initial guess that the person loves troll 2 times the likelihood that they eat 10 grams of popcorn given that they love troll 2.
note the likelihood is the y-axis coordinate on the curve that corresponds to the x-axis coordinate and we multiply that by the likelihood that they drink 500 milliliters of soda pop given that they love troll 2 times the likelihood that they eat 25 grams of candy given that they love troll 2. the initial guess that someone loves troll 2 is 0. 5 the likelihood for popcorn is 0.
06 the likelihood for soda pop is 0. 004 and the likelihood for candy is a really really small number note when we get really really small numbers it's a good idea to take the log of everything to prevent something called underflow the general idea of underflow is every computer has a limit to how close a number can get to zero before it can no longer accurately keep track of that number when a number gets smaller than that limit we run into underflow problems and errors occur so we use the log function to avoid underflow note any log will do but the natural log or log base e is the most commonly used log in statistics and machine learning so we take the log of everything and the log turns the multiplication into the sum of the individual logs the log base e of 0. 5 is negative 0.
69 the log of 0. 06 is negative 2. 8 the log of 0.
004 is negative 5. 5 and the log of this really really small number is negative hundred fifteen now we just add this up and we get negative one hundred twenty four so the log of the love's troll two score is negative one hundred twenty four bam now let's calculate the score for not loving troll 2. we start with the initial guess that someone does not love troll 2 times the likelihood that they eat 20 grams of popcorn given that they do not love troll 2 times the likelihood that they drink 500 milliliters of soda pop times the likelihood that they eat 25 grams of candy so let's plug in the numbers and take the log of everything and that turns the multiplication into the sum of logs now we just do the math and we get negative 48 and since the score for does not love troll 2 is greater than the score for love's troll 2 we will classify this person as someone who does not love troll 2.
double bam note when we look at the raw data it almost looks like we should have classified this person as someone who loves troll 2. after all they ate a lot more popcorn than the average person who doesn't love troll 2 and they drank as much soda as the average person who loves troll 2. however the big thing is that they ate a lot more candy than the people who loved troll 2 and the log of the likelihoods for candy are way different and this difference is what made us classify the new person as someone who does not love troll 2.
