Hey, it's Zara. In this video, I'll be talking about neural networks. Before we dive into neural networks, let's talk about machine learning in general.
Machine learning is a branch of artificial intelligence, or AI, that focuses on building systems that can learn from and make decisions based on data. I'm sure you're hearing AI way too often these days, and you're right to think it's becoming more of a marketing term. Some even argue that John McCarthy, who coined the term in the 1950s, intended it to be a marketing term to attract funding.
Anyways, unlike traditional programing, where we explicitly define the rules and logic, machine learning enables computers to learn patterns and make predictions without being explicitly programed for specific tasks. Let's illustrate this with a real world example. Spam email detection.
Let's say I receive an email and I want to determine whether it's spam or not spam. With traditional programing, I would need to write specific rules. For example, if an email contains words like “lottery”, “free” or “winner”, it's classified as spam.
The limitation here is that I would need to anticipate and encode every possible rule. If spammers change their wording, the program may fail to identify new spam emails. On the other hand, with machine learning, instead of writing specific rules, we feed the machine learning algorithm a large data set of emails, each labeled - in the case of supervised learning - by humans as spam or not spam.
The algorithm learns patterns from this data that differentiate spam from non-spam, and during training, the algorithm will try to build a model that minimizes the classification error, or more generally, something we call a loss function. In this way, the algorithm can adapt to new forms of spam without explicit reprograming. To illustrate this, if I were to plot the data with each email representing a point on a representative components, for example, x1 could be a measure of suspicious words, and x2 could be a measure of how generic the email is.
These features or components could be engineered or learned. My hope is that the emails in my data plot are something like this. Now, this is obviously an oversimplified illustration, but I think you get the general idea.
The stars would represent the spam and the circles would be the non-spam. I could simply draw a hyperplane, or in this case just a line to separate or classify these emails. Remember when I said we want to build a model?
Well, this hyperplane is the current simple model that defines which email is spam or not. So if I receive a new email, it could be simply plotted here to see if it is spam or not, depending on where it's located. With respect to the separating hyperplane.
Since the data, or the emails in this case, used to train the model were labeled, classifying the emails goes under supervised learning, which is one of the three types of machine learning. The other two being unsupervised learning. So for example, clustering news articles into categories.
And the third is reinforcement learning, such as training a car's autopilot through an algorithm that rewards for desirable actions and penalizes for undesirable actions. If you want me to discuss more of these concepts in future videos, please let me know in the comments. What is a neural network?
A neural network is a computational model developed around the mid 20th century, inspired by the way biological neurons in the human brain function. At its core, a neural network consists of layers of interconnected nodes, or neurons, that process data and generate outputs based on that data. The network learns from input data by adjusting the connections, known as weights and biases, between these neurons to improve predictions or classifications over training iterations or time.
The complexity and depth of these networks can vary depending on the problem they are designed to solve. The key components of the neural network are the nodes, the layers, and the connections. Neurons or nodes are the basic unit of a neural network.
Neurons receive inputs, apply a mathematical operation, and pass the result on to the next layer. Each neuron is connected to other neurons, forming a network. We also have layers which are broken down into three categories.
The input layer, which is the first layer that receives the raw data input, such as numerical values, images, or text. Next we have the hidden layers, which are the intermediate layers where the data is transformed by the network. The number and size of these layers can vary depending on the network's design.
And finally we have the output layer, which is the final layer that produces the output of the network, such as a classification label or a predicted value. And finally, the third component of the neural networks are the connections or the weights and biases. Each connection between neurons has an associate weight, which determines the strength of the signal being passed.
Biases are added to each neuron to shift the activation functions. Output. And this allows for more flexibility in the learning process.
Mathematically, the operation within a neuron can be represented as follows. Where wi represents the weights, xi is the input, and b is the bias, and f is the activation function that determines the neurons output. Let's talk about f.
First let's look at the sum inside. None of the operations in X are non-linear. We only see linear operations.
It's just like ax + b where the xi's are simply multiplied by the weights wi’s and added. That's why we use activation functions. Activation functions like the sigmoid function and the ReLU or the rectified linear unit, are commonly used to introduce non-linearity into the network, enabling it to learn complex patterns in the data.
The sigmoid function is often used when the output needs to be interpreted as a probability, as its range is between 0 and 1. ReLU, on the other hand, introduces non-linearity while being computationally efficient, since it involves only a simple thresholding at zero. So if z is below zero, the neuron simply does not get activated.
So maybe you can already see how these networks can represent complex functions by tuning which neurons get activated and which do not given a certain input. we've covered the basics of what neural networks are and how they're structured. I understand that these concepts might seem a bit abstract right now, but in the next lecture I'll introduce different types of neural networks and provide concrete examples to help make these ideas more clear and more practical.