Grad-CAM Explained | FREE XAI Course | L7 - Gradient-weighted Class Activation Mapping

An X-ray AI system displays a result. There is a tumor, but thankfully it is benign. However, the doctor is still hesitant to give this result directly to the patient.

This is because she doesn't know where in the brain the tumor is or how big it is. And to be honest, she doesn't really trust the AI system. This is where Grad Cam comes in.

It is an explainable AI method that can tell the doctor what part of the X-ray image was used to make the classification. It could potentially point out the exact collection of cells that make up the tumor. In general, Grad Cam is used to explain convolutional neural networks.

It does this by highlighting the pixels or regions of an image that are important for a given classification. This is known as a heat map. And due to its speed, reliability, and flexibility, Grad Cam has become the go-to method for creating [Music] these.

Hi, I'm Connor and welcome to ADO. In this lesson, I'm going to explain the theory behind Grad Camp. We're going to understand how it works by looking at both visualizations and the math.

I'm also going to explain the intuition behind the method and why it and other CAM approaches work so well. To end, we'll discuss the advantages and limitations of Grad Cam. This is to understand why it is such a popular method.

And in the next lesson, we'll apply Grad Cam using Python. So, keep an eye out for that video. Before we get started, I'd like to mention that this lesson is part of a wider explainable AI course for computer vision.

It's completely free and you can find a link to it in the description. You'll find the article version for this lesson as well as many coding resources. There's also a paid version of the course which I'll talk about at the end of this video.

Grad Cam produces heat maps, also known as class activation maps or CAMs. These show which part of an input image are used to classify that image as a given class. It does this by waiting feature maps in a model's convolutional layers by that classes gradients.

Let's walk step by step through how Grad Cam creates these heat maps. We start with a trained CNN. This is a simple one with three convolutional layers.

It is used to classify an image as a dog or cat. As we pass an image through the various layers, we reduce the height and width and increase the number of channels. The numbers below a layer give the number of channels.

We also refer to these as feature maps. There will be the same number of these as the number of kernels the layer has. But keep in mind when we talk about feature maps, we mean layers that have been activated by the forward pass.

That is when we do a forward pass through the network. All of the elements of the feature maps will be populated. We will also have the output logets from the model.

This is the raw output before we pass it to a sigmoid or softmax activation function. For now, we will focus on the highest lo which could also be considered the one for the predicted class. We'll call this YC.

Now, let's zoom in on the last convolutional layer con 3. We have 64 feature maps. We refer to an arbitrary map as a k.

To create a grad cam heat map, we first weight each feature so each element has the same waiting. We then do elementwise summation to produce one matrix of elements. We call this the weighted feature map.

This equation gives the generalized formula for the weighted feature map. A KC is the weight we apply to feature map K. All CAMs follow this same procedure using a different approach to calculating the weights.

For grad CAM, they are based on the gradients of the lojit for class C. To understand how we calculate the weights, let's take a closer look at one of the feature maps. Every feature map is a matrix of elements.

The dimensions M and N will depend on how the previous layers have been pulled. AK I J is the value for an arbitrary element in the feature map. Using back propagation, we can find the derivative of the loget YC with respect to an element of the feature map.

This gradient will tell us how much the loget changes with small changes in the element. In other words, larger gradients indicate that this particular element is important for the loget. By taking the average of all gradients, we can understand how important the entire feature map is.

Specifically, we have sum the gradients of every element in the feature map and divided by the total number of elements. This gives us the weight for the given feature map. This process is similar to global average pooling except now we are summing derivatives and not the actual elements value.

So we follow the same procedure for every feature map in the convolutional layer. We then multiply the feature map by its weight and do elementwise summation. Again this gives us one matrix of elements that we call the weighted feature map.

Then only two more steps are needed to create the final grad cam heat map. Firstly some of the elements of the weighted feature map will have negative values. However, we are only interested in the elements that have increased the loit of the predicted class.

So, we use the reloo activation function to ensure all negative elements will have a value of zero. This gives us a coarse heat map. The problem is this will have the same width and height as the feature maps in the convolutional layer.

So, the last step is to increase the dimensions so they are the same as the input image. This is done by upsampling using interpolation techniques. In the end, we can summarize the grad cam heat map using this equation.

It is obtained by applying the reloop activation function to the weighted sum of the feature maps in the final convolutional layer where the weight of each feature map is the average gradient of the loget for the predicted class with respect to the elements in the feature map. for our problem. This will show us which pixels in the input image are used to classify the image as a cat.

When applying the method more generally, there are a few more considerations. Firstly, we've talked about applying the method to the final convolutional layer. However, you can apply to any convolutional layer in the network or even a combination of layers.

The final layer is typically used as it will generally capture the most detailed semantic information while maintaining spatial information. Another consideration is that we can apply this method to any class and not necessarily the one with the highest loit. This can be useful when trying to debug incorrect predictions.

So hopefully the process behind grad cam is clear. However, we didn't cover one crucial aspect and that is why they work so well. Let's move on to understanding the intuition behind the method.

That is why it can reliably identify pixels that are important to a classification. Grad Cam was inspired by another method called class activation maps or CAMs. However, since then the term has become to refer to a collection of methods.

These all create heat maps by weighting the feature maps in convolutional layers by how important they are to a class of interest. The original CAM which we will discuss in a future video was used to explain networks with a global average pooling layer. It uses the weights that connect the GAP values and the loits for the class in the output layer.

Another adaption abilation cam takes a permutation approach to calculating the weights. It does this by deactivating entire feature maps and calculating the impact on the output logets. As we saw, grad cam uses the average gradient of the logets with respect to the elements in the feature maps.

All of the methods have been shown to reliably produce heat maps that can explain how a model is making a classification. They work so well due to the nature of convolutional neural networks. The layers in the network and especially the final layers will capture both important features and their spatial information.

So when we do a forward pass through the network, the elements for these features in their location in the input will be activated. The problem is that often features that are not related to the class of input can be activated. This is because other classes or features related to other classes are in the input.

For example, you can see that the first feature map includes elements for the human space. If we simply summed all feature maps, we would include these irrelevant elements. This is why all CAM methods will weight the feature maps.

By doing so, the heat map will only give the locations of features that have a significant effect on the predicted class. The next question is if all cams do this then why choose grad [Music] cam. The first advantage of grad cam is that it can be applied to any convolutional neural network.

In comparisons can only be applied to GAP networks. This means that grad cam gives you more flexibility when it comes to architecture. Other permutation methods like abilation cam also have this property.

The advantage of grad cam over these is that it is not as computationally expensive. We only need to do one forward pass to calculate the gradients for the lojit. In comparison with abilation cam, we need to do multiple forward passes for every feature map that we deactivate.

Other permutation methods like SHAP are even more computationally expensive. Still, Grad Cam has its limitations. For some applications, we may want more detailed explanations for predictions.

For example, we may want to know that the spots on a leopard or the color of a stop sign has led to a classification. Unfortunately, Grad Cam can only show that pixels within a certain region are contributing to a prediction. In a future video, we'll see how to address this limitation by combining Grad Cam with guided back propagation.

Another limitation is that although Grad Cam can be applied to many architectures, it is not model agnostic. It has been designed to explain CNN's trained on classification tasks. This method can be adapted for regression tasks and they are even adaptions for image segmentation tasks.

However, those processes won't be exactly the same as what we've discussed here. Also, they cannot be applied to transformer-based architectures. In general, we must consider that all CAMs provide approximations for how networks work.

Sometimes those approximations can be misleading or completely wrong. Through user studies, Grad Cam has been shown to provide intuitive explanations for predictions. However, it has also been shown that there are instances or networks where the method doesn't work so well.

It is important to keep this in mind when applying the method. As I mentioned, this video is part of a larger course where I go into detail on explainable AI methods for computer vision. This includes methods like occlusion, shap, grad, cam, guided back propagation, deep lift, and integrated gradients.

You can access all of the course content for free, but there's also a paid version of the course. With that, you'll get access to a certificate for the course, quizzes, all of the videos ad free, and an ebook which will allow you to access all of the course content offline. Fine.