hi everyone welcome to tutorial 52 of our introductory python for image processing tutorial series now in the last tutorial we looked at what thresholding and segmentations were and how to do it in a manual way and how to do in an automated way basically by using otsu to find out what the threshold value is and automatically separating or binarizing your image now in this tutorial let's actually build on that meaning instead of binarizing the image what if we have three four five six different regions of interest that we are interested in segmenting so this tutorial again uh uses a variation of otsu to do exactly the same task and then let's actually take it a bit further by understanding a few morphological operations by cleaning up the segmented image so first of all just to provide again a quick background this is what we saw before right in the previous tutorial we have an image we want to binarize it or we want to segment it so in this case i'm using one single threshold to say okay bright pixels versus not bright pixels but most of the time we have more than one region of interest so how do we get instead of to this binarized state but how do we get our images to this state where we have four different regions that are clearly separated or segmented so again this tutorial is all about getting there but one thing i should also highlight is when you apply your otsu or binarize your image eventually we are going to binarize our image for each segment right we are going to show all the pixels with this bright regions versus not bright dark gray versus not dark gray dark versus not dark and so on so every individual region you're looking at either that's positive or negative you know for that pixel okay so when as you're doing this binarization process of course you'll run into if your images are not clean enough especially if you have a lot of noise first of all you have to clean up the noise but then it's quite common for us to see some of these pixels that are hanging out most of them probably a single pixel or two pixels so to clean this up again you're probably already familiar if you use uh zeiss's zen software zen light again go ahead and download it it's free just google search for zen light zeiss l i t e for light and then it's a great program for you know image processing now uh if you already use image j for example you pro may have actually used this now uh these operations which is erosion okay if you erode it takes one pixel away as you can see in this image and just by doing erosion it cleaned up my image quite a bit because it's removing one pixel now if you want to do quantify object sizes for example and if you want to get accurate representation of your segmentation then you cannot just keep eroding right you have to put the pixels back so one of the tricks that we use is erode the image and then dilate it by one pixel dilation is just adding that pixel back okay so some of the large features they remain intact and the noise gets cleaned so this operation of erosion followed by dilation is called opening and if you do exactly opposite dilate and then erode it's called closing and of course skeletonizing is you keep eroding until you reach the single pixel level so that's called uh uh you know skeletonizing okay so with that knowledge let's actually exit out of the screen and jump into the code and have a quick look at the code again this is very similar to the one we have used the last time except i changed the input image to make it a bit more interesting this is the same image i used in the presentation if you don't have a lot of time to watch this entire tutorial you want one thing out of this let me tell you i'm going to use threshold multi-otsu to actually do multiple thresholding so now there you go you can save your time but i'm going to show you a couple more tricks so please stay tuned so let's go ahead and run these lines uh again nothing new here and let's read an image as grayscale here again this is a grayscale image if it's not a grayscale image and again please watch one of my previous tutorials in terms of how to work with color images again you can work in the rgb space or you can actually convert the image into hsv and only use the value space in the hsv you know value channel in the hsb space okay for now we're reading this image as a grayscale if you look up here it says 653 by 734 and unsigned integer eight everything looks fine black and white image uh or a grayscale image now originally i added a couple of lines of code uh in case the image is noisy when you do segmentation the results would be not very ideal so go ahead and denoise it again i i did uh three or four uh even i think five uh videos on denoising please watch those tutorials to find the best one i like the uh d noise tv shamble up so i tend to use that quite often or non-local means but go ahead and clean up your image if it requires it but first let's see how it looks like without cleaning up now we need to get a quick understanding of how the image looks like so let's go ahead and have a quick look and if i go switch to plots here is how my image looks like so right away i can say i have one two three and a dark region four maybe five but four for sure okay different regions but let's go ahead and look at the histogram i explained this in the last tutorial the way you plot it is take your array and flatten it out okay dot flat and then plot all the values and when you flatten it out again you're converting a 2d array into just one dimension okay and let's do 100 bins and plot all the way from 0 to 255 because this is 8-bit image so let's plot it and here is how the plot looks like okay histogram looks like so like we you know guessed you have probably you know one region right here that represents dark and another one three and four at least four different regions if you want to slice it into more you can but then the result will get a bit messy let's start with four okay now what are these four regions again let's do a manual segmentation and also let's do a multi-otsu which is automatic segmentation the manual you know we did this again in the last tutorial but only for binary case now let's do it for more than binary so region one i'm defining something as region one which is all the values from my image again what is my image it's an array of unsigned integers of size 653 by 734 with values between 0 to 255. now with this i'm saying that create a new array called region1 in fact if i run this line you should see it created a new array of region one which is a boolean array of the same size as the original image size again it's boolean because it's either saying false or true if i open this it should have false values or you see occasionally it's showing me true values what does that mean well because we have a condition here it's returning a boolean result what condition do we have look at the array called image and anything that's greater than or equal to zero all pixel values and pixel value less than 75 okay then this is true okay if this condition holds true then that entry would be true and then i created region 2 75 to 140 why am i doing this because i looked at this 0 to 75ish okay and 75 to 140 i think is somewhere in between these two and uh 140 to 200 is right here and then 200 and above 200 to 255 so i'm basically manually dividing this okay so let's run these lines now we should see all of our four different regions one two three four and they're all boolean arrays now what i'm trying to do is actually put all of these together into one single array again this is nothing but array manipulations you'll get much better at numpy area if you do these type of exercises believe me so i am now creating an array called all regions which is nothing but a bunch of zeros okay and into what shape into the shape of my image okay image shape zero image shape one by the way if you want what is image shape let's go ahead and do what is image shape img dot shape this is shape of our image is 653 and 734 that's exactly what i'm doing here my shape of the image is instead of typing 653 i just did this because my input image may change right so look at the shape of my image and the zeroth dimension which is 653 okay is the value that goes in here and then the value that goes in here is 734 and three channel image so all i'm doing is a numpy array that has all zeros with this shape so if i run this line hopefully that should clear up any if you have any doubts if i run that line and this is called all regions let me open all regions you see if i go to my channel number two which is the rgb channel zeros zeros zeros everything wherever you see it's all zeros why did i do this because i want to fill this with a bunch of other values what am i doing that first of all my all regions region one equals to one zero zero i'm basically saying okay all red pixels is coming from region one green pixels correspond to region two blue pixels correspond to region three and region four is red and green which should be yellow in this case okay and then show all regions image again think about this logic here go ahead and do it yourself so you can see how the image looks like there you go so red green green is all regions too blue and yellow okay now if you you can change the colors if you want again this is just a mixture of these okay so this is a manual way of doing things but let's do this in an automated way and again you need the latest psychic image to do this because this threshold multi-otsu is not available i think on versions 16 and below now i'm using version 17 of psychic image so this is available i think even in 16 this is available but the ones 15 and before you don't have that option so i'm going to use threshold multi also how are we going to use it it's very simple here you see all this exercise that we did in terms of finding out plotting the histogram nothing let's uh go back to our variable explorer and just run this single line right there it's applying this threshold multi auto on our original image or original image right there grayscale image with four different classes so it just divides this into four different regions that's all so we should get three threshold values okay so let's run this line and now you should see our threshold values or three threshold values 85 136 and 197 why only three because obviously it's implied that uh the values go from 0 to 85 85 to 136 136 to 197 and 197 to 255. previously when we looked at the image we said 0 to 75 85 not that bad and we said 75 to 140 it's actually 136 seems to be optimal and we picked the value of 200 197 seems to be optimal this so this is very close to our human way of identifying things probably a bit more repeatable than our human way because if i come back tomorrow and i look at the same histogram i would say well what was i thinking it's not it's not 75 it's probably 70 or something else okay anyway so now that we know the threshold values remember this np dot digitized from my previous tutorial again i i please go ahead and watch it if you haven't already but np dot digitize it assigns values of 0 1 2 3 okay so so far let's clean everything up let's start from clean slate and remember this is all manual you can forget that the reason i showed manual is so you know what is actually happening behind the scenes let's only run these lines so we have an image to work with okay we have an input image and now let's apply the threshold multi also and now we have the threshold values and what do we do once we have threshold values instead of writing all of these lines that we did all we need to do is regions equals to np.
digitize as part of numpy library you have a function called digitize and we are going to apply that on our original image with the number of bins as many as the ones that the thresholds you know reflected basically from our image are calculated from our image and this number of bins should be a numpy array and luckily when you do thresholds equals to threshold multi otsu it already gives you a numpy array look at this up here 85 136 197 in square brackets that's a numpy array so we're all good here so let's go ahead and run this oops run this line and then plot okay so let's look at the plot and there you go it's digitized it looks like this image that we did before but much better okay but in this case the values are automatically coming from mp. digitize in fact if you go ahead and look at our regions or regions here the values are 0 1 2 3 0 1 2 3. so for each region it's assigning a value of either zero one or two or three so this is a great segmentation now you can still say okay my segment one is uh the image with uh where all the regions have value of zero if the values of regions is one two and three you can still do this yeah and assign it to different segments and the reason why i'm doing this is i'll show that in a second if you look at your different segments for example if you only want to apply uh you know erosion and dilation you have morphological operations to one of these regions then you can extract individual regions and then apply that in this case i'm applying opening and closing to each of these image and again how do you apply that opening is part of for example some pi some by scipy nd image library i'm importing it as nd okay and as part of nd we have binary opening we have binary closing they have erosion they have dilation everything again go ahead and look at the documentation of nd image and try to find out what else it can do but so i'm applying nd dot binary opening to my segment 1 with my to my segment one with a kernel of three by three okay again these are the things that you can change and then once it's done i'm going to close apply a closing operation on the opened image you see my input here is the opened image with np dot once three by three okay so this is the kernel i am applying so i did the same to segment one two three and four and then i did exactly the same that i did in the last tutorial which is create an array a blank array sorry click create a blank area with all zeros with the shape of our input image and three channels and merge all the channels so it looks good for visualization purposes so let's run this and then visualize it there you go so this is the final result right there after erosion and dilation you see let me switch back and forth between these images you see a nice difference between these two if i go back and forth between it's tough to see here but there is a difference over there and i'm finally saving this image so i can kind of open it up later and and this is i mean obviously you want to save your images and i'm using plt.
msave to save this image okay so hopefully you learned again the key message here two key messages one practice numpy so you can actually play with these numbers easily and again i wanted to introduce this type of logical way of actually creating your binarizing you know your images so hopefully you learned that but there is an easier way to do your multi thresholding again by using threshold multi also and this literally came out in version 16 of of psychic image before that if you wanted multi multi multiple way of doing this you have to do something of this sort okay but luckily we have this now and once you have this again thanks to np.
