Color Image Segmentation — Image Processing

Segmented Tennis Ball | GitHub

Whenever we see images, it is composed of various elements and objects. In some circumstance, there might come a time that you would want to extract a certain object from the image. What would you do? Ah, I know. The first thing you will think of is to crop it, right? Well, that would somehow work. But there are irrelevant pixels that would be included, and I know for sure you do not want that. What if I tell you that we can obtain objects of interest using image processing techniques?

Welcome to Image segmentation using Python.

Image Segmentation

It is the process of dividing an image into its constituent parts or objects. Common techniques include edge detection, boundary detection, thresholding, region based segmentation, among others. For this blog, let us focus on segmenting our images using Color Image Segmentation through the HSV color space.

But before we do that, install the following libraries, and follow along.

from skimage.io import imread, imshow
from skimage.color import rgb2hsv
from matplotlib.pyplot as plt

We will be using this image moving forward:

Bean bags | Drive (Click to download image)

Our goal here is to segment each bean bag through the HSV color space. But how? As discussed in my previous articles, HSV stands for Hue, Saturation, and Value, and we shall be using information from this color space in segmenting the image later on.

But before the segmentation process, let us first convert the RGB image in HSV form through this code:

bags_hsv = rgb2hsv(bags)
fig, ax = plt.subplots(1, 3, figsize=(12,4))
ax[0].imshow(bags_hsv[:,:,0], cmap='gray')
ax[0].set_title('Hue')
ax[1].imshow(bags_hsv[:,:,1], cmap='gray')
ax[1].set_title('Saturation')
ax[2].imshow(bags_hsv[:,:,2], cmap='gray')
ax[2].set_title('Value');

HSV channel of the image

Wonderful! We were successful in transforming the image. However, this is not helpful to us just yet. We need to obtain the intensity values of each HSV channel to help us in segmenting later. To do that, a colorbar is created through implementing this code:

fig, ax = plt.subplots(1, 3, figsize=(15, 5))
ax[0].imshow(bags_hsv[:,:,0],cmap='hsv')
ax[0].set_title('hue')
ax[1].imshow(bags_hsv[:,:,1],cmap='hsv')
ax[1].set_title('transparency')
ax[2].imshow(bags_hsv[:,:,2],cmap='hsv')
ax[2].set_title('value')
fig.colorbar(imshow(bags_hsv[:,:,0],cmap='hsv')) 
fig.tight_layout()

HSV channel of image in colorbar format

Voila! See the colorbar at the right? We will refer to that in segmenting the bags.

Let us start segmenting!

Before implementing the code, let us first set up our thresholds for the masks:

• Lower Mask (refer to the hue channel)
• Upper Mask (refer to the hue channel)
• Saturation Mask (refer to the transparency channel)

In choosing the thresholds, look at the color bar. For example, if we are segmenting the blue bag. The lower and upper mask values that are appropriate would be 0.6 and 0.7, respectively. So, in other words, it would only get the blue pixel values and neglect the rest. After that, the saturation threshold is decided. This is a bit tricky because you need to consider the colors that are seen in the object. So, play around with the saturation threshold that would return the best segmented image.

Now let us code!

Segmenting the blue bag, we implement this code:

#refer to hue channel (in the colorbar)
lower_mask = bags_hsv[:,:,0] > 0.6 
#refer to hue channel (in the colorbar)
upper_mask = bags_hsv[:,:,0] < 0.7 
#refer to transparency channel (in the colorbar)
saturation_mask = bags_hsv[:,:,1] > 0.3 
 
mask = upper_mask*lower_mask*saturation_mask
red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
imshow(bags_masked)

Segmented blue bag

Perfect! Now let us to the same for the rest. It is a matter of changing the mask values to arrive at your desired output.

Yellow bag

#refer to hue channel (in the colorbar)
lower_mask = bags_hsv[:,:,0] > 0.1
#refer to hue channel (in the colorbar)
upper_mask = bags_hsv[:,:,0] < 0.2
#refer to transparency channel (in the colorbar)
saturation_mask = bags_hsv[:,:,1] > 0.6 
 
mask = upper_mask*lower_mask*saturation_mask
red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
imshow(bags_masked)

Segmented yellow bag

Orange bag

#refer to hue channel (in the colorbar)
lower_mask = bags_hsv[:,:,0] > 0.0
#refer to hue channel (in the colorbar)
upper_mask = bags_hsv[:,:,0] < 0.09
#refer to transparency channel (in the colorbar)
saturation_mask = bags_hsv[:,:,1] > 0.4 
 
mask = upper_mask*lower_mask*saturation_mask
red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
imshow(bags_masked)

Segmented orange Bag

Green bag

#refer to hue channel (in the colorbar)
lower_mask = bags_hsv[:,:,0] > 0.0
#refer to hue channel (in the colorbar)
upper_mask = bags_hsv[:,:,0] < 0.09
#refer to transparency channel (in the colorbar)
saturation_mask = bags_hsv[:,:,1] > 0.4 
 
mask = upper_mask*lower_mask*saturation_mask
red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
imshow(bags_masked)

Segmented green bag

And you are done! Great job!

Brilliant! Now you know how to segment objects in an image using Color Image Segmentation through the HSV color space. In the next article, I am going to show you how to segment images using another technique. Happy coding and segmenting!