catalogue
1. Convert the original image to binary image
3. Determine the background area (expansion) (get the background / maximum pass area)
4. Determine foreground area (distance transformation) (separation) (get seed / foreground)
5. Unknown area found (unknown area = background foreground)
6. Mark the most common domain according to the seed
7. Use watershed algorithm: merge seeds and uncertain areas, and mark the boundary as - 1
1, Basic theory
1. Thought
Prevent assimilation.
Any gray-scale image can be regarded as a terrain surface, in which high intensity represents peaks and low intensity represents valleys. You start filling each isolated valley (local minimum) with water (labels) of different colors. As the water level rises, the water from different valleys will obviously begin to merge and have different colors according to the nearby peaks (slopes). To avoid this, you need to build a barrier where the water merges. You continue to fill the water and build obstacles until all the peaks are underwater. Then the barrier you create will return your segmentation results. This is the "thought" behind Watershed. You can visit the watered CMM page to learn about it with some animation help.
However, this method will produce over segmentation results due to noise or other irregularities in the image. Therefore, OpenCV implements a marker based watershed algorithm. You can specify which valley points to merge and which are not. This is an interactive image segmentation.
2. Principle
What we do is: give different labels to the objects we know. Mark the area we determine as the foreground or object with one color (or intensity), mark the area we determine as the background or non object with another color, and finally mark the area we are not sure with 0. This is our mark. Then watershed algorithm is applied. Then our tag will be updated with the tag we give, and the boundary value of the object will be - 1.
The area near the center of the object is the foreground, the area far from the center of the object is the background, and the remaining area is the uncertain area.
2, Process
First look at the original picture:
Step induction
1. Image binarization
2. Open operation denoising
3. Determine the background area (expansion) (get the background / maximum pass area)
4. Determine foreground area (distance transformation) (separation) (get seed / foreground)
5. Unknown area found (unknown area = background foreground)
6. Mark the most common domain according to the seed
7. Use watershed algorithm: merge seeds and uncertain areas, and mark the boundary as - 1
8. Color and display
1. Convert the original image to binary image
# Convert to binary image
def ToBinary():
global gray, binary
# Grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Binarization
ret, binary = cv.threshold(gray, 0, 255, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
cv.imshow('binary', binary)(for the convenience of later calls, both gray-scale image and binary image are set as global global variables here)
2. Open operation denoising
Open operation to remove noise
# 1. Open operation denoising
opening = cv.morphologyEx(binary, cv.MORPH_OPEN, (3,3), iterations=2)
cv.imshow('opening', opening)
3. Determine the background area (expansion) (get the background / maximum pass area)
After expansion, the object expands and the background decreases. At this time, the object > the original object, and at this time, the background < the original background, then the background at this time can naturally be determined as a part of the original background. (far from the center of the object is the background)
# 2. Determine the background area
sure_bg = cv.dilate(opening, (3,3), iterations=3)
cv.imshow('sure_bg', sure_bg)
4. Determine foreground area (distance transformation) (separation) (get seed / foreground)
Principle: distance transformation reduces the object in the binary graph to obtain a part of the original graph, which can be determined as the foreground. Similar to separation. (if there is no separation, there is no need to change the distance, and only corrosion is enough)
Why not shrink it directly?
Direct reduction will make the picture change. What we want is to reduce the object and keep the picture unchanged.
distanceTransform function: calculates the shortest distance from a non-zero pixel to the nearest zero pixel. It is generally used to solve the bone of the image (binary graph)
def distanceTransform(src: Any,
distanceType: Any,
maskSize: Any,
dst: Any = None,
dstType: Any = None) -> NonedistanceType: the type of solving distance used.
mask_size: the size of the distance transformation mask, which can be ， 3 ， or ， 5. Pair CV_DIST_L1 or CV_ DIST_ In the case of C ， the parameter value is forcibly set to ， 3 because ， 3 × 3 # mask # 5 × 5 ， mask ， the same result, and faster.
# 3. Determine foreground area (distance transform)
# 3-1. Find the maximum width / length (diameter) of the object
dist_transform = cv.distanceTransform(opening, cv.DIST_L2, 5)
# 3-2. Reduce the longest diameter to a certain extent to determine the prospect
ret, sure_fg = cv.threshold(dist_transform, 0.7 * dist_transform.max(), 255, cv.THRESH_BINARY)
# Binarization method of threshold maximum value of foreground threshold function
sure_fg = np.uint8(sure_fg)
cv.imshow('sure_fg', sure_fg)sure_fg is the seed. (the following marks need to be obtained according to the seed)
5. Unknown area found (unknown area = background foreground)
Unknown area = determined background - determined foreground
# 4. Find unknown area (unknown area = determined background - determined foreground)
unknown = cv.subtract(sure_bg, sure_fg)
cv.imshow('unknown', unknown)
6. Mark the most common domain according to the seed
Mark the most common domain according to the seed (greater than 1 is the internal region, Mark 1 is the background region, and 0 is the unknown region)
# 5. Mark the most common domain according to the seed (greater than 1 is the internal region, Mark 1 is the background region, and 0 is the unknown region)
ret, markers = cv.connectedComponents(sure_fg) #Mark the most common domain
markers = markers+1 #The most common domain is marked as 1 (background)
markers[unknown == 255] = 0 #Unknown area marked as 0Display: connected domain (background), uncertain region, seed (foreground)
#Display each area (connected area / background, uncertain area, seed / foreground)
def Show_Markers():
mark = img.copy()
mark[markers == 1] = (255, 0, 0) #Connected domain / background (blue)
mark[markers == 0] = (0, 255, 0) #Uncertain area (green)
mark[markers > 1] = (0, 0, 255) #Foreground / seed (red)
mark[markers == -1] = (0, 255, 0) #Boundary (green)
cv.imshow('Markers', mark) 
7. Use watershed algorithm: merge seeds and uncertain areas, and mark the boundary as - 1
# 6. Using watershed algorithm, the boundary is modified to - 1, and the boundary is painted red (- 1) (boundary: connected domain background - unknown region + seed)
markers = cv.watershed(img, markers) # Watershed algorithm (modified boundary to - 1)
8. Color and display
# 7. Color and display (borders (markers==-1)
dst = img.copy()
dst[markers == -1] = [0, 0, 255] #Border (- 1) coloring
cv.imshow('dst', dst)
Total code and effect
# Distance transformation and watershed algorithm
import numpy as np
import cv2 as cv
# Convert to binary image
def ToBinary():
global gray, binary
# Grayscale
gray = cv.cvtColor(img, cv.COLOR_BGR2GRAY)
# Binarization
ret, binary = cv.threshold(gray, 0, 255, cv.THRESH_BINARY_INV + cv.THRESH_OTSU)
cv.imshow('binary', binary)
#Display each area (connected area / background, uncertain area, seed / foreground)
def Show_Markers():
mark = img.copy()
mark[markers == 1] = (255, 0, 0) #Connected domain / background (blue)
mark[markers == 0] = (0, 255, 0) #Uncertain area (green)
mark[markers > 1] = (0, 0, 255) #Foreground / seed (red)
mark[markers == -1] = (0, 255, 0) #Boundary (green)
cv.imshow('Markers', mark)
# Find boundary
def FindBounding():
global markers
# 1. Open operation denoising
opening = cv.morphologyEx(binary, cv.MORPH_OPEN, (3,3), iterations=2)
cv.imshow('opening', opening)
# 2. Determine the background area (expansion)
sure_bg = cv.dilate(opening, (3,3), iterations=3)
cv.imshow('sure_bg', sure_bg)
# 3. Determine foreground area (distance transform) (seed)
# 3-1. Find the maximum width / length (diameter) of the object
dist_transform = cv.distanceTransform(opening, cv.DIST_L2, 5)
# 3-2. The longest diameter is reduced proportionally to determine the prospect
ret, sure_fg = cv.threshold(dist_transform, 0.7 * dist_transform.max(), 255, cv.THRESH_BINARY)
# Binarization method of threshold maximum value of foreground threshold function
sure_fg = np.uint8(sure_fg)
cv.imshow('sure_fg', sure_fg)
# 4. Find unknown area (unknown area = determined background - determined foreground)
unknown = cv.subtract(sure_bg, sure_fg)
cv.imshow('unknown', unknown)
# 5. Mark the most common domain according to the seed (greater than 1 is the internal region, Mark 1 is the background region, and 0 is the unknown region)
ret, markers = cv.connectedComponents(sure_fg) #Mark the most common domain
markers = markers+1 #The background is marked with 1 (this is the most common domain)
markers[unknown == 255] = 0 #Unknown area marked as 0
Show_Markers() #Display each area (connected area / background, uncertain area, seed / foreground)
# 6. The watershed algorithm is used to merge uncertain regions and seeds, and the boundary is modified to - 1, (boundary: connected domain background -- unknown region + seed)
markers = cv.watershed(img, markers) # Watershed algorithm (modified boundary to - 1)
Show_Markers() #Display each area (connected area / background, uncertain area, seed / foreground)
# 7. Color and display (borders (markers==-1)
dst = img.copy()
dst[markers == -1] = [0, 0, 255] #Border (- 1) coloring
cv.imshow('dst', dst)
if __name__ == '__main__':
img = cv.imread('Resource/test19.jpg')
cv.imshow('img', img)
ToBinary() #To binary graph
FindBounding() #Find boundary
cv.waitKey(0)