Write in front

• Why should machine vision be calibrated with a camera: the direct purpose is to obtain the internal and external parameters of the camera and the distortion parameters
• So what are the use of these parameters: through camera calibration, this lens distortion can be corrected and the corrected image can be generated; Another purpose is to reconstruct the 3D scene from the obtained image.
• How to obtain these parameters: input the calibration board images of various angles and the corner position of each calibration board image, and call various methods in opencv python
• Does it really work after calibration and correction:
• What is the impact of not calibrating: the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the camera results.

Mathematical / physical principles (to be updated)

• Coordinate system
  • World coordinate system
  • Camera coordinate system
  • Calibration plate coordinate system
• parameter
  • Because the influence of the design process can not be changed, this will be the internal parameter of the camera;
  • The influence caused by the environment or installation mode can be changed, which is the external parameter of the camera

Programming implementation

• Tools / materials used: Python 3, opencv Python library, 10-20 chessboard calibration board images from various angles
• API function
  • cv2.findChessboardCorners(), find corners

  The first parameter, Image, is the Mat Image of the chessboard chart, which must be an 8-bit gray or color Image;
  The second parameter, patternSize, is the number of rows and columns of internal corners on each chessboard. Generally, the number of rows and columns is different, which is convenient for subsequent calibration programs to identify the direction of the calibration board;
  The third parameter corners is used to store the coordinate position of the detected inner corner image, which is generally in the form of array;
  The fourth parameter, flag, is used to define different processing methods for finding internal corners on the chessboard. It has a default value.

  • cv2.cornerSubPix() to find sub-pixel corners

  The first parameter image is the pixel matrix of the input image, preferably an 8-bit gray image, which has higher detection efficiency;
  The second parameter corners is the initial corner coordinate vector, which is also the output of sub-pixel coordinate position, so it needs to be floating-point data;
  The third parameter, winSize, is half the size of the search window;
  The fourth parameter, zeroZone, is half the size of the dead zone. The dead zone is the area where the sum operation is not performed at the central position of the search area. It is used to avoid some possible singularities of autocorrelation matrix. When the value is (- 1, - 1), there is no dead zone;
  The fifth parameter criteria defines the termination condition of the iterative process of finding corners, which can be a combination of the number of iterations and corner accuracy;

  • cv2. Drawcheckboardcorners()

  The first parameter image, 8-bit gray or color image;
  The second parameter, patternSize, is the number of rows and columns of internal corners on each calibrated chessboard;
  The third parameter corners is the initial corner coordinate vector, which is also the output of sub-pixel coordinate position, so it needs to be floating-point data;
  The fourth parameter patternwasfold, flag bit, is used to indicate whether the defined inner corner of the chessboard has been completely detected. true indicates that it has not been completely detected. The function will connect all inner corners in sequence with a straight line as a whole. false indicates that there are undetected inner corners. At this time, the function will mark the detected inner corner with a (red) circle;

  • cv2. Calibrate camera()

  The first parameter, objectPoints, is a 3D point in the world coordinate system. It is necessary to calculate (initialize) the world coordinates of each inner corner according to the size of a single black-and-white matrix on the chessboard;
  The second parameter imagePoints is the image coordinate point corresponding to each inner corner;
  The third parameter imageSize is the pixel size of the image, which needs to be used when calculating the camera's internal parameter distortion matrix;
  The fourth parameter cameraMatrix is the internal parameter matrix of the camera;
  The fifth parameter distCoeffs is the distortion matrix;
  The sixth parameter rvecs is the rotation vector;
  The seventh parameter tvecs is the displacement vector;
  The eighth parameter flags is the algorithm used in calibration. There are the following parameters:
  The ninth parameter criteria is the optimal iteration termination condition setting.

  • cv2. Undo

  The first parameter src, the input parameter, represents the distorted original image;
  The second parameter, cameraMatrix, is the internal parameter matrix of the camera obtained before;
  The third parameter distCoeffs is the camera distortion matrix obtained before;
  The fourth parameter dst, the corrected output image, has the same type and size as the input image;
  The fifth parameter newCameraMatrix is consistent with cameraMatrix by default;

• Implementation example: pay attention to the chessboard image path, and pay attention to the different chessboard corners. You need to modify the parameters, otherwise the insertion error will occur

# Camera calibration
import cv2
import numpy as np
import glob

# The parameters for finding sub-pixel corners are set, and the stop criteria are the maximum number of cycles 1 and the maximum error tolerance 0.001
criteria = (cv2.TERM_CRITERIA_MAX_ITER | cv2.TERM_CRITERIA_EPS, 30, 0.001)

# Obtain the position of the corner of the calibration plate
objp = np.zeros((6 * 9, 3), np.float32)
objp[:, :2] = np.mgrid[0:9, 0:6].T.reshape(-1, 2)
# Traverse each chessboard grid to obtain its corresponding number of internal corner points, i.e. nx * ny.
# Save the three-dimensional coordinates of all internal corners in each chessboard grid in the form of array.
# The world coordinate system is built on the calibration board, and the Z coordinates of all points are 0, so only x and y need to be assigned
# print(objp), partial output is as follows:
# [[0. 0. 0.]
#  [1. 0. 0.]
#  [2. 0. 0.]
#  [3. 0. 0.]
#  [4. 0. 0.]
#  [5. 0. 0.]
#  [6. 0. 0.]
#  [7. 0. 0.]
#  [8. 0. 0.]
#  [0. 1. 0.]
#  [1. 1. 0.]
#  [2. 1. 0.]
# ...

obj_points = []  # Store 3D points
img_points = []  # Store 2D points

images = glob.glob("../images/*.jpg")
i = 0
for fname in images:
    img = cv2.imread(fname)
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)    # Convert BGR image into gray value image
    size = gray.shape[::-1]

    ret, corners = cv2.findChessboardCorners(gray, (9, 6), None)    # Extract corners, which specifically refer to the internal corners on the calibration plate, which do not contact the edge of the calibration plate.
    # print(corners)

    # Corner extraction succeeded
    if ret:
        obj_points.append(objp)

        corners2 = cv2.cornerSubPix(gray, corners, (5, 5), (-1, -1), criteria)  # Find sub-pixel corners based on the original corners to improve the precision of corners
        # print(corners2)
        if corners2.any():
            img_points.append(corners2)
        else:
            img_points.append(corners)

        cv2.drawChessboardCorners(img, (9, 6), corners, ret)  # When drawing corners, remember that OpenCV's drawing function generally has no return value
        i += 1
        cv2.imwrite('conimg' + str(i) + '.jpg', img)
        # cv2.waitKey(10)

print(len(img_points))
cv2.destroyAllWindows()

# calibration
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(obj_points, img_points, size, None, None)

print("ret:", ret)
print("mtx:\n", mtx)  # Internal parameter matrix
print("dist:\n", dist)  # Distortion coefficients = (k_1, k_2, p_1, p_2, k_3)
print("rvecs:\n", rvecs)  # Rotation vector  # External parameters
print("tvecs:\n", tvecs)  # Translation vector  # External parameters

print("-----------------------------------------------------")

img = cv2.imread(images[2])
h, w = img.shape[:2]
newcameramtx, roi = cv2.getOptimalNewCameraMatrix(mtx, dist, (w, h), 1, (w, h))  # Display a wider range of images (some images will be deleted after normal remapping)
print(newcameramtx)
print("------------------use undistort function-------------------")
dst = cv2.undistort(img, mtx, dist, None, newcameramtx)
x, y, w, h = roi
dst1 = dst[y:y + h, x:x + w]
cv2.imwrite('calibresult3.jpg', dst1)
print("Method 1:dst The size of the is:", dst1.shape)

• Correction is not equivalent to correction. Correction mathematically emphasizes removing lens distortion, while correction is mathematically arranging the images in order

Reference and thanks

In this article, I refer to the following blogs and express my sincere thanks here!

1. Camera calibration from CSDN
2. Principle and steps of Camera calibration from CSDN seehidre
3. Zhang Zhengyou camera calibration Opencv implementation and calibration process & & calibration result evaluation & & image correction process analysis (with calibration program and checkerboard diagram) csdn-- Makino-
4. OpenCV learning notes (21) -- camera calibration
   -csdn line song er