The following example shows the use of the camera calibration function within EMGU. The function of this library is to allow the automatic calibrate of a cameras FOV. When wide angle 'fisheye' lenses are used in photography a curvature effect can be observed. This type of lens maximises the FOV and is commonly seen in pin-hole camera. The following example shows the the use of the CameraCalibration class in correcting this error. The same class can also be used for producing distortion filters if required.
The code provided should run straight out of the Emgu.Example folder (V2.4.2), extract it to this location. If the code fails to execute re-reference the EMGU libraries and include the required opencv dlls in the bin directory. Note that the project is set to build to the output path "..\..\..\bin\" you may wish to change this if you don't extract to the EMGU.Example folder.
To run calibration you will require a chess board. There is one available here (,pdf) (or png here), print this out and place it onto a flat surface. If you want to generate a distortion filter place it onto a curved surface or try one of the pre filtered chessboard, Filter Chessboard 1, Filter Chessboard 2.
EMGU Coding Level: While the coding is not advanced the rated level for this example is Intermediate. This is not designed as a beginners tutorial and general knowledge of the EMGU video capture example is expected.
The code provided in this sample is basic there is little or no error checking. Support is available through the Forums but please try and examine the code before saying it doesn't work for you. The code is not optimised instead is better formatted to provided an understanding of the stages involved.
To initiate the correction routine fill in the variables and select to go button. This will be re-enabled once the set number of frames has been acquired. The image will initially display the tracked points from the chessboard so you can observe it's ability before calibration.
The Code: Variables
Setting of the chessboard size is achieved using width and heigh variables try playing with these and with a larger chessboard to see what effects they have on calibration.
const int width = 9;//9 //width of chessboard no. squares in width - 1 const int height = 6;//6 // heght of chess board no. squares in heigth - 1 Size patternSize = new Size(width, height); //size of chess board to be detected
These buffers store the relevant information for calculating the calibration there default is set to 100 by the image buffer. This is overwritten by the software remember the larger the value the longer it will take to calculate it will have to deal with 100*(9*6)=5400 point relations if the current maximum buffer size is used.
Bgr line_colour_array = new Bgr[width * height]; // just for displaying coloured lines of detected chessboard static Image<Gray, Byte> Frame_array_buffer = new Image<Gray,byte>; MCvPoint3D32f corners_object_list = new MCvPoint3D32f[Frame_array_buffer.Length]; PointF corners_points_list = new PointF[Frame_array_buffer.Length];
The intrinsic parameters used to remap the image are stored in
Ex_Param. Depending on the calibration method used
CALIB_TYPE some parameters of
IC will need setting up before being used (see here.
IntrinsicCameraParameters IC = new IntrinsicCameraParameters(); ExtrinsicCameraParameters EX_Param;
The Code: Methods
The code is mainly orientated around the
_Capture_ImageGrabbed() method that is called every time a frame is available from the camera. The Form1() load method is used to set up acquisition with the OS default camera. It also fills a Bgr array which is used to display the tracked points of the chessboard. This is displayed manually as the inbuilt method only works for greyscale images. A public enum
Mode is set up to allow managing of each part of the code within the
- Here an image buffer
Frame_array_bufferis filled with frames in which a chessboard is detected. An image is only added when all the chess board coners are found. This can be combined with Mode.Caluculating_Intrinsics to increase speed. Once each successful frame is acquired the process sleeps for 100ms this allows the board to be moved to a different location. Foir a smaller buffer this time should be increased.
- Here the points for the chessboard corners are calculated. this is done by looping through each image of the array. Rather than storing an image buffer
corners_points_listcould be filled directly. Here
object_listis generated to store the relevant real world measurements of the squares this is useful for 3D reconstruction. In the example the chess squares where 20mm x 20mm.
- The call to CalibrateCamera() returns a double, the closer to zero the more accurate the calculations have been. The CALIB_TYPE is important but different kinds require the
IntrinsicCameraParameters ICto be set up before use.
- Here the acquired image is mapped to the IntrinsicCameraParameters using
cvRemap. In this loop
ICis calculated every time. This is not required, once camera calibration is complete you can save the Intrinsic maps generated and load these in any application that uses the camera. If successful camera calibration is only required once.
- CalibrateCamera(Point3D<Single>, Point2D<Single>, MCvSize, IntrinsicCameraParameters, CALIB_TYPE, ExtrinsicCameraParameters)
- FindChessboardCorners(Image<Gray, Byte>, MCvSize, CALIB_CB_TYPE, Point2D<Single>)
- DrawChessboardCorners(Image<Gray, Byte>, MCvSize, Point2D<Single>, Boolean)
- FindExtrinsicCameraParams2(Point3D<Single>, Point2D<Single>, IntrinsicCameraParameters)
- FindHomography(Matrix<Single>, Matrix<Single>)
- FindHomography(Matrix<Single>, Matrix<Single>, Double)
- FindHomography(Matrix<Single>, Matrix<Single>, HOMOGRAPHY_METHOD, Double)
- FindHomography(Point2D<Single>, Point2D<Single>, HOMOGRAPHY_METHOD, Double)
- ProjectPoints2(Point3D<Single>, ExtrinsicCameraParameters, IntrinsicCameraParameters, Matrix<Single>)
- StereoCalibrate(Point3D<Single>, Point2D<Single>, Point2D<Single>, IntrinsicCameraParameters, IntrinsicCameraParameters, MCvSize, CALIB_TYPE, MCvTermCriteria, ExtrinsicCameraParameters, Matrix<Single>, Matrix<Single>)
- Undistort2<C, D>(Image<C, D>, IntrinsicCameraParameters)
- There is a refresh problem when displaying the original image in some cases. This is dues to the demand on processing the main image.
- The table layout panel sometimes mislays controls when form maximisation occurs.