Camera Calibration

From Emgu CV: OpenCV in .NET (C#, VB, C++ and more)
Jump to: navigation, search

Camera Calibration

Namespace

Emgu.CV.CameraCalibration

References

EMGU Reference
OpenCV Reference
OpenCV Use Reference

Downloads

Source Code V1.0

Example

The following example shows the use of the camera calibration function within EMGU. The function of this library is to allow the automatic calibrate of a cameras FOV. When wide angle 'fisheye' lenses are used in photography a curvature effect can be observed. This type of lens maximises the FOV and is commonly seen in pin-hole camera. The following example shows the the use of the CameraCalibration class in correcting this error. The same class can also be used for producing distortion filters if required.


Camera Calibration

BeforeAfter


Camera Distortion

BeforeAfter


Software

Before


Pre-Requisites

The code provided should run straight out of the Emgu.Example folder (V2.4.2), extract it to this location. If the code fails to execute re-reference the EMGU libraries and include the required opencv dlls in the bin directory. Note that the project is set to build to the output path "..\..\..\bin\" you may wish to change this if you don't extract to the EMGU.Example folder.

To run calibration you will require a chess board. There is one available here (,pdf) (or png here), print this out and place it onto a flat surface. If you want to generate a distortion filter place it onto a curved surface or try one of the pre filtered chessboard, Filter Chessboard 1, Filter Chessboard 2.


EMGU Coding Level: While the coding is not advanced the rated level for this example is Intermediate. This is not designed as a beginners tutorial and general knowledge of the EMGU video capture example is expected.


The Code

The code provided in this sample is basic there is little or no error checking. Support is available through the Forums but please try and examine the code before saying it doesn't work for you. The code is not optimised instead is better formatted to provided an understanding of the stages involved.

Camera Calibration works by producing a scene view, formed by projecting 3D points into the image plane using a perspective transformation more informations is available here and here.

To initiate the correction routine fill in the variables and select to go button. This will be re-enabled once the set number of frames has been acquired. The image will initially display the tracked points from the chessboard so you can observe it's ability before calibration.


The Code: Variables

Setting of the chessboard size is achieved using width and heigh variables try playing with these and with a larger chessboard to see what effects they have on calibration.

const int width = 9;//9 //width of chessboard no. squares in width - 1
const int height = 6;//6 // heght of chess board no. squares in heigth - 1
Size patternSize = new Size(width, height); //size of chess board to be detected


These buffers store the relevant information for calculating the calibration there default is set to 100 by the image buffer. This is overwritten by the software remember the larger the value the longer it will take to calculate it will have to deal with 100*(9*6)=5400 point relations if the current maximum buffer size is used.

Bgr[] line_colour_array = new Bgr[width * height]; // just for displaying coloured lines of detected chessboard
static Image<Gray, Byte>[] Frame_array_buffer = new Image<Gray,byte>[100];
MCvPoint3D32f[][] corners_object_list = new MCvPoint3D32f[Frame_array_buffer.Length][];
PointF[][] corners_points_list = new PointF[Frame_array_buffer.Length][];

The intrinsic parameters used to remap the image are stored in IC and Ex_Param. Depending on the calibration method used CALIB_TYPE some parameters of IC will need setting up before being used (see here.

IntrinsicCameraParameters IC = new IntrinsicCameraParameters();
ExtrinsicCameraParameters[] EX_Param;


The Code: Methods

The code is mainly orientated around the _Capture_ImageGrabbed() method that is called every time a frame is available from the camera. The Form1() load method is used to set up acquisition with the OS default camera. It also fills a Bgr array which is used to display the tracked points of the chessboard. This is displayed manually as the inbuilt method only works for greyscale images. A public enum Mode is set up to allow managing of each part of the code within the _Capture_ImageGrabbed() method.

Mode.SavingFrames
Here an image buffer Frame_array_buffer is filled with frames in which a chessboard is detected. An image is only added when all the chess board coners are found. This can be combined with Mode.Caluculating_Intrinsics to increase speed. Once each successful frame is acquired the process sleeps for 100ms this allows the board to be moved to a different location. Foir a smaller buffer this time should be increased.


Mode.Caluculating_Intrinsics
Here the points for the chessboard corners are calculated. this is done by looping through each image of the array. Rather than storing an image buffer corners_points_list could be filled directly. Here object_list is generated to store the relevant real world measurements of the squares this is useful for 3D reconstruction. In the example the chess squares where 20mm x 20mm.
The call to CalibrateCamera() returns a double, the closer to zero the more accurate the calculations have been. The CALIB_TYPE is important but different kinds require the IntrinsicCameraParameters IC to be set up before use.


Mode.Calibrated
Here the acquired image is mapped to the IntrinsicCameraParameters using cvRemap. In this loop IC is calculated every time. This is not required, once camera calibration is complete you can save the Intrinsic maps generated and load these in any application that uses the camera. If successful camera calibration is only required once.


Methods Available

Used

  • CalibrateCamera(Point3D<Single>[][], Point2D<Single>[][], MCvSize, IntrinsicCameraParameters, CALIB_TYPE, ExtrinsicCameraParameters[])
  • FindChessboardCorners(Image<Gray, Byte>, MCvSize, CALIB_CB_TYPE, Point2D<Single>[])


Unused

  • DrawChessboardCorners(Image<Gray, Byte>, MCvSize, Point2D<Single>[], Boolean)
  • FindExtrinsicCameraParams2(Point3D<Single>[], Point2D<Single>[], IntrinsicCameraParameters)
  • FindHomography(Matrix<Single>, Matrix<Single>)
  • FindHomography(Matrix<Single>, Matrix<Single>, Double)
  • FindHomography(Matrix<Single>, Matrix<Single>, HOMOGRAPHY_METHOD, Double)
  • FindHomography(Point2D<Single>[], Point2D<Single>[], HOMOGRAPHY_METHOD, Double)
  • ProjectPoints2(Point3D<Single>[], ExtrinsicCameraParameters, IntrinsicCameraParameters, Matrix<Single>[])
  • StereoCalibrate(Point3D<Single>[][], Point2D<Single>[][], Point2D<Single>[][], IntrinsicCameraParameters, IntrinsicCameraParameters, MCvSize, CALIB_TYPE, MCvTermCriteria, ExtrinsicCameraParameters, Matrix<Single>, Matrix<Single>)
  • Undistort2<C, D>(Image<C, D>, IntrinsicCameraParameters)


Bugs

  1. There is a refresh problem when displaying the original image in some cases. This is dues to the demand on processing the main image.
  2. The table layout panel sometimes mislays controls when form maximisation occurs.