Stereo Imaging

==Stereo Imaging== (Under Construction)

Namespace

Emgu.CV.CameraCalibration

References

EMGU Reference
EMGU Camera Calibration OpenCV

Downloads

[Source Code V1.0]

Example

The following example shows the use of the stereo calibration function within EMGU to produce a matched stereo camera pair for 3D re-construction. This library calibrates the FOV of two cameras and uses their intrinsics to calculate the alignment between the paired images. The following example shows the the use of the CameraCalibration class in 3D re-construction of a pair of images from web cameras.

Software To be uploaded

Pre-Requisites

The code provided should run straight out of the Emgu.Example folder (V2.4.2), extract it to this location. If the code fails to execute re-reference the EMGU libraries and include the required opencv dlls in the bin directory. Note that the project is set to build to the output path "..\..\..\bin\" you may wish to change this if you don't extract to the EMGU.Example folder.

To run 3D re-construction calibration of the cameras must be done first you will require a chess board to do this. There is one available here (,pdf) (or png here), print this out and place it onto a flat surface. For more information on simple camera calibration see here.

EMGU Coding Level: The rated level for this example is Advanced. This is not designed as a beginners tutorial and general knowledge of the EMGU video capture example is expected. While comments are provided in the code you may find some basic detail missed.

The Code

The code provided in this sample is basic there is little or no error checking. Support is available through the Forums but please try and examine the code before saying it doesn't work for you. The code is not optimised instead is better formatted to provided an understanding of the stages involved.

The Code: Variables

Setting of the chessboard size is achieved using width and heigh variables try playing with these and with a larger chessboard to see what effects they have on calibration. The Bgr array line_colour_array is used to draw the detected chessboard corners on the image.

const int width = 9;//9 //width of chessboard no. squares in width - 1
const int height = 6;//6 // heght of chess board no. squares in heigth - 1
Size patternSize = new Size(width, height); //size of chess board to be detected
Bgr[] line_colour_array = new Bgr[width * height]; // just for displaying coloured lines of detected chessboard

These buffers store the relevant calibration information acquired from FindChessboardCorners(), they are set to 100 by default using the static int buffer_length. If you are running a slower system reduce this before executing the program.

MCvPoint3D32f[][] corners_object_Points = new MCvPoint3D32f[buffer_length][];
PointF[][] corners_points_Left = new PointF[buffer_length][];
PointF[][] corners_points_Right = new PointF[buffer_length][];

The intrinsic parameters used to map the pair of images together are stored in IntrinsicCam1 and IntrinsicCam2 . Depending on the calibration method used CALIB_TYPE some parameters of these intrinsics will need setting up before being used (see here.

IntrinsicCameraParameters IntrinsicCam1 = new IntrinsicCameraParameters();
IntrinsicCameraParameters IntrinsicCam2 = new IntrinsicCameraParameters();
ExtrinsicCameraParameters EX_Param;

To apply the StereoCalibrate a Q map is generated using CvInvoke.cvStereoRectify(). The output from his algorithm is stored in the following variables. We only use the Q matrix in producing the disparity map.

Matrix<double> fundamental;
Matrix<double> essential;
Rectangle Rec1 = new Rectangle();
Rectangle Rec2 = new Rectangle();
Matrix<double> Q = new Matrix<double>(4, 4); //This is what were interested in
Matrix<double> R1 = new Matrix<double>(3, 3);
Matrix<double> R2 = new Matrix<double>(3, 3);
Matrix<double> P1 = new Matrix<double>(3, 4);
Matrix<double> P2 = new Matrix<double>(3, 4);
private MCvPoint3D32f[] _points;

Methods Available

Used

• CalibrateCamera(Point3D<Single>[][], Point2D<Single>[][], MCvSize, IntrinsicCameraParameters, CALIB_TYPE, ExtrinsicCameraParameters[])
• FindChessboardCorners(Image<Gray, Byte>, MCvSize, CALIB_CB_TYPE, Point2D<Single>[])
• StereoCalibrate(Point3D<Single>[][], Point2D<Single>[][], Point2D<Single>[][], IntrinsicCameraParameters, IntrinsicCameraParameters, MCvSize, CALIB_TYPE, MCvTermCriteria, ExtrinsicCameraParameters, Matrix<Single>, Matrix<Single>)

Unused

• DrawChessboardCorners(Image<Gray, Byte>, MCvSize, Point2D<Single>[], Boolean)
• FindExtrinsicCameraParams2(Point3D<Single>[], Point2D<Single>[], IntrinsicCameraParameters)
• FindHomography(Matrix<Single>, Matrix<Single>)
• FindHomography(Matrix<Single>, Matrix<Single>, Double)
• FindHomography(Matrix<Single>, Matrix<Single>, HOMOGRAPHY_METHOD, Double)
• FindHomography(Point2D<Single>[], Point2D<Single>[], HOMOGRAPHY_METHOD, Double)
• ProjectPoints2(Point3D<Single>[], ExtrinsicCameraParameters, IntrinsicCameraParameters, Matrix<Single>[])
• Undistort2<C, D>(Image<C, D>, IntrinsicCameraParameters)

Extra Methods

• StereoSGBM(int minDisparity, int numDisparities, int SADWindowSize, int P1, int P2, int disp12MaxDiff, int preFilterCap, int uniquenessRatio, int speckleWindowSize, int speckleRange, bool fullDP);
• MCvPoint3D32f[] ReprojectImageTo3D(Image<Gray, short> disparity, Matrix<double> Q);
• cvStereoRectify(IntPtr cameraMatrix1, IntPtr cameraMatrix2, IntPtr distCoeffs1, IntPtr distCoeffs2, MCvSize imageSize, IntPtr R, IntPtr T, IntPtr R1, IntPtr R2, IntPtr P1, IntPtr P2, IntPtr Q, STEREO_RECTIFY_TYPE flags)

Bugs

1. There is a refresh problem when displaying the disparity map image in some cases. This is dues to the demand on processing the main images.