License Plate Recognition in CSharp

From Emgu CV: OpenCV in .NET (C#, VB, C++ and more)
Jump to: navigation, search

This project is part of the Emgu.CV.Example solution of Version 2.0.0.0 Alpha release

Contents

System Requirement

Component Requirement Detail
Emgu CV Version 2.0.0.0 Alpha
Operation System Windows Only The .net wrapper tessnet2 to the OCR engine Tesseract is written in managed C++
It is NOT compatible with Linux

License Plate Recognition

According to wikipedia

Automatic number plate recognition (ANPR; see also other names below) is a mass surveillance method that uses optical character recognition on images to read the license plates on vehicles. As of 2006, systems can scan number plates at around one per second on cars traveling up to 100 mph (160 km/h).[citation needed] They can use existing closed-circuit television or road-rule enforcement cameras, or ones specifically designed for the task. They are used by various police forces and as a method of electronic toll collection on pay-per-use roads and monitoring traffic activity, such as red light adherence in an intersection.
ANPR can be used to store the images captured by the cameras as well as the text from the license plate, with some configurable to store a photograph of the driver. Systems commonly use infrared lighting to allow the camera to take the picture at any time of the day. A powerful flash is included in at least one version of the intersection-monitoring cameras, serving both to illuminate the picture and to make the offender aware of his or her mistake. ANPR technology tends to be region-specific, owing to plate variation from place to place.

This tutorial's approach to ANPR is divided into two stage

  • In the first stage, we perform license plate region detection
  • In the second stage, we perform OCR on the license plate to recover the license number

Assumption

This tutorial assumes that ANPR is performed on European license plate. Within the source code, you will find the following lines of code that indicates only rectangle with width-height ratio in the range of (3.0, 8.0) is considered.

double whRatio = (double)box.size.Width / box.size.Height;
if (!(3.0 < whRatio && whRatio < 8.0))
...

If you are performing ANPR on different region, you will have to change this threshold to best match the characteristic of the license plate from that region.

Recognition Accuracy

This tutorial is written to demonstrate how a simple ANPR system can be implement. This system is not robust and recognition accuracy might be low. A few places in this algorithm that affect the recognition accuracy includes

  • The license plate region detection is not robust. The contour extraction algorithm requires high contrast. If the car is white or silver, the license plate region is less likely to be recovered.
  • The OCR engine is not tuned for ANPR. If the license plate in your region contains a certain limited set of characters, you should tuned the OCR to be more sensitive to the specific character set. Visit tesseract OCR engine for more information.

Complete Source Code

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using Emgu.Util;
using Emgu.CV;
using Emgu.CV.Structure;
using tessnet2;
using System.Diagnostics;
 
namespace LicensePlateRecognition
{
   /// <summary>
   /// A license plate detector
   /// </summary>
   public class LicensePlateDetector : DisposableObject
   {
      private Tesseract _ocr;
 
      /// <summary>
      /// Create a license plate detector
      /// </summary>
      public LicensePlateDetector()
      {
         //create OCR
         _ocr = new Tesseract();
 
         //You can download more language definition data from
         //http://code.google.com/p/tesseract-ocr/downloads/list
         //Languages supported includes:
         //Dutch, Spanish, German, Italian, French and English
         _ocr.Init("eng", false);
      }
 
      /// <summary>
      /// Detect license plate from the given image
      /// </summary>
      /// <param name="img">The image to search license plate from</param>
      /// <param name="licensePlateList">A list of images where the detected license plate region is stored</param>
      /// <param name="filteredLicensePlateList">A list of images where the detected license plate region with noise removed is stored</param>
      /// <param name="boxList">A list where the region of license plate, defined by an MCvBox2D is stored</param>
      /// <returns>The list of words for each license plate</returns>
      public List<List<Word>> DetectLicensePlate(Image<Bgr, byte> img, List<Image<Gray, Byte>> licensePlateList, List<Image<Gray, Byte>> filteredLicensePlateList, List<MCvBox2D> boxList)
      {
         //Stopwatch w = Stopwatch.StartNew();
         List<List<Word>> licenses = new List<List<Word>>();
         using (Image<Gray, byte> gray = img.Convert<Gray, Byte>())
         using (Image<Gray, Byte> canny = new Image<Gray, byte>(gray.Size))
         using (MemStorage stor = new MemStorage())
         {
            CvInvoke.cvCanny(gray, canny, 100, 50, 3);
 
            Contour<Point> contours = canny.FindContours(
                 Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
                 Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_TREE,
                 stor);
            FindLicensePlate(contours, gray, canny, licensePlateList, filteredLicensePlateList, boxList, licenses);
         }
         //w.Stop();
         return licenses;
      }
 
      private void FindLicensePlate(
         Contour<Point> contours, Image<Gray, Byte> gray, Image<Gray, Byte> canny,
         List<Image<Gray, Byte>> licensePlateList, List<Image<Gray, Byte>> filteredLicensePlateList, List<MCvBox2D> boxList,
         List<List<Word>> licenses)
      {
         for (; contours != null; contours = contours.HNext)
         {
            Contour<Point> approxContour = contours.ApproxPoly(contours.Perimeter * 0.05, contours.Storage);
 
            if (approxContour.Area > 100 && approxContour.Total == 4)
            {
               //img.Draw(contours, new Bgr(Color.Red), 1);
               if (!IsParallelogram(approxContour.ToArray()))
               {
                  Contour<Point> child = contours.VNext;
                  if (child != null)
                     FindLicensePlate(child, gray, canny, licensePlateList, filteredLicensePlateList, boxList, licenses);
                  continue;
               }
 
               MCvBox2D box = approxContour.GetMinAreaRect();
 
               double whRatio = (double)box.size.Width / box.size.Height;
               if (!(3.0 < whRatio && whRatio < 8.0))
               {
                  Contour<Point> child = contours.VNext;
                  if (child != null)
                     FindLicensePlate(child, gray, canny, licensePlateList, filteredLicensePlateList, boxList, licenses);
                  continue;
               }
 
               Image<Gray, Byte> plate = gray.Copy(box);
               Image<Gray, Byte> filteredPlate = FilterPlate(plate);
 
               List<Word> words;
               using (Bitmap bmp = filteredPlate.Bitmap)
                  words = _ocr.DoOCR(bmp, filteredPlate.ROI);
 
               licenses.Add(words);
               licensePlateList.Add(plate);
               filteredLicensePlateList.Add(filteredPlate);
               boxList.Add(box);
            }
         }
      }
 
      /// <summary>
      /// Check if the four points forms a parallelogram
      /// </summary>
      /// <param name="pts">The four points that defines a polygon</param>
      /// <returns>True if the four points defines a parallelogram</returns>
      private static bool IsParallelogram(Point[] pts)
      {
         LineSegment2D[] edges = PointCollection.PolyLine(pts, true);
 
         double diff1 = Math.Abs(edges[0].Length - edges[2].Length);
         double diff2 = Math.Abs(edges[1].Length - edges[3].Length);
         if (diff1 / edges[0].Length <= 0.05 && diff1 / edges[2].Length <= 0.05
            && diff2 / edges[1].Length <= 0.05 && diff2 / edges[3].Length <= 0.05)
         {
            return true;
         }
         return false;
      }
 
      /// <summary>
      /// Filter the license plate to remove noise
      /// </summary>
      /// <param name="plate">The license plate image</param>
      /// <returns>License plate image without the noise</returns>
      private static Image<Gray, Byte> FilterPlate(Image<Gray, Byte> plate)
      {
         Image<Gray, Byte> thresh = plate.ThresholdBinaryInv(new Gray(120), new Gray(255));
 
         using (Image<Gray, Byte> plateMask = new Image<Gray, byte>(plate.Size))
         using (Image<Gray, Byte> plateCanny = plate.Canny(new Gray(100), new Gray(50)))
         using (MemStorage stor = new MemStorage())
         {
            plateMask.SetValue(255.0);
            for (
               Contour<Point> contours = plateCanny.FindContours(
                  Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE,
                  Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_EXTERNAL,
                  stor);
               contours != null; contours = contours.HNext)
            {
               Rectangle rect = contours.BoundingRectangle;
               if (rect.Height > (plate.Height >> 1))
               {
                  rect.X -= 1; rect.Y -= 1; rect.Width += 2; rect.Height += 2;
                  rect.Intersect(plate.ROI);
 
                  plateMask.Draw(rect, new Gray(0.0), -1);
               }
            }
 
            thresh.SetValue(0, plateMask);
         }
 
         thresh._Erode(1);
         thresh._Dilate(1);
 
         return thresh;
      }
 
      protected override void DisposeObject()
      {
         _ocr.Dispose();
      }
   }
}

Result

License Plate Recognition
Personal tools
Namespaces

Variants
Actions
Navigation
others
Toolbox