How to increase fps?

Ask question on using Emgu CV with C#

How to increase fps?

Postby Andy » Sun Sep 18, 2011 2:39 pm

I have written a simple C# code for Face Detection!, But the problem is the frame capture is too slow.. like 2-3 fps.
How to increase fps? Please help.

Code: Select all
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using Emgu.CV;
using Emgu.CV.UI;
using Emgu.CV.Structure;


namespace Faceme
{
    public partial class Form1 : Form
    {
        private Capture capture;
        private HaarCascade haarcascade;

        public Form1()
        {
            InitializeComponent();
        }

        private void Form1_Load(object sender, EventArgs e)
        {
            capture = new Capture();
            haarcascade = new HaarCascade("haarcascade_frontalface_alt.xml");
        }

        private void timer1_Tick(object sender, EventArgs e)
        {
            Image<Bgr, Byte> currentFrame = capture.QueryFrame();

            if (currentFrame != null)
            {
                Image<Gray, Byte> grayFrame = currentFrame.Convert<Gray, Byte>();

                var detectedFaces = grayFrame.DetectHaarCascade(haarcascade)[0];

                foreach (var face in detectedFaces)
                    currentFrame.Draw(face.rect, new Bgr(1, double.MaxValue, 0), 3);

                imageBox1.Image = currentFrame;
            }
        }
    }
}

Kindness, like a boomerang, always returns.
Andy
 
Posts: 54
Joined: Sun Sep 18, 2011 2:35 pm

Re: How to increase fps?

Postby Chris_Johnson » Thu Sep 22, 2011 8:20 am

Hi,

Your code isn't very efficient basically. Your using a timer to gather a frame from your web camera so it's this what is lowing your acquisition rate down. Take a look at the camera capture example.

What you should do is set up a function call to acquire a frame in every down time moment the program has (i.e) its not doing something else. Assuming your Capture is set up using this will attach this process so that the ProcessFrame function is called as much as possible.

Application.Idle += ProcessFrame;


The function when called gets a frame and processes it.

private void ProcessFrame(object sender, EventArgs arg)
{
Image<Bgr, Byte> frame = _capture.QueryFrame();

Image<Gray, Byte> grayFrame = frame.Convert<Gray, Byte>();
Image<Gray, Byte> smallGrayFrame = grayFrame.PyrDown();
Image<Gray, Byte> smoothedGrayFrame = smallGrayFrame.PyrUp();
Image<Gray, Byte> cannyFrame = smoothedGrayFrame.Canny(new Gray(100), new Gray(60));

captureImageBox.Image = frame;
grayscaleImageBox.Image = grayFrame;
smoothedGrayscaleImageBox.Image = smoothedGrayFrame;
cannyImageBox.Image = cannyFrame;
}

This should speed things up however if your using a HD web cam you will probably want to resize the capture image so you aren't working with to much data.

Image<Bgr, Byte> frame = _capture.QueryFrame().Resize(0.5, Emgu.CV.CvEnum.INTER.CV_INTER_AREA);

Hope this helps
Cheers
Chris
Chris_Johnson
 
Posts: 515
Joined: Tue Sep 20, 2011 3:29 pm

Re: How to increase fps?

Postby Andy » Sat Sep 24, 2011 6:56 am

Thank You very much Chris.. :D
I have one more question!
I'm using the latest Emgu, visual studio 2010.

How to get the frame rate? I've tried to do:

double fps = capture.GetCaptureProperty(Emgu.CV.CvEnum.CAP_PROP.CV_CAP_PROP_FPS);

but this always returns zero. However, when I right click on the imageBox and select properties, there is a FPS box, so the property exists somewhere, but how do I get to it!
Kindness, like a boomerang, always returns.
Andy
 
Posts: 54
Joined: Sun Sep 18, 2011 2:35 pm

Re: How to increase fps?

Postby Chris_Johnson » Mon Sep 26, 2011 3:52 am

Hi Andy,

Sorry but I actually don't use the EMGU imagebox, Ive had a look and I have no idea how to get the actual FPS.I would assume it's a simple count within the onPaint event that displays using a time this can be recreated.

Creater a Timer to update you label at any time period lets say a secound to make things easy.

Now I take it your using a ProcessFrame call to display your image and Application.Idle += ProcessFrame; call.

Crate a int to store each frame tasken so
Code: Select all
    int FPS_Count = 0;
    Timer FPS_Timer = new Timer();

    public CameraCapture()
    {
        InitializeComponent();
        FPS_Timer.Interval = 1000; // one second
        FPS_Timer.Tick +=new EventHandler(FPS_Timer_Tick);
    }

    private void captureButtonClick(object sender, EventArgs e)
    {
        Application.Idle += ProcessFrame;
        s.Start();
    }

    private void ProcessFrame(object sender, EventArgs arg)
    {
        //Do somthing
        FPS_Count++;
    }

    private void FPS_Time_Tick(object sender, EventArgs e)
    {
        label.Text = FPS_Count.ToString(); //Display FPS
        FPS_Count = 0;
    }


Now that will give you the FPS in "label" control. But what about if we wanted to update the control a little more often say every half a second. Well we would adjust FPS_Time_Tick accordingly.

FPS_Timer.Interval = 500;

private void FPS_Time_Tick(object sender, EventArgs e)
{
label.Text = (FPS_Count * 2).ToString(); //Display FPS
FPS_Count = 0;
}

//or if your going to change the timer interval

private void FPS_Time_Tick(object sender, EventArgs e)
{
double time_multiply = 1000 / FPS_Timer.Interval;
label.Text = (FPS_Count * time_multiply).ToString(); //Display FPS
FPS_Count = 0;
}

Hope that helps
Cheers
Chris
Chris_Johnson
 
Posts: 515
Joined: Tue Sep 20, 2011 3:29 pm

Re: How to increase fps?

Postby paj006 » Thu Jan 09, 2014 11:32 pm

I have tried this. Does not work. The actual bottleneck is the type of Camera (I think).
I measured the FPS of JUST
Code: Select all
_capture.QueryFrame();

it takes approximately .2 seconds. CPU never goes above 10% so its got to be the webcam or the USB connection.
paj006
 
Posts: 25
Joined: Mon Jun 03, 2013 7:48 pm

Re: How to increase fps?

Postby Chris_Johnson » Fri Jan 10, 2014 3:47 am

Hey paj006,

You are correct to some extent but I don't think you understood the whole point of the original question and on top of this the post was back in 2011 so things have changed a lot. Your frame rate will always be limited by a camera of which most can handle 30 fps at 640*480 on a USB 2.0 connection. You rarely can stream true 1024*769 at 30 FPS on a USB 2.0 connection as the data rate just isn't there to support it (you may however get upscaled images or sub-sampling techniques). You will find most cameras will fall to 15 FPS in HD mode. Now with the introduction of USB 3.0 full HD at 30 FPS is easily achievable but that has only been available since early 2013 onwards and these cameras are only now becoming readilly affordable.

The real bottleneck in the original post is the face detection. I suggested that the Application.Idle event was used as using a timer was not efficient for querying a frame. In the new versions of EMGU this has changed and the acquisition now has a dedicated thread that calls a frame ready event. I would recommend this from now on, see the camera capture example, some altered code below for more advanced use:

also see:
Code: Select all
http://www.emgu.com/wiki/index.php?title=Camera_Capture


Code: Select all
 public CameraCapture()
      {
         InitializeComponent();
         try
         {
            _capture = new Capture();
            _capture.ImageGrabbed += ProcessFrame; //sets up the frame ready event
if (_capture != null)
            {
_capture.Start();
}
         }
         catch (NullReferenceException excpt)
         {
            MessageBox.Show(excpt.Message);
         }
      }

private void ProcessFrame(object sender, EventArgs arg)
        {
            //***If you want to access the image data the use the following method call***/
            //Image<Bgr, Byte> frame = new Image<Bgr,byte>(_capture.RetrieveBgrFrame().ToBitmap());

            if (RetrieveBgrFrame.Checked)
            {
                Image<Bgr, Byte> frame = _capture.RetrieveBgrFrame();
                //because we are using an autosize picturebox we need to do a thread safe update
                DisplayImage(frame.ToBitmap());
            }
            else if (RetrieveGrayFrame.Checked)
            {
                Image<Gray, Byte> frame = _capture.RetrieveGrayFrame();
                //because we are using an autosize picturebox we need to do a thread safe update
                DisplayImage(frame.ToBitmap());
            }
        }

/// <summary>
        /// Thread safe method to display image in a picturebox that is set to automatic sizing
        /// </summary>
        /// <param name="Image"></param>
        private delegate void DisplayImageDelegate(Bitmap Image);
        private void DisplayImage(Bitmap Image)
        {
            if (captureBox.InvokeRequired)
            {
                try
                {
                    DisplayImageDelegate DI = new DisplayImageDelegate(DisplayImage);
                    this.BeginInvoke(DI, new object[] { Image });
                }
                catch (Exception ex)
                {
                }
            }
            else
            {
                captureBox.Image = Image;
            }
        }


As for the face detection the speed is hardware related obviously but to speed things up the easiest way is to reduce resolution of the image where you're looking for the face. A 320*240 Image will process much quicker than 1024*769 image obviously and the position of the face detected in the smaller image can be interpreted by scaling the position up accordingly. I've included an example bellow :

Code: Select all
CascadeClassifier HR_face = new CascadeClassifier(Path.Combine(Application.StartupPath, "Lib//haarcascade_frontalface_default.xml"));

 private void ProcessFrame(object sender, EventArgs arg)
        {
            //***If you want to access the image data the use the following method call***/
            //Image<Bgr, Byte> frame = new Image<Bgr,byte>(_capture.RetrieveBgrFrame().ToBitmap());

    Image<Bgr, Byte> frame = _capture.RetrieveBgrFrame().Resize(384,288, INTER.CV_INTER_CUBIC);
                //Image<Gray, Byte> detection = frame.Convert<Gray, Byte>().Resize(frame.Width / 4, frame.Height / 4, INTER.CV_INTER_CUBIC);
                Image<Gray, Byte> detection = frame.Convert<Gray, Byte>().Resize(384 / 4, 288 / 4, INTER.CV_INTER_CUBIC);
                Rectangle FaceLoc = facedetectsmall(detection, 1.1, 4);
                //int factor = frame.Width / (384 / 4);
                if (FaceLoc != Rectangle.Empty)
                {
                    frame.Draw(FaceLoc, new Bgr(Color.Blue), 2);
 DisplayImage(frame.ToBitmap());
            }
}

Rectangle Face_Position = Rectangle.Empty;
        Rectangle[] This_Face_Pos;
        Point FaceCenter = Point.Empty;

       private Rectangle facedetectsmall(Image<Gray, Byte> Input, double _scalefactor, int _minNeighbours)
        {
            This_Face_Pos = HR_face.DetectMultiScale(Input, _scalefactor, _minNeighbours, new Size(1, 1), Size.Empty);
           
            if (This_Face_Pos.Length > 0)
            {
                Face_Position = new Rectangle(This_Face_Pos[0].X * 4, This_Face_Pos[0].Y * 4, This_Face_Pos[0].Width * 4, This_Face_Pos[0].Height * 4);
                //center face POI
                //Face_Position.X += (int)(Face_Position.Height * 0.25);//0.15
                //Face_Position.Y += (int)(Face_Position.Width * 0.22);//0.22
                //Face_Position.Height -= (int)(Face_Position.Height * 0.3);//0.3
                //Face_Position.Width -= (int)(Face_Position.Width * 0.35);//0.35
                return Face_Position;
            }
            Face_Position = Rectangle.Empty;
            FaceCenter = Point.Empty;
            return Rectangle.Empty;
        }


Cheers,
Chris
Chris_Johnson
 
Posts: 515
Joined: Tue Sep 20, 2011 3:29 pm


Return to C# Help

Who is online

Users browsing this forum: Google [Bot] and 3 guests