cvCamshift fail to track object in EMGU CV

[masangga]

I was trying to implement camshift algorithm using emgu cv, but until now it wont work as it's supposed to be...

http://www.youtube.com/watch?v=gCreJOp2014

here's the explanation of the program

1. by clicking the object, I have set the Object ROI
2. then, from the ROI, it extracts the Hue channel, and calculate the histogram and store it in Pmask
3. after that, it calculates back projection based on extracted hue and calculated Histogram in step 2, then store it in backProject image,
4. treshold the projection with cvAnd
5. then applying Camshift Algorithm (cvCamshift)

when you see the video, when I click the ROI of the object,then the trackwindow begin to track the object based on the color probability distribution that shows in right pannel, but suddenly the trackwindow stop to track the object and always gives the same result(in rectangle) after n frames...

anyone knows the solution for this issue?

nb : Implementation using emgucv..

sorry for my english..

[Andy]

looks like you have made some mistake in program's code sequence!

Can you post the program?


Solution may be you have to draw(update) rectangle for each movement on frame!

[masangga]

ok, I will post the snippet code and explain it carefully...

step 1 : set the image ROI by clicking the ImageBox

Code: Select all

private void CamImageBox_MouseDown(object sender, MouseEventArgs e)
        {
            if (capture != null)
            {
                setROI(e.X, e.Y);
               
            }
        }

        public void setROI(int X, int Y)
        {
            if (checkRange(X, Y))
            {
                //set image roi, roiWidth = 10, roiHeight = 10;
                IMAGE_ROI_Location = new Rectangle(X, Y, roiWidth, roiHeight);
                tracking.trackWindow = IMAGE_ROI_Location;
                initiate = false; //bool variable for initiating the track
                setRoi = true;
            }
        }

the main code of the program

Code: Select all

 private void ProcessFrame(object sender, EventArgs arg)
        {
           // bool initiate = false;
            Image<Bgr, Byte> ImageFrame = capture.QueryFrame();
            frameCount++;
if (test)
            {
                if (initiate == false)
                {
                    tracking.initiateTracking(ImageFrame, IMAGE_ROI_Location, Histhue);
                    ImageFrame.ROI = IMAGE_ROI_Location;
                    CamImageBox2.Image = ImageFrame.Copy(IMAGE_ROI_Location);
                    //CvInvoke.cvResetImageROI(ImageFrame);
                    initiate = true;
                    //tracking.semangatTracking(ImageFrame);
                }
                else
                {
                    try
                    {
                        //when clicked....
                        //tracking.semangatTracking(ImageFrame, IMAGE_ROI_Location);
                        Histhue.setVMIN(trB_Vmin.Value);
                        Histhue.setVMAX(trB_Vmax.Value);
                        Histhue.setSMIN(trB_Smin.Value);
                        //MessageBox.Show(Histhue.getVMIN().ToString());
                        //textBox1.Text = Histhue.getVMIN().ToString();
                        //tracking.initiateTracking(ImageFrame, IMAGE_ROI_Location, Histhue);
                        tracking.semangatTracking(ImageFrame);
                        ImageFrame.Draw(tracking.trackWindow, new Bgr(Color.Blue), 1);
                        //CvInvoke.cvEllipseBox(ImageFrame, tracking.trackbox, new MCvScalar(255, 0, 0), 2, LINE_TYPE.CV_AA, 2);
                        //cvEllipseBox(pVideoFrameCopy, faceBox,CV_RGB(255, 0, 0), 3, CV_AA, 0);
                        CamImageBox.Image = ImageFrame; //original frame
                        CamImageBox3.Image = tracking.backProject; //back projection of the color distribution display
                       
                    }
                    catch (Exception bb)
                    {
                        MessageBox.Show(bb.ToString());
                    }
                }
CamImageBox.Image = ImageFrame;
            }

here's the code from Tracking class (Tracking.cs)

Code: Select all

public void semangatTracking(Image<Bgr, Byte> frameObject)
        {
            //this method where the cvCamshift is implemented
//initiateTracking(frameObject, ROI);
            prevFaceRect = trackWindow;
            backProject = new Image<Gray, byte>(frameObject.Width, frameObject.Height);
            img = new IntPtr[1] { pHue };
            CvInvoke.cvCalcBackProject(img, this.backProject, HistHue.Histo);
            CvInvoke.cvAnd(this.backProject, this.pMask, this.backProject, IntPtr.Zero);

            CvInvoke.cvCamShift(this.backProject, this.prevFaceRect, new MCvTermCriteria(10,1), out trackcomp, out trackbox);
            //trackbox.angle = -1 * trackbox.angle;
            trackWindow = trackcomp.rect;
           
        }

//method for extracting hue channel
 public Image<Gray, Byte> UpdateHueImage(Image<Bgr, Byte> bgrFrame)
        {
            this.pHSV = bgrFrame.Convert<Hsv, Byte>();
            this.pHSV._EqualizeHist();
            Image<Gray, Byte>[] imgChannel = pHSV.Split();
            CvInvoke.cvInRangeS(imgChannel[0], new MCvScalar(0, HistHue.getSMIN(), Math.Min(HistHue.getVMIN(), HistHue.getVMAX()), 0), new MCvScalar(180, 256, Math.Max(HistHue.getVMIN(), HistHue.getVMAX()), 0), pMask);
            this.pHue = imgChannel[0];
            return this.pHue;
           
        }
           
public void initiateTracking(Image<Bgr, Byte> BGRFrame, Rectangle DetectedHand, HistogramHue histo)
        {
            float maxVal,minVal = 0.0f;
            int[] minLocation, maxLocation;
                pHSV = new Image<Hsv, byte>(BGRFrame.Width, BGRFrame.Height);
                pHue = new Image<Gray, byte>(BGRFrame.Width, BGRFrame.Height);
                pMask = new Image<Gray, byte>(BGRFrame.Width, BGRFrame.Height);
                //HistHue = new HistogramHue(); //create histogram representation from ROI
                HistHue = histo;
                pHue = UpdateHueImage(BGRFrame);
                pHue.ROI = DetectedHand;
                pHSV.ROI = DetectedHand;
                pMask.ROI = DetectedHand;   
                HistHue.Histo.Calculate(new Image<Gray, Byte>[] { this.pHue }, true, pMask); //calculate the Histogram
                //CvInvoke.cvConvertScale(
                //HistHue.Histo.MinMax(out minVal, out maxVal, out minLocation, out maxLocation);
                //HistHue.Histo.Normalize(BGRFrame.Width * BGRFrame.Height * 255 / maxVal);
                CvInvoke.cvResetImageROI(pHue);
                CvInvoke.cvResetImageROI(pMask);
               
                //prevFaceRect = DetectedHand;
           
        }

feel free to ask if there's line that you don't understand,sorry for many comments there...

[masangga]

I'm sorry there's mistake for the snippet code..

it should be..

Code: Select all

if (setRoi)
            {
                if (initiate == false)
                {
                    tracking.initiateTracking(ImageFrame, IMAGE_ROI_Location, Histhue);
                    ImageFrame.ROI = IMAGE_ROI_Location;
                    CamImageBox2.Image = ImageFrame.Copy(IMAGE_ROI_Location);

[Andy]

You need to draw rectangle for each movement camshift detects!

Declare a function which performs the movement of rectangle!

[Wafaa Ayman]

Hi masangga,

Could you declare to me what are the trb_Vmin and trb_Smin," i think they are the values of Value and Saturation of the histogram" but how you get them to set them to setVMIN ,setVMAX and setSMIN?

Also what's the HistHue ,"i think also it's the histogram computed for only the hue channel" but i want to be sure that u write it like that or not ??

DenseHistogram HueHist = new DenseHistogram(16,new RangeF(0f,180f));
CvInvoke.cvCalcHist(HueImg,HueHist,false,HMask);
OR
HueHist.Calculate(new Image<Gray, Byte>[] { HueImg}, false,HMask);

Please if U can response me rapidly urgently .

Thanks in advance and sorry for interruption.

[masangga]

HistHue is a class that I build to set the Vmin,vmax, and Smin

yep, you're right, Vmin/Vmax and Smin are the parameters that affects the Camshift Algorithm, so you could tune it up for the best performance..

HistHue also consists of Histogram iniliazation (DenseHistogram class in emgu cv)....

trb_Vmin and trb_Smin are the trackbox Object for tuning up the Vmin and Smin parameter...

here's the reference..

http://www.robinhewitt.com/research/track/camshift.html

and also this one

http://www.cognotics.com/opencv/servo_2007_series/part_3/index.html

sorry for late response...

I did't see any notification for your reply..

cheers up,
Masangga

[Wafaa Ayman]

Thanks too much Masangga for ur consideration and never mind for late me either didn't have any notification about your response.

But now i have a problem i wrote the initiatingTracking method and it worked well , despite the segmentTracking it didn't work and when i insert a break point to check the backproject image i found it empty So it wont track my ROI, So please if U know what are the expected reasons of that please tell me.

Thanks in advance.
Wafaa.

[masangga]

Hi Wafaa..

could you share the code so that I have better understanding about your problem? but if you need privacy then you can PM me..

Cheers
Angga

[Wafaa Ayman]

Hi Angga,

Thanks too much for your help, unfortunately i'm new in this Forum and don't know how to share my code also when i wrote a PM. to U i didn't find Ur name "When clicking on Find a member " .

Can u help me to find the problem in my code i think i don't call my methods right, cause i'm new to camShift Algo. and i want to use it in mouse moving by hand in my graduation project So i'm seriously need to fix it rapidly .

Thanks for Ur help and i'm So sorry for interruption,
Wafaa.