A blobs library (did I say "with code"?)

Request features to be added
AlbertG
Posts: 84
Joined: Fri Jul 25, 2008 8:34 am

A blobs library (did I say "with code"?)

Post by AlbertG » Thu Jan 29, 2009 6:06 am

Hi,

I must admit I never felt comfortable with the blobs detection treatment of OpenCV. For my programs I have my own very specific code, but it could be useful to have a more generic one. There are some libraries like CvBlobsLib http://opencv.willowgarage.com/wiki/cvBlobsLib and CvBlob http://opencv.willowgarage.com/wiki/cvBlobsLib, but wrapping them was not trivial and, also, they were really c++ centric, and didn't fit with a C# philosophy. So I "wrote" my own. You can find the code attached, hope it can make it to EmguCV with the corresponding corrections.

Credit must go to CvBlobsLib authors, as I've blatantly stolen their detection function (keeping their headers acknowledgements, of course) and also their blob class with some minor modifications. The rest of ther project, I wiped out ;-) (Odd fact: Authors of CvBlobsLib woked 20 meters away from where I'm sitting right now. I never got to met them though). Also credit to CvBlob, I got some nice ideas there and tried to replicate their example.

It consists of a c++ project with the heavy stuff and a few functions that can be called from C#. Then some invokes and a Blob class for C#, a couple of image methods to obtain and draw the blobs and an example program. The thing is still buggy (look at those * ellipses...) and most likely leaks memory, but it's a start. Look at the example below. If you can use linq in your programs, the effect is quite interesting:

Code: Select all

static void Main(string[] args)
{
  Image<Gray, Byte> img = new Image<Gray, byte>(@"test.png");
  Image<Bgr, Byte> img2 = img.Convert<Bgr, Byte>();

  // Get the image blobs
  Blob[] blobs = img.GetBlobs(100, IntPtr.Zero, true, true);

  // Skip the one with the biggest area
  IEnumerable<Blob> filtered = blobs.OrderByDescending(b => b.Area).Skip(1);
  // And the one with the worst perimeter-area ratio
  filtered = filtered.OrderBy(b => b.PerimeterLength/b.Area).Skip(1);

  //// Draw them in img2
  img2.DrawBlobs(filtered.ToArray(),
                            true,                                     // Fill
                            true, new Bgr(255, 0, 0),       // Draw Bounding box
                            true, new Bgr(0, 255, 0),       // Draw Convex hull 
                            true, new Bgr(0, 0, 255),       // Draw Ellipse
                            true, new Bgr(255, 0, 0),       // Draw centroid
                            true, new Bgr(100, 100, 100) // Draw Angle
                            );
            
  String win1 = "Test Window1"; //The name of the window
  CvInvoke.cvNamedWindow(win1); //Create the window using the specific name
  CvInvoke.cvShowImage(win1, img); //Show the image
  String win2 = "Test Window2"; //The name of the window
  CvInvoke.cvNamedWindow(win2); //Create the window using the specific name
  CvInvoke.cvShowImage(win2, img2); //Show the image
  CvInvoke.cvWaitKey(0);  //Wait for the key pressing event

  CvInvoke.cvDestroyWindow(win1); //Destory the window
  CvInvoke.cvDestroyWindow(win2); //Destory the window

  // Release the blobs
  foreach (var b in blobs)
    b.Clear();
  }
}
test.png
test.png (87.59 KiB) Viewed 34959 times
blobs.png
blobs.png (148.49 KiB) Viewed 34963 times
Attachments
blobsStuff.zip
(246.17 KiB) Downloaded 5044 times

canming
Site Admin
Posts: 1006
Joined: Thu Feb 28, 2008 4:16 pm

Re: A blobs library (did I say "with code"?)

Post by canming » Thu Feb 05, 2009 9:20 pm

Hi Albert,

This is great stuff.

I have spend some time to compare the CvBlobsLib library with the blob module in OpenCV's cvaux dll. In my opinion, the implementation in cvaux seems to be more attractive, since it contains not only a blob detector, but also implementations of blob trackers, blob tracking post processing modules, blob analysis and foreground detection. So I wrapped the cvaux's blob module under Emgu.CV.VideoSurveilance namespace.

CvBlobsLib is a good blob extraction library, but I don't know how difficult it will be to compile it on Linux, as Emgu CV aims to be cross platform. I believe your wrapper will be very helpful for others trying to use CvBlobsLib in .NET, so I created a Code Gallery link on Emgu CV's side bar. You can post your code in the following link:
http://www.emgu.com/wiki/index.php/Code_Gallery

Regards,
Canming

AlbertG
Posts: 84
Joined: Fri Jul 25, 2008 8:34 am

Re: A blobs library (did I say "with code"?)

Post by AlbertG » Fri Feb 06, 2009 6:54 am

Hi Canming,

I absolutely did not know about the blob tracker module already in cvaux.dll. Is that documented anywhere? I usually check things at the openCV wiki, but this is nowhere to be seen.
After taking a look at the code, I can clearly understand why cvaux's is more appropriate. Anyway I'll take a look at the code gallery and try to upload it there.
As always, thanks for your work,
Albert

chant
Posts: 3
Joined: Thu Mar 12, 2009 9:56 am

Re: A blobs library (did I say "with code"?)

Post by chant » Wed Mar 18, 2009 8:00 am

Hey there,

Thanks Albert for submitting your blob detection wrapper for the cvblobslib. I have tried to use it to detect the blobs in an image, but I have noticed that there is a memory leak which I cannot find.

My code is very simple. I have a Form and a timer to grab an image from a webcam every 50ms. For each frame, I just get the blobs and clear them.

Code: Select all

		Image<Bgr, byte> image;
		Blob[] blob;
		Capture capture = new Capture(); //create a camera captue

		public Form1()
		{
			InitializeComponent();
			timer1.Start();
		}


		private void timer1_Tick(object sender, EventArgs e)
		{
			image = capture.QueryFrame();
			blob = GetBlobs(image, 100, IntPtr.Zero, false, true);

			for (int i = 0; i < blob.Length; i++)
				blob[i].Clear();

			System.GC.Collect();		
		}
As the program runs, the memory used increases. I seem unable to find the leak, can you help me finding it? Or am I doing something wrong?

Thanks in advance.

PS: I have tried to use the blob tracker in cvaux.dll, but I can't do the same as I do with the cvblobslib. Furthermore, the blob tracker is kind of slow.

AlbertG
Posts: 84
Joined: Fri Jul 25, 2008 8:34 am

Re: A blobs library (did I say "with code"?)

Post by AlbertG » Mon Mar 23, 2009 7:36 am

Hi chant,

Glad you found the blob detector useful. About the memory leak, to be fair, it's been ages since I touched any c++ except for this, so it would be surprising if there wasn't any. Sadly, I'm not sure I can find it either.
I have a feeling it has to do with the

Code: Select all

IntPtr vector = BlobsInvoke.CvGetBlobs(Ptr, threshold, maskImage, borderColor, findMoments, out count);
line in GetBlobs, what happens in C++ world and the way I deal it in C#, but I'm not sure how to fix it. Maybe canming, who knows a lot more about unmanaged, can help you there.
By the way, are you completely sure the leak is in the blob detection? The way you use the image, it may seem that it leaks. I would write it with a using, as

Code: Select all

      private void timer1_Tick(object sender, EventArgs e)
      {
         using (image = capture.QueryFrame())
         {
             blob = GetBlobs(image, 100, IntPtr.Zero, false, true);

             for (int i = 0; i < blob.Length; i++)
                blob[i].Clear();
         }
         System.GC.Collect();      
      }
(Though, thinking about it, I don't really know how capture works. Maybe you're not supposed to dispose it ;-))
Hope any of this helps,
Albert

chant
Posts: 3
Joined: Thu Mar 12, 2009 9:56 am

Re: A blobs library (did I say "with code"?)

Post by chant » Mon Mar 23, 2009 8:40 am

Hi Albert,

Thanks for your reply. I'm quite certain that the memory leak is in the blob detection. If I comment the GetBlobs() method then there is no memory leak. I tried your code with the using statement and it still leaks. I think the problem is that the vector that stores the info about the blobs when calling BlobAnalysis() is not "cleaned", i.e. this part of cvBlobsLib.cpp

Code: Select all

CBlob** CvGetBlobs(IplImage* inputImage,
					int threshold,
				    IplImage* maskImage,
				    bool borderColor,
				    bool findMoments,
					int *numberOfDetectedBlobs)
{
	vector<CBlob*> blobVector = vector<CBlob*>();
	BlobAnalysis(inputImage, threshold, maskImage, borderColor, findMoments, [b]blobVector[/b]);
	*numberOfDetectedBlobs = blobVector.size();
	CBlob** blobs = new CBlob*[*numberOfDetectedBlobs];
	std::copy(blobVector.begin(), blobVector.end(), blobs);
	return blobs;
}
As far as I can see, the blobVector variable doesn't free its resources. The only way I could partly solve the memory leak was by adding the code to clear the blobVector as follows:

Code: Select all

CBlob** CvGetBlobs(IplImage* inputImage,
					int threshold,
				    IplImage* maskImage,
				    bool borderColor,
				    bool findMoments,
					int *numberOfDetectedBlobs)
{
	vector<CBlob*> blobVector = vector<CBlob*>();
	BlobAnalysis(inputImage, threshold, maskImage, borderColor, findMoments, [b]blobVector[/b]);
	*numberOfDetectedBlobs = blobVector.size();
	CBlob** blobs = new CBlob*[*numberOfDetectedBlobs];
	std::copy(blobVector.begin(), blobVector.end(), blobs);

	for(int i = 0; i<*numberOfDetectedBlobs; i++)
	{
		blobVector[i]->~CBlob();
	}

	return blobs;
}
The problem is that then I lose the pointer to the blobs, and therefore I cannot use the blobs when coming back to C# world.

I'm not a programmer myself, I just learn by doing, and I can't figure out how to solve this issue. It's not very often that I have to deal with unmanaged code... Any help will be appreciated, I have run out of ideas of what to do to eliminate the memory leak :(

Thanks!

PS: Albert please check your personal messages.

AlbertG
Posts: 84
Joined: Fri Jul 25, 2008 8:34 am

Re: A blobs library (did I say "with code"?)

Post by AlbertG » Mon Mar 23, 2009 11:16 am

The problem is that then I lose the pointer to the blobs, and therefore I cannot use the blobs when coming back to C# world.

I'm not a programmer myself, I just learn by doing, and I can't figure out how to solve this issue. It's not very often that I have to deal with unmanaged code... Any help will be appreciated, I have run out of ideas of what to do to eliminate the memory leak :(

Thanks!
I don't think you should free the blobs there. You do that in C# world later with blob.Clear(), which in the end just does the same. The arrays of blobs, though, probably they should be disposed. As I said, I haven't done any C++ programming in years, but I'm sure canming will help you when he notices this thread. Or maybe you should pm him. Which leads me to...
PS: Albert please check your personal messages.

Wow, sorry. I didn't even know I had private messages! Answering them right now.

chant
Posts: 3
Joined: Thu Mar 12, 2009 9:56 am

Re: A blobs library (did I say "with code"?)

Post by chant » Tue Mar 24, 2009 4:58 am

Freeing the blobs with blob.Clear() is what I was doing in the beginning, but then I noticed the memory leak and tried to track it down. I agree that freeing the blobVector in the C++ code doesn't seem sensible, but doing so helps to reduce the memory leak, so I guess it actually has an effect. But as I said, I don't exactly understand what is going on when moving from C++ to C#.

canming would you have some time to have a quick look at Albert's code? Maybe you are able to spot the source of the problem...

Thanks for your help!

asterisc
Posts: 5
Joined: Wed Apr 21, 2010 11:14 am

Re: A blobs library (did I say "with code"?)

Post by asterisc » Wed May 05, 2010 8:15 am

Hi all,

I have a question about Cvblobs.dll... where can I find it?
Could you help me?

Thank you.

bhegde
Posts: 2
Joined: Sat Jun 26, 2010 3:21 am

Re: A blobs library (did I say "with code"?)

Post by bhegde » Sat Jun 26, 2010 3:26 am

Please help me with the 'Unable to load DLL 'cvblobs.dll' I have compiled cVBlobs and have included in the reference also, and have made all the suggested modifications in the EMGU files ... But I am unable to proceed because of 'Unable to load DLL 'cvblobs.dll' error during run time ...
Please help me...

Regards,

Post Reply