Face Detection using OpenCV

A while back I started playing around with image detection for my drones.  I wanted to see if a drone could go out and find something.  I originally started playing around with a couple of proprietary packages when a friend on mine pointed me toward OpenCV.  Wiki describes OpenCV as an open-source computer vision library geared toward real-time vision applications (launched in 1999).

Below there are 4 sets of images, each consisting of 4 head-shots.  Looking at the first set of images (top-Left) you'll see I have the image detection set for Heads (Blue), Eyes (Green), and Smiles (Blue).  The very first image shows the entire face for which the code detected all three (Head, Eyes and Smile).  For the second image in that group I removed the eyes to see what would happen.  As you can see, not only did the software not detect the eyes, it also did not find the head.  This is because the algorithm used to detect the head uses the eyes as a reference.  For the third image in that group I removed the mouth and again you can see eyes but the head was not recognized.  And for the last image in that first group I removed the head and you can see the eyes and smile were recognized.  So why does it seem the head is dependent on the eyes and mouth?  The answer, Haar Cascades Classifier definition files.  Please read the hyperlink above, there is just too much to type. 

The second group of images I set the program to exclusively look for smiles.  In the first image of the group the software picked up the smile with no problem.  The second image of the group is a half-smile and to my surprise I got a half detection.  Images 3 and 4 of the group did not get detected.  Without looking through the classifier for the smile feature it is easy to see it was trained to look for the shape of a smile.  A couple of times I got a false hit with the software detecting the chin as the smile or detecting both the chin and the smile.  When this happens you can adjust the scaling-factors and neighbor node parameters.       

For the third group of images I just focused on the face.  As mentioned above, the classifier for the face has many more layers and is much more involved than the other classifiers.  Without hair the program detects just a small portion of the face because it does not have the reference(s) of the hairline features.  Images 2 and 3 of the group were detected even though they were a figurine and a drawing.  Image 4 (the pumpkin) was not detected due to the fact most all the trained features do not match the image.  

The last group of images was to highlight detection of eyes.  Again, like the Smile classifier it is pretty straight forward.  I intentionally remove the head features of image 1 and the eyes were still detected.  Image 2 (the dog eyes) was not detected, nor was the spider eye.  I played around with the images to find it had to do with the spacing of the eyes.  Remember, the classifiers are just location based pixel tables.  The dogs eyes are wider than the girls.  When I used a Photo-Shopped image of the dog's eyes moving them closer together the program detected the eyes.  

The attached snippet is just to demonstrate some of OpenCV's image detection routines.  For me, the next step will be to build a classifier to detect Drones.  The plan is to have one of my Drones detect and chase another Drone.  Stay tuned for updates to this article.  

             

OpenCV is written for use with C/C++ but also has existing wrappers for C#, Perl, and Ruby.  I built this OpenCV version in a wrapper for use with VB.NET.  As mentioned, my original intent was to use these libraries for my projects but I also found it useful at work.  However, I was forbidden to write code in C/C++ there (go figure?!?!).  The VB.NET wrapper was a compromise.  In any case, the effort was well worth it.  OpenCV proved itself to be a very useful kit of tools.  The VB wrapper takes a little bit of extra work but performs just as well as the original C/C++ libraries.  Back-to-back performance tests show little to no loss in performance.  And in addition to the other code wrappers, there are bindings available for Python, Java, and even Octave (MatLab).         
  

Face Detection (Head, Eyes, and Smile)
 

Face Detection - Face Detection using OpenCV in VB Wrapper