Thursday, July 14, 2011

Eigenfaces with OpenCV 2

Time to try and make this blog live up to it's domain name. I recently started playing around with face recognition. It seems there are fairly few Open Source libraries for face recognition. What I found after some searching online was:
  • libface, Uses OpenCV's Eigenfaces.
  • The Colorado State University FR Algorithm Evaluation suite. It includes some source code and provides the following algorithms: 
    • Eigenfaces.
    • Eigenfaces with Linear Discriminant Analysis.
    • Bayesian Intrapersonal/Extrapersonal Classifier.
    • Elastic Bunch Graph Matching.
  • OpenCV, provides Eigenfaces.

Since I knew nothing about face recognition I began my quest by reading a pile of CS papers* and (inevitably) settled on starting with "Eigenfaces" which is the simplest of the holistic face recognition methods. There are several tutorials on how to implement Eigenfaces with OpenCV but most of the ones I found use the C AP and not the improved C++ API. On top of that the the C++ API was redesigned which means older tutorials that used the C API cannot always be quickly rewritten.

Some years ago Servo Magazine  published a fine series of OpenCV articles (made available online by The series included two installments on Eigenfaces. The first installment did a good job of explaining Eigenfaces and Principal Component Analysis.  The second installment presented a simple implementation. I set myself the goal of rewriting this code in the new OpenCV C++ API and testing it. I recommend reading the introductory part of the Servo Magazine article and the Eigenfaces paper by Turk & Pentland, "Eigenfaces for Recognition" before proceeding. The paper is quite well written and relatively easy to understand.

* The site has a papers list that is a good place to start.

Eigenfaces in the OpenCV C++ API
In the OpenCV C++ API Eigenfaces is packaged into the PCA object  which in turn is neatly tucked away under array operations. This class can be used to easily implement Eigenfaces and is simpler to use than the old C API:
  • PCA(const Mat& data, const Mat& mean, int flags, int maxComponents=0);
    • data - The set of training faces (Γ = {Γ1, Γ2 ... ΓM}).
    • mean - You can provide a pre-calculated average face (Ψ) of the set of training faces (Γ) or an empty matrix in which case the average face is calculated from the set of training faces you provided.
    • components - How many principal components to retain.
  • Mat project(const Mat& vec) const;
    • Projects a face vector into face-space: (ωk=ukT(Γ-Ψ)).
  • Mat eigenvectors;
    • The eigenfaces (ω) which are the training faces (Γ) after projection into face-space.
  • Mat eigenvalues;
    • The eigenvectors (u) used to project faces to and from face-space.
  • Mat mean;
    • The average face (Ψ) .
The code
I wrote a simple program (not my cleanest code ever) that does more or less the same as the test program in the Servo Magazie tutorial did and it uses the same Cambridge University Computer Lab face database (preview here). The program was written for and compiled on a MacBook so Windows and Linux users may have to do a little tweaking of the includes and the makefile before the code compiles. The test program has several options:

   eigenfacetest [-vs] [-t training image list] [-r test images list]
      -v: Verbose mode.
      -s: The projected face vectors can be retrieved and rendered
          as images. This option shows the average face and a 
          few of the eigenfaces.
      -r: A list of test files with faces to try and recognize.
      -t: A List of training files with faces for the recognizer.

    It expects two files: train_original.txt and test_original.txt which are the files that came with the original Servo Magazine article but you can specify your own test and training file sets with the -t and -r options.

    The average face on the left and three eigenfaces.

    A little experiment
    Running the test sets from the original article gave me an encouraging 100% recognition rate which was way too good to be true. I decided to run the program on the entire set of test images using the first image for each individual as the training image and the rest as test images. The experiment was then repeated with two training images and eight test images etc.


    The results were not quite as good as they were using the original Servo Magazine test-sets but rather consistent with what I thought they would be after reading all those CS papers i.e, that he accuracy would be between 60 and 70%. The accuracy goes up as the size of the training set increases. The accuracy with only one training image per subject is what one would expect from a simple Eigenfaces implementation and the fact that the Cambridge University face database, although small, contains a fair number of non frontal faces. The CU data set is, however, well lit and one would expect that variations in lighting conditions would cause the recognition rate to go down. To improve the results one might try further automatic preprocessing with eye detection, rotation and scaling.

    • The source code, it's makefile and the test set lists can be downloaded here
    • You'll have to download the face database your self, here.

    Wednesday, March 23, 2011

    Infinity blade

    I bought yet another iPhone game, Infinity Blade. It's a 3rd person over-the-shoulder type game but it's not like the ones I usually buy where you have complete freedom of movement. Infinity blade is more like Doom Resurrection, you have limited control over where you want to move except at decision points where you can choose to take one of several routes. The plot is simple, you navigate your way through a castle, dueling a string of monsters on the way until, you reach the throne room of the immortal god-king. Once you get past his champion the dark knight, you get a single crack at killing the boss him self. If you fail you have to start over from the beginning. As you progress through the game you gather money, and experience. The money you can spend on better weapons and armor, the XP points you can spend on better defensive, offensive or magic skills. As you gather XP your skill level increases but since XP points are gathered by "mastering" weapons you are under constant pressure to upgrade your arsenal since your skill level will advance slowly or not at all if you are using a set of mastered armor, shield, sword etc. At certain points in the game, usually before or after duels you can pan to the sides to look around which is important to remember since bonus items are often placed out of sight.

    The game it self has amazingly nice graphics for a mobile game, it should do since it uses the Unreal engine. The movements of the characters are also very well done and convincing. Like all of these 3D games it will burn through your iPhone's battery with blistering speed. All in all, IMHO, it's good value at € 4.99 since it's a game that's reasonably hard to master and in a sense goes on infinitely. So, thumbs up!

    A fire paladin,  his frozen buddy,  an assassin...

    they have feral trolls,  knights,  robots...

    a big ugly baby,  the dark knight and once you get past him...

    you finally get to duel with the boss, who kicks your ass many times forcing you to start over, until you finally manage to kill him.

    You don't get to keep the infinity blade, then you press a button and a hologram appears but only after you watch the credits. After all this the game starts all over again. At least you get to keep your XP and your weapons.

    Saturday, March 12, 2011

    Xcode 4 and App Store access (maybe)

    I just downloaded Xcode 4. Apparently Xcode is now only available through the new "OS X App Store" and the price tag is $4.99. It's not that I mind paying a token $4.99 for downloading Xcode even if it seems a bit cheap of Apple to charge money for a dev suite that used to be freely available (Microsoft charges a much higher price for anything beyond the very basic free edition of Visual Studio). What really annoys me though, is that the App Store is only accessible to residents of a select set of countries. Since I happen to live in Iceland which has up until now not had App Store access I have to jump through hoops to purchase App-Store-only software. The local press reports that we are finally going to get App Store access even if it will only include free apps and no access to media. The same source also reports that delays in Iceland being getting App Store access is a combination of us being a microstate and the fact that our fearless leaders came within a whisker of flushing the national economy down the toilet, so it's not just Apple that's to blame. Having to deal with gift card scalpers in order to be able to purchase software after investing a significant amount of money in an Apple laptop is annoying to say the least. If Apple decides to lock down OS X completely I'll be migrating to another OS. I'll put up with this kind of hoop jumping for acquiring software for an iPhone/iPad or an Android gadget but not for my desktop OS.

    Tuesday, March 8, 2011

    Bluetooth Headphones

    The jacks on the corded headphones I use with my iPhone will usually wear out after 3-4 months so I bought a set of refurbished Sennheiser MM200 bluetooth headphones in the hope they will last longer. The sound quality is excellent and far better than what I expected from such small in-ear speakers. The MM200 doubles as a mobile headset (so far no complaints about people not hearing my voice) and as a remote for the iPhone. The skipping and volume control functions apparently did not work on earlier incarnations of iOS but they both work perfectly on my iPhone 3GS running iOS 4. Furthermore pressing the play button after powering up the MM200 connects it to the iPhone without also activating voice control on the phone unlike every other bluetooth headset I ever had. Reportedly the MM200 suffers from insufficient waterproofing that causes it to break if you wear it while exercising. I haven't experienced this in the month I have had the MM200 even though I cycle 15 km to school. The only downside is that the battery on the iPhone drains quite a bit faster thanks to bluetooth so you might want to switch off Location Services and Wifi or buy an external battery pack. The verdict so far is... thumbs up.

    Thursday, February 24, 2011

    iPod Photo Cache folder

    I finally summoned the energy to figure out how to get rid of that annoying iPod Photo Cache folder that's been bugging me ever since I experimented with syncing some photos to my iPhone a while ago. Even after I switched off photo syncing in iTunes and deleted the iPod Photo Cache folder the damn thing would reappear on my desktop every time I connected the phone to my laptop. The trick is:
    1. Connect your iPhone/iPod to your Mac.
    2. This step is probably unnecessary but I'm paranoid. Back up the photos from your iPhone with the Image Capture utility. Not a perfect tool but it allows you to pull photos from your iPhone to a plain old directory and organize them your self which I like better than using iPhoto. 
    3. Navigate to the "Photos" tab in iTunes.
    4. Check "Sync photos from" and expand the drop down box next to that text. Apparently the photo cache gets re-created in the folder chosen here every time you connect your phone even after you disable photo syncing.
    5. Select some other out of the way folder, even an empty one. I created an empty folder "~/Pictures/iPhone Sync".
    6. Apply the changes and Sync the empty folder once and then deactivate Photo syncing.
    This cured the issue for me.