Raspberry Pi Face Recognition Using OpenCV

Raspberry Pi Face Recognition and Object Detection Using OpenCV

About a year ago, I created a Wall-E robot that does object and face recognition. It uses Arduino as the controller and need to communicate with a computer that runs the face detection program to track the target. Raspberry Pi face recognition has become very popular recently. With the powerful processor on Raspberry Pi, I can connect it with the Arduino using i2c on the robot and run the object recognition program on-board. It could become a truly independent, intelligent Wall-E robot!

However, building such a robot will be a project for near future. In this article, I will be showing you how to do basic Object Recognition on the Raspberry Pi using Python and OpenCV. I will also show you a simple open loop face tracking application using pan-tilt servos to turn the camera around.

Note: Please be careful about the indentation in the Python codes, sometimes my blog decides to mess this up randomly.

I wrote an article on how to use SSH and VNC to control and monitor the Raspberry Pi, and that’s what I will be using in this project.

Installing OpenCV For Python

To install OpenCV for Python, all you have to do is use apt-get like below:

sudo apt-get install python-opencv

To test the installation of OpenCV, run this Python script, it will switch on your camera for video streaming if it is working.

import cv

cv.NamedWindow("w1", cv.CV_WINDOW_AUTOSIZE)
camera_index = 0
capture = cv.CaptureFromCAM(camera_index)

def repeat():
    global capture #declare as globals since we are assigning to them now
    global camera_index
    frame = cv.QueryFrame(capture)
    cv.ShowImage("w1", frame)
    c = cv.WaitKey(10)
    if(c=="n"): #in "n" key is pressed while the popup window is in focus
        camera_index += 1 #try the next camera index
        capture = cv.CaptureFromCAM(camera_index)
        if not capture: #if the next camera index didn't work, reset to 0.
            camera_index = 0
            capture = cv.CaptureFromCAM(camera_index)

while True:
    repeat()

Simple Example of Raspberry Pi Face Recognition

This example is a demonstration for Raspberry Pi face recognition using haar-like features. it finds faces in the camera and puts a red square around it. I am surprised how fast the detection is given the limited capacity of the Raspberry Pi (about 3 to 4 fps). Although it’s still much slower than a laptop, but it would still be useful in some robotics applications.

Raspberry Pi Face Recognition and Object Detection Using OpenCV

You will need to download this trained face file:

http://stevenhickson-code.googlecode.com/svn/trunk/AUI/Imaging/face.xml

#!/usr/bin/python

The program finds faces in a camera image or video stream and displays a red box around them.

import sys
import cv2.cv as cv
from optparse import OptionParser

min_size = (20, 20)
image_scale = 2
haar_scale = 1.2
min_neighbors = 2
haar_flags = 0

def detect_and_draw(img, cascade):
    # allocate temporary images
    gray = cv.CreateImage((img.width,img.height), 8, 1)
    small_img = cv.CreateImage((cv.Round(img.width / image_scale),
			       cv.Round (img.height / image_scale)), 8, 1)

    # convert color input image to grayscale
    cv.CvtColor(img, gray, cv.CV_BGR2GRAY)

    # scale input image for faster processing
    cv.Resize(gray, small_img, cv.CV_INTER_LINEAR)
    cv.EqualizeHist(small_img, small_img)

    if(cascade):
        t = cv.GetTickCount()
        faces = cv.HaarDetectObjects(small_img, cascade, cv.CreateMemStorage(0),
                                     haar_scale, min_neighbors, haar_flags, min_size)
        t = cv.GetTickCount() - t
        print "time taken for detection = %gms" % (t/(cv.GetTickFrequency()*1000.))
        if faces:
            for ((x, y, w, h), n) in faces:
                # the input to cv.HaarDetectObjects was resized, so scale the
                # bounding box of each face and convert it to two CvPoints
                pt1 = (int(x * image_scale), int(y * image_scale))
                pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
                cv.Rectangle(img, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)

    cv.ShowImage("video", img)

if __name__ == '__main__':

    parser = OptionParser(usage = "usage: %prog [options] [filename|camera_index]")
    parser.add_option("-c", "--cascade", action="store", dest="cascade", type="str", help="Haar cascade file, default %default", default = "../data/haarcascades/haarcascade_frontalface_alt.xml")
    (options, args) = parser.parse_args()

    cascade = cv.Load(options.cascade)

    if len(args) != 1:
        parser.print_help()
        sys.exit(1)

    input_name = args[0]
    if input_name.isdigit():
        capture = cv.CreateCameraCapture(int(input_name))
    else:
        capture = None

    cv.NamedWindow("video", 1)

    #size of the video
    width = 160
    height = 120

    if width is None:
    	width = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH))
    else:
    	cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_WIDTH,width)

    if height is None:
	height = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT))
    else:
	cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_HEIGHT,height)

    if capture:
        frame_copy = None
        while True:

            frame = cv.QueryFrame(capture)
            if not frame:
                cv.WaitKey(0)
                break
            if not frame_copy:
                frame_copy = cv.CreateImage((frame.width,frame.height),
                                            cv.IPL_DEPTH_8U, frame.nChannels)

            if frame.origin == cv.IPL_ORIGIN_TL:
                cv.Copy(frame, frame_copy)
            else:
                cv.Flip(frame, frame_copy, 0)

            detect_and_draw(frame_copy, cascade)

            if cv.WaitKey(10) >= 0:
                break
    else:
        image = cv.LoadImage(input_name, 1)
        detect_and_draw(image, cascade)
        cv.WaitKey(0)

    cv.DestroyWindow("video")

To run this program, type in this command in your VNC Viewer’s terminal:

python facedetect.py --cascade=face.xml 0

The number at the end represents the number of your video device.

Face Tracking in Raspberry Pi with pan-tilt Servos

In this example I will be using the Wall-E Robot‘s camera and pan-tilt servo head.

The idea is simple. Raspberry Pi detects the position of the face, sends a command to the Arduino. Arduino will convert the command into servo position and turn the camera. I am using i2c to connect Raspberry Pi and Arduino.

Note: I am still trying to optimize the code for this example, so the result is still not great, but it gives you the idea how it works. I will come back and update this post as soon as I am happy with the result.

Raspberry Pi Face Recognition and Object Detection Using OpenCV

Raspberry Pi Face Recognition and Object Detection Using OpenCV

Arduino Source Code

Here is the Arduino code. Note that this code uses a very dummy and basic open loop control method, I only use this because of its simplicity. For a more optimal control method, please see Color Tracking Using PID.

In this example it basically waits for commands from the Raspberry Pi and turn the head around. The commands are expected to be integer 1, 2, 3 or 4, each represents a direction that it should turn the camera to. While it’s turning, the variable ‘state’ will be set to zero, so Raspberry Pi will stop detecting or sending any more command to avoid turning the camera too far, because of the delays.

Like I mentioned at the beginning, this is an open loop control system, and still has a lot of room to improve. I made it so simple just for some people to pick up more easily.

#include <Wire.h>
#define SLAVE_ADDRESS 0x04
byte command;
byte state;

// Servo code
#include <Servo.h>

Servo servoNeckX;
Servo servoNeckY;

const byte servoNeckX_pin = 3;
const byte servoNeckY_pin = 4;

const int lrServoMax = 2300; // looking right
const int lrServoMin = 700;
const int udServoMax = 2100; // looking down
const int udServoMin = 750;  // looking up

int posX = 1500;
int posY = 1300;

// End of Servo code

void setup() {

	servoNeckX.attach(servoNeckX_pin);
	servoNeckY.attach(servoNeckY_pin);

	servoNeckX.writeMicroseconds(posX);
	delay(100);
	servoNeckY.writeMicroseconds(posY);
	delay(100);

    // initialize i2c as slave
    Wire.begin(SLAVE_ADDRESS);

    // define callbacks for i2c communication
    Wire.onReceive(receiveData);
    Wire.onRequest(sendData);

	Serial.begin(9600);         // start serial for output
    Serial.println("Ready!");

	state = 1;
}

void loop() {
    delay(20);
}

// callback for received data
void receiveData(int byteCount){

	while(Wire.available()) {
		state = 0; // moving servos
		command = Wire.read();
		Serial.print("command received: ");
		Serial.println(command);

		switch (command){

		case 1:
			// lift head (-Y)
			posY = constrain(posY-20, udServoMin, udServoMax);
			servoNeckY.writeMicroseconds(posY);
			break;

		case 2:
			// lower head (+Y)
			posY = constrain(posY+20, udServoMin, udServoMax);
			servoNeckY.writeMicroseconds(posY);
			break;

		case 3:
			// turn head left (+X)
			posX = constrain(posX+20, lrServoMin, lrServoMax);
			servoNeckX.writeMicroseconds(posX);
			break;

		case 4:
			// turn head right (-X)
			posX = constrain(posX-20, lrServoMin, lrServoMax);
			servoNeckX.writeMicroseconds(posX);
			break;
		}
		state = 1; // finished moving servos

	}
}

// callback for sending data

void sendData(){
	Wire.write(state);
}

The Python Code is similar to the first example. I added some code necessary for i2c communication and a few lines after a face is detected, so it sends commands to the Arduino.


#!/usr/bin/python

import sys
import cv2.cv as cv
from optparse import OptionParser

min_size = (20, 20)
image_scale = 2
haar_scale = 1.2
min_neighbors = 2
haar_flags = 0

def detect_and_draw(img, cascade):
    # allocate temporary images
    gray = cv.CreateImage((img.width,img.height), 8, 1)
    small_img = cv.CreateImage((cv.Round(img.width / image_scale), cv.Round (img.height / image_scale)), 8, 1)

	# convert color input image to grayscale
	cv.CvtColor(img, gray, cv.CV_BGR2GRAY)

	# scale input image for faster processing
	cv.Resize(gray, small_img, cv.CV_INTER_LINEAR)
	cv.EqualizeHist(small_img, small_img)

	if(cascade):
		t = cv.GetTickCount()
		faces = cv.HaarDetectObjects(small_img, cascade, cv.CreateMemStorage(0),
		haar_scale, min_neighbors, haar_flags, min_size)
		t = cv.GetTickCount() - t
		print "time taken for detection = %gms" % (t/(cv.GetTickFrequency()*1000.))
		if faces:
			for ((x, y, w, h), n) in faces:
				# the input to cv.HaarDetectObjects was resized, so scale the
				# bounding box of each face and convert it to two CvPoints
				pt1 = (int(x * image_scale), int(y * image_scale))
				pt2 = (int((x + w) * image_scale), int((y + h) * image_scale))
				cv.Rectangle(img, pt1, pt2, cv.RGB(255, 0, 0), 3, 8, 0)

	cv.ShowImage("video", img)

if __name__ == '__main__':

	parser = OptionParser(usage = "usage: %prog [options] [filename|camera_index]")
	parser.add_option("-c", "--cascade", action="store", dest="cascade", type="str", help="Haar cascade file, default %default", default = "../data/haarcascades/haarcascade_frontalface_alt.xml")
	(options, args) = parser.parse_args()

	cascade = cv.Load(options.cascade)

	if len(args) != 1:
		parser.print_help()
		sys.exit(1)

	input_name = args[0]
	if input_name.isdigit():
		capture = cv.CreateCameraCapture(int(input_name))
	else:
		capture = None

	cv.NamedWindow("video", 1)

	#size of the video
	width = 160
	height = 120

	if width is None:
		width = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_WIDTH))
	else:
		cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_WIDTH,width)

	if height is None:
		height = int(cv.GetCaptureProperty(capture, cv.CV_CAP_PROP_FRAME_HEIGHT))
	else:
		cv.SetCaptureProperty(capture,cv.CV_CAP_PROP_FRAME_HEIGHT,height)

	if capture:
		frame_copy = None

	while True:

		frame = cv.QueryFrame(capture)
		if not frame:
			cv.WaitKey(0)
			break
		if not frame_copy:
			frame_copy = cv.CreateImage((frame.width,frame.height),
			cv.IPL_DEPTH_8U, frame.nChannels)

		if frame.origin == cv.IPL_ORIGIN_TL:
			cv.Copy(frame, frame_copy)
		else:
			cv.Flip(frame, frame_copy, 0)

		detect_and_draw(frame_copy, cascade)

		if cv.WaitKey(10) >= 0:
			break
		else:
			image = cv.LoadImage(input_name, 1)
			detect_and_draw(image, cascade)
			cv.WaitKey(0)

	cv.DestroyWindow("video")

Possible Raspberry Pi Face Recognition Improvement

For face recognition on an embedded system, I think LBP is a better choice, because it does all the calculations in integers. Haar uses floats, whick is a killer for embedded/mobile. LBP is a few times faster, but about 10-20% less accurate than Haar.

Subscribe to our Youtube Channel for more videos :D
Don't have much on Google Plus, but follow me anyway :)
Donate any amount, to help us maintain this website.
Love Multicopters? Join our discussion Group on Facebook!

53 thoughts on “Raspberry Pi Face Recognition Using OpenCV

  1. Good write up buddy, however this is face DETECTION not RECOGNITION as you have put in the title.
    Just saves people getting the wrong thing from their google search.

    Cheers

  2. Hello..
    Do you have any suggestion of other trained files? (instead of face, is there any file for objects??)
    Sorry, I am very new to object detection.

Leave a Reply

Your email address will not be published. Required fields are marked *


four × = 20

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>