Face Recognition with OpenCV

The complexity of machines have increased over the years and computers are not an exception. Computers have helped mankind solve lots of problems and complete lots of difficult tasks. Gone are the days when all computers did was simple arithmetic operations, computers now drive the world.

Computers have become so complex, they are being trained to think like humans.
Yes!

We are going to do something of that nature in this article. As humans, recognizing the faces of other people is a simple task and despite the abilities of today's computers it's not as easy for the computer so we have to train it to be able to do the same.

A lot of articles you would see out there get to stop at simple face detection, but in this article would be covering not just face detection but face recognition as well.

This means if the computer is presented with two pictures of me, it would not only recognize what part of the picture is my face, it would also recognize that I am the one in both pictures as well.

To start with, we would have to first install opencv on our machines, which can only be done if you have Python installed. The installation of Python is not the objective of this article, so if you do not already have it on your machine you can get to install Python from the Python Website.

To install Open CV, we can do that using the pip command.

pip install opencv-python

We will also be making use of the numpy package in this article, which should be installed alongside OpenCV using the above command.

If numpy didn't install, you can easily do so using the command below:

pip install numpy

To confirm that your OpenCV is installed, when you activate Python's interactive environment try to import it using:

import cv2

If do not get an error, then you can proceed.

To carry out facial recognition, we would be writing three scripts. One to create a dataset of images, another to train those images and then the last one to recognize the faces based on the results of the training the computer goes through.

We would be needing the Haar Cascade provided by Open CV. This file can be gotten from the opencv directory which is cv2/data/haarcascade_frontalface_default.xml on my machine it should be same on your machine too. Copy the file into the folder where you wish to do the face recognition.

Now let's get into the thick of things.
We would try to get our web cam to get the pictures, needed for the dataset.

import cv2
vid_cam = cv2.VideoCapture(0)
face_detector = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
face_id = 1
count = 0
while(vid_cam.isOpened()):
ret, image_frame = vid_cam.read()
gray = cv2.cvtColor(image_frame, cv2.COLOR_BGR2GRAY)
faces = face_detector.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
cv2.rectangle(image_frame, (x,y), (x+w,y+h), (255,0,0), 2)
count += 1
cv2.imwrite("dataset/User." + str(face_id) + '.' + str(count) + ".jpg", gray[y:y+h,x:x+w])
cv2.imshow('frame', image_frame)
if cv2.waitKey(100) & 0xFF == ord('q'):
break
elif count>100:
break
vid_cam.release()
cv2.destroyAllWindows()

So to explain what each line of code does:

import cv2

Here is the command that tells python to include an external library to be used in this code, in this case it is Open CV.

vid_cam = cv2.VideoCapture(0)

This code calls on the imported Open CV library to begin capturing and the webcam is initiated at this point. If the Open CV doesn't support your webcam, the code will fail here.

face_detector = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

For us to be able carry out image detection, this code is needed. Open CV uses the 'haarcascade_frontalface_default.xml' for Cascade Classification. The resulting object is then stored in the face_detector variable.

face_id = 1

Here is a case of setting the id number of the face, so the first face gets an id of 1.

count = 0

We are going to be taking a couple of images as Open CV needs to train images to be able to recognize faces, the count variable serves as an image count.

while(vid_cam.isOpened()):

This allows the following operations to proceed provided the video camera is opened. The isOpened() method returns True or False.

ret, image_frame = vid_cam.read()

Here, the vid_cam.read() looks into the video capture and then captures the frame which is stored in the image_frame variable, if the operation is successful the boolean True is returned and stored in the ret variable

gray = cv2.cvtColor(image_frame, cv2.COLOR_BGR2GRAY)

The cvtColor() method is used to convert the image frame into the desired color type. In this case we have converted it into grayscale.

faces = face_detector.detectMultiScale(gray, 1.3, 5)

This checks for frames of different sizes and tries to set them to scale, this is applied on the variable to which the Haar Cascade was applied.

for (x,y,w,h) in faces:

Here we loop through the faces and its dimensions, where x and y stand for the coordinates and w and h stand for width and height respectively.

cv2.rectangle(image_frame, (x,y), (x+w,y+h), (255,0,0), 2)

Remember that we are still working with the video camera, the video camera then crops the need part of the image according to the dimensions above.

count += 1

Immediately that is done, the count variable which stands as a counter then increments.

cv2.imwrite("dataset/User." + str(face_id) + '.' + str(count) + ".jpg", gray[y:y+h,x:x+w])

The cropped image is the saved with the name User(face_id).(count).jpg and put into a folder called dataset.

cv2.imshow('frame', image_frame)

After save, this code ensures the image is video frame is displayed with a rectangle on the individual's face after face detection has been done.

if cv2.waitKey(100) & 0xFF == ord('q'):
break

After the each picture, the user is allowed to stop the program from taking more pictures which can be done by pressing the 'q' on the keyboard for atleast 100ms.

elif count>100:
break

What this code does is to stop the video from working the moment 100 pictures have been taken, regardless if user wants to take more or not.

vid_cam.release()

Here, the web cam is closed and not just stopped from taking pictures.

cv2.destroyAllWindows()

Then all the windows OpenCV has opened have been destroyed and the code runs to conclusion.

Now that we are done with that, we can then get to train the image dataset:

import cv2, os
import numpy as np
from PIL import Image
recognizer = cv2.face.createLBPHFaceRecognizer()
detector = cv2.CascadeClassifier("haarcascade_frontalface_default.xml");
def getImagesAndLabels(path):
imagePaths = [os.path.join(path,f) for f in os.listdir(path)]
faceSamples=[]
ids = []
for imagePath in imagePaths:
PIL_img = Image.open(imagePath).convert('L')
img_numpy = np.array(PIL_img,'uint8')
id = int(os.path.split(imagePath)[-1].split(".")[1])
faces = detector.detectMultiScale(img_numpy)
for (x,y,w,h) in faces:
faceSamples.append(img_numpy[y:y+h,x:x+w])
ids.append(id)
return faceSamples,ids
faces,ids = getImagesAndLabels('dataset')
recognizer.train(faces, np.array(ids))
recognizer.save('trainer/trainer.yml')

Let's go ahead and explain this code too:

import cv2, os

Just like the other code, here we are importing OpenCV and os which we would be needing for file path.

import numpy as np

We are also importing the numpy library which would be used for matrix calculation(a matrix is a just an arrangement of arrays).

from PIL import Image

We are importing the Python Image Library and then from it we are getting the Image library from this package too.

recognizer = cv2.face.createLBPHFaceRecognizer()

What this does is to apply the createLBPHFaceRecognizer() method to the cv2.face object, this would help make the recognition of faces easy as we do not have to come up with our own set of algorithms.

detector = cv2.CascadeClassifier("haarcascade_frontalface_default.xml");

If you have been following the tutorial, you would have come across this before. It helps with face detection using the “haarcascade_frontalface_default.xml” for the Cascade Classification.

def getImagesAndLabels(path):

Now, we are about to begin the image training proper, so we create a function.

imagePaths = [os.path.join(path,f) for f in os.listdir(path)]

This code checks into the current directory of the file and checks for the image files then adds them to this list.

faceSamples=[]

This initializes a list of samples, it is empty at this point but faces would be added as the code runs.

ids = []

Initialize a list of ids, which is initially empty.

for imagePath in imagePaths:

Remember the code that checked for the image files in the directory? Yes? Now, we are going to loop through each of those files and carry out operations on them.

PIL_img = Image.open(imagePath).convert('L')

Now the first thing we do to the image is to convert it to grayscale, and this code does that.

img_numpy = np.array(PIL_img,'uint8')

The grayscaled image is just a series of numbers all in one place, so we create a numpy array out of them and assign it to a variable.

id = int(os.path.split(imagePath)[-1].split(".")[1])

If you recall the file that gets the images, you would recall that we named the files User(face_id).count.jpg. So here we are splitting the names with the “.” and then we extract the face_id and assign to a variable here. We would need the id for recognition.

faces = detector.detectMultiScale(img_numpy)

From the numpy array, the detectMultiScale() method is going to try to detect the faces from the pattern it finds in the numpy array. Then it assigns the values in the faces variable.

for (x,y,w,h) in faces:

Here, we are looping through the values assigned to the variable. The values here are the x and y coordinates which we could take as the origin, and then w and h standing for width and height respectively.

faceSamples.append(img_numpy[y:y+h,x:x+w])

Earlier we created a list of face samples, but it was empty. Here we get to add faces to that list, and we are adding the y to h so as to get the two values of the y coordinates and same is done to x.

ids.append(id)

We now have a face in the face sample list, so we get its id and append it to the ids list too.

return faceSamples,ids

Then after it all, we return the list of face samples and the list of ids.

faces,ids = getImagesAndLabels('dataset')

Remember that getImagesAndLabels() is just a function. So we get to call the function here, and the return values are saved in the faces and ids variables.

recognizer.train(faces, np.array(ids))

Here is where the real training happens. We applied the createLBPHFaceRecognizer() method sometime earlier and assigned to a recognizer variable. It's training time!

recognizer.save('trainer/trainer.yml')

After training, we get to save the results from the training.
After running the code, it creates a file called trainer.yml which would then be used by the face recognition code.

Here is the face recognition code:

import cv2
import numpy as np
recognizer = cv2.face.createLBPHFaceRecognizer()
recognizer.load('trainer/trainer.yml')
cascadePath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(cascadePath)
font = cv2.FONT_HERSHEY_SIMPLEX
cam = cv2.VideoCapture(0)
while True:
ret, im =cam.read()
gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(gray, 1.2,5)
for(x,y,w,h) in faces:
cv2.rectangle(im, (x-20,y-20), (x+w+20,y+h+20), (0,255,0), 4)
Id = recognizer.predict(gray[y:y+h,x:x+w])
if(Id == 1):
Id = "Nazmi"
else:
Id = "Unknown"
cv2.rectangle(im, (x-22,y-90), (x+w+22, y-22), (0,255,0), -1)
cv2.putText(im, str(Id), (x,y-40), font, 2, (255,255,255), 3)
cv2.imshow('im',im)
if cv2.waitKey(10) & 0xFF == ord('q'):
break
cam.release()
cv2.destroyAllWindows()

If you have been following the article from the beginning, we have done this before. If you haven't kindly do.

recognizer.load('trainer/trainer.yml')

Remember we trained the recognizer and saved a file? Yes? We are loading that file now.

cascadePath = "haarcascade_frontalface_default.xml"

We would be working with the haarcascade file, and here we have assigned the file name to a variable.

# Create classifier from prebuilt model
faceCascade = cv2.CascadeClassifier(cascadePath)

Here we get to carry out Cascade classification on the haarcascade file.

font = cv2.FONT_HERSHEY_SIMPLEX

We set the font type that would be used when the code recognizes the face in an image and displays the name.

cam = cv2.VideoCapture(0)

We've been here before, but this time it's time to recognize the faces. If you do not know what this code does, it launches the webcam.

while True:
ret, im =cam.read()
gray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
faces = faceCascade.detectMultiScale(gray, 1.2,5)
for(x,y,w,h) in faces:

All these have been done before, kindly check the code that was used to save images if you do not know what the code does.

cv2.rectangle(im, (x-20,y-20), (x+w+20,y+h+20), (0,255,0), 4)

So this helps the webcam detect where the faces are and places a rectangle on to indicate a face.

Id = recognizer.predict(gray[y:y+h,x:x+w])

We have alrerady loaded the train file into the recognizer, so it is able to to recognize the face now.

if(Id == 1):
Id = "Myself"
else:
Id = "Unknown"

After trying to recognize what face it is, it checks for the id and sees if it exists. Here, the value of the Id would be the name of whoever was owned the faced with such an id when the image dataset was being created.

cv2.rectangle(im, (x-22,y-90), (x+w+22, y-22), (0,255,0), -1)
cv2.putText(im, str(Id), (x,y-40), font, 2, (255,255,255), 3)

The code after finding the owner of the Id, draws a rectangle round the face and places the name of the owner of the face. Face recognized!

cv2.imshow('im',im)

Here, the video frame is displayed with the bounded rectangle.

if cv2.waitKey(10) & 0xFF == ord('q'):
break
cam.release()
cv2.destroyAllWindows()

So when done, you can stop the program by pressing 'q' key, and it stops the webcam and closes it.

There you have it, your webcam can now recognize faces and you can use it whenever you wish. Asides using the webcam you can also load an image, however that requires some other steps than those taken in this article.

You can find the source code used on its github repo. Also tweet us if you have comments or want to discuss @linuxhint