Class 4

This class covers video stream processing and face recognition in OpenCV.

We have learned how to insert decorations to any locations in a image. The next step is to use face recognition to locate the face in camera. In common sense, a video is simply a stack of a serious continuous images called frame. By extracting those frames, we can process the whole video in the same way we process the image. Hence, our plan will be:

• Stream video from camera
• Extract every single frame
• Locate human face on each frame
• Insert our decoration
• Display and loop

Processing Video

To process video file or video stream from camera, we need to use a class provided by OpenCV called VideoCapture. This class provides convinent member functions that can access camera and extract frames easily.

• Member functions: ```cpp VideoCapture capture;

void capture.open(int deviceID);
//open the camera, for built-in camera, device ID is 0

bool capture.isOpen();
//return false if camera is not opened

bool capture.read(Mat frame); //capture frames from the stream, store frame image in Mat frame //return false if failed to read the frame

- Program structure of reading frames:
```cpp
Mat frame;  //initialize a Mat object to store the frame read in
capture.open( camera_ID );  //open the camera, the primary ID for built-in camera on computer is 0

if (!stream1.isOpened())  //check if video device has been initialised
{
  cout << "cannot open camera";
}

while ( capture.read(frame) )   //break the loop if frame reading fails
{
	if( frame.empty() ){    //check if the frame being successfully read in
		cout << "--(!) No captured frame -- Break!\n";
		break;
	}
	
  imshow("camera",frame); //process your frame here
	
  if( waitKey(1) == 2 ) //control the imshow refresh rate
	{
		break;
	}
}

Processing Video

In this project, we will use the Haar-feature based cascade classifier to detect human face and eyes. Haar-feaure is a series of patterns that describ edges, lines, and center-surround relations in the image. The classifier can be trained to detect the Haar-features in the image. OpenCV has prepared the classifier to us. The class is called CascadeClassifier. All we need to do is to load the machine learning file into the classifier, and do recognition on the pre-processed images.

bool classifier.load(string cascadeFilePath);	
//load the xml file in system to the classifier, read in file path
//return false if file not read in

void classifier.detectMultiScale(Mat imgROI, vector<Rect> matches);	
//detect the targeting object of multiple scales in the image
//imgROI can be selected manually, can also be the whole picture
//matches is a Rect vector that stores the coordinate, width, and height of the object detect in the image

Display the result

In order to result of face recognition, we can draw a rectangle around the face we detected. Here is the implementation:

//specify the two diagonal corner points of the detected region
Point corner1( matches[i].x, matches[i].y );
Point corner2( matches[i].x + matches[i].width, matches[i].y + matches[i].height);

//draw the rectangle
rectangle( img, corner1, corner2, 1, LINE_8, 0);