ABOUT
Future improvement & Conclusion
1. Better detection with less restriction (detect faces with various orientations)
2. Better overlay of the sticker (use algorithms taught in class that blur the edge between stickers and the original image)
3. More precise orientation of the sticker (which is glass in this case). In other words, when faces have some strange orientation, an angle that is calculated from positions of eyes would not be enough to form a full transformation matrix that describes the entire orientation.
4. A detection algorithm that detects left and right eye separately even if the image contains a pair of eyes
5. Real-time modification based on one time detection and keep the real-time tracking of the detected area
6. More stickers to test based on not only eyes, but more features like hair, ears that are not included in the Matlab functions
Introduction
The whole project will be implemented in two parts: the first part is the face detection, using the Cascade Object Detector which is based on Viola-Jones algorithm. The reason for using this algorithm is due to its good accuracy and straightforward implementation in Matlab. Since our focus is not on the perfect accuracy of face detection, Viola-Jones algorithm is our best choise. The original detector can detect people’s face, nose, eyes, mouth or upper body, and we also implemented our own implementation to detect single eye from an image; The second part is the modification of detected faces: upon detecting the eyes, we will replace the detected area with a 2D model. For example, after mapping the eyes, we can replacing the area around the eyes with a pair of glasses. The key procedure is to figure out the degree for rotation of the stickers so that we could rotate the sticker to best fit the face. After several process of the stickers, we need to scale them according to different faces. Our current approach would be based on distance between eyes since this is the variable that does not modify significantly when the orientation changes. For this project, we will use 2D models applied on frontal faces.
Motivation
There are now quite a lot of real-time face modification mobile apps such as Facewin, Snow, Faceu that transform faces by adding motion stickers and effects on your face. We find that these apps have their basic algorithm in common - using the face detection technology that detects eyes and nose, more specifically, these apps only detect faces that are parallel with horizontal direction. The later tracking part allows these apps to perform real-time modifications. Therefore, by simply detecting eyes and nose, we are able to do hundreds of modifications on faces. This evokers our intention to work on the same modifications. For this project, we will not work on the real-time tracking, instead, we will focus on the modification on the static images based on our own calculated data.
Goal
Given a photo of people, we seek to detect the human faces and add our own choice of modifications (stickers) on all the faces. The stickers are transformed in size and orientation to fit the faces.