Configuration Of Generic Face Recognition System

Published Date: 02 Nov 2017

CHAPTER 2

LITERATURE SURVEY

Face recognition system is major part of biometric research. Biometric research is a part of statistical pattern recognition. Pattern recognition is part of computer vision. So mainly when deal with computer vision there is a comparison of human vision and computer vision systems.

Face Recognition has been a fascinating issue for two different fields dealing with the Artificial Intelligence. Two different fields are neuroscientists and Computer Engineers. A human vision can easily detect the face, identify the face of particular person and remember it, too. As far as the computer vision is concern it recognizes the faces, the face area which should be detected first and recognition or classification next. Therefore for computer vision to recognize the faces from photographs is the important task and all the photographs should be taken in the controlled and appropriate environment. It also requires uniform background and identical poses which provides more information to solve the problem. Here the photographs means images are applied to the face recognition system. There may be a video sequence instead of photographs, which may provide better or much amount of information for face recognition system. Videos should taken into uniform and controlled environment, too.

Application of face recognition system ranges from static, controlled-format photographs to the uncontrolled video images, posing a wide range of technical challenges and requiring an equally wide range of techniques from image processing, analysis, understanding and pattern recognition.

Here the differences in terms of quality, background, variability of the images of a particular individual person that must be recognized. In general face recognition system requires proper input with some different kinds of requirements because it is a computer vision. So that here to build a system which can be compare to the human vision system. Also it requires availability of a well-defined recognition or matching criterion, and the nature, type, and amount of input from a user [3]. A list of some commercial systems is given in Table 2.1

Areas

Specific applications

Entertainment

Video game, virtual reality, training programs

Human-robot-interaction, human-computer-interaction

Smart cards

Drivers’ licenses, entitlement programs

Immigration, national ID, passports, voter registration

Welfare fraud

Information security

TV Parental control, personal device logon, desktop logon

Application security, database security, file encryption

Intranet security, internet access, medical records

Secure trading terminals

Law enforcement and surveillance

Advanced video surveillance, CCTV control

Portal control, post event analysis

Shoplifting, suspect tracking and investigation

Table 2.1 Applications of Face Recognition

In the face recognition system the flow of the system is defined as below:

Input to the system e.g.) images or video sequence

Then find out the face from it.

Find out important features from it.

Recognition is the last step.

Mainly verification is done on the input image with the use of stored images in the system databases. Above listed steps are defined in the figure 2.1. In the recognition of the face we already have a face database from where we find the match to the input of the system. The flow of face recognition defines that input images or video sequence has to be matched with the available database to the system and defines that particular person is this. Sometimes system identifies the different one than the actual present in the input.

Fig. 2.1 Configuration of generic face recognition system.

Face Detection

Nowadays some applications of Face Recognition don’t require face detection. In some cases, face images stored in the data bases are already normalized. There is a standard image input format, so there is no need for a detection step. However, the conventional input images of computer vision systems are not that suitable[5] . For example, video surveillance systems try to include face detection, tracking and recognizing.

Face detection must deal with several well known challenges[3]. They are usually present in images captured in uncontrolled environments, such as surveillance video systems. These challenges can be attributed to some factors[5]:

Pose variation: The ideal scenario for face detection would be one in which only frontal images were involved. But, as stated, this is very unlikely in general uncontrolled conditions. Moreover, the performance of face detection algorithms drops severely when there are large pose variations. It’s a major research issue. Pose variation can happen due to subject’s movements or camera’s angle.

Feature occlusion: The presence of elements like beards, glasses or hats introduces high variability. Faces can also be partially covered by objects or other faces.

Facial expression: Facial features also vary greatly because of different facial gestures.

Imaging conditions: Different cameras and ambient conditions can affect the quality of an image, affecting the appearance of a face.

There are some problems closely related to face detection besides feature extraction and face classification[3]. For instance, face location is a simplified approach of face detection. It’s goal is to determine the location of a face in an image where there’s only one face. We can differentiate between face detection and face location.

Face Detection is a concept that includes many sub-problems. Some systems detect and locate faces at the same time, others first perform a detection routine and then, if positive, they try to locate the face[5]. Then, some tracking algorithms may be needed - see Figure 2.2.

Fig. 2.2 Face detection process

Face detection algorithms usually share common steps. Firstly, some data dimension reduction is done, in order to achieve a admissible response time. Some pre-processing could also be done to adapt the input image to the algorithm prerequisites. Then, some algorithms analyze the image as it is, and some others try to extract certain relevant facial regions. The next phase usually involves extracting facial features or measurements. These will then be weighted, evaluated or compared to decide if there is a face and where is it. Finally, some algorithms have a learning routine and they include new data to their models.

Face detection is, therefore, a two class problem where we have to decide if there is a face or not in a picture. This approach can be seen as a simplified face recognition problem. Face recognition has to classify a given face, and there are as many classes as candidates. Consequently, many face detection methods are very similar to face recognition algorithms.

Approaches to face detection

In this section, two classification criteria will be presented. One of them differentiates between distinct scenarios. Depending on these scenarios different approaches may be needed. The other criterion divides the detection algorithms into four categories[5].

Detection depending on the scenario.

Controlled environment: It’s the most straightforward case. Photographs are taken under controlled light, background, etc. Simple edge detection techniques can be used to detect faces [3].

Color images: The typical skin colors can be used to find faces. They can be weak if light conditions change. Moreover, human skin color changes a lot, from nearly white to almost black. But, several studies show that the major difference lies between their intensity, so chrominance is a good feature [3]. It’s not easy to establish a solid human skin color representation. However, there are attempts to build robust face detection algorithms based on skin color. ˆ

Images in motion: Real time video gives the chance to use motion detection to localize faces. Nowadays, most commercial systems must locate faces in videos. There is a continuing challenge to achieve the best detecting results with the best possible performance [3].

Detection methods divided into categories

Methods are divided into four categories. These categories may overlap, so an algorithm could belong to two or more categories. This classification can be made as follows[5]:

Knowledge-based methods: Ruled-based methods that encode our knowledge of human faces.

Feature-invariant methods: Algorithms that try to find invariant features of a face despite its angle or position.

Template matching methods: These algorithms compare input images with stored patterns of faces or features. ˆ

Appearance-based methods: A template matching method whose pattern database is learnt from a set of training images.

Face Tracking

Many face recognition systems have a video sequence as the input. Those systems may require being capable of not only detecting but tracking faces. Face tracking is essentially a motion estimation problem. Face tracking can be performed using many different methods, e.g., head tracking, feature tracking, image-based tracking, model-based tracking. These are different ways to classify these algorithms[5].

Head tracking/Individual feature tracking: The head can be tracked as a whole entity, or certain features tracked individually.

2D/3D: Two dimensional systems track a face and output an image space where the face is located. Three dimensional systems, on the other hand, perform a 3D modelling of the face. This approach allows to estimate pose or orientation variations.

The basic face tracking process seeks to locate a given image in a picture. Then, it has to compute the differences between frames to update the location of the face. There are many issues that must be faced: Partial occlusions, illumination changes, computational speed and facial deformations.

Feature Extraction

The feature extraction process can be defined as the procedure of extracting relevant information from a face image. This information must be valuable to the later step of identifying the subject with an acceptable error rate. The feature extraction process must be efficient in terms of computing time and memory usage. The output should also be optimized for the classification step.

Feature extraction involves several steps - dimensionality reduction, feature extraction and feature selection. This steps may overlap, and dimensionality reduction could be seen as a consequence of the feature extraction and selection algorithms. Both algorithms could also be defined as cases of dimensionality reduction.

Dimensionality reduction is an essential task in any pattern recognition system. The performance of a classifier depends on the amount of sample images, number of features and classifier complexity.

We can make a distinction between feature extraction and feature selection. Both terms are usually used interchangeably. Nevertheless, it is recommendable to make a distinction.

A feature extraction algorithm extracts features from the data. It creates those new features based on transformations or combinations of the original data. In other words, it transforms or combines the data in order to select a proper subspace in the original feature space.

On the other hand, a feature selection algorithm selects the best subset of the input feature set. It discards non-relevant features. Feature selection is often performed after feature extraction. So, features are extracted from the face images, then a optimum subset of these features is selected.

The dimensionality reduction process can be embedded in some of these steps[5], or performed before them – see Figure 2.

Fig. 2.3 Feature Extraction process

Feature Extraction methods

There are many feature extraction algorithms Most of them are used in other areas than face recognition. Researchers in face recognition have used, modified and adapted many algorithms and methods to their purpose.

See Table 2.2 for a list of some feature extraction algorithms used in face recognition.

Method

Notes

Principal Component Analysis (PCA)

Eigenvector-based, linear map

Kernel PCA

Eigenvector-based , non-linear map, uses kernel methods

Weighted PCA

PCA using weighted coefficients

Linear Discriminant Analysis (LDA)

Eigenvector-based, supervised linear map

Kernel LDA

LDA-based, uses kernel methods

Independent Component Analysis(ICA)

Linear map, separates non-Gaussian

distributed features

Neural Network based methods

Diverse neural networks using PCA,etc.

Table 2.2 Feature Extraction Methods

Principal Components Analysis (PCA):

The advantage of PCA comes from its generalization ability. It reduces the feature space dimension by considering the variance of the input data. The method determines which projections are preferable for representing the structure of the input data. Those projections are selected in such a way that the maximum amount of information (i.e. maximum variance) is obtained in the smallest number of dimensions of feature space[7].

In order to obtain the best variance in the data, the data is projected to a subspace (of the image space) which is built by the eigenvectors from the data. In that sense, the eigenvalue corresponding to an eigenvector represents the amount of variance that eigenvector handles[9]. The mathematical formulation of PCA is discussed in Chapter 3.1.

Linear Discriminant Analysis (LDA):

While PCA tries to generalize the input data to extract the features, LDA tries to discriminate the input data by dimension reduction[12].

LDA searches for the best projection to project the input data, on a lower dimensional space, in which the patterns are discriminated as much as possible. For this purpose, LDA tries to maximize the scatter between different classes and minimize the scatter between the input data in the same class. LDA uses generalized eigenvalue equation to obtain this discrimination. The mathematical aspects of the LDA can be found in Chapter 3.2

Feature Selection methods

Feature selection algorithm’s aim is to select a subset of the extracted features that cause the smallest classification error. The importance of this error is what makes feature selection dependent to the classification method used.

The most straightforward approach to this problem would be to examine every possible subset and choose the one that fulfils the criterion function[5]. Some effective approaches to this problem are based on algorithms like branch and bound algorithms. See Table 2.3 for selection methods.

Method

Definition

Comments

Exhaustive search

Evaluate all possible subsets of Features

Optimal, but too complex

Branch and bound

Use branch and bound algorithm

Can be optimal. Complexity of max O(2n)

Best individual features

Evaluate and select features individually

Not very effective.

Simple algorithm

Sequential Forward Selection (SFS)

Evaluate growing feature

sets (starts with best feature)

Retained features can’t be discarded. Faster than SBS.

Sequential Backward Selection (SBS)

Evaluate shrinking feature.

sets (starts with all the features)

Deleted features can’t be re-evaluated.

"Plus l -take away r" selection

First do SFS then SBS

Must choose l and r values

Sequential Forward Floating Search (SFFS) and Sequential Backward Floating Search (SBFS)

Like "Plus l -take away r", but l and r values automatic pick and dynamic update.

Close to optimal Affordable computational cost.

Table 2.3 Feature Selection Methods

Face Classification

Once the features are extracted and selected, the next step is to classify the image. Appearance-based face recognition algorithms use a wide variety of classification methods. Sometimes two or more classifiers are combined to achieve better results .One way or another, classifiers have a big impact in face recognition. Classification methods are used in many areas like data mining, finance, signal decoding, voice recognition, natural language processing or medicine[5].

Classification algorithms usually involve some learning supervised, unsupervised or semi-supervised. Unsupervised learning is the most difficult approach, as there are no tagged examples. However, many face recognition applications include a tagged set of subjects. Consequently, most face recognition systems implement supervised learning methods. There are also cases where the labelled data set is small. Sometimes, the acquisition of new tagged samples can be infeasible. Therefore, semi-supervised learning is required.

Classifiers

There are three concepts that are key in building a classifier - similarity, probability and decision boundaries [4]. We will present the classifiers from that point of view.

Similarity Based Classifiers: This approach is intuitive and simple. Patterns that are similar should belong to the same class. These approach has been used in the face recognition algorithms implemented later. The idea is to establish a metric that defines similarity and a representation of the same-class samples.

For example, the metric can be the Euclidean distance. The representation of a class can be the mean vector of all the patterns belonging to this class. The 1-NN decision rule can be used with these parameters. Its classification performance is usually good. See Table 2.4 for similarity based classifiers.

Method

Notes

Template matching

Assign sample to most similar template. Templates must be normalized.

Nearest Mean

Assign pattern to nearest class mean.

Subspace Method

Assign pattern to nearest class subspace.

1-NN

Assign pattern to nearest pattern’s class

k-NN

Like 1-NN, but assign to the majority of k nearest patterns.

Vector Quantization

methods Assign pattern to nearest centroid. There are various learning methods.

Table 2.4 Similarity Based Classifiers

Probabilistic classifiers: Some classifiers are building based on a probabilistic approach. For example, Bayes decision rule is often used. The rule can be modified to take into account different factors that could lead to miss-classification. Bayesian decision rules can give an optimal classifier, and the Bayes error can be the best criterion to evaluate features. Therefore, a posterior probability functions can be optimal. See Table 2.5 for probabilistic classifiers.

Method

Notes

Bayesian

Assign pattern to the class with the highest estimated posterior probability.

Logistic Classifier

Predicts probability using logistic curve method

Parzen Classifier

Bayesian classifier with Parzen density estimates

Table 2.4 Probabilistic Classifiers

Classifiers using decision boundaries: This approach can become equivalent to a Bayesian classifier. It depends on the chosen metric. The main idea behind this approach is to minimize a criterion (a measurement of error) between the candidate pattern and the testing patterns. One example is the Fisher’s Linear Discriminant (often FLD and LDA are used interchangeably). It’s closely related to PCA. A special type of classifier is the decision tree. It is trained by an iterative selection of individual features that are most salient at each node of the tree. During classification, just the needed features for classification are evaluated, so feature selection is implicitly built-in. The decision boundary is built iteratively. See Table 2.6 for classifiers using Decision Trees.

Method

Notes

Fisher Linear Discriminant (FLD)

Linear classifier. Can use MSE optimization

Binary Decision Tree

.

Nodes are features. Can use FLD

Could need pruning

Perceptron

Iterative optimization of a classifier

(e.g. FLD)

Multi-layer Perceptron

Two or more layers. Uses sigmoid transfer functions.

Radial Basis

Network Optimization of a Multi-layer perceptron. One layer at least uses

Gaussian transfer functions.

Support Vector Machines

Maximizes margin between two classes.

Table 2.6 Classifiers using Decision Trees

Broadly face recognition methods can be categorized into two groups: feature-based and appearance-based.

In feature-based approach, a set of local features is extracted from the image such as eyes, nose, mouth etc. and they are used to classify the face. The major benefit of this approach is its relative robustness to variations in illumination, contrast, and small amounts of out-of-plane rotation. But there is generally no reliable and optimal method to extract an optimal set of features. Another problem of this approach is that it may cause some loss of useful information in the feature extraction step.

The appearance-based approaches use the entire image as the pattern to be classified, thus using all information available in the image. However, they tend to be more sensitive to image variations. Thus major issue of designing an appearance-based approach is the extraction of useful information which can be used for efficient face recognition system that is robust under different constraints (pose, illumination, expressions etc.)

When using appearance-based approach, an image of size mxn pixels is represented as a vector in mn-dimensional space. But for an efficient and fast recognition system, the mn-dimensional space is quite large. This generates the need for dimension reduction algorithms. While reducing the dimension, these algorithms must also possess enhanced discrimination power.

Some of the most used algorithms are principal component analysis (PCA), linear discriminant analysis (LDA),and independent component analysis (ICA). These linear algorithms project data linearly from high dimensional image space to a low dimensional subspace.

Since the entire image space along with constraints is highly non-linear, they are unable to preserve the non-linear variations necessary to differentiate among different classes[16]. Due to this, the linear methods fail to achieve high face recognition accuracy.

Soft computing techniques (artificial neural networks, fuzzy logic and genetic algorithms) have emerged as an important methodology for analysis in computer vision research [16].

Artificial neural network is a powerful tool to resolve the nonlinearity imposed by different constraints.

Similarly, fuzzy logic is used for modelling human thinking and perception. In place of using crisp set (theory of binary propositions), fuzzy systems reason with fuzzy set of multi-values. It is well established that the effectiveness of human brain is not only from precise cognition, but also from analysis based on fuzzy set and fuzzy logic.