Facial Point Detection using Boosted Regression and Graph Models Authors: Michel Valstar,Brais Martinez, Xavier Binefa, Maja Pantic 讲解人: 赵小伟

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

第一作者 Michel Valstar Research Interest Publication Homepage Research associate in Maja Pantic's HCI^2 lab at the Computing Department of Imperial College London, UK Research Interest Automatically recognize facial expressions from face video Publication CVPR’06, CVPR’10 Homepage http://www.doc.ic.ac.uk/~mvalstar/index.html

第二作者 Brais Martinez Research Interest Publication Homepage PhD Student, Universitat Pompeu Fabra Research Interest Object Tracking Facial Feature Detection and Tracking Thermal Imagery Publication 2 CVPR’10 , PR’08, ICIP’06 Homepage http://cmtech.upf.edu/?page_id=90

第三作者 Xavier Binefa Valls Homepage Research Interest Associate Professor, Information Technology and Telecommunication Department of the Universitat Pompeu Fabra Research Interest Motion Detection and tracking, Machine Learning Face and Gesture recognition, Digital Libraries Human computer interaction, Sensor Fusion Homepage http://cmtech.upf.edu/?page_id=84

第四作者 Maja Pantic Research Interest HomePage Imperial College London: Reader in Multimodal Human-Computer Interaction University of Twente: Professor in Affective  Behavioural Computing Research Interest Face and body gesture recognition, Human-computer interaction (HCI), Affective computing, Educational software, E-learning tools, Intelligent systems, Machine learning HomePage http://www.doc.ic.ac.uk/~maja/

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

文章信息 文章出处 CVPR 2010 相关文献 [23] D. Vukadinovic and M. Pantic, “Fully automatic facial feature point detection using gabor feature based boosted classifiers,” In Proc. Systems, Man and Cybernetics, vol. 2, pp. 1692–1698, 2005.

Abstract Finding fiducial facial points in any frame of a video showing rich naturalistic facial behavior is an unsolved problem. Yet this is a crucial step for geometric-feature-based facial expression analysis, and methods that use appearance-based features extracted at fiducial facial point locations. In this paper we present a method based on a combination of Support Vector Regression and Markov Random Fields to drastically reduce the time needed to search for a point’s location and increase the accuracy and robustness of the algorithm. Using Markov Random Fields allows us to constrain the search space by exploiting the constellations that facial points can form.

Abstract The regressors on the other hand learn a mapping between the appearance of the area surrounding a point and the positions of these points, which makes detection of the points very fast and can make the algorithm robust to variations of appearance due to facial expression and moderate changes in head pose. The proposed point detection algorithm was tested on 1855 images, the results of which showed we outperform current state of the art point detectors.

摘要 在具有丰富的自然面部行为的视频帧中进行面部关键特征点的定位是一个尚未解决的问题。然而，对基于几何特征的面部表情分析以及需要从面部关键特征点提取表观特征的方法而言，面部关键特征点的定位是一个很重要的步骤。 本文提出了一种结合SVR和MRF的面部关键特征点定位方法。该方法大大降低了搜索特征点的时间，并且提高了算法的精度和鲁棒性。 一方面，使用MRF对面部关键特征点的分布进行建模，以此来限制特征点的搜索范围。 另一方面，通过SVR学习到了特征点周围区域的表观信息与特征点位置的映射关系。该方法可以更快的检测特征点，并且对由面部表情和头部姿态的适度变化引起的表观变化比较鲁棒。 我们在1855幅图像上测试了提出的面部特征点检测算法，实验表明，本文的算法超越了当前state-of-the-art的算法。

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

AdaBoost-based Facial Landmark Localization Preparing Samples Negative Samples Positive Samples Real AdaBoost Learning Feature Extraction

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

拟解决的问题 None but [23] is able to detect all 20 facial points necessary for automatic expression recognition No previous work has reported to be able to robustly handle large occlusions such as glasses, beards, and hair that covers part of the eyebrows and eyes None have reported to detect facial points robustly in the presentence of facial expressions 22 fiducial facial feature points (including pupils)

本文的主要思想 Iteratively using Support Vector Regression and local appearance based features to provide an initial predictions of 22 points Then, the Markov Network is applied to ensure the new locations predicted by SVR regressors form correct point constellations SVR regression The output of the SVRs to detect an pupil MRF points model

文章结构 Introduction BoRMaN point detection Experiments A priori probability Regression prediction Spatial relations Point detection algorithm Local appearance based features and AdaBoost feature selection Experiments Conclusions and future work

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

实现细节 A priori probability Regression prediction Local appearance based features and AdaBoost feature selection Spatial Relations Point detection algorithm

Regression prediction The localization problem is formulated as finding the vector that relates a patch location to the target point . This problem is decomposed into two separate regression problem Regressor is tasked with finding the angle of Regressor is tasked with finding the length of As we can see, the regressors give a good yet not a perfect indication of where the target point is. Note that although the location of the pupil is a global minimum, the predicted distance at that location is not zero.

Regression prediction Such errors can be removed by using a iterative procedure. The error of the estimates Impression of the regressors output Great errors which are not merely impressions Spatial restrictions on the location of each facial point depending on the other facial points are applied to solve this problem. The output of the SVRs to detect an pupil

实现细节 A priori probability Regression prediction Local appearance based features and AdaBoost feature selection Spatial Relations Point detection algorithm

Local appearance based features and AdaBoost feature selection Haar-like filters are adopted as the descriptors of local appearance The reason for this is Show that the success of the proposed approach is due to the idea of tuning the point detection problem from a classification procedure into a regression procedure, and not due to asome highly descriptive appearance feature Exploring the integral image The regression performance decrease when the dimensionality of the training set is too large AdaBoost is used to select features

实现细节 A priori probability Regression prediction Local appearance based features and AdaBoost feature selection Spatial Relations Point detection algorithm

Spatial Relations Each relative position of a pair of points is a vector pointing from one facial point to another The relation between two vectors and is described by two parameters The relation between their angles The relation between their lengths (0,0) Relation between two vectors

Spatial Relations Variables such as and are modeled as a Sigmoid function. If a variable takes its value in , then Illustration of Sigmoid function, cited from Wiki

Spatial Relations Once the pairwise relations are defined, the joint probability of a configuration is modeled by a Markov Random Field. The nodes correspond to each of the relative positions Relation between and is modeled as

实现细节 A priori probability Regression prediction Local appearance based features and AdaBoost feature selection Spatial Relations Point detection algorithm

Point detection algorithm Flow of algorithm

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

本文方法与已有方法的对比 Distance Metric:

实验结果

提纲 作者信息 文章信息 背景知识 拟解决的问题与采用的思路 实现细节 实验 结论

本文可以借鉴的地方 Regression instead of classification Markov Random Field to model the constellation of facial points Select features by AdaBoost

谢谢！

附录

Introduction of AdaBoost(1/6) AdaBoost通过对一些弱分类器(weak classifier)的组合来形成一个强分类器(strong classifier), “提升(boost)”弱分类器得到一个分类性能好的强分类器 每一个弱分类器都对前一个分类器错误分类的样本给与更多的重视

Introduction of AdaBoost(2/6) 其中，h表示弱分类器的响应值，θ为正例反例判别阈值，f表示特征响应值 Positive Negative

Introduction of AdaBoost(3/6) 输入 样本集合 (x1,y1), (x2,y2), ..., (xn,yn) 训练参数：样本权值wi、分类器层数T等等 输出 一个由很多弱分类器线性组合得到的强分类器

Introduction of AdaBoost(4/6) 分类错误率是否达到？ 选择错误率最小的弱分类器 更新强分类器 样本权值更新，分类正确的样本权值减小 遍历所有特征，分别计算以每个特征 作为弱分类器的分类错误率 否 输出强分类器 是

Introduction of AdaBoost(5/6) For t=1,...,T 1. 归一化权重，使得wt为一个概率分布： 2. 对每个特征j, 训练一个弱分类器hj, 计算其带权重的错误率 3. 选择误差最小的弱分类器ht加入强分类器 4. 更新每个样本的权重

Introduction of AdaBoost(6/6)

Haar-like Feature(1/2) Haar-like feature 白色矩形像素和减去黑色矩形像素和

Haar-like Feature(2/2) Haar-like feature 计算矩形内部像素灰度值的和 定义积分图 计算D内部像素灰度和 4 + 1 - 2 - 3