Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 第六章 模式分析导论 Section One Features, patterns and classifiers （第一节 特征、模式与分类器） Section Two Components of a pattern recognition system （第二节 模式识别系统构成） Section Three Review of probability theory （第三节 概率论复习） Section Four Bayes Theorem （第四节 贝叶斯定理） Section Five Gaussian densities （第五节 高斯密度） Intelligent Sensors System 6-1 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers （第一节 特征、模式与分类器） 1、Feature (1) Feature definition Feature is any distinctive aspect, quality or characteristic （特征是有与众不同的外表、质量或特性） Features may be symbolic (i.e., color) or numeric (i.e., height) （特征可以是符号（如颜色）或数字（如高度） (2) Feature vector The combination of d features is represented as a d-dimensional column vector called a feature vector (特征的组合表达成一个d维列矢量称为特征矢量) (3) Feature space The d-dimensional space defined by the feature vector is called feature space (由d维特征矢量定义的d维空间称为特征空间） Intelligent Sensors System 6-2 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers 1、Feature （续） (4) scatter plot Objects are represented as points in feature space. This representation is called a scatter plot （在特征空间空间中，目标对象用点来表达，这种表达方式称为散斑图） Intelligent Sensors System 6-3 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers 2、Patterns （1）Patterns definition Pattern is a composite of traits or features characteristic of an individual。 （模式是由一组有显著特点或有明显特征个体组成） （2）In classification, a pattern is a pair of variables {x, ω} where， x is a collection of observations or features (feature vector)， ωis the concept behind the observation (label) （模式在分类中，模式是一对变量{x, ω} ，X是观察值或特征值的 集合，ω是观测值之后的概念） Intelligent Sensors System 6-4 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers 2、Patterns （续） （3） What makes a “good” feature vector?（什么是“好的”特征矢量？） The quality of a feature vector is related to its ability to discriminate examples from different classes （特征矢量的质量与它能区别不同类型样本的能力相关） （a）Examples from the same class should have similar feature values（来自同一类型样本，应该有相似的特征值） （b）Examples from different classes have different feature values （来自不同类型样本，应该有不同的特征值） Intelligent Sensors System 6-5 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers 2、Patterns （续） （4） More feature properties (a) Linear separability (线性可分离） (b) Non-linear separability (非线性可分离） (c) Multi-modal（多形态） (d) Highly correlated features（高相关特征） Intelligent Sensors System 6-6 School of Information Engineering

Section One Features, patterns and classifiers Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section One Features, patterns and classifiers 3、Classifiers （1）The goal of a classifier is to partition feature space into class- labeled decision regions （分类器的目的是将特征空间分割成类型标志的决策区域） （2）Borders between decision regions are called decision boundaries （决策区域之间的边界称为决策分界线） Intelligent Sensors System 6-7 School of Information Engineering

Section Two Components of a pattern recognition system Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section Two Components of a pattern recognition system （第二节 模式识别系统构成） 1、A typical pattern recognition system contains (a) A sensor (b) A preprocessing mechanism(预处理装置） Intelligent Sensors System 6-8 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 1、A typical pattern recognition system contains (Cont.) (c) A feature extraction mechanism (manual or automated) （特征提取装置（手动或自动） (d) A classification or description algorithm（分类或运算） (e) A set of examples (training set) already classified or described （一组已经被分类或被描述的样本） Intelligent Sensors System 6-8 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example Consider the following scenario*(考虑如下假定） （1）A fish processing plan wants to automate the process of sorting incoming fish according to species (salmon （鲑鱼）or sea bass （黑鲈鱼）) Intelligent Sensors System 6-9 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example Consider the following scenario*(考虑如下假定） （2）The automation system consists of a conveyor belt for incoming products （一条用于输入产品的传送带） two conveyor belts for sorted products （二条用于分类产品的传送带） a pick-and-place robotic arm（一个用于挑选和放置产品的机械手） a vision system with an overhead CCD camera（一个CCD摄像视觉系统）a computer to analyze images and control the robot arm（一台用于分析图象和控制机械手的计算机） Intelligent Sensors System 6-9 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （1）Sensor The camera captures an image as a new fish enters the sorting area （摄像机捕捉进入存储区的鱼） （2）Preprocessing Adjustments for average intensity levels Segmentation to separate fish from background （调整平均密度，将鱼从平 台分散） （3） Feature Extraction Suppose we know that, on the average, sea bass is larger than salmon （假设已知黑鲈鱼平均体积大于鲑鱼） Intelligent Sensors System 6-10 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （4） Classification (A) Collect a set of examples from both species（从两种鱼中采集一组样本） Plot a distribution of lengths for both classes （两种鱼长度分布状态图） (B) Determine a decision boundary (threshold) that minimizes the classification error.（以最小误差确定决策边界线） (C) We estimate the system’s probability of error and obtain a discouraging result of 40%(估计系统的概率误差，得到40%分类结果） What is next? Intelligent Sensors System 6-11 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （D） Improving the performance of our PR system (a) Committed to achieve a recognition rate of 95%, we try a number of features Width, Area, Position of the eyes and mouth… (为达到95%识别率，尝试许多特征，如宽度，面积、眼睛和嘴巴的位置等） Intelligent Sensors System 6-12 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （D） Improving the performance of our PR system (b) only to find out that these features contain no discriminatory information（找出这些特征中不含歧视区别的信息） (c) Finally we find a “good” feature: average intensity of the scales (最后找到“好的”特征：识别范围内的平均密度） Intelligent Sensors System 6-12 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （D） Improving the performance of our PR system （续） (d) We combine “length” and “average intensity of the scales” to improve class separability （将长度和识别范围内的平均密度结合作为改进的类别可分性） Intelligent Sensors System 6-13 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （D） Improving the performance of our PR system （续） (e) We compute a linear discriminant function to separate the two classes and obtain a classification rate of 95.7% （计算线性判别函数以分离两类鱼，并得到95.7%正确分辨率） Intelligent Sensors System 6-13 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （E） The issue of generalization (a) The recognition rate of our linear classifier (95.7%) met the design specs（规格）, but we still think we can improve the performance of the system (b) We then design an artificial neural network with five hidden layers, a combination of logistic and hyperbolic tangent activation functions, train it with the Levenberg-Marquardt algorithm and obtain an impressive classification rate of 99.9975 % with the following decision boundary length Intelligent Sensors System 6-14 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo 2、Pattern recognition example（续） （E） The issue of generalization (c) Satisfied with our classifier, we integrate the system and employ it to the fish processing plant（满满意我们的分类器，将其 集成系统并应用于鱼加工厂） A few days later the plant manager calls to complain that the system is misclassifying an average of 25%of the fish。 What went wrong? Intelligent Sensors System 6-14 School of Information Engineering

Section Three Review of probability theory Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section Three Review of probability theory （第三节 概率论复习） Intelligent Sensors System 6-15 School of Information Engineering

Section Three Review of probability theory Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section Three Review of probability theory （第三节 概率论复习） Intelligent Sensors System 6-16 School of Information Engineering

Section Four Bayes Theorem Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section Four Bayes Theorem （第四节 贝叶斯定理） Intelligent Sensors System 6-17 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo ⁞ Bayes Theorem（续） Intelligent Sensors System 6-18 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo ⁞ Bayes Theorem（续） Intelligent Sensors System 6-19 School of Information Engineering

Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo ⁞ Bayes Theorem（续） Intelligent Sensors System 6-20 School of Information Engineering

Section Five Gaussian densities Chapter 6 Introduction to Pattern Analysis Prof. Dehan Luo Section Five Gaussian densities （第五节 高斯密度） Intelligent Sensors System 6-21 School of Information Engineering