1、1Pattern Recognition北京交通大學電子信息工程學院Chapter 2 Bayesian Decision Theory Introduction Bayesian decision theory - error/risk minimum Discriminant function The normal density Discriminant function for normal density Pattern Recognition System4An example character Recognitionab?5An example character Recogn

2、itionabTwo classes , denoted by C1, C2C1C27舉例 字符識別Corresponding to a;Corresponding to b;8舉例 字符識別10Features 輸入矢量的維數(shù)很高 設法降低維數(shù)特征字符高與寬的比率 v11Features字符高與寬的比率 v如何選擇合適的閾值？12Features 特征T直方圖字符高與寬的比率14The overall system can be viewed as a mapping from a set of input variables , to an output variables .The pr

3、oblem is how to get the mapping function.Classify （分類）15ClassifyClassifyx1x217Classify18Classify19模式識別系統(tǒng)的主要構成怎樣才能得最好的分類效果呢？20模式識別系統(tǒng)的主要構成統(tǒng)計模式識別神經(jīng)網(wǎng)絡支持向量機 21Bayes theorem Supposing that we wish to classify a new character but as yet we have made no measurements on the image of that character. The goal

4、is to classify the character in such a way as to minimize the probability of misclassification. 22Bayes theoremIf we have collected a large number of examples of the character,we could find the fractions which belong in each of the two classes. 24Bayes theoremPriori probabilities （先驗概率）If the letter

5、 a occurs three times as often as the letter b 25Bayes theoremIf we were forces to classify a new character without being allowed to see the corresponding imageThen the best we can do is to assign it to the class having higher prior probability. 先驗概率27Bayes theoremNow suppose that we have measured t

6、he value of the feature variables It is clear that this give us further information on which to base our classification decision基于特征分類28Bayes theorem直方圖29Bayes theorem Priori probabilities （先驗概率）30Bayes theorem Joint probability - the probability that the image has the feature value and belongs to c

7、lass 聯(lián)合概率 - 屬于類 而擁有特征值 的概率31Bayes theoremConditional probability - specifies the probability that the observation falls in column of the array given that it belongs to the class條件概率 - 類 ，同時特征值為 32Bayes theorem33Bayes theorem34Bayes theorem35Bayes theorem36Bayes theoremWhat isThe probability that the

8、 class is given that the measured value falls in the cell Posterior probability/ 后驗概率37Bayes theorem in general 38Bayes theoremThe posterior probability gives the probability of the pattern belonging to class once we have observed the feature vector . The probability of misclassification is minimize

9、d by selecting the class having the largest posterior probability39Bayes theorem in general 40Decision boundaries A feature vector x is assigned to class if for all or41Decision boundaries 42R1R243R1R2分類錯誤最小！44Bayes DecisionThe probability of misclassification is minimized by selecting the class hav

10、ing the largest posterior probability45Example:假設在某個局部地區(qū)細胞識別中正常 和異常 兩類的先驗概率分別為 正常： 異常： 現(xiàn)有一待識別的細胞，其觀察值為 ，從類條件概率密度曲線上查得 試對該細胞分類。 46Example:利用貝葉斯公式，分別計算 及 的后驗概率。 歸類于正常狀態(tài)47字符識別問題48Bayes DecisionThe probability of misclassification is minimized by selecting the class having the largest posterior probabil

11、ity49Decision functions A feature vector x is assigned to class if for all分類錯誤最小50Decision functions A feature vector x is assigned to class if for all51Decision functions A feature vector x is assigned to class if for all52Decision functionsA feature vector x is assigned to class if53Decision funct

12、ions54字符識別問題基于最小風險的貝葉斯決策55基于最小錯誤率的決策基于最小風險的貝葉斯決策56基于最小風險的貝葉斯決策57基于最小風險的貝葉斯決策58基于最小風險的貝葉斯決策59基于最小風險的貝葉斯決策60Take action : decide , if Bayesian decision rule is stated as:基于最小風險的貝葉斯決策61Take action : decide , if 基于最小風險的貝葉斯決策62Bayes decision rule : Take action : decide , If equal to: is called : Likeliho

13、od ratio. 基于最小風險的貝葉斯決策63 Likelihood ratio : Decision rule: If the likelihood ratio of class and exceeds a threshold value (that is independent of the input pattern ), the optimal action is to decide . 64Decision functions A feature vector x is assigned to class if 分類風險最小Minimum-error rate classifica

14、tion65正確類別的后驗概率=正確率True state : , correct rate=錯誤率=1 - 正確率Minimum-error rate classification66Minimum-error rate classification67error rateMinimum-error rate classification6869Bayes Decision: error minimumThe probability of misclassification is minimized by selecting the class having the largest post

15、erior probability70Bayes Decision: risk minimum If the likelihood ratio of class and exceeds a threshold value (that is independent of the input pattern ), the optimal action is to decide . 71小結 分類問題： 數(shù)學問題： 如何解決這個數(shù)學問題？數(shù)學問題從輸入到輸出的映射概率、 Bayes 決策最小錯誤率：后驗概率最小風險：似然比72作業(yè) 二1. 兩類樣本C1， C2的特征分布直方圖如下：試求每類樣本的先驗

16、概率、(類)條件概率及后驗概率。73作業(yè) 二2. 已知甲類：P(1) = 0.7和類條件概率密度函數(shù)p(x|1) ,乙類：P(2) = 0.3和類條件概率密度函數(shù)p(x|2)今有待分類樣本特征觀察值x = 10，且由函數(shù)曲線查得p(10|1) = 0.2， p(10|2) = 0.5試用最小錯誤率Bayes決策對樣本x = 10進行分類試用最小風險Bayes決策對該樣本進行分類，設11=22=0，12=2，21=174Bayesian Decision TheoryBayesian decision theory is a fundamental statistical approach to the problem