局部圖像特征提?。宏P(guān)鍵點(diǎn)檢測(cè)關(guān)鍵點(diǎn)提?。ń屈c(diǎn)）關(guān)鍵點(diǎn)提取（角點(diǎn)）Keypointextraction:Corners9300HarrisCornersPkwy,Charlotte,NC為什么提取關(guān)鍵點(diǎn)Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Step1:extractkeypoints提取關(guān)鍵點(diǎn)Step2:matchkeypointfeatures匹配關(guān)鍵點(diǎn)Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Step1:extractkeypointsStep2:matchkeypointfeaturesStep3:alignimages好的關(guān)鍵點(diǎn)Goodkeypoints?好的關(guān)鍵點(diǎn)Characteristicsofgoodkeypoints緊致&高效CompactnessandefficiencyManyfewerkeypointsthanimagepixels關(guān)鍵點(diǎn)數(shù)目比像素少很多顯著性SaliencyEachkeypointisdistinctive關(guān)鍵點(diǎn)是獨(dú)特的、有特色的局部特性LocalityAkeypointoccupiesarelativelysmallareaoftheimage;robusttoclutterandocclusion重復(fù)性/再現(xiàn)性RepeatabilityThesamekeypointcanbefoundinseveralimagesdespitegeometricandphotometrictransformations無論幾何或光學(xué)變換，

同一關(guān)鍵點(diǎn)都能被檢測(cè)到應(yīng)用ApplicationsKeypointsareusedfor:Imagealignment對(duì)齊3Dreconstruction三維重建Motiontracking運(yùn)動(dòng)跟蹤Robotnavigation機(jī)器人導(dǎo)航Databaseindexingandretrieval數(shù)據(jù)庫檢索Objectrecognition目標(biāo)識(shí)別角點(diǎn)檢測(cè)：基本思想Q:Corner角點(diǎn)是什么？角點(diǎn)CornerWeshouldeasilyrecognizethepointbylookingthroughasmallwindow在小窗口中就可以很容易識(shí)別出Shiftingawindowinany

directionshouldgivealargechangeinintensity在任意方向移動(dòng)，強(qiáng)度都應(yīng)該變化巨大“edge”:

nochangealongtheedgedirection“corner”:

significantchangeinalldirections“flat”region:

nochangeinalldirections角點(diǎn)檢測(cè)

Cornerdetection:Basicidea角點(diǎn)檢測(cè)CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:I(x,y)E(u,v)E(0,0)角點(diǎn)檢測(cè)CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:兩個(gè)圖像塊之間的L2距離I(x,y)E(u,v)E(3,2)角點(diǎn)檢測(cè)CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:兩個(gè)圖像塊之間的L2距離Wewanttofindouthowthisfunctionbehavesforsmallshifts小幅移動(dòng)時(shí)該函數(shù)的表現(xiàn)?E(u,v)Considertheaxis-alignedcase(gradientsareeitherhorizontalorvertical):InterpretingthesecondmomentmatrixHarris角點(diǎn)檢測(cè)算子Harris角點(diǎn)檢測(cè)算子TheHarriscornerdetector計(jì)算步驟計(jì)算圖像的局部梯度局部圖像梯度減去平均值計(jì)算局部梯度的協(xié)方差矩陣計(jì)算特征值和特征向量利用閾值處理特征向量，從而判斷是否為角點(diǎn)Harris角點(diǎn)檢測(cè)算子在一個(gè)很小的局部窗口內(nèi)計(jì)算每個(gè)像素點(diǎn)的梯度值：Harris角點(diǎn)檢測(cè)算子將梯度減去平均值，則是為了做一定程度的歸一化。利用這個(gè)歸一化的梯度值，我們可以計(jì)算協(xié)方差矩陣，從而擬合一個(gè)拋物面（下圖的P，代表梯度計(jì)算的局部圖像塊）基礎(chǔ)知識(shí)回顧：協(xié)方差矩陣方差、協(xié)方差基礎(chǔ)知識(shí)回顧：協(xié)方差矩陣方差、協(xié)方差縮放旋轉(zhuǎn)黑色線條表明特征向量方向基礎(chǔ)知識(shí)回顧：協(xié)方差矩陣方差、協(xié)方差Harris角點(diǎn)檢測(cè)算子這個(gè)協(xié)方差矩陣的特征向量有著特殊的意義，它可以寫作下式，從而反應(yīng)出圖像局部點(diǎn)的特點(diǎn)：只有兩個(gè)特征值都比較

大且沒有明顯的差距時(shí)，

才是角點(diǎn)。Harris角點(diǎn)檢測(cè)算子方法1：通過來確定是否是角點(diǎn)。方法2：使用確定是否是角點(diǎn)。這個(gè)算子計(jì)算量更小。兩者檢測(cè)結(jié)果很相似TheHarriscornerdetector計(jì)算偏導(dǎo)Computepartialderivativesateachpixel計(jì)算局部二階矩矩陣ComputesecondmomentmatrixMinaGaussianwindowaroundeachpixel:計(jì)算角點(diǎn)響應(yīng)函數(shù)ComputecornerresponsefunctionRC.HarrisandM.Stephens,ACombinedCornerandEdgeDetector,Proceedingsofthe4thAlveyVisionConference:pages147—151,1988.

HarrisDetector:StepsHarrisDetector:StepsComputecornerresponseRTheHarriscornerdetector計(jì)算偏導(dǎo)Computepartialderivativesateachpixel計(jì)算局部二階矩矩陣ComputesecondmomentmatrixMinaGaussianwindowaroundeachpixel:計(jì)算角點(diǎn)響應(yīng)函數(shù)ComputecornerresponsefunctionR閾值過濾ThresholdR局部最大值Findlocalmaximaofresponsefunction(nonmaximumsuppression)C.HarrisandM.Stephens,ACombinedCornerandEdgeDetector,Proceedingsofthe4thAlveyVisionConference:pages147—151,1988.

HarrisDetector:StepsFindpointswithlargecornerresponse:R>thresholdHarrisDetector:StepsTakeonlythepointsoflocalmaximaofRHarrisDetector:Steps角點(diǎn)特征的魯棒性RobustnessofcornerfeaturesWhathappenstocornerfeatureswhentheimageundergoesgeometricorphotometrictransformations?當(dāng)圖像發(fā)生幾何或光學(xué)變換時(shí)，角點(diǎn)特征？AffineintensitychangeOnlyderivativesareused,soinvarianttointensityshiftI

I

+

b只利用了梯度，多一對(duì)于亮度偏移具有不變性

Intensityscaling:I

a

IRx

(imagecoordinate)thresholdRx

(imagecoordinate)亮度變化部分不變性PartiallyinvarianttoaffineintensitychangeI

a

I+bImagetranslationDerivativesandwindowfunctionareshift-invariantCornerlocationiscovariantw.r.t.translation與平移協(xié)變ImagerotationSecondmomentellipserotatesbutitsshape(i.e.eigenvalues)remainsthesame二階矩橢圓旋轉(zhuǎn)但形狀（特征值）保持不變Cornerlocationiscovariantw.r.t.rotation與旋轉(zhuǎn)協(xié)變ScalingAllpointswillbeclassifiedasedgesCornerCornerlocationisnotcovariantw.r.t.scaling!與尺度不協(xié)變總結(jié)SummaryHarris角點(diǎn)檢測(cè)算子計(jì)算圖像的局部梯度局部圖像梯度減去平均值計(jì)算局部梯度的協(xié)方差矩陣計(jì)算特征值和特征向量利用閾值處理特征向量，從而判斷是否為角點(diǎn)參考資料計(jì)算機(jī)視覺：原理、算法、應(yīng)用及學(xué)習(xí)（第五版）ComputerVision:AModernApproachbyDavidForsythandJeanPonce(2nded.)ComputerVision:AlgorithmsandApplications,byRichardSzeliski,/Book/計(jì)算攝影學(xué)@/column/hawkcpProgrammingComputerVisionwithPython,JanErikSolemCS543/ECE549ComputerVision,UIUC,/CS231n，斯坦福大學(xué)，李飛飛，/計(jì)算攝影學(xué)CS543/ECE549ComputerVisionComputerVision:AlgorithmsandApplicationsCS231n,CNNsforCVConsidertheaxis-alignedcase(gradientsareeitherhorizontalorvertical):InterpretingthesecondmomentmatrixIfeitheraorbiscloseto0,thenthisisnotacorner,sowewantlocationswherebotharelargeInterpretingthesecondmomentmatrix對(duì)角化Inthegeneralcase,needtodiagonalizeM:特征值決定橢圓軸距，R決定方向TheaxislengthsoftheellipsearedeterminedbytheeigenvaluesandtheorientationisdeterminedbyR:directionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2Visualizationofsecondmomentmatricesdirectionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2Visualizationofsecondmomentmatrices特征值決定橢圓軸距，R決定方向directionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2ClassificationofimagepointsusingeigenvaluesofM:Interpretingtheeigenvalues

1

2“Corner”

1and

2arelarge,

1~

2;

Eincreasesinalldirections

1and

2aresmall;

Eisalmostconstantinalldirections“Edge”

1>>

2“Edge”

2>>

1“Flat”regiondirectionoft