局部圖像特征提取：關鍵點檢測關鍵點提取（角點）關鍵點提取（角點）Keypointextraction:Corners9300HarrisCornersPkwy,Charlotte,NC為什么提取關鍵點Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Step1:extractkeypoints提取關鍵點Step2:matchkeypointfeatures匹配關鍵點Whyextractkeypoints?Motivation:panoramastitching全景圖拼接Wehavetwoimages–howdowecombinethem?Step1:extractkeypointsStep2:matchkeypointfeaturesStep3:alignimages好的關鍵點Goodkeypoints?好的關鍵點Characteristicsofgoodkeypoints緊致&高效CompactnessandefficiencyManyfewerkeypointsthanimagepixels關鍵點數目比像素少很多顯著性SaliencyEachkeypointisdistinctive關鍵點是獨特的、有特色的局部特性LocalityAkeypointoccupiesarelativelysmallareaoftheimage;robusttoclutterandocclusion重復性/再現性RepeatabilityThesamekeypointcanbefoundinseveralimagesdespitegeometricandphotometrictransformations無論幾何或光學變換，

同一關鍵點都能被檢測到應用ApplicationsKeypointsareusedfor:Imagealignment對齊3Dreconstruction三維重建Motiontracking運動跟蹤Robotnavigation機器人導航Databaseindexingandretrieval數據庫檢索Objectrecognition目標識別角點檢測：基本思想Q:Corner角點是什么？角點CornerWeshouldeasilyrecognizethepointbylookingthroughasmallwindow在小窗口中就可以很容易識別出Shiftingawindowinany

directionshouldgivealargechangeinintensity在任意方向移動，強度都應該變化巨大“edge”:

nochangealongtheedgedirection“corner”:

significantchangeinalldirections“flat”region:

nochangeinalldirections角點檢測

Cornerdetection:Basicidea角點檢測CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:I(x,y)E(u,v)E(0,0)角點檢測CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:兩個圖像塊之間的L2距離I(x,y)E(u,v)E(3,2)角點檢測CornerDetection:DerivationChangeinappearanceofwindowWfortheshift[u,v]:兩個圖像塊之間的L2距離Wewanttofindouthowthisfunctionbehavesforsmallshifts小幅移動時該函數的表現?E(u,v)Considertheaxis-alignedcase(gradientsareeitherhorizontalorvertical):InterpretingthesecondmomentmatrixHarris角點檢測算子Harris角點檢測算子TheHarriscornerdetector計算步驟計算圖像的局部梯度局部圖像梯度減去平均值計算局部梯度的協方差矩陣計算特征值和特征向量利用閾值處理特征向量，從而判斷是否為角點Harris角點檢測算子在一個很小的局部窗口內計算每個像素點的梯度值：Harris角點檢測算子將梯度減去平均值，則是為了做一定程度的歸一化。利用這個歸一化的梯度值，我們可以計算協方差矩陣，從而擬合一個拋物面（下圖的P，代表梯度計算的局部圖像塊）基礎知識回顧：協方差矩陣方差、協方差基礎知識回顧：協方差矩陣方差、協方差縮放旋轉黑色線條表明特征向量方向基礎知識回顧：協方差矩陣方差、協方差Harris角點檢測算子這個協方差矩陣的特征向量有著特殊的意義，它可以寫作下式，從而反應出圖像局部點的特點：只有兩個特征值都比較

大且沒有明顯的差距時，

才是角點。Harris角點檢測算子方法1：通過來確定是否是角點。方法2：使用確定是否是角點。這個算子計算量更小。兩者檢測結果很相似TheHarriscornerdetector計算偏導Computepartialderivativesateachpixel計算局部二階矩矩陣ComputesecondmomentmatrixMinaGaussianwindowaroundeachpixel:計算角點響應函數ComputecornerresponsefunctionRC.HarrisandM.Stephens,ACombinedCornerandEdgeDetector,Proceedingsofthe4thAlveyVisionConference:pages147—151,1988.

HarrisDetector:StepsHarrisDetector:StepsComputecornerresponseRTheHarriscornerdetector計算偏導Computepartialderivativesateachpixel計算局部二階矩矩陣ComputesecondmomentmatrixMinaGaussianwindowaroundeachpixel:計算角點響應函數ComputecornerresponsefunctionR閾值過濾ThresholdR局部最大值Findlocalmaximaofresponsefunction(nonmaximumsuppression)C.HarrisandM.Stephens,ACombinedCornerandEdgeDetector,Proceedingsofthe4thAlveyVisionConference:pages147—151,1988.

HarrisDetector:StepsFindpointswithlargecornerresponse:R>thresholdHarrisDetector:StepsTakeonlythepointsoflocalmaximaofRHarrisDetector:Steps角點特征的魯棒性RobustnessofcornerfeaturesWhathappenstocornerfeatureswhentheimageundergoesgeometricorphotometrictransformations?當圖像發生幾何或光學變換時，角點特征？AffineintensitychangeOnlyderivativesareused,soinvarianttointensityshiftI

I

+

b只利用了梯度，多一對于亮度偏移具有不變性

Intensityscaling:I

a

IRx

(imagecoordinate)thresholdRx

(imagecoordinate)亮度變化部分不變性PartiallyinvarianttoaffineintensitychangeI

a

I+bImagetranslationDerivativesandwindowfunctionareshift-invariantCornerlocationiscovariantw.r.t.translation與平移協變ImagerotationSecondmomentellipserotatesbutitsshape(i.e.eigenvalues)remainsthesame二階矩橢圓旋轉但形狀（特征值）保持不變Cornerlocationiscovariantw.r.t.rotation與旋轉協變ScalingAllpointswillbeclassifiedasedgesCornerCornerlocationisnotcovariantw.r.t.scaling!與尺度不協變總結SummaryHarris角點檢測算子計算圖像的局部梯度局部圖像梯度減去平均值計算局部梯度的協方差矩陣計算特征值和特征向量利用閾值處理特征向量，從而判斷是否為角點參考資料計算機視覺：原理、算法、應用及學習（第五版）ComputerVision:AModernApproachbyDavidForsythandJeanPonce(2nded.)ComputerVision:AlgorithmsandApplications,byRichardSzeliski,/Book/計算攝影學@/column/hawkcpProgrammingComputerVisionwithPython,JanErikSolemCS543/ECE549ComputerVision,UIUC,/CS231n，斯坦福大學，李飛飛，/計算攝影學CS543/ECE549ComputerVisionComputerVision:AlgorithmsandApplicationsCS231n,CNNsforCVConsidertheaxis-alignedcase(gradientsareeitherhorizontalorvertical):InterpretingthesecondmomentmatrixIfeitheraorbiscloseto0,thenthisisnotacorner,sowewantlocationswherebotharelargeInterpretingthesecondmomentmatrix對角化Inthegeneralcase,needtodiagonalizeM:特征值決定橢圓軸距，R決定方向TheaxislengthsoftheellipsearedeterminedbytheeigenvaluesandtheorientationisdeterminedbyR:directionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2Visualizationofsecondmomentmatricesdirectionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2Visualizationofsecondmomentmatrices特征值決定橢圓軸距，R決定方向directionoftheslowestchangedirectionofthefastestchange(

max)-1/2(

min)-1/2ClassificationofimagepointsusingeigenvaluesofM:Interpretingtheeigenvalues

1

2“Corner”

1and

2arelarge,

1~

2;

Eincreasesinalldirections

1and

2aresmall;

Eisalmostconstantinalldirections“Edge”

1>>

2“Edge”

2>>

1“Flat”regiondirectionoft