1、Image Registration in Remote Sensingwhen images are from different modalitiesJean SEQUEIRA LSIS Aix-Marseille UniversityShenzhen March 2016Remote Sensing: satellites, airplanes, UFA EnvisatRemote Sensing imagesImages of a same region (radar, multispectral and hyperspectral)Agriculture controlTwo ima

2、ges in C band, of DLR-SAR system, on April 19th and 24th of 2006, in GermanyNon-controled urbanization in the area of Iguazu FallsPollution controlENVISAT (ASAR) image showing the pollution generated by the tanker Prestige in november 2002Subsidence controlSubsidence map of MexicoData, Information,

3、Knowledge, IntelligenceData: Raw elements (Image)Information: What can be extracted from data Knowledge: World representationIntelligence: Action on this WorldExemple : Data: image in which each pixel is a vector (p scalar values)Information: band composition (e.g. using PCA or ICA)Knowledge: multis

4、pectral image, frequencies, reflectivity of scene elements, problem to be solved we can use knowledge to extract information but we have to take careRegistrationFig III.1 page 36Matching pixels of both images through their position in the sceneRegistrationWe have to know:Which kind of transformation

5、A set of landmarks to provide the matching processWe have to characterize the common information and the location where information is differentAn exampleImage deformationSensor location and orientation but also FigNeed for a global registration ( Image Map et Digital Terrain)Kind of transformationW

6、hich transformations ?Translation Rotation Translation Rotation Scaling 2D Homography3D HomographyPiecewise 3D HomographyKnowledge: satellite or plane parameters Digital Terrain Rigid transformationsTranslation Rotation (scaling)Equivalence between the order of transformation process:Rotation Transl

7、ation vs. Translation Rotation z = a.(z+t) or z = a.z + t with a, z and t complex 3 variables (tx, ty and q) or 4 (with the scaling factor s)Finding common landmarksSeveral problems:Characterization of common information areasFinding landmarksWhich landmarks?Points or structures Pixel content and ne

8、ighborhood (texture, mutual information, ) Different modalities2D clouds of points obtained from imagesClassical approaches for registering 2D clouds of points using a Rigid Transformation (Translation + Rotation)Different approaches:ICP (Iterated Closest Point Chen & Medioni, 1991 ; Besl & Mc Kay 1

9、992)Geometric Invariants (Hu Moments, Shape Context - Hu, 1962)RANSAC (RANdom SAmple Consensus Fischer & Bolles, 1981)Hough Transform (Robust Estimators Hough 1959,1962 Duda & Hart 1972)Fig IV.2 page 63Using the Hough Transform for 2D clouds of points registrationWhich Hough Transform? TH from m to

10、1Drawback: Very sensitive to noiseConclusion: We need to introduce knowledge to make it more robustAn important remark: parts of 2D clouds of points can be associated with 1D structuresWhy is it like that? Whatever the modality or the physical property used to capture and represent these images, poi

11、nts that represent relevant information are located on lines (boundaries)SAR (Synthetic Aperture Radar) ImagesAndaman Island (Bay of Bengal) ENVISAT / ASARClouds of pointsBinary images obtained by applying a Touzi operatorNoise and Common InformationCharacterization of a 1D underlying structurePrimi

12、tive : (x , y , q , s)(location, orientation, score)Different kinds of possible situations a unique representationCharacterization of an initial set of primitivesUse of an incremental maskCharacterization of the final set of primitivesFrom the initial cloud of points to the final set of primitives:D

13、efinition of mask parameters (radius) and threshold valueApplication to each point it provides or not a new primitive (x , y , q , s)Sort of all these primitives by decreasing scoresFinal set of primitives: we go through the sorted list and we only keep those primitives whose distance to all the sel

14、ected ones is over dminProblem evolutionInitialy (rough mode): two images of different modalitiesProblem: not the same information about the content, and not the same representationThen ( unified mode): two representations in the same way (clouds of points)Problem: important sensitivity, even using

15、the best estimatorFinally (introducing knowledge): two representations as a reduce set of primitives that characterize the local orientation of the information where it is relevantMethod: comparison of these two sets of primitivesFrom a problem of image registration to a problem of similarity betwee

16、n the neighborhood of two locationsFrom a pair of primitives:a = q1 q2z1 = eia . z2 + twith t = tx + i.ty An important remark: Variable separation: a is obtained independently from tx and ty This separation is only apparent: a calculation is valid only if primitives are in corresponding areas! Thus,

17、 the validity of this calculation only depends on the translation! But in this case, it provides a 1D peak at the a value (whatever a) Conclusion : The study of the robustness of this estimator (of a) provide a similarity measure beetwen two neigborhoods, and it is invariant by rotationSome results and some problemsCharacterization of a peak: Integration on neighbors when the rotation has an intermediate value:Some results and some problemsStudy of peak detection as a function of n, k and kb/k :The sampling level is very importantTaking care of