Chap01D ImageMatching

Candidate Moving window
subarray A subarray B
▪ Pengertian Image Matching

▪ Klasifikasi Digital Image Matching
▪ Area-based Matching Principle Search array
▪ Normalized Cross-correlation Method

▪ Normalized Cross-correlation as Linear Regression
▪ Computing Correlation Coefficient Using a Moving Window within
a Search Array
▪ Maximum Correlation Factor
▪ Matching Challenges
▪ Least Squares Matching
▪ Aplikasi Image Matching dalam Fotogrametri
DIGITAL IMAGE MATCHING
2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 1

Pengertian Image Matching
Digital Image Matching / Pencocokkan Citra secara Dijital
Salah satu kegiatan yang umum sehubungan dengan penggunaan ste-
reoplotter adalah menempatkan titik apung pada posisi model 3-D
dari suatu titik obyek.
▪ Ini berarti half-mark kiri dan kanan berada pada titik/obyek yang sama di
foto kiri dan kanan.
▪ Ini memerlukan kemampuan untuk mengenali karakteristik citra (tekstur,
bentuk, dsb.) yang mirip/sama dalam daerah yang kecil/terbatas dari
citra kiri dan kanan suatu pasangan stereo.
Sistem penglihatan manusia mampu melakukan tugas ini dengan

mudah, yaitu dengan persepsi 3-D mata kiri dan kanan.
▪ Ini adalah manual image matching.
Dalam fotogrametri dijital, pencarian titik/obyek yang sama dari suatu

foto di foto(-foto) lainnya dengan menggunakan perangkat lunak
komputer disebut sebagai digital image matching.

Klasifikasi Digital Image Matching
Ada tiga kategori : area-based, feature-based, dan hybrid.
Metode area-based : melakukan perbandingan secara numeris dari di-
gital numbers dalam subarrays yang kecil/terbatas dari masing-masing
citra.
▪ pendekatan ini sangat langsung dan umum digunakan dalam sistem soft-
copy.
▪ lower level
▪ yang akan dibahas dalam kuliah ini.
Metode feature-based : mencakup ekstraksi features (tepi/edges pada
skala yang berbeda), dilanjutkan dengan perbandingan berdasarkan
karakteristik feature seperti bentuk dan ukuran.
▪ memerlukan teknik-teknik dari kecerdasan buatan dalam ilmu komputer
▪ higher level, lebih rumit
Metode Hybrid : merupakan kombinasi dari kedua pendekatan di atas.
▪ pra-pengolahan dari citra kiri dan kanan untuk menandai feature (tepi)
▪ setelah features terlokasi, maka pencocokkan citra dilakukan dengan meto-
de area-based.

Area-based matching basic principle
Finding pixel-to-pixel correspondences.
▪ For each pixel in the left image, search for the most similar pixel in the right
image
▪ Using neighbourhood windows.

Visualisasi Metode Area-Based
Pada contoh ini titik
BM-01 pada bagian fo-
to kiri akan dicari pa-
sangannya pada foto
kanan menggunakan
cara pencocokan citra
(image matching)
Search image (array)
Moving window (array)
BM-01
Candidate image (array) search window (array)

Area-Based Matching
Choice of Window Size Some Methods
Factors to be considered : Similarity measures for two
▪ Ambiguity windows.
▪ Noise sensitivity ▪ SAD (sum of absolute differences)
▪ Sensitivity towards viewpoint re- ▪ SSD (sum of squared differences)
lated distortions ▪ CC (cross-correlation)
▪ Expected object sizes ▪ Least Squares Matching
▪ Frequency of depth jumps ▪ ...

Normalized Cross-Correlation Method (1)
The simplest area-based digital image-matching method.
A statistical comparison is computed from digital numbers taken from

same-size subarrays in the left and right images.

A correlation coefficient is computed by the following equation, using
digital numbers from subarrays A dan B.
m n
 __ __

  ( A
 ij
i 1 j 1 
 A )(Bij  B )
  AB
c  
 m n  __

2
  m n
 __
 
2  A B
   Aij  A      Bij  B  
 i 1 j 1 
    i 1 j 1   
c : correlation coefficient ;
m and n : the numbers of rows and columns in the subarrays ;
Aij : the digital number from subarray A at row i, column j ;
Bij : the digital number from subarray B at row i, column j ;
A : the average of all digital numbers in subarray A.
B : the average of all digital numbers in subarray B.

The correlation coefficient : -1 ≤ c ≤ +1
▪ c = +1, perfect correlation

• an exact match
▪ c = -1, negative correlation

• identical images from negative & positive
▪ c near zero indicate a nonmatch

• comparison of any two sets of random numbers
▪ Usually c < +1, e.g. because of image noise
▪ General threshold value : 0.7

• If the c > 0.7, the subarrays are assumed to match.

Contoh Hitungan Koefisien Korelasi
A B
35 35 30 34 33 31
45 60 40 46 58 39
40 50 35 38 52 34
Rata-rata A 41.11 Rata-rata B 40.56
Aav = Ai,j - A Bav = Bi,j - B
6.1 6.1 11 6.56 7.56 9.56
3.9 18.9 1.1 5.44 17.4 1.56
1.1 8.9 6.1 2.56 11.4 6.56
m n
 __ __

  
i 1 j 1 
( Aij  A )(Bij  B )

c  = 0.9871
 m n  __
  m n 
2
__
 
2
   Aij  A      Bij  B  
 i 1 j 1 
    i 1 j 1 
  
c > 0.7 maka kedua citra sesuai (matched)

Normalized Cross-correlation
as Linear Regression (1)
E.g., assume the follow- (b) y = α + βx
ing pair of 3 X 3 arrays
of digital numbers are to
be analyzed using linear
regression.
25 48 89 
A  43 94 47  =β
76 21 57 
Linear regression
33 56 81
B  40 98 54 
=α
84 16 49  (a)

Normalized Cross-correlation as Linear Regression (2)
Tabular form for computing linear regression
S   ( x i  x )  x
2 2 2

xi 
2
 33,210 
500 2
 5432 .2
x i
n 9
S   ( y i  y )  y
2 2 2

y i 
2
 34,619 
5112
 5605 .6
y i
n 9
Normalized Cross-correlation as Linear Regression (3)
Sxy   xi  x y i  y    x i y i  

xi y i   33,705  500  511  5316 .1
n 9
Sxy 5316 .1
   0.979 (slope)
Sx2 5432 .2
y i x i
511  500 
  
 0.979    2.41 ( y intercept)
n n 9  9 
Sample correlatio n coefficien t (normalize d cross - correlatio n coefficien t) :
Sxy 33,705
r   0.963
2 2
S x Sy 33,210  34,619
Rumus ini tidak menghitung harga rata - rata dan selisih terhadap harga rata -
rata, sehingga perhitunga n akan lebih cepat.
Kelemahann ya, nominal angka - angka dalam  menjadi lebih besar.

Computing Correlation Coefficients Using a
Moving Window within a Search Array (1)
A candidate subarray from the left
photo is chosen, and a search will be
performed for its corresponding sub-
array in the right image.
Since the exact position of the image

in the right image is not initially
known, a search array is selected
with dimensions much larger than
those of the candidate subarray.
A moving window approach is then

used, comparing the candidate sub-
array from the left image with all pos-
sible window locations within the
search array from the right image.

Computing Correlation Coefficients Using a
Moving Window within a Search Array (2)
At each window location in the search
array, the correlation coefficient is
computed, resulting in a correlation
matrix C.
The largest correlation value in C is

tested against threshold.
▪ If it exceeds the threshold, the corres-
ponding location within the search array
is considered to be the match.

Contoh Hitungan Image Matching (CC Technique) (1)
The candidate array A is an ideal template for fiducial cross, and the following
search array S is a portion of a digital image containing s fiducial cross.
Compute the position of the fiducial within the search array by correlation.
 41 43 43 49 60 43 41 40 44 
43 44 45 50 64 45 43 43 45 

 0 0 50 0 0  42 43 44 48 63 49 45 42 42 
 0 0 50 0 0   
  42 45 47 50 65 45 45 41 41
A  50 50 50 50 50  S  59 62 62 64 69 64 62 63 60 
   
 0 0 50 0 0  50 48 48 51 68 55 50 54 53 
 0 0 50 0 0  42 41 44 48 63 42 47 47 45 
 
42 44 42 45 62 44 44 45 43 
42 43 44 48 60 47 44 38 35 


Contoh Hitungan Image Matching (CC Technique) (2)
3
The correlatio n matrix :
 0.24  0.09 0.35  0.19  0.19 
 0.24  0.16 0.32  0.21  0.32  Select the maximum correlation
 
C   0.25 0.37 0.94 0.29 0.27  coefficient.
3
  ▪ The maximum value, 0.94, occurs
  0 . 08 0 . 02 0 . 50  0 . 06 0 . 03  at row 3, column 3 of the C array.
 0.28  0.23 0.27  0.18  0.22 
3
 41 43 43 49 60 43 41 40 44  This value was computed when
43 44 45 50 64 45 43 43 45 
the upper left element of the
 3 moving window was at that
42 43 44 48 63 49 45 42 42 
  position (row =3, column = 3) in
42 45 47 50 65 45 45 41 41 the search array.
S  59 62 62 64 69 64 62 63 60  ▪ Since the center of the cross in the
 
50 48 48 51 68 55 50 54 53  template is 2 columns to the right
42 41 44 48 63 42 47 47 45  and 2 rows down from the upper
  left corner, the center of the cross
42 44 42 45 62 44 44 45 43  in the search array is at row 3 + 2
42 43 44 48 60 47 44 38 35 
 = 5 and column 3 + 2 =5.

Maximum Correlation Factor (1)
The solid circles are the correlation factors obtained at different locations of the
matching window.
Fitting a parabola through the points and determining its maximum would allow
determination of the matched position to subpixel accuracy.

Maximum Correlation Factor (2)
Typical problems occurring in the determination of the maximum correlation value

(a).The SNR of the template and correlation window is low.
(b).May be caused by repetitive patterns in the windows.
(c).Difficulty with global thresholds is manifest : although a distinct peak, the
maximum is far less than in (a).

Matching Challenges
Scene elements do not always look the same in the two images
because of :
1. Camera/sensor-related problems : (radiometric problems)
▪ Image noise,
▪ differing gain, (perekaman yg berbeda)
▪ contrast, etc..
2. Viewpoint-related problems :
▪ Perspective distortions (geometric problems)
▪ Occlusions (halangan)
▪ Specular reflections (pantulan seperti cermin)

Distorsi Geometri pada Entitas Matching
(Perspective Distorsions)
Distorsi Geometri akibat parameter orientasi :
a) Perbedaan skala antar dua foto/ image
b) Perbedaan rotasi (kappa, phi, omega) antar dua foto/ image
c) Effek kemiringan permukaan.
d) Perbedaan bentuk akibat efek relief
➢ Distorsi b) dapat dipecahkan dengan epipolar resampling.

➢ Distorsi a), b), dan c) dapat dipecahkan dengan least squares
matching. (untuk daerah yang kecil, yaitu candidate subarray A
dan moving window subarray B)

Ideal Case
(a). A flat surface patch with point P in its center is imaged in two image
patches, whose centers are in P’ and P”.
(b). The two image patches are identical in this ideal case.

Scale Effect
(a). Stereopair with different image scale.

(b). The projection of two conjugate image patches onto a flat surface.
The dark shaded pixels of the left-upper corner of both image patches are assumed to be
conjugate in area-based matching.
Upaya
Their differentmengatasi dengan
position on the surfacemenyeragamkan skala
illustrates the geometric melalui
distortion introduced by a
penyamaan jarak antar TU 1-TU2’ dan TU1’-TU2) pada
(TUdifference.
scale
kedua foto yang bertampalan
Rotations Effect
Effect of rotation differences between the left and right image of a stereopair.
(a). Effects of rotation about x-axis (flight direction).
(b). Effects of rotation about the y-axis.
(c). Effects of rotation about the z-axis.

Tilted Surface Effect
(a).The surface is tilt-
ed about the air base
(C’C”).
▪ There is no ill effect
because the pixels still
refer to the same
ground pixel.
(b).The surface is tilt-
ed perpendicular to
the air base, a situati-
on referred to as fore-
shortening.
▪ Here, “conjugate” pix-
els point to different a-
reas in object space
and the similarity mea-
sure becomes pro-
blematic.
Effect of tilted surface on the similarity measure in area-based matching.

Relief Effect

Least Squares Matching
Least squares matching is closely related to the correlatio n method, with the added advantage of being able
to obtain the match location to a fraction of a pixel (split pixel accuracy).
The following form being commonly used.
A(x, y)  h0  h1B( x , y )
x  a0  a1x  a2 y
y  b0  b1x  b2 y
A(x, y) : the digital number from the candidate subarray of the left image.
B(x, y) : the digital number from a subarray in the search area of the right image at location (x, y).
h0 : the radiometri c shift.
h1 : the radiometri c scale.
ai , bi : parameters of affine relationsh ip between the coordinate s of the pixel on the left photo and the coor -
dinates of the correspond ing pixel on the right photo.
This functional models correct geometric (different ial scale, kappa rotation, shear) and radiometri c (camera/
sensor related problems) distorsion between left and right image relatively .
Combinatio n of epipolar resampling and least squares matching will increase the speed and (geometric ) ac -
curacy of digital image matching.

Positions of subarrays for least squares matching.
The original matching window m0 is transformed to a new position and
new shape after every iteration.
The gray values of the transformed window m1 are found by resampling.

Aplikasi Image Matching dalam Fotogrametri
1. Pada proses Orientasi Dalam dan Relatif (saat mencari pasangan
titik untuk diamati).
2. Pada proses Triangulasi Udara (saat mencari pasangan titik ikat
pada semua foto dimana titik tersebut muncul).
3. Pembentukan DTM (DTM Generation) (saat mencari pasangan
titik foto untuk posisi titik DTM).
4. Proses produksi ortofoto dijital
5. Digitasi pada layar monitor (onscreen digitizing)

Stereo restitution of a stereopair is
Analytical
realized by intersecting 5 pairs of rays Stereo Model &
coming from left and right photograph
at standard points. Image Matching
Stereo model achieved after relative
and absolute orientation is construct-
ed from unlimited intersection points,
where the coordinates (photos and
model) of five intersected points i.e.
the standard points are known as the
result of the restitution process.
But for the rest of the intersection/
model points have to be searched and
computed as required. Here, the
image matching will play role.

Aerial
Triangulation
Process
Tie points selection &
indentification :
Tie points are required to connect
adjacent foto or models in block
adjustment.
Common point in one photo has to be
transferred into other photo(s), here
comes the image matching
application.

Finding corresponding
elements in the two
images by matching small
regions from the left and
right images of a
stereopair in aerial
triangulation

Coordinate Measurements in AT
Coordinate
measurements for
control points, tie
points and single
points.
In softcopy
photogrammetric
instrument coordinate
measurements mostly
are done
stereoscopically.
Stereoscopic vision for
any point in the stereo
model is automatically
created based on
collinearity condition

Automatic DTM Generation
Digital Terrain Model
generation
To be able to measure
height of a certain DTM
point(s), related left and
right image at their
surrounding area have
to be searched and
matched. (after photo
orientation)
DTM

Slit/ pixel
Digital taken from

left photo
p'
p''
Orthophoto
In digital orthophoto if the Height data
Guide profile
input from
process is guided by every profile Scanning guide
point dH 3D model
stereo model, the stereo Scale and
Position correction
viewing needs the help of mechanism
image matching. P
Rectified
P' Orthophoto
Pixels plotted
after correction
building up
orthophoto

On
Screen
Digitizing
In “on screen”
digitizing,
observation is
based on stereo
model, the stereo
viewing involving
image matching
process.

Tugas Hitungan Image Correlation
Diberikan candidate array A dan search window array S di bawah ini.
 48 62 68 39 42 
 31 38 41 44 
 47 51 53   52
A   43 49 50  ; S   49 67 55 79 74 
 
39 52 67   31 42 74 87 70 
32 61 42 51 66 
a. Tentukan posisi piksel yang paling cocok dengan array A pada search window array S !
b. Jika posisi kolom 1 dan baris 1 pada search window array A berada pada kolom 300 dan baris 200 dari citra
dijital original, tentukan posisi soal a.) pada citra original !

References
▪ I.S.M. International Systemap Corp., 2000. The Fundamentals of Digital
Photogrammetry, Vancouver, British Columbia, Canada V6E 4A2.
▪ McGlone, J.C., ed., 2004. Manual of Photogrammetry, 5th ed.,
American Society of Photogrammetry and Remote Sensing, Maryland
20814, USA, 1151 p.
▪ Mikhail, E.M., J.S. Bethel, and J.C. McGlone, 2001. Introduction to
Modern Photogrammetry, John Wiley & Sons, Inc., New York, 479 p.
▪ Schenk, T., 1999. Digital Photogrammetry, vol. I, TerraScience, OH
43135, USA, 428 p.
▪ Wolf, P.R., and B.A. Dewitt, 2000. Elements of Photogrammetry : with
Application in GIS, 3rd ed., McGraw-Hill, New York, 608p.
Aknowledgements
Some data, information, illustrations, and diagrams in this presentation are
quoted from .ppt files developed by Dr. Ir. Bobby S. Dipokusumo, M.Sc. and
Dr.Ir. Andri Hernandi, MT.

PROBLEMS / EXERCISES


Chap01D ImageMatching

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Chap01D ImageMatching

Diunggah oleh

Hak Cipta:

Format Tersedia

Candidate Moving window

▪ Pengertian Image Matching

▪ Normalized Cross-correlation Method

DIGITAL IMAGE MATCHING

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 1

Sistem penglihatan manusia mampu melakukan tugas ini dengan

Dalam fotogrametri dijital, pencarian titik/obyek yang sama dari suatu

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 2

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 3

▪ Using neighbourhood windows.

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 4

Search image (array)

Moving window (array)

Candidate image (array) search window (array)

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 5

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 6

A statistical comparison is computed from digital numbers taken from

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 7

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 8

▪ c = +1, perfect correlation

▪ c = -1, negative correlation

▪ c near zero indicate a nonmatch

▪ Usually c < +1, e.g. because of image noise

▪ General threshold value : 0.7

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 9

Rata-rata A 41.11 Rata-rata B 40.56

Aav = Ai,j - A Bav = Bi,j - B

6.1 6.1 11 6.56 7.56 9.56

3.9 18.9 1.1 5.44 17.4 1.56

1.1 8.9 6.1 2.56 11.4 6.56

c > 0.7 maka kedua citra sesuai (matched)

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 10

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 11

Tabular form for computing linear regression

Sxy   xi  x y i  y    x i y i  

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 13

Since the exact position of the image

A moving window approach is then

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 14

The largest correlation value in C is

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 15

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 16

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 17

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 18

Typical problems occurring in the determination of the maximum correlation value

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 19

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 20

➢ Distorsi b) dapat dipecahkan dengan epipolar resampling.

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 21

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 22

(a). Stereopair with different image scale.

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 24

Effect of tilted surface on the similarity measure in area-based matching.

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 25

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 26

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 27

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 29

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 31

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 32

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 33

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 34

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 35

2/3/2020 1:12 PM Digital Photogrammetry Principles, SHM 36

Digital taken from