Imagenes Coincidentes PDF

DEVELOPMENT AND STATUS OF IMAGE MATCHING
IN PHOTOGRAMMETRY
Armin Gruen (armin.gruen@geod.baug.ethz.ch)
Institute of Conservation and Building Research, ETH Zurich, Switzerland
(Based on a presentation at the Ian Dowman Retirement Symposium, entitled
Progress and Prospects for Photogrammetry and Remote Sensing in a Changing World,
held at University College London on 28th June 2010)
Abstract
Image and template matching is probably the most important function in digital
photogrammetry and also in automated modelling and mapping. Many approaches
for matching have evolved over the years, but the problem is still unsolved in general
terms. This paper describes the development of image matching techniques in
photogrammetry over the past 50 years, addresses the results of some empirical
accuracy studies and also provides a critical account of some of the problems that
remain. Although automated approaches have quite a number of advantages, the
quality of the results is still not satisfactory and, in some cases, far from acceptable.
Even with the most advanced techniques, it is not yet possible to achieve the quality of
results that a human operator can produce. There is an urgent need for further
improvements and innovations, be it through more powerful multi-sensor ap-
proaches, thereby enlarging the information spectrum, and/or through advancements
in image understanding algorithms, thus coming closer to human capabilities of
reading and understanding image content.
Keywords: DSM generation, empirical tests, image matching, least squares,
multi-image, multiple image features
Introduction
Professor Ian Dowman was among the first to propose the use of fully digital systems in
photogrammetry (Dowman, 1984), in this case for topographic mapping from satellite data. In
any of those systems image matching is a crucial function, upon which many other follow-up
products will depend.
Image matching is a key component of many tasks in photogrammetry, computer vision and
image analysis; it is also crucial to a wide range of applications such as navigation, guidance,
automatic surveillance, robot vision, medical image analysis and to the modelling and mapping
sciences. For more than 50 years, image matching has been an issue of research, development
and practical implementation in software systems. Nevertheless, a critical assessment of the
current status of image matching shows that the problemhas not yet been solved in general terms.
The Photogrammetric Record 27(137): 3657 (March 2012)
DOI: 10.1111/j.1477-9730.2011.00671.x
2012 The Author. The Photogrammetric Record 2012 The Remote Sensing and Photogrammetry Society and Blackwell Publishing Ltd.
Blackwell Publishing Ltd. 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street Malden, MA 02148, USA.
This paper aims to describe the major lines of development and achievements as they can
be traced in the mapping community. With a topic so important to many disciplines and the
limited space provided, it is clear that not all developments can be described and given proper
credit here.
There is a large body of publications on image matching in the computer vision literature;
however, no particular developments, except some basic interest operators or other image
analysis algorithms, have transferred successfully into photogrammetric systems. Lately,
PhotoSynth (Snavely et al., 2006) has become well known for its simultaneous and automated
orientation of hundreds of non-calibrated (Internet) images and its associated derivation of
sparse point clouds. While this is a quite interesting development, the focus of this approach is
not on image matching for surface model generation; indeed no high-quality dense and
complex surface models have been shown up to now. Some other recent developments,
however, show some promising matching results (Hirschmueller, 2008; Vu et al., 2009), albeit
that strictly controlled tests are still missing.
This contribution aims at tracing the development in image matching in photogrammetry
from the middle of the 1950s until the present day. Since the author and his group of
researchers have worked in this eld for about 30 years and contributed a great number of
publications, there will be a certain focus on this work. The chronology in developments will
be structured into the following periods: the Early Years (1960s and 1970s); the New
Approaches (1980s); the Time of Consolidation and Extensions (1990s); and nally the
Time of Acceptance (2000s).
The discussion on the development and status of image matching must also take into
consideration that this technique is used for a variety of different tasks, with different
prerequisites and expectations. The prime, though not exclusive, applications of image
matching in our elds are for surface generation, tie and control point measurement for
orientation and triangulation, industrial quality control (targeted and non-targeted points),
feature (edge) extraction and feature/object tracking. The type, size and quality of images used
(satellite, aerial, terrestrial) and the expected accuracies of the results vary greatly.
Accordingly, a critical analysis will lead to different results. Therefore, the arguments will
be based on the achievable results and not necessarily on those required by a certain
application. Research in this context is not only about developing some new methodology, but
also about providing a clear understanding of its properties, which means pushing this
methodology to its performance limits in order to gain an insight into its potential and
limitations.
Three Basic Matching Techniques
Image matching has been a major research issue in computer vision and digital
photogrammetry for many years; accordingly, many different approaches have evolved. Three
basic matching techniques can be distinguished:
(a) intensity-based;
(b) feature-based; and
(c) relational.
In intensity-based matching the original, or slightly modied (enhanced), image data is
used in the form of a matrix of grey values. The most prominent methods are cross-correlation
and least squares matching (LS matching or LSM), which are also called area-based matching.
They provide sub-pixel accuracy, in extreme cases 1/10 pixel and even better. LS matching is a
highly non-linear process and therefore requires very good approximate values.
The Photogrammetric Record
2012 The Author
The Photogrammetric Record 2012 The Remote Sensing and Photogrammetry Society and Blackwell Publishing Ltd 37
Feature-based matching requires, rstly, the extraction of basic image features, such as
patches, corners, junctions, edges and so on. In a second step, matching is performed between
these features. Features are sometimes, but not always, more stable with regard to reectance
characteristics. On the other hand, information that is lost during the feature extraction phase
can no longer be recovered. Some methods provide for sub-pixel accuracy, but not at the level
of the intensity-based methods.
Feature-based matching has been performed with:
(1) relaxation;
(2) dynamic programming;
(3) robust estimation;
(4) cross-correlation; and
(5) graph matching.
The solution space may be reduced by constraints such as:
(1) use of epipolar images;
(2) use of more than two images;
(3) limits on the magnitude of changes in parallax;
(4) a priori modelling of objects (coarse description of object);
(5) hierarchical coarse-to-ne strategies;
(6) best-rst strategies, using features sequentially, according to the relevance of their
information content;
(7) thin-to-thick or thick-to-thin strategies (either starting with just a few saved
match points as a skeleton and densifying, or starting with a dense point eld and
thinning out by blunder detection); and
(8) observation of behaviour of parallaxes (inections are not allowed).
Relational matching uses geometric or other relations between features and structures
(combination of features). Correspondence is established by tree-search techniques. These
methods are not very accurate but are usually robust; they do not require good approximations.
Their use in digital photogrammetry for digital terrain model (DTM) generation is rather scarce.
There are several more or less exhaustive descriptions of these various techniques
available (Lemmens, 1988; Baltsavias, 1991, Chap. 3).
The Early Years (1960s and 1970s)
Image matching was rst introduced in the early 1950s. It started as an analogue
procedure using electrical circuits for solving the matching equations (see, for example,
Hobrough, 1959). A good survey of the very early efforts of analogue cross-correlation is
given in Hobrough (1965). It clearly shows that equipment manufacturers were the driving
force behind the development, rather than university groups. A particularly successful and
much discussed system was the Gestalt Photomapper GPM I and GPM II (Kelly et al., 1977;
Alberich, 1985). In Mikhail et al. (1978), Fred Doyle reports the experiences of the USGS with
the GPM. They used it only with small scale photography (1:80 000) for orthophoto
production and reported a height accuracy of 5 m. Hobrough (1965) already lists two attempts
at digital correlation (Williams, 1959; DeMeter, 1963). What is amazing is the great optimism
that accompanied these developments. The automated mapping problem was considered to be
practically solved; Hobrough (1965) states: Within the next three to ve years many of the
present automation programs should be completed and several of the conventional
photogrammetric operations will probably be automated on a more or less routine basis.
Gruen. Development and status of image matching in photogrammetry
2012 The Author
38 The Photogrammetric Record 2012 The Remote Sensing and Photogrammetry Society and Blackwell Publishing Ltd
By the middle of the 1970s, the increasing computational power of digital computers
allowed for the fully digital treatment of the matching problem. Helava (1972) already
discussed the epipolar principle in matching and a little later (Helava, 1976, 1978) presented an
in-depth study of auto- and cross-correlation based on linear system theory. He listed 11 cross-
correlation principles and was critical of the traditional cross-correlation approach, which is
insufcient for photogrammetric stereo applications and requires image data shaping to
account for the phase differences between different image frequencies and within each
frequency. The image power spectrum also reveals cases of multiple correlation peaks,
whereby the highest peak does not necessarily represent the correct solution. Helava also
referred to the new AS-11B-X system, an automated stereo mapper, which used parallel
processing of neighbouring terrain proles in order to perform the matching of one aerial
model within 10 minutes, a speed that is dreamt of even today. For more technical details of
the system, see Scarano and Brumm (1976). In an interesting variant, Masry (1974) used cross-
correlation on an analytical plotter with epipolar constraints for change detection. Dowman and
Haggag (1977) also worked on this method.
By the early 1980s, the literature on image analysis and matching had grown
tremendously. Therefore, only very few publications can be referenced here, most of which
possess an overview character.
The approaches and achievements within the photogrammetric community of those early
years are described in Makarovic (1980), Konecny and Pape (1981) and Baltsavias (1984).
Computer vision scientists also had an early and signicant impact on matching techniques (as
an example, see Baker and Binford, 1981). Overviews and summaries of the state-of-the-art
methods can be found in Aggarwal et al. (1977), Andrews (1978), Bernstein (1978) and
Chellappa and Sawchuk (1985).
It was obvious for those closely involved in matching problems that the existing
approaches, which were mostly based on cross-correlation, had signicant deciencies. Helava
(1976) states that the human operator is far superior and addresses with this remark the lack
of suitable image understanding algorithms.
The characteristics of cross-correlation were well understood by the early 1980s and its
deciencies were identied:
(1) discrepancy between conjugate images, caused by geometrical distortions (terrain
slope, height differences, positional and attitude differences of sensors), radiometric
problems (illumination, reectance, varying material properties) and imaging arte-
facts;
(2) discretisation of trial step size; and
(3) lack of good methods for the assessment of results, gure-of-merit (quality of match).
The shortcomings of this class of image matching methods nally, after early excitement
and false predictions, caused a slow-down in the development of operational automated
correlation systems. This problem was addressed at a panel session at the annual ASP
Convention in Denver, Colorado in March 1982. Five short articles in Photogrammetric
Engineering & Remote Sensing (PE&RS, 1983) reect a part of this discussion.
Cross-correlation cannot respond appropriately to a number of facts that are inseparably
related to stereo-images of three-dimensional and sometimes even two-dimensional objects.
The conjugate images created under the laws of perspective projection might differ
considerably from each other. Terrain slope, height differences, and positional and attitude
differences of the sensors cause geometrical distortions. Illumination and reectance conditions
might distort the images radiometrically. Under certain circumstances, this may even trigger a
geometrical displacement in the matching. Noise from the electrical components and the
2012 The Author
sampling rate (pixel size) could also inuence both the geometric and the radiometric
correspondence of the images.
Cross-correlation is very simple conceptually, easy to implement and computationally
fast. The main problem with conventional cross-correlation is that it allows only for two shift
parameters between template and patch. Rotations, scale and other deformations between
template and patch cannot be accommodated. Therefore, the following rules should be
observed when applying cross-correlation: use it only with epipolar images and use small patch
sizes.
Cross-correlation works well and is fast if the patches to be matched contain sufcient
signal without too much high-frequency content, and if geometric and radiometric distortions
are kept to a minimum. Both conditions are not often encountered in aerial and terrestrial
images. On the other hand, in satellite images, due to their smaller image scale, these problems
are less inuential and cross-correlation is more likely to succeed. Therefore, some efforts were
made very early on to design matching techniques that are more efcient than cross-
correlation. Concepts that were suggested by the articial intelligence community included:
rst- and second-order derivative matching; relaxation methods; segmentation and graph
structure matching; transform (Hough transform) matching; and feature (edge) matching.
Thus, a tendency to switch from area-based to edge-based analysis could be observed.
However, Rosenfeld (1984) remarked, in an excellent critical review, that these new methods
also did not solve the problems referred to above.
It is important to note that the concept of epipolar line matching, for the purpose of
reduction of the matching solution space and thus the number of false matches, was already
proposed by Helava (1972). Panton (1978) clearly expressed the need for epipolar line
constraints, patch shaping, algorithmic tuning, reliability monitoring and even parallel
processing, by showing the rst implementations of these concepts. He demonstrated that
parallel processing can result in amazing performance: 270 match points per second,
35 minutes for a full stereomodel consisting of 562 500 points on a CDC 1700 minicomputer.
Compared to the single processor mainframe computer CDC 6400 this gave an increase in
speed by a factor of 34.
In a much later study, Zheltov and Sibiryakov (1997) have shown that cross-correlation
can be modied into a version that takes care of all six parameters of an afne transformation
between template and patch, albeit at higher computational expense. It also has been shown
that this modied version is equivalent to least squares matching.
New Approaches (1980s)
The 1980s saw a long period of very active development of new, more powerful,
matching approaches. Originally, the driving force behind the new developments was the
equipment industry. Now researchers from universities took charge. At the same time, the
system manufacturers started to offer software solutions for digital matching that were
incorporated in photogrammetric equipment, rstly in analytical plotters and later in digital
stations.
Probably the most signicant contribution, the least squares matching technique (LS
matching, LSM), was developed in the early 1980s. Due to its exibility and accuracy, it has
turned out to have a major impact on image matching, with many extensions, and is currently
used in many digital photogrammetric matching tasks.
Early investigations were reported by Forstner (1982), Ackermann (1984) and Pertl
(1984). The author also investigated this technique in 1982 as adaptive least squares matching
(ALSM). The method was called adaptive because it can be executed in a self-tuning mode,
2012 The Author
meaning the parameter set to be estimated can be corrected automatically in order to obtain a
most appropriate estimation model set-up with respect to the specic signal content of the
patches to be matched.
Early concepts and software were developed in a project funded by Helava Associates,
Inc. Tests were performed with synthetic and real images. The method was found to be of great
potential for a variety of image- and template-matching problems (Baltsavias, 1984; Grun,
1984; Gruen and Baltsavias, 1984).
If properly used, least squares matching combines the advantages of area-based and edge-
based matching. The basic equations are set up in the context of a statistical estimation model.
The estimation itself is performed as least squares estimation. The familiar apparatus of the
least squares approach with respect to parameter estimation and hypothesis testing can be
favourably utilised. Precision and reliability measures are readily available and allow an
assessment of the quality of the match in a better way than is feasible with other matching
techniques. Algorithmic, computational and numerical aspects can also be studied in a well-
known environment.
ALSM has great potential in different respects, as recognised from the very beginning and
reiterated here:
(a) high matching accuracy;
(b) geometrical/stochastical constraints: stabilisation, reliability, speed;
(c) multi-image matching (reliability);
(d) simultaneous matching/point positioning;
(e) multi-patch matching: neighbourhood conditions;
(f) multispectral, multitemporal matching;
(g) monitoring of quality (precision, reliability);
(h) simultaneous image reshaping, radiometric adjustment;
(i) combination of area-based and edge-based analysis;
(j) usable in hierarchical mode (coarse-to-ne);
(k) usable as derivative-operator-based matching procedure (rst-order slope vari-
ables, second order);
(l) rule-based matching: patch selection (good signal content);
(m) incomplete data patches (for example, triggered by occlusions);
(n) computational performance: parallel implementation possible;
(o) usable for pattern recognition (template matching), feature extraction, image feature
measurement (ducials, tie points, control points), change detection, line following;
and
(p) general matching technique (beyond images): DTM/DSM analysis/co-registration,
image/map registration.
A comprehensive description of the basic algorithm and its multiphoto geometrically
constrained (MPGC) extension and many results are given in Baltsavias (1991).
The quality of a matching procedure depends mainly on the type and content of the image
signal. Given the images of an object, there is not much room for improvement of the signal.
Very often there is, however, additional information available that could support the matching.
Important categories of information are geometrical and radiometric conditions. They relate to
the imaging geometry of the sensor, orientation and positional data of the sensor, image feature
radiometry and geometric constraints, and to object constraints. They have to be set up as linear
or linearised observation equations in the least squares context and are, as such, added to the
observation equations for the grey values of the pixels. The resulting hybrid system is of the
combined adjustment type. It leads not only to a much improved matching procedure but, in
2012 The Author
addition, provides for a simultaneous matching/object point positioning technique. Thus, the
two-stage process of image measurement object positioning is replaced by a one-stage
solution that is capable of utilising all available radiometric and geometric information at once.
The MPGC technique offers considerable advantages with respect to precision and
reliability. Because of the use of all the geometric information available and the internal
consistency of the algorithm, the success rate increases and many problematic situations are
signalised:
(1) The use of the geometrical constraints increases the convergence radius and rate
because the search is one-dimensional. The use of multiple scenes has the same
effect. In addition, the search is constrained with respect to direction and step size, so
that image patches subject to small displacements can support those that are subject
to larger displacements.
(2) In many cases, occlusions do not prevent correct convergence. The less occluded
patches benecially inuence those that are more occluded. The quality measures of
the algorithm allow for the detection of occlusions.
(3) Multiple solutions are drastically reduced because of the conditional one-
dimensional search. Mismatches can be detected unless all image patches hit false
maxima along epipolar lines simultaneously, which would be a very rare case.
Other extensions are summarised in Gruen (1996b). In the following some modications
and extensions are addressed, which make this approach so powerful and usable in a variety of
different forms. In particular, the following stages of algorithmic development are
distinguished:
(1) Stereo (two-image) adaptive least squares matching (ALSM) (Gruen, 1985b).
(2) Multiphoto geometrically constrained (MPGC) matching (Gruen, 1985a; Gruen and
Baltsavias, 1985, 1988b; Baltsavias, 1991):
(a) collinearity constraint;
(b) forward intersection constraint (interior and exterior orientations known; X, Y, Z
estimated simultaneously);
(c) epipolar constraint (interior and exterior orientations known; X, Y, Z derived in a
separate step); and
(d) bundle constraint (interior and exterior elements are simultaneously estimated
with X, Y, Z coordinates).
(3) Digital surface model (DSM) constraints:
(a) XY constraint (Gruen, 1985a; interior and exterior elements known; given X, Y,
only Z is estimated simultaneously; this also became known as the vertical line
locus method); and
(b) Z (contour) constraint (Gruen, 1985a; Gruen and Baltsavias, 1986; interior and
exterior elements known; given Z, only X, Y are estimated simultaneously; this is
equivalent to drawing contours froma stereomodel or a multi-image arrangement).
(4) Image feature constraints:
(a) edge constraint (Gruen and Stallmann, 1991).
(5) Globally enforced least squares matching:
(a) multiple patch matching;
(b) 2D patches (Gruen, 1985a);
(c) 3D (volume element) patches (Maas et al., 1994);
(d) linear feature extraction with LS template matching (Gruen and Agouris, 1994);
(e) linear feature extraction with LSB-snakes (Gruen and Li, 1997);
2012 The Author
(f) object-space-oriented LSM (Gruen and Zhang, 2002; Zhang, 2005); and
(g) neighbourhood constraints by stochastic relaxation (Gruen and Zhang, 2002;
Zhang, 2005).
(6) 3D surface matching:
(a) 3D surface and space curve matching (Gruen and Akca, 2005; Akca, 2007).
A particular type of image feature constraint is the image edge constraint. Whenever edges
have to be measured, such as breaklines in DTMs, this may constitute an appropriate solution.
Essentially MPGC is an area-based matching technique. For high accuracy edge matching, the
method is transformed into a combination of an area-based and feature-based technique. This is
achieved by introducing, as a reference template, a synthetic (or real) edge pattern, which is to
be matched with the actual image edges. Compared to the conventional feature-based matching
techniques, this method does not require the extraction of image edges, but matching is done
directly by using the original grey value edges. For algorithmic details, see Gruen and
Stallmann (1991). An efcient automatic measurement procedure can be realised via
implementation of a tracking technique, which tracks the edges either in object or in image
space.
If larger image regions have to be processed by matching, image patches with low or no
signal content pose a serious problem. For such cases the technique of globally enforced least
squares matching has been developed. Here the basic idea is to establish geometrical
neighbourhood conditions between adjacent patches in order to give stabilising support to the
weak patches by the strong ones. Ideally, the weak regions would be bridged and a stable
global solution would be obtained. The rst such solutions were introduced as multipoint
matching or multi-patch matching techniques. The introduction of neighbourhood constraints
leads to a simultaneous solution for all patches. Thus, a full image format can be processed in
one sweep. Early approaches to this concept have been presented by Gruen (1985a), Rauhala
(1986), Rosenholm (1986) and Li (1989).
Another globally enforced technique is object space oriented least squares matching as
introduced by Wrobel (1987), Ebner and Heipke (1988) and Helava (1988b). It represents the
most generalised approach to least squares matching. Due to its complexity with respect to
implementation and handling, it has been used only occasionally and under laboratory
conditions (Kempa and Schlueter, 1993).
In the 1980s, image matching was mostly used in close-range applications. Examples
include: camera calibration (Beyer, 1987, 1992); human face measurements (Gruen and
Baltsavias, 1988a); object tracking (Baltsavias and Stallmann, 1990); and industrial quality
control (Gruen and Stallmann, 1991). Often it was applied in the form of template matching
(for example, Lue et al., 1987). Many more tests and applications are reported in Gruen (1988)
and Baltsavias (1991), the latter noting that scientic investigations with aerial imagery were
rather scarce. Otto and Chau (1988) demonstrated an early application to SPOT satellite stereo-
images using the techniques of region growing. For the results of an ISPRS WGIII/4 test on
image matching, see Guelch (1988).
The many activities in algorithmic development were occasionally also accompanied by
accuracy studies. Schewe and Forstner (1986) reported industrial car measurements on the
analytical plotter Planicomp, with an accuracy of 005 to 02 pixels. Gruen and Baltsavias
(1986) derived DTMs from 1:5300 scale aerial images with a height error of 002 to 004%
of ying height for natural points. This compares with an accuracy of manual measurements
of signalised points of 0003% of ying height (Trinder, 1986). Rosenholm (1986)
processed aerial images at scales between 1:4000 and 1:50 000 with an accuracy of 04 to
06 pixels.
2012 The Author
In summary, the following accuracies have been attained with LSM:
(1) laboratory test, targeted points: 001 to 002 pixels;
(2) close-range applications, project conditions, targeted points: 01 to 02 pixels; and
(3) aerial photogrammetry, natural points: 03 to 05 pixels.
In parallel to these many research and development efforts, commercial systems came
onto the market. Correlators were implemented on analytical plotters, for example, by Kern in
the form of the vertical line locus method (Bethel, 1986; Almroth and Hendriks, 1987) or least
squares matching (Pertl, 1984) and on mono-comparators (Helava, 1988a, with least squares
matching).
From its very beginnings, image matching was not isolated from practice, but always
developed with the aim of integrating it into a photogrammetric processing system. As
explained earlier, these efforts go back to the early 1950s and have been pursued ever since. In
addition, the ISP (International Society of Photogrammetry) had established a working group on
Automated and Analytical Instruments in 1969, well before analytical plotters entered the
civilian market. In 1976, at the ISP Helsinki Congress, this working group was split into two and
a new working group entitled Automated Instruments and Systems was formed. The
International Archives of Photogrammetry, Volume 24 (Proceedings of ISPRS Technical
Commission II Symposium held in Ottawa in 1982) contains a number of papers addressing this
issue. Of particular interest is Case (1982). He describes the concept of a fully digital station
DSCC (digital comparator/correlator) and gives detailed numbers on expected performance
parameters. This is the time when university groups were also investigating fully digital systems
and a number of studies and implementations were revealed (Albertz and Koenig, 1984;
Dowman, 1984; Gruen and Beyer, 1986, 1990; Haggren, 1986; Gruen, 1989), although not all
were supported by image matching functions. A system for the processing of SPOT images
was under development by Dowman et al. (1987). Four different image matching algorithms
were tested, with least squares matching nally implemented (Otto and Chau, 1988).
Many efforts in digital photogrammetry at that time were driven by the fact that the US
Defence Mapping Agency had launched a large research and development programme with the
goal of executing its mapping operations fully automatically and, thus, fully digitally by the
year 1990. The rst commercial photogrammetric digital station was presented as DSP1 by
Kern at the ISPRS Congress in Kyoto in 1988, but at this time without a matching module
(Cogan et al., 1988).
Time of Consolidation and Extension (1990s)
The idea of globally enforced matching was later generalised such that 3D image data-sets
(voxel cuboids) could be matched. The related method was used for the measurement of laser-
induced uorescence ow elds in a technical chemistry application (Maas et al., 1994). A
further modication was suggested for the extraction of linear features (Gruen and Agouris,
1994). This, again, was further developed into LSB-snakes, which combine the powerful tools
of least squares estimation with the determination of energy-minimising functions (Gruen and
Li, 1997).
A particularly promising and successful implementation has been presented by Maas
(1996). This approach is derived from the concept of MPGC matching. It is a multi-image
matching technique, whereby features are searched for along epipolar lines. The computational
effort grows exponentially with the number of images, but specic search strategies help to
keep the computing times within reasonable bounds. Compared to MPGC matching, this is a
linear procedure requiring no approximate values and no iterations. On the other hand, it is
2012 The Author
essentially a searching technique for features whose locations are dened by epipolar lines.
There is some room for improvement in the sense that feature attributes could be considered for
matching and/or least squares area-based matching could be done as a last step.
DTM Generation on Digital Workstations
A large number of commercial digital systems emerged in the 1990s, but not all of them
survived for very long. Only a few have made a commercial impact. Automated DTM
generation, in particular for ortho-image production, is a major function of digital
photogrammetric stations. Most systems and approaches work hierarchically with image
pyramids. They use epipolar images rather than the original images and apply either cross-
correlation or feature-based techniques. The sampling mode is often on a regular grid in either
object space or image space, or based on arbitrarily distributed points. In the last two cases, the
data is often transformed into a regular DTM grid before being presented to the user. In the
authors opinion, this is not an appropriate solution. DTM generation by image matching and
DTM interpolation should be separated. Otherwise, the effects of both are inseparably
combined and the user has no indication of the quality of either procedure.
The most popular digital photogrammetric stations at that time had the following DTM
software and approaches installed:
(1) Leica/Helava DPW, Automated Terrain Extraction (AATE), cross-correlation
(Miller and De Venecia, 1992; Zhang and Miller, 1997);
(2) Zeiss PHODIS ST, Topo SURF (MATCH-T), feature based;
(3) Intergraph ImageStation, MATCH-T, feature based (Krzystek, 1995);
(4) PCI Geomatics, DEM extraction;
(5) SOCET SET, BAE Systems digital DTM generation; and
(6) VirtuoZo, cross-correlation with reshaping, global matching with probabilistic
relaxation (Zhang et al., 1992, 1996).
The results of studies comparing the performance of different digital stations with respect
to DTM generation are referenced in Baltsavias et al. (1996), Gruen (1996a) and Smith and
Smith (1996).
In general, most high-end workstations delivered results of similar quality, although the
underlying algorithms, strategies, robustness and ease of use of the software varied. Most
systems required the user to set a large number of input parameters (up to 28 in a particular
system). Even small changes in seemingly harmless parameters often led to signicant
alterations in the results. There was no logical visible or predictable connection between the
change of parameters and the results. Major problems occurred in the case of homogeneous
texture, shadows, dense vegetation (trees), dark and very steep slopes, water features, urban
environments, and so on. In Gruen (1996a), the main problems were listed as:
(1) lack of recognition of object edges and geomorphologically important features;
(2) no bridging of regions with poor signal content;
(3) insufcient handling of occlusions and shadow areas;
(4) unreliable reduction of DSM to DTM; and
(5) missing quality assessment; no good internal quality control.
In another study (Gruen, 1999), it was shown by empirical tests that the accuracy results
of automated DTM generation were worse by a factor of ve or more compared with those
obtained from analytical plotters. The results of yet another empirical accuracy study were
published in Gruen et al. (2000). The matching software of three commercial systems was
2012 The Author
tested with three different aerial image data-sets of different image scales. The rms errors
achieved were worse by factors of 16 to 16 than the theoretical expectations for manual
measurements. This was largely due to very large numbers of blunders. Gong et al. (2000)
have also carried out an interesting assessment. The low-quality results were caused by the fact
that the capabilities of existing matching algorithms had not been fully utilised. Signicant
improvements could be expected through:
(1) use of more than two images for matching;
(2) making use of all available geometrical constraints;
(3) development of methods for internal quality control;
(4) improvement of user interface;
(5) explanation of functions of parameter settings;
(6) integration of a priori knowledge (such as existing DTMs); and
(7) integration of image understanding algorithms and use of multi-sensor data.
Empirical, controlled testing of matching procedures and software, when applied to
surface model generation, is not a simple task. The need to generate reference data-sets with an
accuracy at least three times better than the expected system performance (a conservative
number) indeed causes some headaches. If a system performance of 03 pixels is assumed then
this translates, in the case of medium scale aerial images (20 cm footprint), into a required
reference point accuracy of 2 cm. In the case of close-range applications with a footprint of
1 mm this value would be 01 mm. For satellite images there are not such stringent
requirements, although when considering the latest generation with a footprint of 50 cm,
problems with reference data generation are also faced. This situation is complicated by the
fact that the matcher will generate hundreds of thousands, or even millions, of points per
stereomodel, thus requiring a very large number of reference points. This is surely one of the
reasons why little suitable empirical test data is available, even up to the present time. As a
result it can be said that the matching procedures and software have developed not only
without a solid theoretical basis (company approaches are usually not even published), but also
without much empirical scientic control. It is up to the individual user to judge the suitability
of his resultsnot a very comforting situation.
Time of Acceptance (2000 to Now)
Automatic DTM/DSM generation through image matching has gained much attention in
recent years. A wide variety of approaches have been developed and automatic DSM
generation packages have, in the meanwhile, been commercially available on several digital
photogrammetric workstations. At the turn of the century, it was still noted that commercial
image matching software did not have the required performance in terms of high-quality
results. This applies in particular to the processing of aerial images, but also to the recently
available high-resolution stereo satellite images.
Close-range applications are considered by most system manufacturers as niche markets.
Therefore, no effort is made to provide suitable software and this area is left entirely to the
activities of academic research groups.
Although the algorithms and the matching strategies of commercial systems may differ
from one another, the accuracy performance and the problems encountered are very similar
in the major systems. Furthermore, the performance of commercial image matchers does
not live up, by far, to the standards set by manual measurements (Gruen, 1996a, 1999;
Gruen et al., 2000). The main problems in automated DTM generation are encountered
with:
2012 The Author
(a) little or no texture;
(b) distinct object discontinuities;
(c) local object patches are not sufcient approximations of planar faces;
(d) repetitive objects;
(e) occlusions;
(f) moving objects, including shadows;
(g) multi-layered and transparent objects;
(h) radiometric artefacts such as specular reections; and
(i) reduction from DSM to DTM.
In the year 2000, the author started development cooperation with Starlabo Inc., Tokyo.
As part of a larger software package, a new matching approach and module was developed for
an aerial three-line scanner (TLS) system. It was later modied to also be able to handle
satellite images (SAT-PP) and terrestrial close-range cases (CLORAMA). The following refers
to descriptions of the TLS system without, however, any restriction in the generality of the
chosen algorithms.
The TLS matcher aims to generate DSMs by considering specically the problems (a) to (f)
above. The matcher is described in detail in several publications (for example, Gruen and
Zhang, 2002; Zhang, 2005). The raw level TLS images were used together with the given, or
previously triangulated, orientation elements. After the generation of image pyramids, the
matcher uses three kinds of image features, namely, general feature points, edge points and grid
points. A triangular irregular network (TIN) based DSM is constructed from the matched points
on each level of the pyramid, which in turn is used in the subsequent pyramid level for
approximation and adaptive computation of the matching parameters. Finally, the modied
MPGC matching is used optionally to achieve more accurate results for all the matched features.
Among the usual matching techniques, area-based matching (ABM) and feature-based
matching (FBM) are the two main ones applied in automatic DSM generation, but additionally
relational matching is sometimes used. All basic matching techniques have advantages and
disadvantages with respect to the problems presented above. The key to successful matching is
an appropriate matching strategy, making use of all available and explicit knowledge
concerning the sensor model, network structure and image content. However, even then the
lack of an image understanding capability will lead to problems, whose impact must be judged
by the project specications.
The matching approach is a hybrid method that combines ABM and relational matching.
It uses a coarse-to-ne hierarchical strategy with a combination of several image matching
algorithms and automatic quality control. ABM (both in the form of a modied cross-
correlation and least squares matching) is employed to match feature points and grid points
(see also Hsia and Newton, 1999). Generally, the performance and success rate of ABM
mainly depends on: the existence of sufcient image texture; the quality of the approximations
and a set of matching parameters, such as the matching window size; the search distance; and
the acceptance threshold for the correlation coefcient. How to select a set of correct matching
parameters is problematic, because the requirements for these parameter values are conicting.
These matching parameters are functions of many factors, including terrain type, image
texture, image scale, disparity variations and image noise. The TLS matcher uses a set of
adaptively determined matching parameters. This is done by analysing the results of the higher
level image pyramid matching and using them at the current pyramid level.
The performance of ABM is not good if there is insufcient image texture, in the case of
repetitive patterns, and at surface discontinuities. Unfortunately, these problems are very
typical of large scale images as provided by TLS. In the rst case, because of missing points,
2012 The Author
the ABM may lead to holes in the DSM. To overcome this problem, a global image matching
technique, based on probabilistic relaxation (Hancock and Kittler, 1990), is employed to match
grid points in order to bridge the poor texture areas. This relational matching uses local
smoothness constraints. In the last case, ABM generates smoothing effects at surface
discontinuities. ABM is also employed to match the edges located on such discontinuities, but
the matched edges are used as breaklines to control the weights of the surface smoothness
constraints in the global image matching procedure. As such, they prohibit the smoothness
constraints from crossing the edges. The quasi-epipolar curves derived from the TLS sensor
model are used to restrict the search range to only one direction. The residual y parallax grid is
used to compensate some of the errors in the raw image data.
In summary, the matching approach can be characterised by the following aspects:
(1) Multiple image matching and different matching algorithms. Anewexible and robust
matching algorithm, the geometrically constrained cross-correlation (GC
3
) method,
has been developed in order to take advantage of multiple images. The algorithm is
based on the concept of multi-image matching guided fromthe object space and allows
the reconstruction of 3D objects by matching all available images simultaneously,
without having to match all individual stereopairs separately and merge the results.
Besides this special form of cross-correlation, LSM is also used as an option.
(2) Matching with multiple primitives. More robust hybrid image matching algorithms
have been developed by taking advantage of both ABM and FBM techniques and
utilising both local and global image information. In particular, an edge matching
method is combined with a grid point matching method through a probability relax-
ation-based relational matching process. The use of edges leads to the preservation of
surface discontinuities, while grid points bridge areas with little or no texture.
(3) Self-tuning matching parameters. The adaptive determination of the matching
parameters results in a higher success rate and fewer mismatches. These parameters
include the size of the correlation window, the search distance and the correlation
threshold values. This is done by analysing the matching results at the previous
image pyramid level and using them at the current level.
(4) High matching redundancy. With this matching approach, highly redundant
matching is achieved, so that points and edges can be generated. Highly redundant
matching results are suitable for representing very steep and rough terrain and allow
the terrain microstructures and surface discontinuities to be well preserved. More-
over, this high redundancy also allows for better automatic blunder detection.
(5) Efcient surface modelling. The object surface is modelled by a TIN generated by a
constrained Delauney triangulation of the matched points and edges. A TIN is suitable
for surface modelling because it integrates all the original matching results, including
points and edge features, without any interpolation. It is adapted to describe complex
terrain types that contain many surface microstructures and discontinuities.
(6) Coarse-to-ne hierarchical strategy. The algorithm works in a coarse-to-ne multi-
resolution image pyramid structure, and obtains intermediate DSMs at multiple
resolutions. Matches on low-resolution images serve as approximations to restrict the
search space and to adaptively compute the matching parameters for the subsequent
levels.
Results of Controlled Accuracy Tests
Results, based on controlled tests, have been published at several times, such as in Gruen
and Zhang (2002) for aerial images. In Eisenbeiss et al. (2005) and Lambers et al. (2007) the
2012 The Author
results of a comparison of image matching from aerial UAV images (using SAT-PP) with
terrestrial laser scan data showed no clear superiority of either technique. Image matching,
however, turned out to be of greater generality and more exible use.
Recently, the German Society of Photogrammetry and Remote Sensing (DGPF)
conducted a test on evaluation of digital photogrammetric aerial camera systems. As part
of this test the accuracy of DSMs, derived automatically, was also investigated. Some of the
results have been reported in Wolff (2009). The key problem with such tests, which were done
with aerial images with footprints of 8 and 20 cm, is the generation of sufciently good
reference data.
Tests in close range applications have been reported, for example, by Remondino et
al. (2008, 2009) and Remondino and Menna (2008), although not always supported by
accurate reference data. In some cases, terrestrial laser scanning was compared with image
matching results, with no clear indication as to which technique would deliver the better
results.
The performance of the matching software SAT-PP for DSM generation has been veried
extensively with several high-resolution satellite imagery data-sets, such as SPOT-5, IKONOS,
QuickBird, ALOS/PRISM, Cartosat-1 and WorldView-1, over different terrain types; these
include hilly and rugged mountainous areas, and rural, suburban and urban areas. A detailed
analysis of the results of IKONOS was presented in Gruen et al. (2005) and Zhang and Gruen
(2006). Other processing and evaluation results of IKONOS and SPOT-5 HRS/HRG can be
found in Zhang and Gruen (2004), Poli et al. (2004), Baltsavias et al. (2006), Poon et al. (2005)
and Crespi et al. (2008). ALOS/PRISM results are described in Gruen and Wolff (2007) whilst
WorldView-1 results can be found in Poli et al. (2009). A general summary of satellite image-
matching results is given in Wolff and Gruen (2008). In a special application, the performance
of image matching with respect to the generation of 3D tree canopy models from aerial images
was tested and compared to lidar data. Image matching results (computed with SAT-PP) turned
out to give better results (Baltsavias et al., 2008).
What has been observed in all the controlled tests is that an accuracy of 1 to 2 pixels can
be achieved in DSM generation, but there are many blunders in the results. These blunders are
usually not single spikes (which could be easily detected), but rather groups of gross errors that
act like local systematic errors. Therefore, they are not easily detected automatically. In future
research and development activities emphasis should be on the understanding of the underlying
reasons for these blunders and on the development of techniques to avoid them. The main
sources of errors are:
(1) object features (edges, height differences, steepness of slopes, repetitive objects, role
of vegetation);
(2) illumination and reectance properties (lack of image texture, shadows, specular
reections), image quality (signal-to-noise ratio, image artefacts); and
(3) network problems (insufcient design, partial occlusions).
Image matching must be seen in the context of automation. The potential advantages of
automation are:
(1) increased accuracy;
(2) reduced equipment costs;
(3) increased throughput;
(4) faster availability of results (online capability);
(5) new kinds of products; and
(6) better quality products.
2012 The Author
The last promise, especially, has not yet been fullled. The quality of DSMs produced by
image matching still cannot compare with those derived from manual measurements.
Advancements in image understanding algorithms are therefore urgently needed.
Quality Control and Editing
Internal quality control should be a key feature of any supposedly automatic procedure. In
DSM/DTM generation this quality control is either non-existent or comes with great
deciencies. In some systems the trafc light system is being used, which classies matched
points in three colours, red, yellow and green, according to their quality. However, how these
classiers are computed remains largely unknown. In addition, in real data-sets, many
misclassications are observed. Therefore, the only reliable quality control procedure currently
available is visual checking, for example, by overlaying the generated point cloud and the
stereomodel. This, however, turns the automated procedure into a semi-automated one.
In connection with these problems, the editing of data must be considered. Editing is done
in the form of pre- and post-editing. Pre-editing includes the denition of dead areas such as
water surfaces, clouds or very steep (and possibly vegetation-covered) slopes. Procedures for
post-editing are mainly concerned with blunder and systematic error removal, but also with
reduction from DSM to DTM. Although there are supposedly automated procedures offered
for this latter task, there is a latent danger that serious errors are introduced, because the
decisions are based only on purely geometric considerations. Automated blunder detection is
only possible at very coarse levels, such as through the ltering of single spikes. Usually the
results returned need heavy manual editing.
With commercial digital workstations, editing and quality control tools are available in
terms of:
(1) stereo-superimpositioning of point and/or contours;
(2) correction of points, lines and areas;
(3) geomorphological editing;
(4) denition of dead areas (where no matching is performed);
(5) analysis of wireframes and contours;
(6) stereoscopic ortho-images;
(7) colour marking of points (red, yellow, green for quality status); and
(8) representation of parallaxes or heights in terms of grey values.
Relation to Laser Scanning
Laser scanning has generated much interest in recent years and is used on many projects.
There is an ongoing discussion as to which method is more suitableimage matching or laser
scanning. Despite the relevance of this topic, it is surprising how few comprehensive
investigations are available up to now. Gruen (2007) has addressed some problems when
dealing with laser scan point clouds for 3D modelling and in Gruen (2009) the advantages of
photogrammetry were emphasised and advertised. Lately, the superiority of laser scanning has
been increasingly questioned; Leberl et al. (2010) show how multipoint matching with
Ultracam images with 90% overlap can provide high-quality point clouds. Some test examples
from satellite, aerial and terrestrial images have been listed in the previous text. There has been
a trend to consider laser scan data superior in terms of accuracy, and so used as reference data
for the checking of the image matching results; however, this assumption has been found to be
wrong in quite a few examples. Laser scan and image matching data have quite different error
2012 The Author
properties, which have never been compared comprehensively. This is still a useful task to be
performed in the future. In 3D city modelling, a particularly important application, a recent
survey by Haala and Kada (2010), considering the use of image as well as laser scan data,
indicates that full automation in high-resolution building modelling is still a widely unsolved
problem, no matter whether image or laser scan data is used.
Concluding Remarks
The development of image matching in photogrammetry has been described chrono-
logically. While the original approaches to image matching were derived from signal processing
and image transfer requirements, specic models trying to consider the photogrammetric
imaging and network aspects were introduced later. The underlying models for matching
became more complex over time. With the advent of digital techniques, there was no longer any
actual limit to the renement of models but computing time considerations became critical
issues. This is where the research stands today. Despite the fact that the matching problem has
not yet been solved in general terms, there are quite rened models available, with millions of
match points being produced per stereomodel. Least squares matching, especially, can be used
in many different modes and variations. As such, it provides for an algorithmic framework that
is highly adaptive to various types of image content, network structures, processing
requirements and accuracy expectations. It has also been demonstrated in the past that least
squares matching allows exploitation of the full accuracy potential of images and systems. It
provides for a measurement accuracy that goes way beyond the capabilities of a human operator.
The advantages of LS matching lie in its inherent high accuracy. For DSM generation, an
accuracy limit of 01 pixels can be expected under favourable conditions. Furthermore,
precision and reliability measures and statistical tests can readily be derived for internal quality
control. This comes at high computational costs. Therefore, researchers are currently looking at
techniques for speeding up the computations. This may be either done by grid computing and/
or integration of GPU (graphics processing unit) technology into the algorithmic ow (Ernst
and Hirschmuller, 2008). This, however, does not solve the basic problems in image matching,
as stated in this contribution. There are essentially two ways in which further progress can be
achieved: (a) adding more information by using a multi-sensor approach; and (b) advancing
image understanding algorithms. While the realisation of the former is currently ongoing, the
latter is the harder option to full.
In Haala (2009), the comeback of digital image matching has been proclaimed. In my
understanding, image matching was always present and available, but up to now its full
potential has not been fully realised.
In an outstanding article, Rosenberg (1955) remarks on the future of automated mapping,
of which image matching is a crucial component: The engineering problems in electronic
photogrammetry are very considerable It will be a long time before completely automatic,
electronic photogrammetry is actually at hand. Despite all the progress that has been
achieved and the availability of new technologies, more than 50 years later this statement is
still essentially correct.
references
Ackermann, F., 1984. Digital image correlation: performance and potential application in photogrammetry.
Photogrammetric Record, 11(64): 429439.
Aggarwal, J. K., Duda, R. O. and Rosenfeld, A. (Eds.), 1977. Computer methods in image analysis. IEEE
Press, New York. 466 pages.
2012 The Author
Akca, D., 2007. Least squares 3D surface matching. Ph.D. thesis, Institute of Geodesy and Photogrammetry, ETH
Zurich, Switzerland, Mitteilungen 92. Dissertation No. 17136. 78 pages.
Alberich, L., 1985. Orthophotomapping with an automated system at 1:50 000 map scale. Proceedings ASP
ACSM Convention, Washington, DC, 1: 222228.
Albertz, J. and Koenig, G., 1984. A digital stereophotogrammetric system. International Archives of Photo-
grammetry and Remote Sensing, 25(A2): 17.
Almroth, U. and Hendriks, L., 1987. High accuracy digital matching of close-range objects on the analytical
plotter Kern DSR-11. Proceedings of ISPRS Intercommission Conference on Fast Processing of Photo-
grammetric Data, Interlaken, Switzerland. 437 pages: 193203.
Andrews, H. C. (Ed.), 1978. Digital image processing. IEEE Press, New York. 737 pages.
Baker, H. H. and Binford, T. O., 1981. Depth from edge and intensity based stereo. Proceedings of the 7th
International Joint Conferences on Articial Intelligence. 631636.
Baltsavias, E., 1984. Digital image correlation in photogrammetry. M.Sc. technical report, Department of
Geodetic Science and Surveying, Ohio State University, Columbus, Ohio.
Baltsavias, E., 1991. Multiphoto geometrically constrained matching. Ph.D. thesis, Institute of Geodesy and
Photogrammetry, ETH Zurich, Switzerland, Mitteilungen 49. 221 pages.
Baltsavias, E. P. and Stallmann, D., 1990. Trinocular vision for automatic and robust 3-D determination of the
trajectories of moving objects. International Archives of Photogrammetry and Remote Sensing, 28(5/1): 620
629.
Baltsavias, E., Li, H., Gaschen, S. and Sinning, M., 1996. DTM generation with the Leica/Helava DPW 770
and VirtuoZo digital photogrammetric systems. Paper presented at GeoInformatics 96, Wuhan, China, 16th to
19th October. 1017.
Baltsavias, E., Zhang, L. and Eisenbeiss, H., 2006. DSM generation and interior orientation determination of
IKONOS images using a testeld in Switzerland. Photogrammetrie, Fernerkundung, Geoinformation, (1): 41
54.
Baltsavias, E., Gruen, A., Eisenbeiss, H., Zhang, L. and Waser, L. T., 2008. High quality image matching
and automated generation of 3D tree models. International Journal of Remote Sensing, 29(5): 12431259.
Bernstein, R. (Ed.), 1978. Digital image processing for Remote Sensing. IEEE Press/Wiley, New York. 473
pages.
Bethel, J., 1986. The DSR11 image correlator. American Congress on Surveying and Mapping & American
Society of Photogrammetry and Remote Sensing. Annual Convention, IV: 4449.
Beyer, H. A., 1987. Some aspects of the geometric calibration of CCD-cameras. Proceedings of ISPRS Inter-
commission Conference on Fast Processing of Photogrammetric Data, Interlaken, Switzerland. 437 pages:
6881.
Beyer, H. A., 1992. Geometric and radiometric analysis of a CCD-camera based photogrammetric close-range
system. Ph.D. thesis, Institute of Geodesy and Photogrammetry, ETH Zurich, Switzerland. Dissertation No.
9701. 186 pages.
Case, J. B., 1982. The digital stereo comparator/compiler (DSCC). International Archives of Photogrammetry,
24(2): 2329.
Chellappa, R. and Sawchuk, A. A. (Ed.), 1985. Digital image processing and analysis, Volume 2. IEEE
Computer Society Press, New York. 721 pages.
Cogan, L., Gugan, D., Hunter, D., Lutz, S. and Peny, C., 1988. Kern DSP1 digital stereo photogrammetric
system. International Archives of Photogrammetry and Remote Sensing, 27(B2): 7183.
Crespi, M., De Vendictis, L., Poli, D., Wolff, K., Colosimo, G., Gruen, A. and Volpe, F., 2008.
Radiometric quality and DSM generation analysis of Cartosat-1 stereo imagery. International Archives of
Photogrammetry, Remote Sensing and Spatial Information Sciences, 37(B13): 13491355.
DeMeter, E. R., 1963. Latest advances in automated mapping. Photogrammetric Engineering, 29(6): 10271036.
Dowman, I., 1984. Instrumentation for topographic mapping from satellite data. International Archives of
Photogrammetry, 25(2): 155163.
Dowman, I. J. and Haggag, A. A. Y., 1977. Digital image correlation along epipolar lines. International
Symposium on Image Processing, Interactions with Photogrammetry and Remote Sensing, Graz, Austria.
4749.
Dowman, I. J., Gugan, D. J., Muller, J.-P., ONeill, M. and Paramananda, V., 1987. Digital processing of
SPOT data. Proceedings of ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data,
Interlaken, Switzerland. 437 pages: 318330.
Ebner, H. and Heipke, C., 1988. Integration of digital image matching and object surface reconstruction. Inter-
national Archives of Photogrammetry and Remote Sensing, 27(B11): 534545.
Eisenbeiss, H., Sauerbier, M., Zhang, L. and Gru n, A., 2005. Mit dem Modellhelikopter uber Pinchango Alto.
Geomatik Schweiz, 2005(9): 510515 (in German).
2012 The Author
Ernst, I. and Hirschmu ller, H., 2008. Mutual information based semi-global stereo matching on the GPU.
Advances in Visual Computing. Proceedings 4th International Symposium on Visual Computing (ISVC08).
Springer Lecture Notes in Computer Science, 5358: 228239.
Fo rstner, W., 1982. On the geometric precision of digital correlation. International Archives of Photogrammetry,
24(3): 176189.
Gong, J., Li, Z., Zhu, Q., Sui, H. and Zhou, Y., 2000. Effects of various factors on the accuracy of DEMs: an
intensive experimental investigation. Photogrammetric Engineering & Remote Sensing, 66(9): 11131117.
Gruen, A., 1985a. Adaptive kleinste Quadrate Korrelation und geometrische Zusatzinformationen. Vermessung,
Photogrammetrie, Kulturtechnik, 83(9): 309312 (in German).
Gruen, A., 1985b. Adaptive least squares correlation: a powerful image matching technique. South African
Journal of Photogrammetry, Remote Sensing and Cartography, 14(3): 175187.
Gruen, A., 1988. Towards real-time photogrammetry. Photogrammetria, 42(56): 209244.
Gruen, A. W., 1989. Digital photogrammetric processing systems: current status and prospects. Photogrammetric
Engineering & Remote Sensing, 55(5): 581586.
Gruen, A., 1996a. Digital photogrammetric stations revisited. International Archives of Photogrammetry and
Remote Sensing, 31(B2): 127134.
Gruen, A., 1996b. Least squares matching: a fundamental measurement algorithm. Chapter 8 in Close Range
Photogrammetry and Machine Vision (Ed. K. B. Atkinson). Whittles, Caithness, Scotland. 371 pages: 217255.
Gruen, A., 1999. Digital photogrammetric stations. Promises, performance, perspectives. GeoInformatics, 2(3):
2227.
Gruen, A., 2007. Laser scanning: blackbox or braintwister? GeoInformatics, 10(7): 4951.
Gruen, A., 2009. Aerial photogrammetryrevival of a discipline. GIS Development, 13(1): 3031.
Gruen, A. and Baltsavias, E. P., 1984. Adaptive least squares correlation. Project Report for Helava Assoc.,
Inc., USA.
Gruen, A. and Baltsavias, E. P., 1985. Adaptive least squares correlation with geometrical constraints. SPIE,
595: 7282.
Gruen, A. and Baltsavias, E. P., 1986. High precision image matching for digital terrain model generation.
International Archives of Photogrammetry and Remote Sensing, 26(3/1): 284296.
Gruen, A. and Beyer, H. A., 1986. Real-time photogrammetry at the digital photogrammetric station (DIPS) of
ETH Zurich. Canadian Surveyor, 41(2): 181199.
Gruen, A. and Baltsavias, E. P., 1988a. Automatic 3-D measurement of human faces with CCD-cameras. SPIE,
1030: 106115.
Gruen, A. and Baltsavias, E. P., 1988b. Geometrically constrained multiphoto matching. Photogrammetric
Gruen, A. and Beyer, H. A., 1990. DIPS IIturning a standard computer workstation into a digital photo-
grammetric station. International Archives of Photogrammetry and Remote Sensing, 28(2): 247255.
Gruen, A. and Stallmann, D., 1991. High accuracy edge matching with an extension of the MPGC-matching
algorithm. SPIE, 1526: 4255.
Gruen, A. and Agouris, P., 1994. Linear feature extraction by least squares template matching constrained by
internal shape forces. International Archives of Photogrammetry and Remote Sensing, 30(3/1): 316323.
Gruen, A. and Li, H., 1997. Semi-automatic linear feature extraction by dynamic programming and LSB-snakes.
Photogrammetric Engineering & Remote Sensing, 63(8): 985995.
Gruen, A., Ba r, S. and Buehrer, Th., 2000. DTMs derived automatically from DIPSwhere do we stand?
GeoInformatics, 3(5): 3639.
Gruen, A. and Zhang, L., 2002. Automatic DTM generation from three-line-scanner (TLS) images. International
Archives of Photogrammetry and Remote Sensing, 34(2A): 131137.
Gruen, A. and Akca, D., 2005. Least squares 3D surface and curve matching. ISPRS Journal of Photogrammetry
and Remote Sensing, 59(3): 151174.
Gruen, A., Zhang, L. and Eisenbeiss, H., 2005. 3D precision processing of high-resolution satellite imagery.
Proceedings of the ASPRS 2005 Annual Conference, Baltimore, Maryland (on CD-ROM).
Gruen, A. and Wolff, K., 2007. DSM generation with ALOS/PRISM data using SAT-PP. Proceedings of the
IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Barcelona, Spain. 4 pages (on CD-
ROM).
Gru n, A., 1984. Adaptive least squares correlationconcept and rst results. Intermediate Research Project
Report to Helava Assoc., Inc., Ohio State University, Columbus, Ohio. 13 pages.
Guelch, E., 1988. Results of test on image matching of ISPRS WGIII/4. International Archives of Photogram-
metry and Remote Sensing, 27(B3): 254271.
Haala, N., 2009. Comeback of digital image matching. In Photogrammetric Week 2009 (Ed. D. Fritsch).
Wichmann, Heidelberg, Germany. 289301.
2012 The Author
Haala, N. and Kada, M., 2010. An update on automatic 3D building reconstruction. ISPRS Journal of Photo-
grammetry and Remote Sensing, 65(6): 570580.
Haggren, H., 1986. Real-time photogrammetry as used for machine vision applications. International Archives of
Photogrammetry and Remote Sensing, 26(5): 374382.
Hancock, E. R. and Kittler, J., 1990. Discrete relaxation. Pattern Recognition, 23: 711733.
Helava, U. V., 1972. Digital processing and analysis of image data. International Archives of Photogrammetry,
20(1). 354 pages.
Helava, U. V., 1976. Digital correlation in photogrammetric instruments. International Archives of Photogram-
metry, 21(2). 26 pages.
Helava, U. V., 1978. Digital correlation in photogrammetric instruments. Photogrammetria, 34(1): 1934.
Helava, U. V., 1988a. Digital comparator correlator system. International Archives of Photogrammetry and
Helava, U. V., 1988b. Object-space least squares correlation. Photogrammetric Engineering & Remote Sensing,
54(6): 711714.
Hirschmueller, H., 2008. Stereo processing by semiglobal matching and mutual information. IEEE Transactions
on Pattern Analysis and Machine Intelligence, 30(2): 328342.
Hobrough, G. L., 1959. Automatic stereo plotting. Photogrammetric Engineering, 25(5): 763769.
Hobrough, G. L., 1965. Automation in photogrammetric instruments. Photogrammetric Engineering, 31(4): 595
603.
Hsia, J.-S. and Newton, I., 1999. A method for the automated production of digital terrain models using a
combination of feature points, grid points, and lling back points. Photogrammetric Engineering & Remote
Sensing, 65(6): 713719.
Kelly, R. E., McConnell, P. R. H. and Mildenberger, S. J., 1977. The Gestalt photomapping system.
Photogrammetric Engineering & Remote Sensing, 43(11): 14071417.
Kempa, M. and Schlueter, M., 1993. DEM evaluation by an operator and facets stereo vision: a comparison
based on close range imagery. In Optical 3-D Measurement Techniques II (Eds. A. Gruen & H. Kahmen).
Wichmann, Karlsruhe, Germany. 624 pages: 502509.
Konecny, G. and Pape, D., 1981. Correlation techniques and devices. Photogrammetric Engineering & Remote
Sensing, 47(3): 323333.
Krzystek, P., 1995. Generation of digital elevation models. Chapter 7 in Second Course in Digital Photogram-
metry. Landesvermessungsamt Nordrhein-Westfalen and Institut fur Photogrammetrie, Universitat Bonn.
Unpaginated.
Lambers, K., Eisenbeiss, H., Sauerbier, M., Kupferschmidt, D., Gaisecker, T., Sotoodeh, S. and
Hanusch, T., 2007. Combining photogrammetry and laser scanning for the recording and modelling of the
late intermediate period site of Pinchango Alto, Palpa, Peru. Journal of Archaeological Science, 34(10): 1702
1712.
Leberl, F., Irschara, A., Pock, T., Meixner, P., Gruber, M., Scholz, S. and Wiechert, A., 2010. Point
clouds: lidar versus 3D vision. Photogrammetric Engineering & Remote Sensing, 76(10): 11231134.
Lemmens, J. P. M., 1988. A survey on stereo matching techniques. International Archives of Photogrammetry and
Li, M. X., 1989. Hierarchical Multi-point Matching with simultaneous detection and location of breaklines. Ph.D.
thesis, Department of Photogrammetry, Royal Institute of Technology, Stockholm, Sweden. 177 pages.
Lue, Y., Zhang, Z. and Zhang, J., 1987. Calibration of a drum scanner with least squares correlation. Pro-
ceedings of ISPRS Intercommission Conference on Fast Processing of Photogrammetric Data, Interlaken,
Switzerland. 437 pages: 109119.
Maas, H.-G., 1996. Automatic DEM generation by multi-image feature based matching. International Archives of
Photogrammetry and Remote Sensing, 31(3): 484489.
Maas, H.-G., Stefanidis, A. and Gruen, A., 1994. From pixels to voxels: tracking volume elements in
sequences of 3-D digital images. International Archives of Photogrammetry and Remote Sensing, 30(3/2):
539546.
Makarovic, B., 1980. Image correlation algorithms. International Archives of Photogrammetry, 23(B2): 139158.
Masry, S. E., 1974. Digital correlation principles. Photogrammetric Engineering, 40(3): 303308.
Mikhail, E. M., Helava, U. A., Jancaitis, J. R., Doyle, F. J., Helmering, R. J., Ackerman, F. and
Turner, A. K., 1978. Panel discussion: the future of DTM. Photogrammetric Engineering & Remote
Sensing, 44(12): 14871497.
Miller, S. and De Venecia, K., 1992. Automatic elevation extraction and the digital photogrammetric work-
station. ASPRSACSM Annual Convention, Albuquerque, New Mexico, 1: 572580.
Otto, G. P. and Chau, T. K. W., 1988. A region-growing algorithm for matching of terrain images. Proceedings
4th Alvey Vision Conference, University of Manchester, UK. 123128.
2012 The Author
Panton, D. J., 1978. A exible approach to digital stereo mapping. Photogrammetric Engineering & Remote
Sensing, 44(12): 14991512.
PE&RS, 1983. Five short articles on automated correlation of stereo data. Photogrammetric Engineering & Remote
Sensing, 49(4): 527544.
Pertl, A., 1984. Digital image correlation with the analytical plotter Planicomp C100. International Archives of
Photogrammetry, 25(3b): 874882.
Poli, D., Zhang, L. and Gruen, A., 2004. SPOT-5/HRS stereo images orientation and automated DSM
generation. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences,
35(B1): 421432.
Poli, D., Wolff, K. and Gruen, A., 2009. Evaluation of Worldview-1 stereo scenes. International Archives of
Photogrammetry, Remote Sensing and Spatial Information Sciences, 38(1-4-7/W5). 6 pages (on CD-ROM).
Poon, J., Fraser, C. S., Zhang, C., Zhang, L. and Gruen, A., 2005. Quality assessment of digital surface
models generated from IKONOS imagery. Photogrammetric Record, 20(110): 162171.
Rauhala, U. H., 1986. Compiler positioning of array algebra technology. International Archives of Photo-
grammetry and Remote Sensing, 26(3/3): 173198.
Remondino, F., el-Hakim, S. F., Gruen, A. and Zhang, L., 2008. Turning images into 3-D models. IEEE
Signal Processing Magazine, 25(4): 5565.
Remondino, F. and Menna, F., 2008. Image-based surface measurement for close-range heritage documentation.
International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 37(B5): 199206.
Remondino, F., Gruen, A., von Schwerin, J., Eisenbeiss, H., Rizzi, A., Girardi, S., Sauerbier, M. and
Richards-Rissetto, H., 2009. Multi-sensor 3D documentation of the Maya site of Copan. Proceedings of the
XXIIth CIPA International Symposium, Kyoto, Japan. 6 pages.
Rosenberg, P., 1955. Information theory and electronic photogrammetry. Photogrammetric Engineering, 21(4):
543555.
Rosenfeld, A., 1984. Image analysis: problems, progress and prospects. Pattern Recognition, 17(1): 312.
Rosenholm, D., 1986. Accuracy improvement of digital matching for evaluation of digital terrain models.
International Archives of Photogrammetry and Remote Sensing, 26(3/2): 573587.
Scarano, F. A. and Brumm, G. A., 1976. A digital elevation data collection system. Photogrammetric
Schewe, H. and Fo rstner, W., 1986. The program PALM for automatic line and surface measurement using
image matching techniques. International Archives of Photogrammetry and Remote Sensing, 26(3/2): 608
622.
Smith, M. J. and Smith, D. G., 1996. Operational experiences of digital photogrammetric systems. International
Archives of Photogrammetry and Remote Sensing, 31(B2): 357362.
Snavely, N., Seitz, S. M. and Szeliski, R., 2006. Photo tourism: exploring photo collections in 3D. ACM
Transactions on Graphics, 25(3): 835846.
Trinder, J. C., 1986. Precision of stereoscopic height measurements. Photogrammetric Engineering & Remote
Sensing, 52(1): 7579.
Vu, H. H., Keriven, R., Labatut, P. and Pons, J.-P., 2009. Towards high-resolution large-scale multi-view
stereo. Paper presented at CVPR09. http://imagine.enpc.fr/publications/publications1.html#CVPR09a [Ac-
cessed: 8th January 2011].
Williams, R. E., 1959. The automatic map compilation system. Photogrammetric Engineering, 25(1): 103110.
Wolff, K., 2009. DGPF project: evaluation of digital photogrammetric aerial based imaging systemsgeneration
of digital surface models. International Archives of Photogrammetry, Remote Sensing and Spatial Information
Sciences, 38(1-4-7/W5). 5 pages (on CD-ROM).
Wolff, K. and Gruen, A., 2008. Up-to-date DSM generation using high resolution satellite image data. Inter-
national Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 37(B13): 1103
1108.
Wrobel, B., 1987. Facets stereo vision (FAST vision)a new approach to computer stereo vision and to digital
photogrammetry. Proceedings of ISPRS Intercommission Conference on Fast Processing of Photogrammetric
Data, Interlaken, Switzerland. 437 pages: 231258.
Zhang, B. and Miller, S., 1997. Adaptive automatic terrain extraction. SPIE, 3072: 2736.
Zhang, J., Zhang, Z., Shen, W. and Wang, Z., 1996. VirtuoZo digital photogrammetric system and its theo-
retical foundation and key algorithms. International Archives of Photogrammetry and Remote Sensing, 31(B2):
424429.
Zhang, L., 2005. Automatic digital surface model (DSM) generation from linear array images. Institute of
Geodesy and Photogrammetry, ETH Zurich, Switzerland, Mitteilungen 88. Dissertation No. 16078. 199 pages.
Zhang, L. and Gruen, A., 2004. Automatic DSM generation from linear array imagery data. International
Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 35(B3): 128133.
2012 The Author
Zhang, L. and Gruen, A., 2006. Multi-image matching for DSM generation from IKONOS imagery. ISPRS
Journal of Photogrammetry and Remote Sensing, 60(3): 195211.
Zhang, Z., Zhang, J., Wu, X. and Zhang, H., 1992. Global image matching with relaxation method.
Proceedings of the International Colloquium on Photogrammetry, Remote Sensing and Geographic Infor-
mation Systems, Wuhan, China, 11th to 14th May. 175188.
Zheltov, S. Y. and Sibiryakov, A. V., 1997. Adaptive subpixel cross-correlation in a point correspondence. In
Optical 3D Measurement Techniques (Eds. A. Gruen & O. Ku hbler). Wichmann, Zurich, Switzerland. 86
95.
Resume
Lappariement dimages et de formes est sans doute loperation la plus
importante en photogrammetrie numerique et en modelisation et cartographie
automatiques. De nombreuses approches se sont succede depuis des annees mais le
proble`me nest toujours pas totalement resolu. Cet article decrit le developpement
des techniques dappariement dimages en photogrammetrie pendant les 50
dernie`res annees, presente les resultats obtenus dans quelques etudes de precision
empiriques, et dresse un bilan critique des proble`mes qui subsistent. Bien que les
approches automatiques aient un grand nombre davantages, la qualite des resultats
nest toujours pas satisfaisante, et meme loin detre acceptable dans certains cas.
Meme avec les techniques les plus avancees, nous sommes toujours dans lincapacite
datteindre la qualite des resultats obtenus par un operateur humain. Il y a un besoin
urgent dameliorations et dinnovations, soit a` travers des approches multi-capteurs
plus puissantes et consistant a` elargir le spectre dinformation, soit a` travers une
amelioration des algorithmes de comprehension dimages visant a` les rendre plus
proches des possibilites humaines de lecture et de comprehension du contenu des
images.
Zusammenfassung
Bild- und Musterkorrelation gehoren zu den wichtigsten Grundfunktionen der
Digitalen Photogrammetrie und somit auch der automatischen 3D Modellierung und
Kartierung. Viele Ansatze zur Korrelation wurden uber die Jahre entwickelt, aber
das Problem gilt grundsatzlich noch immer als ungelost. Dieser Beitrag beschreibt
die Entwicklung der Verfahren der Bildkorrelation in der Photogrammetrie uber die
letzten 50 Jahre, verweist auf die Ergebnisse einiger empirischer Genauigkeitsstudien
und diskutiert einige der immer noch bestehenden Probleme.Obwohl automatische
Verfahren eine ganze Reihe von Vorteilen aufweisen, ist doch die Qualitat der
Ergebnisse meist nicht ausreichend, teilweise ja sogar weit entfernt von jeglicher
Akzeptanz. Selbst mit den hochstentwickelten Verfahren sind wir immer noch nicht in
der Lage, die Qualitat der Ergebnisse eines menschlichen Operateurs zu erreichen.
Wir benotigen dringend weitere Verbesserungen und Innovationen. Dazu gibt es
gegenwartig zwei grundsatzlich gangbare Wege: (a) Nutzung von Multi-Sensor
Informationen und somit Erweiterung der Informationsgrundlagen und/oder (b) durch
Fortschritte bei den Algorithmen des Bildverstehens und somit besserer Modellierung
des menschlichen Prozesses des Bildverstehens.
2012 The Author
Resumen
La correspondencia de imagenes y muestras es, probablemente, la funcion mas
importante en la fotogrametr a digital, en el modelado 3D y en la cartograf a
automatica. Muchos metodos de correspondencia han evolucionado a lo largo de los
anos pero, en terminos generales, el problema se considera aun no resuelto
completamente. Este art culo describe la evolucion de las tecnicas de corre-
spondencia de imagenes en la fotogrametr a a lo largo de los ultimos 50 anos,
analiza los resultados de algunos estudios emp ricos de la exactitud, y ofrece una
valoracion cr tica de los problemas aun sin resolver. Aunque los metodos
automaticos poseen un gran numero de ventajas, la calidad de los resultados no
es todav a satisfactoria y, en algunos casos, incluso esta lejos de ser aceptable.
Incluso con las mas avanzadas tecnicas no somos capaces de lograr la calidad de los
resultados que un operador humano puede conseguir. Hay una necesidad urgente de
continuar las mejoras e innovaciones, ya sea mediante la utilizacion de multiples
sensores que incrementen el espectro de la informacion, o por avances en los
algoritmos de comprension de la imagen que permitan acercarnos mas a la
capacidad humana de lectura e interpretacion de su contenido.
2012 The Author

Imagenes Coincidentes PDF

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Imagenes Coincidentes PDF

Diunggah oleh

Hak Cipta:

Format Tersedia

DEVELOPMENT AND STATUS OF IMAGE MATCHING

Anda mungkin juga menyukai