Jean Ponce
Ecole Normale Superieure, Paris, France
Abstract: This paper addresses the problem of characterizing a general class of cameras under reasonable, linear assumptions. Concretely, we use the formalism and
terminology of classical projective geometry to model cameras by two-parameter linear families of straight linesthat
is, degenerate reguli (rank-3 families) and non-degenerate
linear congruences (rank-4 families). This model captures
both the general linear cameras of Yu and McMillan [16]
and the linear oblique cameras of Pajdla [8]. From a geometric perspective, it affords a simple classification of all
possible camera configurations. From an analytical viewpoint, it also provides a simple and unified methodology for
deriving general formulas for projection and inverse projection, triangulation, and binocular and trinocular geometry.
Figure 1. What a camera is. See text for details.
1. Introduction
eras of Yu and McMillan [16] and the linear oblique cameras of Pajdla [8], but never, to the best of our knowledge, in
a fully unified manner. From an analytical perspective, our
model also provides a simple and unified methodology for
deriving general formulas for direct and inverse projection,
triangulation, and binocular and trinocular geometry.
1.1. Contributions
The main contributions of this paper are theoretical.
They can be summarized as follows:
A unified framework for modeling many types of central and non-central imaging devices. A camera as defined in this presentation is (roughly) a device for associating straight lines with points (Figure 1, bottom). In essence,
a camera has at its disposal a linear bag of lines from
which it picks, for any point x, the corresponding line .
As will be shown later in this paper, this process can be decomposed into first finding a point z = Ax on the line ,
which in turn defines this line. This is the essential part of
imaging. As before, there is a secondary part that associates
with x its image y in some retinal plane r. Conversely, an
inverse projection process associates with each image point
y its preimage .
A geometric classification of the possible camera configurations. As shown in Section 2, classical (line) projective geometry [10, 15] provides a complete characteriza-
tion of all physical cameras captured by our model, including linear oblique (or bilinear) cameras [8, 16] and stereo
panoramas or cyclographs [12], two-slit [4, 17] and linear
pushbroom cameras [5], and pencil cameras [16].
A unified analytical framework for describing the
mapping between points and the corresponding lines.
We show in the appendix that this mapping can always be
represented in terms of a 44 matrix A. In turn, this affords
simple and general formulas for direct and inverse projection as well as triangulation (Section 3).
A general approach to multi-view geometry. We show
in Sections 2 and 3 that the fundamental matrix and trifocal
tensor characterizing admissible binocular or trinocular correspondences for pinhole cameras are readily generalized to
arbitrary pairs or triples of cameras in our model.
As stated earlier, these contributions are (so far) mainly
of theoretical interest. Experiments will follow, and we
hope that the formulas afforded by our framework will
prove of interest for practitioners interested in stereopsis
with rigs that include some non-traditional cameras. In the
mean time, we also hope they will be of interest to people
who want to know what is under the hood, so to speak, of
the cameras that are the bread and butter of our profession.
Figure 3. A two-slit camera and a linear pushbroom camera. Projectively, the two models are equivalent, the parallel planes of
the pushbroom camera meeting along a line at infinity.
Consider two arbitrary cameras in our model. By construction, a line 1 associated with the first camera is already linearly dependent on three or four lines depending
on the rank k of the corresponding family. For the point
where 1 pierces the image plane to match a point in the
second image, 1 must intersect the corresponding line 2 ,
and thus satisfy one additional linear constraint. In particular, the lines from the first camera intersecting 2 must form
a rank k 1 familythat is, depending on k, a regulus or a
flat pencil of lines. The lines from the second family intersecting 1 also form a flat pencil or a regulus, but the two
line sets may not sweep a common surface, and the epipolar
locus will consist of lines or conics. We will give analytical
formulas characterizing these in the next section.
3.1. Plucker
coordinates
The join operator associates with two points x and
y the line x y joining them. This geometric operator has
an analytical counterpart, and we will use the same notation
for both: we define the Pl u cker coordinate vector of the line
joining x and y as x y = (u; v), where
x2 y3 x3 y2
x4 y1 x1 y4
u = x4 y2 x2 y4 , v = x3 y1 x1 y3 ,
x4 y3 x3 y4
x1 y2 x2 y1
and, in Matlab fashion, we use the ; symbol to indicate
that the coordinates of u and v have been stacked onto each
other to form a vector of R 6 . We will from now on identify
lines with their Plucker coordinate vectors in some fixed but
otherwise arbitrary coordinate system. Plucker coordinates
are homogeneous, and lines form a quadratic hypersurface
L4 of dimension 4the Klein quadricin the projective
space P5 : Indeed, it follows immediately from the definition
of the join that the Plucker coordinate vector = (u; v)
of a line satisfies the quadratic constraint u v = 0. It
is in fact possible to define an inner product in L 4 by the
def
formula (|) = u t + v s, where = (u; v) and
def
x = [ ]x
where [ ] = pq T qpT ,
p = [ ]p
def
where [ ] = xy T yxT .
T
u = Yz+ y,
(y 2 y 3 z)
def
where Yz+ = (y 3 y 1 z)T
T
(1)
(y 1 y 2 z)
i=1
u2i ii +
,
= P T u
where
i<j3
ui uj ( ij + ji ),
P T = [1 , . . . , k ],
(2)
3.6. Triangulation
Let us consider n 2 image points y i (i = 1, . . . , n)
with coordinates u i . The triangulation problem is to find
the point x that projects onto these cameras. We write that
i must pass through x, or
the corresponding rays i = PiT u
[ i ]x = 0 for i = 1, . . . , n. This is a 4n 4 system of
homogeneous linear equations in x, of which 2n are independent since each matrix [ i ] has only rank 2, and the corresponding homogeneous linear least-squares problem can
be solved as an eigenvalue problem.
where F = P1 P2T .
for i = 1, . . . , 4. These polynomial constraints are the generalization of the trifocal constraints associated with pinhole cameras. The maximum size of the corresponding tensor is 5 5 5, with at most triquadratic constraints.
4. Discussion
We have presented a unified framework for representing
many of the cameras discussed in the literature as linear
families of lines of rank 3 or 4. In turn, this has allowed us
to derive simple and general formulas for monocular and
multi-view geometry. Besides practical applicationse.g.,
an implementation of the multi-view tensors presented in
this papermany open theoretical problems remain. For
example, is it possible to find a simple parameterization of
linear parabolic congruences in terms of four fixed screws,
as was done for elliptic and hyperbolic ones? This would
bring down the size of all fundamental matrices to 4 4.
Our approach has been purely projective. What about
Euclidean cameras, their internal parameters, and self
calibration? Conversely, what about totally uncalibrated
cameras, for which the (inverse) projection matrices are
unknown? In this case, the parameters of the quadratic
forms associated with inverse projection and multi-view
geometry are also unknown, leading back to 6 6 fundamental matrices. Does knowing these determine the
cameras projective parameters? Another interesting issue
is to go back to photometry and study how our cameras
sample the light field, as proposed in [7]. Much remains to
be done.
Acknowledgments: This research was supported in part by
the Agence Nationale de la Recherche under grants Hfibmr
and Triangles. The author thanks H. Havlicek for providing
several of the figures, as well as X. Goaoc and the Willow
team for useful discussions and comments.
5. Appendix
We derive in this appendix the equation relating, for each
non-degenerate line congruence, any point x to the corresponding line. In each case, we show that it can be written
xT (p1 pT2 )x
xT (p q T )x
= T 1 2T .
x
x (q 1 p2 )x
xT (q 1 q T2 )x
X = [p1 p2 , p2 q 1 + q 2 p1 , p1 q 1 + p2 q 2 , q 1 q 2 ]
(3)
=
=
(q x)(
q x)p p
p x)p q
+ (p x)(
.
(p x)(
q x)q p
p x)q q
and
(q 1 x)2 + (q 2 x)2
(q x)(p1 x) + (q 2 x)(p2 x)
= 1
x
.
(q 2 x)(p1 x) (q 1 x)(p2 x)
2
2
(p1 x) + (p2 x)
The second and third column of X are not lines but only
screws since the lines p2 q 1 and q 2 p1 are mutually
skew, and so are p1 q 1 and p2 q 2 . It follows that the
preimage of an image point y, with coordinates u in some
basis (y 1 , y 2 , y 3 ) of the image plane, can be written in the
, with P = X T .
usual form = P T u
References
[1] R. Bolles, H. Baker, and D. Marimont. Epipolar-plane image
analysis: An approach to determining structure from motion.
IJCV, 1:755, 1987.
[2] A. Dandurand. The rigidity of compound spatial grid. Structural Topology, 10, 1984.
[3] O. Faugeras, Q.-T. Luong, and T. Papadopoulo. The Geometry of Multiple Images. MIT Press, 2001.
[4] D. Feldman, T. Pajdla, and D. Weinshall. On the epipolar geometry of the crossed-slits projection. In Proc. ICCV, 2003.
[5] R. Gupta and R. Hartley. Linear pushbroom cameras. IEEE
Trans. PAMI, 19(9):963975, 1997.
[6] D. Hilbert and S. Cohn-Vossen. Geometry and the Imagination. Chelsea, 1952.
[7] A. Levin, W. Freeman, and F. Durand. Understanding camera trade-offs through a Bayesian analysis of light field projections. In Proc. ECCV, 2008.
[8] T. Pajdla. Stereo with oblique cameras. IJCV, 47(123):161
170, 2002.
[9] J. Ponce, K. McHenry, T. Papadopoulo, M. Teillaud, and
B. Triggs. The absolute quadratic complex and its application to camera self calibration. In Proc. CVPR, 2005.
[10] H. Pottmann and J. Wallner. Computational Line Geometry.
Springer-Verlag, 2001.
[11] P. Rademacher and G. Bishop. Multiple-center-of-projection
images. In Proc. SIGGRAPH, 1998.
[12] S. Seitz and J. Kim. The space of all stereo images. IJCV,
48(1):2128, 2002.
[13] P. Sturm. Multi-view geometry for general camera models.
In Proc. CVPR, 2005.
[14] P. Sturm and J.P. Barreto. General imaging geometry for
central catadioptric cameras. In Proc. ECCV, 2008.
[15] O. Veblen and J. Young. Projective Geometry. The
Athaenum Press, 1910.
[16] J. Yu and L. McMillan. General linear cameras. In Proc.
ECCV, 2004.
[17] A. Zomet, D. Feldman, S. Peleg, and D. Weinshall. Mosaicing new views: The crossed-slits projection. IEEE Trans.
PAMI, 25(6):741754, 2003.