SCHOOL of ELECTRICAL ENGINEERING & COMPUTER SCIENCE FACULTY of ENGINEERING & BUILT ENVIRONMENT The UNIVERSITY of NEWCASTLE

Comp3320/6370 Computer Graphics

Course coordinator: Dr Joe Ryan

Semester 2, 2012

LECTURE 1
Fundaments of the Computer Graphics Pipeline
Version 1.2

July 25, 2012

2

OVERVIEW Geometrical Background Some Typical Maths Topics In Computer Graphics 6 8

Some Basic Algebraic Concepts 9 Notes about Vector Spaces . . . . . . . . . . . . . . . . 12 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Example subspace . . . . . . . . . . . . . . . . . . . . . . 16 Basis of a vector space . . . . . . . . . . . . . . . . . . . 18 Linear Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Equations for Lines in 2D-Space [with (x,y)-coordinates] 20 Geometrical Denitions
3

22

The Computer Graphics Rendering Pipeline 23 Virtual Camera and View Frustum . . . . . . . . . . . 24 The Three Conceptual Stages . . . . . . . . . . . . . . 25 The Application Stage The Geometry Stage Model and View Transform . . Lighting and Shading . . . . . . Lighting . . . . . . . . . . . . . . . Projection . . . . . . . . . . . . . . Orthographic and Perspective Clipping . . . . . . . . . . . . . . . . Clipping . . . . . . . . . . . . . . . . Screen Mapping . . . . . . . . . . Screen Mapping Process . . . .
4

27 28
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 31 32 33 34 35 36 37 38

Projections

Summary of the Geometry Stage

39

The Rasteriser Stage 40 Buers, Visibility . . . . . . . . . . . . . . . . . . . . . . . 41 Texturing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Summary of the Pipeline Exercise 1 Exercise 2 Exercise 3 43 44 45 46

Geometrical Background In Computer Graphics we typically work with objects that live in a three-dimensional world. The aim is to draw them in two dimensions on the screen. Special cases are 2-dimensional objects and worlds. Using dimensionality reduction we can also attempt to understand 4D or higher dimensional objects and worlds. Some elementary maths can help us to make some of the calculations required in computer graphics simpler, clearer and much faster. Linear algebra covers properties of vectors and matrices required for displaying and modifying 3D objects in 3D space. Basic dierential geometry helps to deal with non-linear curves and surfaces.
6

Notation R, R+ , Rn N Z Q C H {...; ...} (a , b ) [a , b ] (a , b ], [a , b ) V, W, U v, w v w = v, w 0, 0V real numbers, reals greater than 0, n-tuples of reals natural numbers: {0, 1, 2, ...} integer numbers: {..., 1, 0, 1, 2, ...} rational numbers, i.e fractions complex numbers Hamiltonian quaternions set of ... such that ... open interval of reals between a and b closed interval of reals between a and b half-open intervals of reals between a and b vector spaces vectors dot product of two vectors zero vector, zero vector of V
7

Some Typical Maths Topics In Computer Graphics Some topics which can be relevant for Computer Graphics Vector spaces, vectors, points Points, lines, planes, 3-space, ... Systems of linear equations Dot product, cross product Curves, surfaces, manifolds Matrices, transformations Triangulations, splines Complex numbers, quaternions More ..

Some Basic Algebraic Concepts Please try to look up the denitions of the mathematical structures below in an Algebra book in the library or on-line (e.g. [?] or [?]). sets monoids and semi-groups groups rings elds division rings, division algebras modules vector spaces

DEF. An abelian group is a set, V , together with an operation that combines any two elements a and b to form another element denoted a b. The following axioms have to be satised: Closure For all u, v V we have u v V . Associativity For all u, v, w V we have (u v ) w = u (v w). Identity element There exists a identity element Id V such that v Id = Id v = v for all v V Inverse element For each v A there exists an inverse w V such that w v = v w = Id. Commutativity For all u, v A we have v w = w v Note: The operation could be replaced by + or anything similar that satises the axioms above.
10

DEF. By a vector space over the real numbers R we shall mean a set V along with two operations + and such that if v, w V then v + w V and if r R and v V then each scalar multiple r v of v is in V and the following ve properties are satised: VS 1. For all u, v, w V we have associativity, namely (u + v ) + w = u + (v + w). VS 2. There exists a zero vector 0 V such that v + 0 = 0 + v = v for all v V VS 3. For each v V there exists an additive inverse w V such that w + v = v + w = 0. VS 4. For all u, v V we have v + w = w + v VS 5. If r, s R are scalars and v, w V then 1 v = v and ( r + s) v = r v + s v r (v + w) = r v + r w (rs) v = r (s v )

11

Notes about Vector Spaces VS 14 say that V is an abelian group under vector addition. The additive inverse w in VS 3 is uniquely determined and usually denoted by v . For all r R and for all v V r0=0v =0 and (r v ) = (r) v = r (v )

12

Terminology The elements of a vector space V are called vectors. To determine if a set V is a vector space one must specify the set V , and dene vector addition and scalar multiplication in V . Then if V satises the properties VS 15 it is a vector space over the real numbers R. A vector space over the set of real numbers R, is called a real vector space. Instead of using real numbers R any eld can be employed such as the complex numbers C. A vector space over the set of complex numbers C, is called a complex vector space.

13

Examples 1. Let V = Rn be the set of n-tuples of real numbers. If x = (x1, x2, ..., xn) and y = (y1, y2, ..., yn) are two such n-tuples, dene x + y = (x1 + y1, x2 + y2, ..., xn + yn) and if a R, dene a x = (ax1, ax2, ..., axn) (Verify the vectorspace axioms.) 2. Let V be the set of all real valued functions on a non-empty set S . If f, g are functions, we can dene f + g in the usual way, and af in the usual way. 3. Let V = M (n n, R) be the set of all n nMatrices with real elements. Then V together with component-wise addition and component-wise multiplication with a scalar a R is a 14 vector space over R.

Subspaces Given a vector space V over R, a subspace of V is any nonempty subset W of V with 1. 0 W and 2. W is closed under addition (i.e., if x, y, W then x + y W ) and 3. W is closed under scalar multiplication (i.e., if x W and a R the ax W ) It is easy to see that subspaces of V are vector spaces (over the same eld) in their own right.

15

Example subspace 1. Let V be a vector space over R, and let v1, v2, ..., vr V . The subset of all linear combinations x1v1 + ... + xr vr , with xi R is a subspace (verify !). It is called the subspace generated by v1, v2, ..., vr . 2. Let n > 1 and let j be a xed integer, 1 j n. Let V = Rn and let W be the set of all n-tuples in Rn whose j -th component is zero, i. e. xj = 0. Then W is a subspace which is sometimes identied with Rn1 because it essentially consists of (n 1)-tuples.

16

DEF. A norm on a vector space V over the real numbers R is a function . : V R, v v such that following three properties are satised: N1. For all v V N3. For all v, w V Examples v 1= v 2=
n i=1 |vi| (Manhattan norm or 1-norm) n 2 (Euclidean norm or 2-norm) v i=1 i n p) p , where p 1 is a real number (p-norm) | v | i i=1
1

v = 0 v = 0. v = || v . v + w v + w (triangle inequality).

N2. For all v V, R

v p=( v = max(|v1|, ..., |vn|) (maximum norm) Note.

v v

=1
17

Basis of a vector space DEF. The intersection of all subspaces of a vector space V containing a given set of vectors is called their span. The set of vectors is called linearly independent if no vector can be removed without diminishing the span. A linearly independent set whose span is the whole vector space V is called a basis of V . Note. It can be shown that every vector space has a basis, and vector spaces over a given eld are xed up to isomorphism by a single cardinal number (called the dimension of the vector space) representing the size of the basis. For instance the real vector spaces are just R0, R1, R2, R3, ..., R.

18

Linear Maps DEF. Let V and W be two Rvector spaces. A linear map from V to W is a map V W which preserves the vector space structure, i.e. it preserves sums of vectors and products of scalars with vectors. The set of all linear maps from V to W is denoted L(V, W ). It is a Rvector space. Given the bases of both vector spaces V and W , elements of L(V, W ) can be identied with matrices. An isomorphism is a linear map that is one-to-one (monomorphism) and onto (epimorphism). Two vector spaces V and W are called isomorphic if and only if there exists an isomorphism between them.

19

Equations for Lines in 2D-Space [with (x,y)-coordinates] Basic form: y = mx + b (m = slope, b = section on y -axis) Point & direction form: y p2 = m(x p1) p1 + tv1 Parametric vector form: (x(t), y (t)) = p + tv = p2 + tv2 (through point p = (p1, p2) in direction of vector v = (v1, v2))
xp1 y p2 = Two-Point form: q q1p1 , through points p and q . 2p2 Implicit form: ax + by + c = 0 (a, b, c are real constants). x + y =1 Axis section form: p qy x (line through points p = (px, 0) and q = (0, qy ))

Hessian Normal Form (HNF): x cos + y sin d = 0 (d = closest distance from origin, = angle between normal and x-axis)
20

Some literature and sources This lecture uses a number of texts as sources, including: [?], [?], [?], and [?]. The remaining slides and pictures are mostly based on the book: Tomas Akenine-M oller and Eric Haines and Natty Homan, Real-Time Rendering, 3rd Edition, 2008, isbn 987-1-56881-424-7, A. K. Peters, Ltd., Natick, MA, USA. [?] or [?]. For this book there is an excellent support page at: http://www.realtimerendering.com/

21

Geometrical Denitions Basic rendering primitives used by most graphics hardware: points, lines, triangles. A model or an object is a collection of geometric entities. A scene is a collection of models comprising that everything that is included in the environment is rendered. It can include material descriptions, lighting, and viewing specications. Examples of objects: A car, a building, a line. Objects can consist of other objects.

22

The Computer Graphics Rendering Pipeline Main function of the pipeline: Given a virtual camera, 3D objects, light sources, lighting models, textures and other features generate, or render, a 2D image. Locations and shapes of objects in the image depend on: their geometry, placement of camera, and characteristics of environment. Appearance of objects depends on: material properties, light sources, textures, and lighting models. The pipeline has dierent stages. Some of them are implemented in hardware. Only primitives inside the view frustum are rendered.

23

Virtual Camera and View Frustum

  6JG )TCRJKEU 4GPFGTKPI 2KRGNKPG

3$Z 1d

yt Y jP $8s s %$Zsj {s8s $R jB{sI s Y $} BP Y }ks8$I wbY PBZ j$tR {Bt%D !tjk Y }$8$$%R $tR$I Y %$b %BjZ8 s tII 3B st $8s Ys $R tII $t }R}{$% wsR $R Y {sR YD Y %$b %BjZ8 $R s PZRZ8 $ s Zt{sI }ks8$I b$Y s {stZjs AsR qY $Y $8s RYBbR bYs Y {s8s XRR" WB Y ABB88BR {ZA $t Y jP $8s $R tB $t Y tI$t B Y $Y A{sZR $ $R jB{sI BZR$I Y %$b PZRZ8 VjRB Y $stj $t Y jP $8s $R {j$}}I ss$tR Y R8sjj wtsD }jst BP Y PZRZ8 bY${Y RZjR $t s JZsI$jssj

qY Bsj BP Y$R {Ys} $R YZR B }B%$I s Is$jI ZtIRstI$t 24 BP Y PZt{$Bt BP Y tI$t BP $8sR bY$j B8$$t I$R{ZRR$Bt BP $8}j8ts$Bt $RRZR

The Three Conceptual Stages There are three conceptual stages: 1. Application stage: driven by the application and therefore implemented in software; it may contain collision detection, animations, acceleration algorithms 2. Geometry stage: implemented in soft- or hardware; transforms, projections, lighting, etc. 3. Rasteriser stage: draws an image using the data provided by the previous stage Each of these stages typically is a pipeline in itself, i.e. it consists of several substages.

25

 #TEJKVGEVWTG



3$Z 11

qY AsR${ {BtRZ{$Bt BP Y tI$t }$}j$t {BtR$R$t BP Y RsRU s}}j${s$Bt B8k stI Y sR$9 es{Y BP YR RsR 8sk A s }$}j$t $t $RjP sR $jjZRsI AjBb Y B8k Rs B s Rs 8sk A w}sjkD }ssjjj$9I sR RYBbt AjBb Y sR$9 Rs yt Y$R $jjZRs$Bt Y s}}j${s$Bt Rs $R s R$tj }B{RR AZ Y$R Rs {BZjI sjRB A }$}j$tI B }ssjjj$9I

%k Y 8$tZR Y BY RsR 8ZR A $Ij PB Bt 8$tZ bY$j Y R$t bYj ss{Y8t $R {B8}jI 3B Y$R }s${Zjs }$}j$t Y R$t bYj Rs $R Y ABjt{, R$t{ $ I8$tR Y R}I BP Y t$ }BIZ{$Bt qY$R ,$tI BP }$}j$t {BtRZ{$Bt $R sjRB PBZtI $t Y {Bt- BP sjT 26 $8 {B8}Z s}Y${R V {BsR I$%$R$Bt BP Y sjT$8 tI$t }$}T j$t $tB Y {Bt{}Zsj RsR $R RYBbt $t 3$Z 11 qY RsR s

The Application Stage It is software based and the developer has full control (The other stages are built upon hardware). Takes care of inputs from mouse, joystick keyboard, touchscreen, VR helmet, ... At the end of the application stage, the geometry (points, lines, triangles) is fed into the next stage. Typical processes of the application stage: Collision detection ; response is fed back into colliding objects Texture animation, animations via transforms, geometry morphing Acceleration algorithms (e.g. hierarchical view frustum culling)
27

The Geometry Stage

It is subdivided in ve functional stages:

  6JG )TCRJKEU 4GPFGTKPI 2KRGNKPG

3$Z 1n

qY B8k Rs RZAI$%$II $tB s }$}j$t BP PZt{$Btsj RsR

PB8 s R$tj }$}j$t Rs wbY${Y ZtR $t }ssjjj b$Y Y BY }$}j$t RsRD yt BY {sRR s PZt{$Btsj Rs 8sk A RZAI$%$II $tB R%sj R8sjj }$}j$t RsR 3B -s8}j s Bt -8 sjj RsR $t Y t$ tI$t }$}j$t 8sk Zt $t RBPbs stI Yt kBZ {BZjI Rsk Ys kBZ t$ }$}j$t {BtR$RR BP Bt }$}j$t Rs V Y 28 BY -8 s{Y PZt{$Btsj Rs {BZjI A RZAI$%$II $tB R%sj R8sjj }$}j$t RsR stI s{Y RZ{Y

Model and View Transform

Model transform: applied to vertices and normals of the model for positioning and orienting. View transform: places camera at origin and aims it in direction of neg. z -axis. model transform (4 4)-matrix model space model coordinates model ; world space world coordinates several instances camera neg. z axis view transform (4 4)-matrix ; camera space

29

 6JG )GQOGVT[ 5VCIG



3$Z 1;

yt Y jP $jjZRs$Bt Y {s8s $R jB{sI stI B$tI sR Y ZR bstR $ B A qY %$b stRPB8 jB{sR Y {s8s s Y B$$t jBB,$t sjBt Y ts$% FTs-$R sR RYBbt Bt Y $Y qY$R $R IBt B 8s, Y {j$}}$t stI }B`{$Bt B}s$BtR R$8}j stI PsR qY j$Y sk ss $R Y %$b %BjZ8 < }R}{$% %$b$t $R sRRZ8I R$t{ Y %$b %BjZ8 $R s PZRZ8 7$8$js {Yt$JZR s}}jk B stk ,$tI BP }B`{$Bt

Y B$$t stI s$8 $ B 8s, $ jBB, $t Y I${$Bt BP Y ts$% F Ts-$R1 b$Y Y rTs-$R }B$t$t Z}bsIR stI Y ?Ts-$R }B$t$t B Y $Y qY s{Zsj }BR$$Bt stI I${$Bt sP Y %$b stRPB8 YsR At s}}j$I $R I}tIt Bt Y ZtIjk$t V}}j${s$Bt _Bs8 ytPs{ wV_yD qY R}s{ YZR Ij$tsI $R {sjjI Y {s8s R}s{ B 8B {B88Btjk Y k R}s{ Vt -s8}j BP Y bsk $t 30 bY${Y Y %$b stRPB8 s&{R Y {s8s stI Y 8BIjR $R RYBbt $t 3$Z 1; BY Y 8BIj stRPB8 stI Y %$b stRPB8 s $8}j8tI sR ; d ; 8s${R bY${Y $R Y

Lighting and Shading One or more light sources. The geometric model may have a colour associated with each vertex. Texture can give a 3D eect without lighting. Lighting equation: Calculate colour for each vertex of the model. Gouraud shading: Use location and properties of light sources, position and normal of the vertex and material properties to compute the colour at each vertex of the surface. Then interpolate the colours at the vertex of each triangle over the triangle. Pixel shading can be used for more elaborate lighting eects. Normally, lighting is calculated in world space.
31

Lighting
  6JG )TCRJKEU 4GPFGTKPI 2KRGNKPG

3$Z 1x

V R{t b$YBZ j$Y$t $R Bt Y jP s j$ R{t Bt Y $Y

8sk sjRB Ys% s {BjB sRRB{$sI b$Y s{Y %- B s -Z wst $8sD XjZI BtB" Y8 3$Z 1x $%R st -s8}j WB${ YBb s -Z $%R Y RZPs{ s YTI$8tR$Btsj &{ b$YBZ j$Y$t AZ RBj$I {BjBR 32 s Zt{Bt%$t{$t 3B YBR 8BIjR Ys s B A s&{I Ak j$Y RBZ{R s j$Y$t JZsT

Projection After lighting the view volume is transformed into a unit cube with extreme points at (1,1,1) and (-1,-1,-1), called the canonical volume. Two projection methods 1. Orthographic: parallel lines remain parallel; combine translation and scaling 2. Perspective: The farther away the smaller; parallel lines may converge at horizon; mimics how we perceive object size. View frustum = truncated pyramide with rectangular base; is transformed into the unit cube. Both projections can be constructed with (44)-matrices. After projection the model is called to be in normalised device coordinates.
33 z -coordinate is stored in the Z-buer.

 6JG )GQOGVT[ 5VCIG Orthographic and Perspective Projections



3$Z 1E

!t Y jP $R st BYBs}Y${ B }ssjjj }B`{$Bt Bt Y $Y $R s }R}{$% }B`{$Bt



2TQLGEVKQP

34

Clipping Only primitives wholly or partially inside the view volume need to be passed to the rasterizer stage, which then draws them on the screen. Due to the projection transform the transformed primitives are always clipped against the unit cube. The view volume has six clipping planes, but additional ones can be dened.

35

Clipping
 6JG )GQOGVT[ 5VCIG 

3$Z 1

VP Y }B`{$Bt stRPB8 Btjk Y }$8$$%R $tR$I Y Zt$ {ZA wbY${Y {BR}BtI B }$8$$%R $tR$I Y %$b PZRZ8D s IR$I PB {Bt$tZI }B{RR$t qYPB Y }$8$$%R BZR$I Y Zt$ {ZA s I$R{sII }$8$$%R Bsjjk $tR$I s ,} stI }$8$$%R $tR{$t b$Y Y Zt$ {ZA s {j$}}I ss$tR Y Zt$ {ZA stI YZR tb %${R s tsI stI BjI BtR s I$R{sII

t$t Y$R Rs qY ?T stI r T{BBI$tsR BP s{Y }$8$$% s stRT 36 PB8I B PB8 R{t {BBI$tsR 7{t {BBI$tsR BY b$Y Y F T{BBI$tsR s sjRB {sjjI b$tIBb {BBI$tsR VRRZ8 Ys Y R{t

Screen Mapping Only the clipped primitives are passed on to the screen mapping stage. Still 3D. x-, and y -coordinates of each primitive are transformed to form screen coordinates. Translation and scaling. z -coordinate not aected ; z [-1,1]. Screen coordinates together with the z -coordinate are passed on to the rasteriser stage.

37

R{t 8s}}$t $R s stRjs$Bt PBjjBbI Ak s R{sj$t B}s$Bt qY F T {BBI$ts $R tB s&{I Ak Y$R 8s}}$t qY tb ?T stI rT{BBI$tsR s Rs$I B A R{t {BBI$tsR qYR sjBt b$Y Y F T{BBI$ts Screen Mapping Process wbd F dD s }sRRI Bt B Y sR$9 Rs qY R{t 8s}T }$t }B{RR $R I}${I $t 3$Z 1M

3$Z 1M

qY }$8$$%R j$ $t Y Zt$ {ZA sP Y }B`{$Bt stRPB8 stI Y R{t 8s}}$t }B{IZ s,R {s BP htI$t Y {BBI$tsR Bt Y R{t

38

Summary of the Geometry Stage

In summary the geometry stage performs a very demanding task. With a single light source each vertex requires about 100 individual precision oating point operations.
  6JG )TCRJKEU 4GPFGTKPI 2KRGNKPG

3$Z 1n

qY B8k Rs RZAI$%$II $tB s }$}j$t BP PZt{$Btsj RsR

PB8 s R$tj }$}j$t Rs wbY${Y ZtR $t }ssjjj b$Y Y BY }$}j$t RsRD yt BY {sRR s PZt{$Btsj Rs 8sk A RZAI$%$II $tB R%sj R8sjj }$}j$t RsR 3B -s8}j s Bt -8 sjj RsR $t Y t$ tI$t }$}j$t 39 8sk Zt $t RBPbs stI Yt kBZ {BZjI Rsk Ys kBZ t$ }$}j$t {BtR$RR BP Bt }$}j$t Rs V Y BY -8 s{Y PZt{$Btsj Rs

The Rasteriser Stage Rasterization or Scan Conversion: Given the transformed and projected vertices, colors, texture coordinates, assign correct colors to the pixels. Unlike the geometry stage which handles per polygon operations, the rasterizer stage handles per-pixel operations. Pixel information is stored in the colour buer which is a rectangular array of colours. For high performance the rasterizer stage should be implemented in hardware.

40

Buers, Visibility Double buering uses a front and a back buer to avoid that the viewer observes the rastering. Z-buer = depth buer. It stores the z -value from the camera to the closest primitive. Z-buer algorithm: If the polygon point being scan converted at (x, y ) is no farther from the viewer than is the point whose colour and depth are currently in the buers, then the new points colour and depth replace the old values. has O(n) convergence allows primitives to be rendered in any order works for any primitive for which a z -value can be computed for each pixel Texturing means glueing an image to an object.
41

j$tR JZsI$jssjR R}YR {kj$tIR }ss8${ RZPs{R { G Ys% 8t$BtI Ys Y {BjB AZ& $R ZRI B RB {BjBR stI Ys Y . TAZ& RBR F T%sjZR PB s{Y }$-j <Bb% Y s BY Texturing

V {Bb 8BIj b$YBZ -ZR $R RYBbt Bt Y jP qY bB -ZR $t Y 8$IIj s XjZI" BtB Y {Bb stI Y RZj $R RYBbt Bt Y $Y qY B} -Z $R PB Y kR bY$j Y ABB8 -Z $R PB Y ABIk BP Y {Bb w]Bb 8BIj ZRI
3$Z 1

{BZRk BP FtR NsRRBtD

42

Summary of the Pipeline A model is built from points, lines and triangles; texture; light source. Application: User can interact with the model. Camera can move. view transform and model transform. Geometry: Put models into eye space. Lighting, textures Projection; unit cube; clipping. Map into a window on the screen. Rasterizer: All primitives are rasterized. Render primitives with texture. Visibility: Z-buer algorithm.
43

Exercise 1 Calculate the midpoint of the circle in 2D that contains the following three points: A = (1, 1) B = (3, 0) C = (5, 1)

44

Exercise 2 Show that the following two denitions of the dot product are equivalent (we restrict it to 3D now but is works the same in n dimensions): Def 1: Def 2: v w = v1w1 + v2w2 + v3w3 v w = |v | |w| cos ,

where is the angle between v and w.

45

Exercise 3 Select a plane and a line in 3D and calculate the hitpoint: For the plane you require a normal vector n and an anchor point B . For the line you require a direction vector c and an anchor point A. Use the example: n = (1, 0, 1), B = (1, 2, 2), c = (1, 2, 0), A = (0, 1, 1).

46