JinHo Kwak

Sungpyo Hong
Linear Algebra
Second Edition
Springer Science+Business e d i ~ LLC
JinHo Kwak Sungpyo Hong
Department of Matbematics
Pohang University of Science
Department of Mathematics
Pohang University of Science
and Technology and Technology
Pohang, Kyungbuk 790-784
South Korea
Pohang, Kyungbuk 790-784
SouthKorea
Library of Cougress Cataloging-in-PubHeation Data
Kwak, lin Ho, 1948-
Linear algebra I lin Ho Kwak, Sungpyo Hong.-2nd ed.
p.cm.
Includes bibliographical references and index.
ISBN 978-0-8176-4294-5 ISBN 978-0-8176-8194-4 (eBook)
DOI 10.1007/978-0-8176-8194-4
1. Algebras, Linear. I. Hong, Sungpyo, 1948- . Title.
QAI84.2.K932004
512' .5-dc22
AMS Subject Classifications: 15-01
ISBN 978-0-8176-4294-5 Printed on acid-free paper.
@2004 Springer Science+Business Media New York
Originally published by Birkhlluser Boston in 2004
2004043751
CIP
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher Springer Science+Business Media, LLC,
except for brief excerpts in connection with reviews or
scholarly analysis. Use in connection with any form of information storage and retrievaI, electronic
adaptation, computer software, or by similar or dissimilar methodology now known or hereafter
developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to property rights.
987654321 SPIN 10979327
www.birkhasuer-science.com
Preface tothe Second Edition
This second edition is based on many valuable comments and suggestions from
readers of the first edition. In this edition, the last two chapters are interchanged
and also several new sections have been added. The following diagram illustrates the
dependencies of the chapters.
Chapter 1
Linear Equations and Matrices
Chapter 4
Linear Transformations
Chapter 5
Inner Product Spaces
Chapter 8
Jordan Canonical Forms
Chapter 2
Determinants
Chapter 6
Diagonalization
Chapter?
Complex Vector Spaces
vi Preface to the Second Edition
The major changes from the first edition are the following.
(1) In Chapter 2, Section 2.5.1 "Miscellaneous examples for determinants" is
added as an application.
(2) In Chapter 4, "A homogeneous coordinate system" is introduced for an appli-
cation in computer graphics.
(3) In Chapter 5, Section 5.7 "Relations of fundamental subspaces" and Section 5.8
"Orthogonal matrices and isometries" are interchanged. "Least squares solutions,"
"Polynomial approximations" and "Orthogonal projection matrices" are collected
together in Section 5.9-Applications.
(4) Chapter 6 is entitled "Diagonalization" instead of "Eigenvectors and Eigen-
values." In Chapters 6 and 8, "Recurrence relations," "Linear difference equations"
and "Linear differential equations" are described in more detail as applications of
diagonalizations and the Jordan canonical forms of matrices.
(5) In Chapter 8, Section 8.5 "The minimal polynomial of a matrix" has been
added to introduce more easily accessible computational methods for Anand e
A
, with
complete solutions of linear difference equations and linear differential equations .
(6) Chapter 8 "Jordan Canonical Forms" and Chapter 9 "Quadratic Forms" are
interchanged for a smooth continuation of the diagonalization problem of matrices.
Chapter 9 "Quadratic Forms" is extended to a complex case and includes many new
figures.
(7) The errors and typos found to date in the first edition have been corrected.
(8) Problems are refined to supplement the worked-out illustrative examples and
to enable the reader to check his or her understanding of new definitions or theorems.
Additional problems are added in the last exercise section of each chapter. More
answers, sometimes with brief hints, are added, including some corrections.
(9) In most examples, we begin with a brief explanatory phrase to enhance the
reader's understanding.
This textbook can be used for a one- or two-semester course in linear algebra. A
theory oriented one-semester course may cover Chapter 1, Sections 1.1-1.4, 1.6-1.7;
Chapter 2 Sections 2.1-2.3; Chapter 3 Sections 3.1-3.6; Chapter 4 Sections 4.1-4.6;
Chapter 5 Sections 5.1-5.4; Chapter 6 Sections 6.1-6.2; Chapter 7 Sections 7.1-7.4
with possible addition from Sections 1.8, 2.4 or 9.1-9.4. Selected applications are
included in each chapter as appropriate. For a beginning applied algebra course, an
instructor might include some ofthemin the syllabus at his or her discretion depending
on which area is to be emphasized or considered more interesting to the students.
In definitions, we use bold face for the word being defined, and sometimes an italic
or shadowbox to emphasize a sentence or undefined or post-defined terminology.
Preface to the Second Edition vii
Acknowledgement: The authors would like to express our sincere appreciation
for the many opinions and suggestions from the readers of the first edition including
many of our colleagues at POSTECH. The authors are also indebted to Ki Hang Kim
and Fred Roush at Alabama State University and Christoph Dalitz at Hochschule
Niederrhein for improving the manuscript and selecting the newly added subjects in
this edition. Our thanks again go to Mrs . Kathleen Roush for grammatical corrections
in the final manuscript, and also to the editing staff of Birkhauser for gladly accepting
the second edition for publication.
JinHo Kwak
Sungpyo Hong
E-mail: jinkwak@postech.ac.kr
sungpyo@postech.ac.kr
January 2004, Pohang, South Korea
Preface to the First Edition
Linear algebra is one of the most important subjects in the study of science and engi-
neering because of its widespread applications in social or natural science, computer
science, physics, or economics . As one of the most useful courses in undergradu-
ate mathematics , it has provided essential tools for industrial scientists. The basic
concepts of linear algebra are vector spaces, linear transformations, matrices and
determinants, and they serve as an abstract language for stating ideas and solving
problems .
This book is based on lectures delivered over several years in a sophomore-level
linear algebra course designed for science and engineering students. The primary
purpose of this book is to give a careful presentation of the basic concepts of linear
algebra as a coherent part of mathematics, and to illustrate its power and utility through
applications to other disciplines . We have tried to emphasize computational skills
along with mathematical abstractions , which have an integrity and beauty of their
own. The book includes a variety of interesting applications with many examples not
only to help students understand new concepts but also to practice wide applications
ofthe subject to such areas as differential equations, statistics, geometry, and physics.
Some of those applications may not be central to the mathematical development and
may be omitted or selected in a syllabus at the discretion of the instructor. Most
basic concepts and introductory motivations begin with examples in Euclidean space
or solving a system of linear equations, and are gradually examined from different
points of view to derive general principles .
For students who have finished a year of calculus, linear algebra may be the first
course in which the subject is developed in an abstract way, and we often findthat many
students struggle with the abstractions and miss the applications . Our experience is
that , to understand the material, students should practice with many problems, which
are sometimes omitted. To encourage repeated practice, we placed in the middle of
the text not only many examples but also some carefully selected problems, with
answers or helpful hints. We have tried to make this book as easily accessible and
clear as possible , but certainly there may be some awkward expressions in several
ways. Any criticism or comment from the readers will be appreciated .
x Preface to the First Edition
We are very grateful to many colleagues in Korea, especially to the faculty mem-
bers in the mathematics department at Pohang University of Science and Technology
(POSTECH), who helped us over the years with various aspects of this book. For
their valuable suggestions and comments, we would like to thank the students at
POSTECH, who have used photocopied versions of the text over the past several
years. We would also like to acknowledge the invaluable assistance we have received
from the teaching assistants who have checked and added some answers or hints
for the problems and exercises in this book. Our thanks also go to Mrs. Kathleen
Roush who made this book much more readable with grammatical corrections in the
final manuscript. Our thanks finally go to the editing staff of Birkhauser for gladly
accepting our book for publication.
Jin Ho Kwak
Sungpyo Hong
April 1997, Pohang, South Korea
Contents
Preface to the Second Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v
Preface to the First Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1 Linear Equations and Matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Systems of linear equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Gaussian elimination. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.3 Sums and scalar multiplications of matrices. . . . . . . . . . . . . . . . . . . . . 11
1.4 Products of matrices .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.5 Block matrices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.6 Inverse matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.7 Elementary matrices and finding A-I . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.8 LDUfactorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 29
1.9 Applications..... . .. . . . .. . . .. . .. .... . . . .. .. . . . . ....... . ... . 34
1.9.1 Cryptography.. . . .. . . . . . . . .. . .. . .. .. .. . .. . .. . . .. .. .. 34
1.9.2 Electrical network 36
1.9.3 Leontief model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.10 Exercises 40
2 Determinants. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 45
2.1 Basic properties of the determinant. . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.2 Existence and uniqueness of the determinant. . . . . . . . . . . . . . . . . . .. 50
2.3 Cofactor expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 56
2.4 Cramer's rule. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 61
2.5 Applications .... . ....... ... . . . .. . ... ............. ... ... .... 64
2.5.1 Miscellaneous examples for determinants. . . . . . . . . . . . . . .. 64
2.5.2 Area and volume. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 67
2.6 Exercises ............. ..... . . .. . . .. . ...... .... .. .. . .. .. . . . 72
xii Contents
3 Vector Spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 75
3.1 The n-space jRn and vector spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.2 Subspaces. . . . .. . . . ... . . .. . . . .. . . . . . ... . ... ... . . . . ... .... .. 79
3.3 Bases. . . .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.4 Dimensions . ... . ... .. .. ... .. .. . .. .. . . . . .... . .. ... . . . ... . .. 88
3.5 Rowand columnspaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 91
3.6 Rank and nullity 96
3.7 Bases for subspaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 100
3.8 Invertibility.... .. ...... ... .. ... . .. . . ... . . . . . . . . . . . . . . . . . . . 106
3.9 Applications 108
3.9.1 Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.9.2 The Wronskian 110
3.10 Exercises 112
4 Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.1 Basic propertiesof linear transformations 117
4.2 Invertiblelinear transformations . . . . . . . . . . . . . . . . . . . . . 122
4.3 Matrices of linear transformations. . . . . . . . . . . . . . . . . . . . . . . . . . . .. 126
4.4 Vector spaces of linear transformations . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.5 Change of bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.6 Similarity. . . . .. .. . . . . . . . . . .. . . . . . .. .... .. . . . ... .. . . . ... . .. 138
4.7. Applications 143
4.7.1 Dual spaces and adjoint " 143
4.7.2 Computer graphics 148
4.8 Exercises ... . . . . . .. . ... . . . .. . .. . . . . . . . . . . . .. . .. . . . . . . . . . .. 152
5 Inner Product Spaces 157
5.1 Dot products and inner products 157
5.2 The lengths and angles of vectors.. . . . .. . .. .. . . .. . . .. .. . . .. . .. 160
5.3 Matrix representations of inner products 163
5.4 Gram-Schmidt orthogonalization 164
5.5 Projections. .... . .. . . ... .. . . ... .... .... ................. . . . 168
5.6 Orthogonal projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
5.7 Relations of fundamental subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
5.8 Orthogonal matrices and isometries 177
5.9 Applications 181
5.9.1 Least squares solutions 181
5.9.2 Polynomial approximations 186
5.9.3 Orthogonal projectionmatrices. . . . . . . . . . . . . . . . . . . . . . . . . 190
5.10 Exercises 196
Contents xiii
6 Diagonalization 201
6.1 Eigenvalues and eigenvectors 201
6.2 Diagonalization of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
6.3 Applications 212
6.3.1 Linear recurrence relations . . . . . . . . . .. 212
6.3.2 Linear difference equations 221
6.3.3 Linear differential equations I . . . . . . . . . . . . . . . . . . . . . . . . . . 226
6.4 Exponential matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 232
6.5 Applications continued . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 235
6.5.1 Linear differential equations II 235
6.6 Diagonalization of linear transformations . . . . . . . . . . . . . . . . . . . . . . . 240
6.7 Exercises . .. .. . .. .. .... . ... .... .... .. ....... . .. .. . ....... . 242
7 Complex Vector Spaces 247
7.1 The n-space en and complex vector spaces " 247
7.2 Hermitian and unitary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 254
7.3 Unitarily diagonalizable matrices 258
7.4 Normal matrices 262
7.5 Application . ...... .. .... . ........ ..... .............. .. .. .. . 265
7.5.1 The spectral theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 265
7.6 Exercises.. ...... .. .... . .. ..... . ... .... .... . . ........ ... . . 269
8 Jordan Canonical Forms 273
8.1 Basic properties of Jordan canonical forms . . . . . . . . . . . . . . .. 273
8.2 Generalized eigenvectors " 281
8.3 The power A
k
and the exponential e
A
. . . . .. . 289
8.4 Cayley-Hamilton theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 294
8.5 The minimal polynomial of a matrix " 299
8.6 Applications. .... .. ... ...... .... ...... .. ..... .. .. .. .. ...... 302
8.6.1 The power matrix A
k
again 302
8.6.2 The exponential matrix e
A
again 306
8.6.3 Linear difference equations again . . . . . . . . . . . . . . . . . . . . . .. 309
8.6.4 Linear differential equations again. . . . . . . . . . . . . . . . . . . . . . 310
8.7 Exercises 315
9 Quadratic Forms 319
9.1 Basic properties of quadratic forms " 319
9.2 Diagonalization of quadratic forms 324
9.3 A classification of level surfaces 327
9.4 Characterizations of definite forms " 332
9.5 Congruence relation 335
9.6 Bilinear and Hermitian forms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
9.7 Diagonalization of bilinear or Hermitian forms 342
9.8 Applications 348
9.8.1 Extrema of real-valued functions on jRn . . . 348
xiv Contents
9.8.2 Constrainedquadratic optimization 353
9.9 Exercises.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
SelectedAnswersand mots 361
Bibliography 383
Index 385
Linear Algebra