Vector spaces, Normed spaces, bases

Eugenia Malinnikova, NTNU, Institutt for matematiske fag

17.-18. september 2014

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Vector space
A real vector space is a set V endowed with operations called
addition and scalar multiplication,
V × V → V , (x, y ) 7→ x + y , R × V → V , (λ, x) 7→ λx, such
that there exists an element 0 ∈ V called the zero vector, and for
each x ∈ V there exists an additive inverse −x ∈ V , and
∀ x, y , z ∈ V and ∀ λ, µ ∈ R the following properties hold:
(i) x + 0 = x, (additive identity)
(ii) x + (−x) = 0, (additive inverse)
(iii) x + y = y + x, (symmetry)
(iv) x + (y + z) = (x + y ) + z, (associativity)
(v) 1x = x, (multiplicative identity)
(vi) λ(µx) = (λµ)x, (compatibility)
(vii) λ(x + y ) = λx + λy , (distributivity)
(viii) (λ + µ)x = λx + µx, (distributivity)

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples
The elements of V are called vectors. If the field of scalars R is
replaced with C one obtains instead a complex vector space.
I (R, +, ·), the set of real numbers R endowed with the usual
addition and multiplication is a real vector space.
I More generally,

Rn = {(x1 , . . . , xn ) : xj ∈ R for j = 1, 2, . . . , n.}

endowed with component-wise addition
(x1 , . . . , xn ) + (y1 , . . . , yn ) = (x1 + y1 , . . . , xn + yn )
and component-wise scalar product
λ(x1 , . . . , xn ) = (λx1 , . . . , λxn )
is a real vector space for any natural number n ∈ N.
I Similarly Cn is a complex vector space, but, it can also be
considered as a real vector space.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Further examples

I Q1: Write down an example of a vector space that didn’t

appear on the previous slide.
I The set of real-valued continuous functions on an interval
I ⊂ R,

C (I , R) = {f : I → R such that f is continuous}

is a real vector space with the zero function f ≡ 0 as additive

identity and −f as additive inverse, when one defines

(f + g )(t) = f (t) + g (t), (λf )(t) = λf (t).

I Sequence spaces l∞ , l1 , l2 are vector spaces with

component-wise addition and scalar multiplication.
I Matrices of a given size form a vector space.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Subspaces
Let V be a vector space. A non-empty subset W ⊂ V is a
subspace of V if it is closed under linear operations, i.e.,
W ⊂ V subspace of V ⇔ λx + µy ∈ W ∀ x, y ∈ W and ∀µ, λ ∈ R (C)
In particular, 0 ∈ W , and W is itself a vector space (the axioms
for a vector space follow from those of V ).
I In any vector space, {0} (the set consisting only of the zero
element) is a subspace, since λ0 + µ0 = 0 ∈ {0} for all
scalars λ, µ.
I Consider R = {(x, 0, 0) : x ∈ R} as a subset of

R3 = {(x, y , z) : x, y , z ∈ R}.
Then R is a subspace of R3 , since it is non-empty and
λ(x1 , 0, 0) + µ(x2 , 0, 0) = (λx1 + µx2 , 0, 0) ∈ R ⊂ R3 .
I Q2: a) give another example of a subspace of R3 , b) give an
example of a subset of R3 that is not a subspace.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples of subspaces
I Q3: give an example of a proper subspace of
Mn×m , l∞ , C (I , R)
I Similarly, the set of real-valued continuous functions on R
which vanish on some set S ⊂ R is a subspace of C (R, R):

{f ∈ C (R, R) : f ≡ 0 on S} is a subspace of C (R, R),

since

µf (x) + λg (x) = 0 if f (x) = 0 and g (x) = 0.

I The vector space of polynomials of degree at most n, Pn (R),
endowed with the usual addition and scalar multiplication, is a
subspace of the set of polynomials of degree at most
S n + 1.
P0 (R) ⊂ P1 (R) ⊂ . . . ⊂ Pn (R) ⊂ . . . ⊂ P(R) := ∞ n=0 Pn (R)
are all subspaces of each other and, ultimately, of the vector
space of all polynomials, P(R).
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Linear combinations and span
A linear combination of vectors u1 , . . . , un is a finite sum
n
X
aj uj ,
j=1

where a1 , . . . , an are scalars. The (linear) span of S ⊂ V is the

set of all linear combinations of vectors in S:
def. 
X
span(S) = aj xj : xj ∈ S, aj scalars .
finite

If W = span(S) we say that S generates W .

The linear span of a set S is the smallest subspace containing S
For any S ⊂ V , span(S) is a subspace of V , and
\
span(S) = {W : W is a subspace of V }.
S⊂W

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples of linear spans
I Let x = (1, 0), y = (2, 0) and z = (1, 1) be vectors in R2 .
Then

span{x} = span{y } = span{x, y } = {(λ, 0) : λ ∈ R},

span{z} = {(λ, λ) : λ ∈ R},

span{x, z} = span{y , z} = span{x, y , z} = R2 .

I The vectors e1 = (1, 0, . . .), e2 = (0, 1, 0, . . .), . . .,

en = (0, . . . , 0, 1) generate Rn .
I In general, the span of a set differs between real and complex
vector spaces:

spanR {1} = R but spanC {1} = C.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Linear dependence

A family of vectors u1 , u2 , . . . is called linearly dependent if one

of them is linear combination of some of the others:

{u1 , u2 , . . .} linearly dependent ⇔

Xn
aj uj = 0 for some n ∈ N and at least one aj 6= 0.
j=1

Else, the family is linearly independent:

{u1 , u2 , . . .} linearly independent ⇔

X n
 
for any n ∈ N : aj uj = 0 =⇒ aj = 0 ∀ j .
j=1

Q4: Find three linear dependent vectors such that each pair is
linearly independent.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples of linear independent sets
I The vectors x = (1, 0), y = (2, 0) and z = (1, 1) are linearly
dependent in R2 , since
   
2 1
=2 .
0 0

But both the sets {x, z} and {y , z} are linearly independent,

since
         
1 1 0 a1 + a2 0
a1 x + a2 z = 0 ⇐⇒ a1 + a2 = ⇐⇒ =
0 1 0 a2 0

and similarly for {y , z}.

I If 0 ∈ S, then S is linearly dependent.
I {1, x, x 2 , . . .} is linearly independent in P(R).
I {1, cos(x), sin(x), cos(2x), sin(2x), . . .} is linearly independent
in C (I , R).
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Bases

Let V be a vector space. A linearly independent set which

generates V is called a (Hamel) basis for V .

S ⊂ V Hamel basis for V ⇔ span(S) = V and S lin. indep.

Equivalently, S is a Hamel basis for V if every vector x ∈ V has a

unique and finite representation
X
x= aj u j , uj ∈ S.
finite

Q5: Give some examples of Hamel bases, can you find an infinite
example?

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples of bases

I {e1 , . . . , en }, with

ej = (0, . . . , 1
|{z} , 0 . . .)
jth position

is called the standard basis for Rn .

I {1, x, x 2 , . . .} is an ordered Hamel basis for P(R): every real
polynomial can be uniquely expressed as a finite sum,
X
p(x) = aj x j , aj ∈ R.
finite

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Dimension

If V has a basis consisting of finitely many vectors, V is said to be

finite-dimensional. Else, V is infinite-dimensional.
The dimension of any finite-dimensional vector space is unique
All bases of a finite-dimensional vector space have the same
number of elements. This number is called the dimension of the
space. [PROOF]
I Rn has dimension n.
I Pn (R), has dimension n + 1.
I Cn has dimension n when considered as a complex vector
space, but 2n when considered a real vector space.
I The lp -, BC -, and L2 -spaces are all infinite-dimensional.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Isomorphism of vector spaces

A vector space isomorphism is a bijective linear map between

two vector spaces, i.e. an invertible function T : V → U such that

T (x + y ) = Tx + Ty and T (λx) = λTx for all x, y ∈ V , λ ∈ R (C).

Two vector spaces which allow for such a mapping are called
isomorphic, and we write

V ∼
= U ⇔ ∃ isomorphism T : V → U.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Isomorphism examples
I Regarded as a real vector space, the space Cn of complex
n-tuples, z = (z1 , . . . , zn ), z1 , . . . , zn ∈ C is isomorphic to
Euclidean space R2n via the isomorphism
z = (x1 + iy1 , . . . , xn + iyn ) 7→ (x, y ) = (x1 , . . . , xn , y1 , . . . , yn ).
I Q6: Write down bases for C 2 over C and over R.
I The set of polynomials with real coefficients of degree at most
n, Pn (R), is isomorphic to Rn+1 . The mapping
T : Pn (R) → Rn+1 , an x n + . . . + a1 x + a0 7→ (a0 , a1 , . . . , an )
is both bijective and linear,
n
X
n+1
∀ (a0 , . . . , an ) ∈ R ∃! p(x) = ak x k ∈ Pn (R),
k=0
 Xn n
X  n
X 
k k k
T λ ak x + µ bk x =T (λak + µbk )x
k=0 k=0 k=0
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Finite-dimensional vector spaces
If V is a real vector space with a finite basis {e1 , . . . , en } then
V ∼= Rn .
By the definition of a basis, any x ∈ V has a unique representation
n
X
x= aj ej .
j=1

Let T : V → Rn
be P
the mapping defined
P by Tx = (a1 , . . . , an ).
T is linear: if x = aj ej and y = bj ej ,
T (λx + µy ) = (λa1 + µb1 , . . . , λan + µbn )
= λ(a1 , . . . , an ) + µ(b1 , . . . , bn ) = λTx + µTy ,
surjective: for any (a1 , . . . , an ) ∈ Rn there exists
T is P
x = nj=1 aj ej ; Tx = (a1 , . . . , an )
T is injective:
Tx = Ty ⇐⇒ ∀j : aj = bj =⇒ x = y.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Normed space

A normed space is a vector space V endowed with a function

V → [0, ∞), x 7→ kxk,

called the norm on V , which satisfies:

(i) kλxk = |λ| kxk, (positive homogeneity)

(ii) kx + y k ≤ kxk + ky k, (triangle inequality)
(iii) kxk = 0 if and only if x = 0, (positive definiteness)

for all scalars λ and all elements x, y ∈ V . A vector space may

allow for many different norms.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Examples of normed spaces

I The vector space Rn with the usual addition and scalar

multiplication allows for several norms, for example:
the Euclidean norm

1/2
k(x1 , . . . , xn )kl2 = x12 + . . . + xn2

the maximum norm

k(x1 , . . . , xn )kl∞ = max{|x1 |, . . . , |xn |},

and the summation norm

k(x1 , . . . , xn )kl1 = |x1 | + . . . + |xn |.

I These are all special cases of the (finite-dimensional) lp -norm

p 1/p , 1 ≤ p ≤ ∞.
Pn

k(x1 , . . . , xn )klp = j=1 |xj |

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Further examples of Normed spaces
The space of real- (or complex-) valued bounded and continuous
functions on an interval (open or closed), BC (I , R), becomes a
normed vector space when endowed with the supremum norm
kf k∞ = sup |f (x)|.
x∈I

If I = [a, b] it follows from the extreme value theorem that

BC ([a, b], R) = C ([a, b], R) (as sets and linear spaces) and
kf k∞ = sup |f (x)| = max |f (x)|.
x∈[a,b] x∈[a,b]

If I = (a, b) is either infinite or does not contain its end points,

then BC ((a, b), R) ( C ((a, b), R). An example of this strict
inclusion is the function x 7→ 1/x on (0,1). It is continuous, but
[x 7→ 1/x] 6∈ BC ((0, 1), R),
since supx∈(0,1) |1/x| = ∞.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Normed spaces are metric spaces
If k · k is a norm on X , then d(x, y ) := kx − y k is a metric on X .
Proof.
The distance is non-negative and well defined, since
0 ≤ kx − y k ≤ kxk + ky k < ∞, for x, y ∈ (X , k · k).
| {z }
d(x,y )

Symmetry: d(x, y ) = kx − y k = ky − xk = d(y , x).

Triangle inequality: d(x, y ) = kx − y k
≤ kx − zk + kz − y k = d(x, z) + d(z, y ).
Non-degeneracy: d(x, y ) = kx − y k = 0 ⇔ x − y = 0 ⇔ x = y .

N.b. Metric spaces need not be vector spaces. The set of positive
real numbers, R+ = (0, ∞), with the metric given by
d(x, y ) := |x − y | is a metric space, but it is not a linear space.
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Equivalence of norms
Two norms k · k1 and k · k2 on a vector space X are said to be
equivalent if there exists a number c ∈ R such that
c −1 kxk1 ≤ kxk2 ≤ ckxk1 for all x ∈ X.

I The maximum and summation norms are equivalent on Rn ,

since
n
X n
X
max |xj | ≤ |xj | and |xj | ≤ n max |xj |.
1≤j≤n 1≤j≤n
j=1 j=1

Hence
n−1 kxkl∞ ≤ kxkl1 ≤ nkxkl∞ for x = (x1 , . . . , xn ).
I On a finite-dimensional vector space, any two norms are
equivalent. In particular, any norm on Rn is equivalent to the
Euclidean norm.[PROOF]
Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Unit balls

The unit ball may look quite different depending on the underlying
metric/norm. The following illustration captures this in the case of
the lp -norm on R2 . Homogeneity and the triangle inequality
however imply that a ball in any metric given by a norm will always
be a convex set in the underlying space.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Banach space

A complete normed space is called a Banach space.

I R is complete with respect to the metric d(x, y ) = |x − y |.
Hence, (R, | · |) is a Banach space.
I Both Rn P
and Cn are Banach spaces with respect to the norm
kxk2 = ( nj=1 |xj |2 )1/2 .
I The space of square-summable sequences l2 is complete with
respect to the norm
∞
X
1/2
kxk2 = |xj |2 .
j=1

So is l∞ , and lp , for any p ≥ 1.

I For any interval I ⊂ R, BC (I , R) and BC (I , C), with norms
given by kxk = supt∈I |x(t)|, are Banach spaces.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
Hamel and Schauder basis

Infinite-dimensional Banach spaces have only uncountable Hamel

bases.
Let X be an infinite-dimensional Banach space. Then a sequence
{ej }j∈N cannot be a Hamel basis for X .

Let (X , k · k) be a Banach space. A sequence {ej }j∈N is called a

Schauder basis for X if every vector x ∈ X has a unique
representation
X
x= xj ej ,
j∈N

PN
meaning that limN→∞ kx − j=1 xj ej k = 0. The scalars xj are the
coordinates of x.

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
A basis for the sequence spaces
Let ej = (0, . . . , 1, 0 . . .). Then {ej }∞
j=1 is a (Schauder) basis for lp ,
1 ≤ p < ∞.
Approximation property:
X ∞ X∞
p
x = {xj }j∈N ∈ lp ⇒ |xj | < ∞ ⇒ |xj |p → 0 as N → ∞.
j=1 N+1

N
X
xj ej − x p = k(x1 , . . . , xN , 0, . . .) − (x1 , . . . , xN , xN+1 , . . .)kp
j=1
∞
X
1/p
= |xj |p → 0 as N → ∞.
N+1
Uniqueness of coordinates:
∞
X ∞
X N
X
xj ej = yj ej ⇔ lim |xj − yj |p = 0 ⇒ xj = yj , for all j ∈ N.
N→∞
j=1 j=1 j=1

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10
This week summary, Sections 3.1-3.4 in Lecture notes

I Definitions and examples of vector spaces, subspaces, (Hamel)

bases
I Isomorphism of vector spaces, finite-dimensional spaces are
isomorphic to Rn
I Definition and example of normed spaces
I Normed spaces are metric spaces
I Any two norms on Rn are equivalent
I Banach spaces, examples
I Schauder bases

Eugenia Malinnikova, NTNU, Institutt for matematiske fag TMA4145 Linear methods, Lectures 9&10