Convex Set
Given an affine space (𝐴, 𝑉) where 𝑉 = (𝐹, 𝑉) is a vector space with an ordered field 𝐹 (when discussing
affine combination, the orderedness of 𝐹 is not required), a subset 𝐶 ⊆ 𝐴 is a convex set if for any two
points 𝑥, 𝑦 ∈ 𝐶 and any 0 < 𝜃 < 1, we have 𝜃𝑥 + (1 − 𝜃)𝑦 ∈ 𝐶. Obviously, an affine subspace of (𝐴, 𝑉) Commented [TC1]: Can be 0 ≤ 𝜃 ≤ 1 but it is not
is a convex set. The dimension of a convex set 𝐶 is the dimension of the affine hull of 𝐶. Given a subset necessary, since 𝑥, 𝑦 are already in 𝐶.
of points 𝑆 ⊆ 𝐴, the smallest convex set that contains 𝑆 is called the convex hull of 𝑆, denoted as 𝐶(𝑆).
Given a set of points 𝑆 = {𝑝 , … , 𝑝 }, the (finite) convex combination of 𝑆 is defined as ∑ 𝜆 𝑝 s.t.
∑ 𝜆 = 1 and 𝜆 ≥ 0.
One implication of convex sets for optimization search is that once a search along a direction leaves a
convex set, it never returns. However, for the search might re-enter a non-convex set.
See Theorem 20-5 for an application: the situation of a linear search is reflected by the level set of the
current position, where a worse and bigger level set contains a better and smaller level set. If all level
sets are convex, once a search leaves a better one, it never returns to it in future search, and hence
miss the minimum for sure.
2) Property 17-2 The induced vectors set 𝑉 = {0𝑥⃗: 𝑥 ∈ 𝐶} w.r.t. any origin is a convex set, although
it is origin dependent. For any 𝜃 ∈ (0,1) and 0𝑥⃗, 0𝑦⃗ ∈ 𝑉 , then by Property 16-4 we have for any
𝜃 ∈ [0,1]
3) Property 17-3 Given two convex set 𝐶, recall its negative set −𝐶 ≔ 0 − 0𝑥⃗ : 𝑥 ∈ 𝐶 = 0 − 𝑉 . If
𝐶 is convex, then −𝐶 is convex. For example, given a polyhedron 𝑃: 𝐚 [𝑥] ≤ 𝑏, then it is not hard
to find −𝑃: 𝐚 [𝑥] ≥ 𝑏. For any 𝑥, 𝑦 ∈ 𝐶, 𝜃 ∈ [0,1]
where 𝜃0𝑥⃗ + (1 − 𝜃)0𝑦⃗ ∈ 𝑉 since 0𝑥⃗ , 0𝑦⃗ ∈ 𝑉 and 𝑉 is convex, then 𝜃𝑥 + (1 − 𝜃)𝑦 ∈ 0 −
𝑉 = −𝐶 and we see −𝐶 is convex.
4) Property 17-4 Given two convex set 𝐶 , 𝐶 , recall their sum is
𝐶 + 𝐶 ≔ 0 + 0𝑥 ⃗ + 0𝑥 ⃗, 𝑥 ∈ 𝐶 , 𝑥 ∈ 𝐶 = 0+𝑉 +𝑉
𝐶 + 𝐶 is origin dependent, but it is always convex. First note
𝑉 = 𝑉 + 𝑉 = 0𝑥 ⃗ + 0𝑥 ⃗, 𝑥 ∈ 𝐶 , 𝑥 ∈ 𝐶
Then for any 𝑥, 𝑦 ∈ 𝐶 + 𝐶 , 𝑥 = 0 + 0𝑥 ⃗ + 0𝑥 ⃗ for some 𝑥 ∈ 𝐶 , 𝑥 ∈ 𝐶 , and 𝑦 = 0 + 0𝑦⃗ +
0𝑦 ⃗ for some 𝑦 ∈ 𝐶 , 𝑦 ∈ 𝐶 , and
Note this halfspace is the preimages of (−∞, 𝑏) under 𝑓(𝐱) = 𝐚 𝐱. 𝑓(𝐱) is a linear map and hence a
continuous ℝ → ℝ function w.r.t. any norms defined on ℝ and ℝ by EX 23, then the halfspace is an
open set, since the preimages of an open set under a continuous function is also open. Halfspace is a
convex set since 𝐚 (𝜃𝐱 + (1 − 𝜃)𝐲) < 𝜃𝑏 + (1 − 𝜃𝑏) = 𝑏 for any 𝑥, 𝑦 ∈ 𝐻 , 0 ≤ 𝜃 ≤ 1. An example
of such a low open half space is shown below
Figure 17-1 A lower open half space 2𝑥 − 0.5𝑦 + 𝑧 < 2 in ℝ
Similarly, we can define upper open half space 𝑈𝐚, , lower closed half space 𝐿𝐚, , upper closed half
space 𝑈𝐚, as the following, and they are all convex.
𝑈𝐚, = {𝑥 ∈ ℝ , 𝐚 𝐱 > 𝑏}
𝐿𝐚, = {𝑥 ∈ ℝ , 𝐚 𝐱 ≤ 𝑏}
𝑈𝐚, = {𝑥 ∈ ℝ , 𝐚 𝐱 ≥ 𝑏}
Define the (closed) polyhedral 𝑃 = ⋂ 𝐿𝐚 , as a non-empty intersection of finite many lower closed
hyperplanes. Since intersection of convex sets are convex, we know a polyhedral is convex. Halfspaces
and polyhedral can be defined for general affine spaces over field ℝ by technique exactly the same as
in Lemma 16-6, and the definition is affine frame independent.
We should note if a halfspace is “upper” or “lower” depends how we “view” the halfspace. For
example, 𝐚 𝐱 = 𝑏 and −𝐚 𝐱 = −𝑏 stands for the same hyperplane 𝐻, however, {𝑥 ∈ ℝ , 𝐚 𝐱 >
𝑏} is an upper halfspace of 𝐻 in view of “𝐚 𝐱 = 𝑏” a lower halfspace of 𝐻 in view of “−𝐚 𝐱 = −𝑏”.
1.0
Lemma 17-1 A subset 𝐶 of 𝐴 is convex iff the convex combination of any finite many points
𝑝 , … , 𝑝 ∈ 𝐶 is still in 𝐶. Proof is the same as Lemma 16-1. Necessity is trivial; the closedness of
convex combination immediately implies 𝜃𝑥 + (1 − 𝜃)𝑦 ∈ 𝐶 for any 𝑥, 𝑦 ∈ 𝐶 . Sufficiency by
showing that every convex combination can be computed as the convex combination of two
points at a time,
𝜆
𝜆 𝑝 = 𝜆 𝑝 + (1 − 𝜆 ) 𝑝
1−𝜆
∑
where ∑ 𝑝 is another convex combination since ∑ = = 1 and ≥ 0.
A corollary of this lemma is that for any subset 𝑆 ⊆ 𝐴, its convex hull 𝐶(𝑆) is the set of all possible
finite convex combinations, i.e.
Obviously, if we take out any element from above-defined 𝐶(𝑆), it will become a non-convex set
by above lemma.
Property 17-7 With the same proof as Property 16-26, given another affine space 𝐵 and any affine
map 𝒜: 𝐴 → 𝐵, then the image of any convex set 𝐶 of 𝐴 under 𝒜, i.e. 𝒜(𝐶) is convex. As a special
case, ℒ(𝐶) is convex for any linear map ℒ. Conversely, given any convex set 𝐷 in 𝐵, the preimage of 𝐷,
denoted by 𝒜 (𝐷), is also convex. Like the counterexample in Property 16-27, given an arbitrary set
𝑆 in 𝐴, 𝒜(𝑆) being a convex set does not imply 𝑆 is convex.
Property 17-8 Exchangeability of affine map and convex hull operator. For any 𝑆 ⊆ 𝐴, 𝒜(𝐶(𝑆)) =
𝐶 𝒜(𝑆) , analogous to Property 16-27. For any finite many points 𝑝 , … , 𝑝 in 𝑆, we have 𝒜(𝑝 ) ∈
𝒜(𝑆) and by Lemma 17-1 ∑ 𝜆 𝑝 ∈ 𝐶(𝑆) for any convex combination {𝜆 } s.t. 𝜆 ≥ 0, ∑ 𝜆 = 1,
then
𝒜 𝜆𝑝 = 𝜆 𝒜(𝑝 ) ∈ 𝐶 𝒜(𝑆)
Conversely, for any 𝒜(𝑝 ), … , 𝒜(𝑝 ) ∈ 𝒜(𝑆) with 𝑝 , … , 𝑝 as corresponding preimages in 𝑆 (each
𝑝 might have multiple preimage, but just choose one), then for any {𝜆 } s.t. 𝜆 ≥ 0, ∑ 𝜆 = 1 we
have ∑ 𝜆 𝒜(𝑝 ) ∈ 𝐶(𝒜(𝑆)) and
𝜆 𝒜(𝑝 ) = 𝒜 𝜆𝑝 ∈ 𝒜(𝐶(𝑆))
As a special case, for any 𝑆 ⊂ 𝐴, if 𝒜(𝑆) is a convex set in 𝐵, then 𝒜 𝐶(𝑆) = 𝒜(𝑆), simply due
to 𝒜(𝐶(𝑆)) = 𝐶 𝒜(𝑆) = 𝒜(𝑆).
As a corollary, for any point 𝑎 ∈ 𝑆, we have 𝐶 𝑎𝑆⃗ = 𝑎 𝐶(𝑆) ⃗. We define affine function 𝒜: 𝐴 → 𝑉
by 𝒜(𝑝) = 𝑎𝑝⃗, ∀𝑝 ∈ 𝐴. Thus, 𝐶 𝑎𝑆⃗ = 𝐶 𝒜(𝑆) , and 𝑎 𝐶(𝑆) ⃗ = 𝒜(𝐶(𝑆)), and the claim follows
immediately from above property.
Lemma 17-2 Given a set of points 𝑆 = {𝑝 , … , 𝑝 }, 𝐶(𝑆) is the set of all convex combinations of 𝑆.
If 𝑆 is an affine independent set of size 𝑛, then 𝐶(𝑆) is of dimension 𝑛 − 1 and named an 𝑛-
simplex, denoted as Δ . Proof is the same as Theorem 16-3. By Lemma 17-1, a convex set
containing 𝑝 , … , 𝑝 cannot be smaller, otherwise at least one convex combination of 𝑝 , … , 𝑝 is
not in 𝐶(𝑆). We only need to show that 𝐶(𝑆) is a convex set. Let 𝑥, 𝑦 be any two points in 𝐶(𝑆),
then for any 0 ≤ 𝜃 ≤ 1 we have
=𝜃 𝜆 + (1 − 𝜃) 𝜇 = 𝜃+1−𝜃 = 1
0.5
1.0
1.5
2.0
2.0
1.5
affine hull
1.0
convex hull
0.5
0.0
0.5
1.0
1.5
2.0
Figure 17-4 An example of affine hull and convex hull of three points in ℝ
Define the convex hull of two different points 𝑥, 𝑦 ∈ 𝐶 as a line segment, denoted as [𝑥, 𝑦], i.e.
[𝑥, 𝑦] = {𝜃𝑥 + (1 − 𝜃)𝑦: 𝑥, 𝑦 ∈ 𝐶, 𝜃 ∈ [0,1]}. The definition of convex set can be hence re-written as:
a set 𝐶 is convex if for any two points 𝑥, 𝑦 ∈ 𝐶 we have [𝑥, 𝑦] ⊆ 𝐶. Define the relative interior (a more
general concept with the same name will be defined later) of [𝑥, 𝑦] as (𝑥, 𝑦) = {𝜃𝑥 + (1 − 𝜃)𝑦: 𝑥, 𝑦 ∈
𝐶, 𝜃 ∈ (0,1)]} , i.e. (𝑥, 𝑦) = [𝑥, 𝑦]\{𝑥, 𝑦} . Similarly, define [𝑥, 𝑦) = [𝑥, 𝑦]\{𝑦}, (𝑥, 𝑦] = [𝑥, 𝑦]\{𝑥} .
Note the line segment here is an abstract concept – a convex hull of two points, or a Δ – without any
additional machinery like metric.
Lemma 17-4 Given a point 𝑝 ∈ 𝐶 and a line segment [𝑎, 𝑏] s.t. 𝑝 ∈ (𝑎, 𝑏) , then [𝑎, 𝑏] =
(𝑎, 𝑝)⋃(𝑝, 𝑏)⋃{𝑎, 𝑏, 𝑝} = [𝑎, 𝑝]⋃[𝑝, 𝑏] . For any 𝑞 ∈ 𝐶\{𝑎, 𝑏, 𝑝}, we have 𝑞 = 𝜃𝑎 + (1 − 𝜃)𝑏 for
some 0 < 𝜃 < 1, also
1 1 𝜆−1
𝑎 = (𝑝 − (1 − 𝜆)𝑏) = 𝑝 + 𝑏
𝑝 = 𝜆𝑎 + (1 − 𝜆)𝑏 ⇒ 𝜆 𝜆 𝜆
1 1 𝜆
𝑏= (𝑝 − 𝜆𝑎) = 𝑝+ 𝑎
1−𝜆 1−𝜆 𝜆−1
for some 0 < 𝜆 < 1, 𝜆 ≠ 𝜃. Rewrite 𝑞 as affine combination of 𝑎, 𝑝 and 𝑝, 𝑏 respectively and we have
1 𝜆 (1 − 𝜃)𝜆 1−𝜃 𝜃−𝜆 1−𝜃
𝑞 = 𝜃𝑎 + (1 − 𝜃) 𝑝+ 𝑎 = 𝜃+ 𝑎+ 𝑝= 𝑎+ 𝑝
1−𝜆 𝜆−1 𝜆−1 1−𝜆 1−𝜆 1−𝜆
1 𝜆−1 𝜃 𝜃(𝜆 − 1) 𝜃 𝜆−𝜃
𝑞=𝜃 𝑝+ 𝑏 + (1 − 𝜃)𝑏 = 𝑝 + + (1 − 𝜃) 𝑏 = 𝑝 + 𝑏
𝜆 𝜆 𝜆 𝜆 𝜆 𝜆
Note either 𝜆 > 𝜃 or 𝜆 < 𝜃. When 𝜆 > 𝜃 we find 𝑞 is a convex combination of 𝑝, 𝑏 and hence 𝑞 ∈
(𝑝, 𝑏); when 𝜆 < 𝜃, we find 𝑞 is a convex combination of 𝑎, 𝑝 and hence 𝑞 ∈ (𝑎, 𝑝). It is easy to further
verify that 𝜆 < 𝜃 iff 𝑞 ∈ [𝑎, 𝑝) and 𝜆 > 𝜃 iff 𝑞 ∈ (𝑝, 𝑏] (note 𝑞 = 𝑎 ⇒ 𝜃 = 1 ⇒ 𝜆 < 𝜃, and 𝑞 = 𝑏 ⇒ Commented [TC9]: 𝑞 is nearer to 𝑎 than 𝑝, so its
𝜃 = 0 ⇒ 𝜆 > 𝜃). “weight” 𝜃 on 𝑎 should be larger than 𝑝’s “weight” 𝜆.
Lemma 17-5 If 𝑝 ∈ (𝑎, 𝑏), then any 𝑥 ∈ [𝑎, 𝑝), 𝑦 ∈ (𝑝, 𝑏] we have 𝑝 ∈ (𝑥, 𝑦). By Property 16-5 of
affine combination, we can solve the following as if it were a normal equation group,
(𝜃 − 1)𝑥 + (1 − 𝜃 )𝑦
𝑥 = 𝜃 𝑎 + (1 − 𝜃 )𝑏 ⎧ ⎪𝑎 = 𝜃 −𝜃
𝑦 = 𝜃 𝑎 + (1 − 𝜃 )𝑏 ⇒
⎨ 𝜃 𝑥−𝜃 𝑦
𝜃 ≠𝜃 ⎪𝑏 = 𝜃 − 𝜃
⎩
in which 𝑎, 𝑏 are both expressed by 𝑥, 𝑦 by valid affine combinations, then
(𝜃 − 1)𝑥 + (1 − 𝜃 )𝑦 𝜃 𝑥−𝜃 𝑦
𝑝 = 𝜆𝑎 + (1 − 𝜆)𝑏 = 𝜆 + (1 − 𝜆)
𝜃 −𝜃 𝜃 −𝜃
𝜆(𝜃 − 1) + (1 − 𝜆)𝜃 𝜆(1 − 𝜃 ) − (1 − 𝜆)𝜃
= 𝑥+ 𝑦
𝜃 −𝜃 𝜃 −𝜃
𝜃 −𝜆 𝜆−𝜃
= 𝑥+ 𝑦
𝜃 −𝜃 𝜃 −𝜃
By previous lemma, 𝑥 ∈ [𝑎, 𝑝), 𝑦 ∈ (𝑝, 𝑏] ⇒ 𝜃 < 𝜆 < 𝜃 ⇒ > 0, > 0 ⇒ 𝑝 ∈ (𝑥, 𝑦).
Theorem 17-1 Carathéodory’s theorem. For any subset 𝑆 ⊆ 𝐴, if dim 𝐶(𝑆) = 𝑛, then 𝐶(𝑆) can be
written as the following,
(𝜆 + 𝛼𝜇 ) = 𝜆 + 𝛼𝜇 = 𝜆 +𝛼 𝜇 = 𝜆 +𝛼 ×0 = 1
Then the objective is to find some 𝛼 s.t. at least one of 𝜆 + 𝛼𝜇 is zero but 𝜆 + 𝛼𝜇 ≥ 0 for 𝑖 =
1, … , 𝑚 so that ∑ (𝜆 + 𝛼𝜇 )𝑝 is still a convex combination but has only 𝑚 − 1 effective terms with
non-zero coefficients. Such 𝛼 can be found by
𝜆∗ ∗ 𝜆
𝛼=− , 𝑖 = max − :𝜇 > 0
𝜇∗ ,…, 𝜇
Note − : 𝜇 > 0 is non-empty since there must be some positive 𝜇 s as mentioned above. Clearly
𝛼 < 0 and 𝜆 ∗ + 𝛼𝜇 ∗ = 0. Note 𝛼 < 0, thus for 𝜇 ≤ 0, 𝜆 + 𝛼𝜇 ≥ 0 holds; for 𝜇 > 0,
𝜆 𝜆∗ 𝜆 𝜆∗ 𝜆∗ 𝜆∗
− ≤− ⇒ ≥ ⇒𝜆 ≥ 𝜇 ⇒𝜆 − 𝜇 ≥0
𝜇 𝜇∗ 𝜇 𝜇∗ 𝜇∗ 𝜇∗
This implies 𝑎 can be expressed as a convex combination of {𝑝 , … , 𝑝 }\{𝑝 ∗ }, which completes the
proof.
Given an affine space (𝐴, 𝑉), if we define a metric 𝑑 on 𝐴, we have a metric-induced topology on 𝐴. Recall
that given a set of points 𝑆 ⊆ 𝐴 and a distance metric 𝑑, a point 𝑥 in 𝑆 is an interior point if there exists
𝜖 > 0 s.t. 𝐵 (𝑥), which an open ball centered at 𝑥 with radius 𝜖, is a subset of 𝑆.
However, this definition leads to a problem – whether a point is interior depends on the dimension. For
example, every point in an open 2-d ball {(𝑥, 𝑦): 𝑥 + 𝑦 < 1} is interior in ℝ , however they are no
interior in ℝ . Thus, we define the concept of relative interior: 𝑥 is a relative interior point if there exists
𝜖 > 0 s.t. 𝐵 (𝑥)⋂𝐴(𝑆) ⊆ 𝑆, where 𝐴(𝑆) is the affine hull of 𝑆. Thus, for example, every point in an open
2-d ball is relative interior in ℝ . Define the relative interior of 𝑆 as the set of all relative interior points
of 𝑆, denoted as ri 𝑆, and 𝑆 is called relative open if ri 𝑆 = 𝑆. Recall the closure of 𝑆 is 𝑆 union all its limit
points, denoted as 𝑆̅, and 𝐴(𝑆) = 𝐴(𝑆̅) by Property 16-23. The interior of 𝑆 is denoted as int 𝑆. Relative
interior and relative open sets can be viewed as extension of the concept of interior and open sets,
because they equate the latter two if 𝐴(𝑆) is the whole space 𝐴. We focus our discussion on a convex set
𝐶 of 𝐴, and we assume a normed affine space, an affine space whose associated vector space is normed.
Property 17-9 Let 𝒜 be an affine map, then 𝒜(𝑆̅) ⊆ 𝒜(𝑆) for any set 𝑆 (not just convex set). For any
𝑥 ∈ 𝑆̅ , 𝒜(𝑥) ∈ 𝒜(𝑆̅) , and suppose 𝑥 → 𝑥 where 𝑥 ∈ 𝑆, then 𝒜(𝑥 ) → 𝒜(𝑥) by Property 16-35, Commented [TC11]: Affine map is continuous and
thus we have 𝒜(𝑥) is a limit point of {𝒜(𝑥 )} ⊂ 𝒜(𝑆) and 𝒜(𝑥) ∈ 𝒜(𝑆). bounded w.r.t. any norms defined on the affine spaces.
Further, if 𝑆̅ is compact, then 𝒜(𝑆̅) = 𝒜(𝑆). Recall from real
analysis that any sequence in a compact set of a metric-induced
topological space have a convergent subsequence. For any 𝑦 ∈
𝒜(𝑆) , it is a limit of a sequence {𝒜(𝑥 )} ⊂ 𝒜(𝑆) . If 𝑆̅ is
𝑆 = 𝑆̅ compact, then {𝑥 } has a subsequence converging to some limit
1 𝑥 ∈ 𝑆̅ , and {𝒜(𝑥 )} has a subsequence convergent to 𝒜(𝑥)
= (𝑥, 𝑦): 𝑥 ∈ (0,1], 𝑦 ≥
𝑥 since affine map is continuous by Property 16-35. By
uniqueness of limit, 𝑦 = 𝒜(𝑥) ∈ 𝒜(𝑆̅) , implying 𝒜(𝑆) ⊆
𝒜(𝑆̅).
We present a counterexample for a non-compact set 𝑆 that the
equality does not hold. On ℝ , consider 𝒜: ℝ → ℝ defined by
𝒜(𝑆̅) = (0,1] 𝒜(𝑥, 𝑦) = 𝑥 , and let 𝑆 = (𝑥, 𝑦): 𝑥 ∈ (0,1], 𝑦 ≥ , which is
actually the epigraph of 𝑓(𝑥) = , 𝑥 ∈ (0,1] to be introduced
later, then 𝑆̅ = 𝑆 , so 𝒜(𝑆̅) = 𝒜(𝑆) = (0,1] , but 𝒜(𝑆) =
[0,1] ⊃ 𝒜(𝑆̅).
Property 17-10 𝑆 × 𝑆 = 𝑆 × 𝑆 w.r.t. any norm defined on the product space. For any (𝑥, 𝑦) ∈
𝑆 × 𝑆 , let (𝑥 , 𝑦 ) → (𝑥, 𝑦) where {(𝑥 , 𝑦 )} ⊂ 𝑆 × 𝑆 , then {𝑥 } ⊂ 𝑆 , {𝑦 } ⊂ 𝑆 and by
𝑥 → 𝑥, 𝑦 → 𝑦 and thus 𝑥 ∈ 𝑆 , 𝑦 ∈ 𝑆 ⇒ (𝑥, 𝑦) ∈ 𝑆 × 𝑆 . Conversely, for any (𝑥, 𝑦) ∈ 𝑆 × 𝑆 , let
𝑥 → 𝑥, 𝑦 → 𝑦 where {𝑥 } ⊂ 𝑆 , {𝑦 } ⊂ 𝑆 , and thus (𝑥 , 𝑦 ) → (𝑥, 𝑦); since (𝑥 , 𝑦 ) ∈ 𝑆 × 𝑆 for
every 𝑘, we have (𝑥, 𝑦) ∈ 𝑆 × 𝑆 .
Property 17-11 𝑆 + 𝑆 ⊆ 𝑆 + 𝑆 for any 𝑆 , 𝑆 ⊆ 𝔸 . We use 𝑆 × 𝑆 = 𝑆 × 𝑆 for any two sets
𝑆 , 𝑆 , and use Property 17-9. Also for any 𝑆 , 𝑆 ⊆ 𝔸 . Define linear map ℒ: 𝐹 → 𝐹 as ℒ(𝑥, 𝑦) =
𝑥 + 𝑦 for any (𝑥, 𝑦) ∈ 𝐹 , then it is easy to verify ℒ(𝑆 × 𝑆 ) = 𝑆 + 𝑆 . Check that
𝑆 + 𝑆 = ℒ(𝑆 × 𝑆 ) = ℒ(𝑆 × 𝑆 ) ⊆ ℒ(𝑆 × 𝑆 ) = 𝑆 + 𝑆 Commented [TC12]: This is application of Property 73.
If at least one of 𝑆 , 𝑆 are bounded, then 𝑆 + 𝑆 = 𝑆 + 𝑆 . WLOG suppose 𝑆 is bounded, then for any 𝑧 ∈
𝑆 + 𝑆 , there exists 𝑥 + 𝑦 → 𝑧 where {𝑥 } ⊂ 𝑆 , {𝑦 } ⊂ 𝑆 . Since 𝑆 is bounded, then {𝑥 } is bounded, and
hence {𝑦 } is bounded, and (𝑥 , 𝑦 ) has a convergent subsequence 𝑥 , 𝑦 → (𝑥, 𝑦) for some 𝑥 ∈ 𝑆 , 𝑦 ∈ Commented [TC13]: {𝑥 + 𝑦 } is bounded since it is
convergent, the {𝑥 } being bounded implies {𝑦 } being
𝑆 , thus by 𝑥 → 𝑥, 𝑦 →𝑦 ⇒𝑥 +𝑦 → 𝑥 + 𝑦, which implies 𝑧 = 𝑥 + 𝑦 (a subsequence, if it converges, bounded.
must converges to the limit of the whole sequence). This means 𝑧 ∈ 𝑆 + 𝑆 and completes the proof.
Property 17-12 𝑆 ⋂𝑆 ⊆ 𝑆 ⋂𝑆 . For any 𝑥 ∈ 𝑆 ⋂𝑆 , let 𝑥 → 𝑥 where {𝑥 } ⊂ 𝑆 ⋂𝑆 , then {𝑥 } ⊂
𝑆 ⇒ 𝑥 ∈ 𝑆 , and {𝑥 } ⊂ 𝑆 ⇒ 𝑥 ∈ 𝑆 , implying 𝑥 ∈ 𝑆 ⋂𝑆 .
However, 𝑆 ⋂𝑆 ≠ 𝑆 ⋂𝑆 in general. Consider 𝑆 = (0,1) and 𝑆 = (1,2) , then 𝑆 ⋂𝑆 = ∅ , but
𝑆 ⋂𝑆 = {1}. Property 17-22 proves the equality holds for two convex sets when they have non-empty
interior intersection.
Theorem 17-2 Line segment principal. Let 𝐶 be a non-empty convex set, if 𝑥 ∈ ri 𝐶 and 𝑦 ∈ 𝐶̅ , then
[𝑥, 𝑦) ⊆ ri 𝐶. In other words, for any line segments in 𝐶̅ , if one of the points is in ri 𝐶, then at most the
other end point is not in ri 𝐶 (but at the boundary of 𝐶).
By definition, ∃𝜖 > 0 s.t. 𝐵 (𝑥)⋂𝐴(𝐶) ⊆ 𝐶 . If 𝑦 ∈ 𝐶 as well,
𝑦 then for any 𝜃 ∈ (0,1) and 𝑧 = 𝜃𝑥 + (1 − 𝜃)𝑦, draw a ball
𝐵 (𝑧 ) , and ∀𝑝 ∈ 𝐵 (𝑧 ) we have 𝑝 = 𝑧 + 𝜃𝐰 for some
‖𝐰‖ < 𝜖, then 𝑥 + 𝐰 ∈ 𝐵 (𝑥)⋂𝐴(𝐶) and
𝑝 𝜃(𝑥 + 𝐰) + (1 − 𝜃)𝑦 = 𝑝 ⇒ 𝑝 ∈ 𝐶 Commented [TC14]: 𝜃(𝑥 + 𝐰) + (1 − 𝜃)𝑦 = 𝜃𝑥 +
𝑧 ⇒ 𝐵 (𝑧 )⋂𝐴(𝐶) ⊆ 𝐶 ⇒ 𝑧 ∈ ri 𝐶 (1 − 𝜃)𝑦 + 𝜃𝐰 = z + 𝜃𝐰 = 𝑝. This is consistent with the
𝑥+𝐰 geometric property of a triangle: [𝑝, 𝑧 ] is parallel with
Now for the general case of 𝑦 ∈ 𝐶̅ , let 𝑦 → 𝑦 where {𝑦 } ⊂ 𝐶. [𝑥, 𝑥 + 𝑤], then [
[ , ]
=[
[ , ]
=
[ , ]
= 𝜃.
𝑥 For any 𝜃, let 𝑧 , = 𝜃𝑥 + (1 − 𝜃)𝑦 , then 𝑧 , → 𝑧 as 𝑦 → 𝑦. , 𝐰] , 𝐰] [ . ]
A direct corollary is that ri 𝐶 is convex when 𝐶 is convex. For any two points 𝑥, 𝑦 ∈ ri 𝐶, we have both
(𝑥, 𝑦] ⊆ ri 𝐶 , [𝑥, 𝑦) ∈ ri 𝐶 ⇒ [𝑥, 𝑦] ∈ ri 𝐶, and thus ri 𝐶 is convex.
Theorem 17-3 Let 𝐶 be a non-empty convex set, then ri 𝐶 ≠ ∅. If 𝐶 is a singleton, then 𝐴(𝐶) = 𝐶 and
it is easy to see the single point in 𝐶 is the sole interior point.
If 𝐶 contains at least two points, then dim 𝐴(𝐶) = 𝑛 ≥ 1. The strategy is to construct a non-empty
relative open set 𝐺 in 𝐶 s.t. every point in 𝐺 is an interior point of 𝐶. Let 𝑝 , … , 𝑝 be the maximum
affine independent set of 𝐶, then let
where 𝑝 is chosen as the origin and 𝑝 𝑝⃗ form a basis of the induced vector space of 𝐴(𝐶). Note
Λ = (𝜆 , … , 𝜆 ): 𝜆 < 1, 𝜆 ∈ (0,1)
can be easily verified as an open set in 𝐹 w.r.t. any norm. Now for any 𝑥 ∈ 𝐺, where 𝑥 has a vector
coordinate 𝛌 w.r.t. origin 𝑝 , then for another point 𝑦 ∈ 𝐴(𝐶) sufficiently close to 𝑥, its coordinate
𝛌 would be sufficiently close to 𝛌 by Lemma 16-7 and Theorem 3-10, so that 𝛌 ∈ Λ, which in turn
gives 𝑦 ∈ 𝐺, then 𝐺 is relative open w.r.t. 𝐴(𝐶), which proves our original claim.
For above proof, recall that a set 𝐺 is open means for any point 𝑥 ∈ 𝐺 , if another point 𝑦 is
sufficiently close to 𝑥 w.r.t. to the defined metric, then we must have 𝑦 ∈ 𝐺; this is formalized as
the equivalent open ball argument. For a set 𝐺 open relative to another set 𝐻, it means for any
point 𝑥 ∈ 𝐺, if another point 𝑦 ∈ 𝐻 is sufficiently close to 𝑥 w.r.t. to the defined metric, then we
must have 𝑦 ∈ 𝐺.
.
Property 17-13 As a corollary, we have 𝐴(ri 𝐶) = 𝐴(𝐶). Let 𝑥 = 0.6𝑝 + ∑ 𝑝 ∈ 𝐺 and 𝑦 =
.
0.2𝑝 + ∑ 𝑝 ∈ 𝐺 , then 𝑝 = 2𝑥 − 𝑦 ∈ 𝐴(𝐺). Similarly, 𝑝 ∈ 𝐴(𝐺) for any 𝑖 = 0, … , 𝑛, and
this means 𝐴(𝐺) contains the maximum affine independent set 𝑝 , … , 𝑝 of 𝐶, so 𝐴 𝐴(𝐺) = 𝐴(𝐺) =
𝐴(𝐶) by Property 16-16. Since 𝐺 ⊆ ri 𝐶 ⊆ 𝐶 ⇒ 𝐴(𝐺) ⊆ 𝐴(ri 𝐶) ⊆ 𝐴(𝐶) , then 𝐴(𝐺) = 𝐴(ri 𝐶) = Commented [TC16]: 𝐺 and 𝐶 share a same maximum
𝐴(𝐶). affine independent set, then 𝐴(𝐺) = 𝐴(𝐶) are both span of
that maximum affine independent set.
Theorem 17-4 Let 𝐶 be a non-empty convex set. A point 𝑥 is a relative interior point of 𝐶 iff ∀𝑦 ∈ 𝐶,
there exists 𝜃 > 0 s.t. 𝑥 − 𝜃𝑥𝑦⃗ ∈ 𝐶. In plain words, for any 𝑥, 𝑦 ∈ 𝐶, then there exists a sufficient small
movement of 𝑥 in the direction of 𝑦𝑥⃗ (the opposite direction of 𝑥𝑦⃗) s.t. it keeps 𝑥 in 𝐶. Note it is not
𝑥 + 𝜃𝑥𝑦⃗ , e.g. on interval [0,1] , 𝑥 = 0 satisfies ∀𝑦 ∈ [0,1], ∃𝜃 > 0, 𝑥 + 𝜃𝑥𝑦⃗ ∈ 𝐶 , but 𝑥 = 0 is a
boundary point.
Necessity. If 𝑥 ∈ ri 𝐶, clearly there exists a sufficiently small 𝜃 < ‖ s.t. 𝑥 − 𝜃𝑥𝑦⃗ lies in an open ball
⃗‖
𝐵 (𝑥) ⊆ ri 𝐶.
Sufficiency. Let 𝑥 be a point in 𝐶 s.t. ∃𝜃 > 0 s.t. 𝑧 = 𝑥 − 𝜃𝑥𝑦⃗ ∈ 𝐶, and let 𝑦 ∈ ri 𝐶. If 𝑥 = 𝑦 then we
are done. If 𝑦 ≠ 𝑥, then 𝑧 = 𝑥 + 𝜃(𝑥 − 𝑦) ⇒ 𝑥 = 𝑧+ 𝑦, where again we note 𝑧 ∈ 𝐶 and 𝑦 ∈
ri 𝐶, then by line segment principal Theorem 17-2, we have 𝑥 ∈ ri 𝐶.
Property 17-14 As a corollary, an affine subspace is always relative open to itself, i.e. ri 𝐴 = 𝐴 for
any affine subspace 𝐴 of 𝐴. Recall an affine subspace is a convex set. For any 𝑥, 𝑦 ∈ 𝐴 , 𝑥 − 𝜃𝑥𝑦⃗ =
𝑥 − 𝜃(𝑦 − 𝑥) = (1 + 𝜃)𝑥 − 𝜃𝑦 ∈ 𝐴 since it is an affine combination, then ri 𝐴 = 𝐴 since 𝑥 is an
arbitrary point in 𝐴 .
In general, some seemingly intuitive properties like “int 𝑆̅ = int 𝑆”, “𝑆̅ = ınt 𝑆”, “int 𝑆 = int 𝑆 ⟺ 𝑆 =
𝑆 ” do not hold. For example, on [0,1] ⊆ ℝ, the set of rational numbers ℚ[ , ] has empty interior, but
ℚ[ , ] = [0,1], then
int ℚ[ , ] = (0,1) ≠ ∅ = int ℚ[ , ]
ℚ[ , ] = [0,1] ≠ ∅ = ınt ℚ[ , ]
int ℚ ,
⋃ ,1 = , 1 = int , 1 but ℚ ,
⋃ , 1 = [0,1] ≠ ,1
We prove that such properties will hold for convex sets. In following discussion, letter 𝐶, 𝐶 , 𝐶 , etc.
denotes convex sets in 𝔸 = (𝐴, 𝑉), unless otherwise stated.
Property 17-15 If 𝐶 ⊆ 𝐶 and dim 𝐶 = dim 𝐶 , then ri 𝐶 ⊆ ri 𝐶 . Note 𝐴(𝐶 ) = 𝐴(𝐶 ) since they
share some maximum affine independent set of 𝐶 , and so ∀𝑥 ∈ ri 𝐶 , ∃𝐵 (𝑥)⋂𝐴(𝐶 ) =
𝐵 (𝑥)⋂𝐴(𝐶 ) ⊆ 𝐶 ⊆ 𝐶 . However, if dim 𝐶 < dim 𝐶 , then the claim is false. Consider line segment
[(0,0), (0,1)] in ℝ whose relative interior is (0,0), (0,1) , but all points in the open interval are
boundary points of 𝐶([(0,0), (0,1)]⋃[(0,1), (1,1)]).
Property 17-16 ri(𝐶 × 𝐶 ) = ri 𝐶 × ri 𝐶 . Let 𝑥 ∈ ri(𝐶 ) , 𝑦 ∈ ri(𝐶 ), then ∃𝜆, 𝜇 > 0 s.t. 𝑧 = 𝑥 −
𝜆𝑥𝑥⃗ ∈ 𝐶 and 𝑧 = 𝑦 − 𝜆𝑦𝑦⃗ ∈ 𝐶 for any 𝑥 ∈ 𝐶 , 𝑦 ∈ 𝐶 by Theorem 17-4, then
𝐶 (𝑥) 𝐶
𝐶
𝑥
𝔸
Note 𝐶 = {𝑥: ∃𝑦, (𝑥, 𝑦) ∈ 𝐶}. Let 𝒜 be the projection of points in 𝔸 × 𝔹 onto 𝔸, then it is easy
to verify 𝒜(𝐶) = 𝐶 . By Property 17-20 we can exchange the projection and interior operator,
and find
ri 𝐶 = ri 𝒜(𝐶) = 𝒜(ri 𝐶) = {𝑥: ∃𝑦, (𝑥, 𝑦) ∈ ri 𝐶}
Let 𝑀 = {𝑥} × 𝐵 = {(𝑥, 𝑦): 𝑦 ∈ 𝐵} for any 𝑥 ∈ ri 𝐶 , then the projection of 𝑀 ⋂𝐶 on 𝔹 is
clearly 𝐶 (𝑥) since
𝑀 ⋂𝐶 = ({𝑥} × 𝐵)⋂({𝑥} × 𝐶 ) = {𝑥} × 𝐶 (𝑥) = {(𝑥, 𝑦): 𝑦 ∈ 𝐶 (𝑥)}
Note 𝑀 is an affine subspace so ri 𝑀 = 𝑀 by Property 17-14, then
ri(𝑀 ⋂𝐶) = ri 𝑀 ⋂ ri 𝐶 = 𝑀 ⋂ ri 𝐶 = {(𝑥, 𝑦): 𝑦 ∈ ri 𝐶 (𝑥)}
⇒ 𝑦 ∈ ri 𝐶 (𝑥) ⟺ (𝑥, 𝑦) ∈ ri(𝑀 ⋂𝐶) , ∀𝑥 ∈ ri 𝐶
Further verify that ri 𝐶 = ⋃ ∈ (𝑀 ⋂ ri 𝐶) . (𝑥, 𝑦) ∈ ri 𝐶 means 𝑥 ∈ ri 𝐷 and so (𝑥, 𝑦) ∈ 𝑀 ,
thus (𝑥, 𝑦) ∈ 𝑀 ⋂ ri 𝐶 ; conversely (𝑥, 𝑦) ∈ ⋃ ∈ (𝑀 ⋂ ri 𝐶) directly implies (𝑥, 𝑦) ∈ ri 𝐶 .
Finally,
Cone
A point in the conical hull of a convex set can be written in a much simpler form.
Any direction in the hyperplane is in the linearity space of the two halfspaces.
Vector sum of two polyhedrons is a polyhedron.
Sum of two cone is a cone.
Intersection of two cones is a cone.
If a hyperplane contains the entire relative boundary of a convex set 𝐶, and either it contains entire 𝐶, or
it contains 𝐶 in one closed halfspace. Further, it one of its closed halfspace contains some point in ri 𝐶,
then 𝐶 is entirely contained in that halfspace.