Analysis of Algorithms

CS 477/677

Lecture 10
Instructor: Monica Nicolescu
 The Heap Data Structure
• Def: A heap is a nearly complete binary tree with
the following two properties:
– Structural property: all levels are full, except
possibly the last one, which is filled from left to right
– Order (heap) property: for any node x
Parent(x) ≥ x

It doesn’t matter that 4 in

7 4
level 1 is smaller than 5 in
5 2 level 2
Heap
CS 477/677 - Lecture 10 2
 Maintaining the Heap Property
• Suppose a node is smaller than a
child
– Left and Right subtrees of i are max-heaps
• Invariant:
– the heap condition is violated only at that
node
• To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller than
children

CS 477/677 - Lecture 10 3
 Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
1

4
2 3

1 3
4 5 6 7

8
2 9 10
16 9 10
14 8 7

A: 4 1 3 2 16 9 10 14 8 7

CS 477/677 - Lecture 10 4
 Heapsort
• Goal:
– Sort an array using heap representations

• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains

CS 477/677 - Lecture 10 5
 HEAP-EXTRACT-MAX
Goal:
– Extract the largest element of the heap (i.e., return the max
value and also remove that element from the heap
Idea:
– Exchange the root element with the last
– Decrease the size of the heap by 1 element
– Call MAX-HEAPIFY on the new root, on a heap of size n-1

Heap A: Root is the largest element

CS 477/677 - Lecture 10 6
 HEAP-INCREASE-KEY
• Goal:
– Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper
place for the newly increased key
16

14 10
8 7 9 3
i
Key [i] ← 15 2 4 1

CS 477/677 - Lecture 10 7
 MAX-HEAP-INSERT
• Goal:
16
– Inserts a new element into a max-
heap 14 10
8 7 9 3
• Idea:
2 4 1 -
– Expand the max-heap with a new
16
element whose key is -
– Calls HEAP-INCREASE-KEY to 14 10

set the key of the new node to its 8 7 9 3

correct value and maintain the 2 4 1 15

max-heap property
CS 477/677 - Lecture 10 8
 The Search Problem
• Find items with keys matching a given search key
• Dictionary = data structure that supports two basic
operations: insert a new item and return an item with a
given key
• Queries: return information about the set
– Search (S, k)
– Minimum (S), Maximum (S)
– Successor (S, x), Predecessor (S, x)
• Modifying operations: change the set
– Insert (S, k)
– Delete (S, k)

CS 477/677 - Lecture 10 9
 Binary Search Trees
• Tree representation:
– A linked data structure in which
each node is an object
• Node representation: parent
L R
– Key field key data
– Satellite data
– Left: pointer to left child
Left child Right child
– Right: pointer to right child
– p: pointer to parent (p [root [T]] =
NIL)
• Satisfies the binary-search-tree
property
CS 477/677 - Lecture 10 10
 Binary Search Tree Example

• Binary search tree property:

– If y is in left subtree of x, 5

then key [y] ≤ key [x]

3 7
2 5 9
– If y is in right subtree of x,
then key [y] ≥ key [x]

CS 477/677 - Lecture 10 11
 Searching for a Key
• Given a pointer to the root of a tree and a key k:
5
– Return a pointer to a node with key k
if one exists 3 7
– Otherwise return NIL 2 4 9

• Idea
– Starting at the root: trace down a path by comparing k with the
key of the current node:
• If the keys are equal: we have found the key
• If k < key[x] search in the left subtree of x
• If k > key[x] search in the right subtree of x

CS 477/677 - Lecture 10 12
 Successor
Def: successor (x ) = y, such that key [y] is the
smallest key > key [x] 15
• E.g.: successor (15)17=
successor (13)15
= 6 18

successor (9) =
13 3 7 17 20
2 4 13
• Case 1: right (x) is non empty 9

– successor (x ) = the minimum in right (x)

• Case 2: right (x) is empty
– go up the tree until the current node is a left child:
successor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the largest element
CS 477/677 - Lecture 10 13
 Finding the Successor
Alg: TREE-SUCCESSOR(x)
1. if right [x]  NIL
2. then return TREE-MINIMUM(right [x])
3. y ← p[x] 15
4. while y  NIL and x = right [y] y
6 18
5. do x ← y
3 7 17 20
6. y ← p[y] x
2 4 13
7. return y 9

Running time: O (h), h – height of the tree

CS 477/677 - Lecture 10 14
 Predecessor
Def: predecessor (x ) = y, such that key [y] is the
biggest key < key [x]
• E.g.: predecessor (15)13
= 15

predecessor (9) =7 6 18
predecessor (13)9= 3 7 17 20
2 4 13
• Case 1: left (x) is non empty 9
– predecessor (x ) = the maximum in left (x)
• Case 2: left (x) is empty
– go up the tree until the current node is a right child:
predecessor (x ) is the parent of the current node
– if you cannot go further (and you reached the root):
x is the smallest element
CS 477/677 - Lecture 10 15
 Insertion
• Goal:
– Insert value v into a binary search tree
• Idea:
– If key [x] < v move to the right child of x,
Insert value 13
else move to the left child of x
12
– When x is NIL, we found the correct position
– If v < key [y] insert the new node as y’s left child 5 18
else insert it as y’s right child 2 9 15 19
1 3 13 17
– Begining at the root, go down the tree and maintain:
• Pointer x : traces the downward path (current node)
• Pointer y : parent of x (“trailing pointer” )

CS 477/677 - Lecture 10 16
 Example: TREE-INSERT
x, y=NIL y
Insert 13: 12 12
x
5 18 5 18
2 9 15 19 2 9 15 19
1 3 17 1 3 17

12 12

x y
5 18 5 18
2 9 15 19 2 9 15 19
1 3 17 1 3 13 17
x = NIL
y = 15
CS 477/677 - Lecture 10 17
 Alg: TREE-INSERT(T, z)
1. y ← NIL
2. x ← root [T]
12
3. while x ≠ NIL
4. do y ← x 5 18
5. if key [z] < key [x]
2 9 15 19
6. then x ← left [x]
1 3 13 17
7. else x ← right [x]
8. p[z] ← y
9. if y = NIL
10. then root [T] ← z Tree T was empty
11. else if key [z] < key [y]
12. then left [y] ← z
13. else right [y] ← z Running time: O(h)
CS 477/677 - Lecture 10 18
 Deletion
• Goal:
– Delete a given node z from a binary search tree
• Idea:
– Case 1: z has no children
• Delete z by making the parent of z point to NIL, instead of to z
15 15

5 16 5 16

3 12 20 3 12 20
z
10 13 18 23 10 18 23

6 delete 6

7 CS 477/677 - Lecture 10 7 19
 Deletion
• Case 2: z has one child
– Delete z by making the parent of z point to z’s child,
instead of to z
– Update the parent of z’s child to be z’s parent
15 delete 15
z
5 16 5 20

3 12 20 3 12 18 23

10 13 18 23 10

6 6

7 7

CS 477/677 - Lecture 10 20
 Deletion
• Case 3: z has two children
– z’s successor (y) is the minimum node in z’s right subtree
– y has either no children or one right child (but no left child)
– Delete y from the tree (via Case 1 or 2)
– Replace z’s key and satellite data with y’s.
6 15 15
delete z
5 16 6 16

3 12 20 3 12 20

10 13 18 23 10 13 18 23

y 6 7

7 CS 477/677 - Lecture 10 21
 Idea for TREE-DELETE(T, z)
• Determine a node y that has to be deleted
– If z has only 1 child  y = z (case 2)
– If z has 2 children  y = TREE-SUCCESSOR(z) (case 3)
– In any case y has at most 1 child!!!
• Set a node x to the non-nil child of y
• Delete node y: set the parent of x to be the parent of y
• If the y is the root x becomes the new root
otherwise, update parent pointers accordingly
• If the deleted node was the successor of z: move y’s key
and satellite data onto z
• The deleted node y is returned for recycling

CS 477/677 - Lecture 10 22
 TREE-DELETE(T, z)
1. if left[z] = NIL or right[z] = NIL
2. then y ← z z has one child

3. else y ← TREE-SUCCESSOR(z) z has 2 children

4. if left[y]  NIL 15 y

5. then x ← left[y] 5 16
x
3 12 20
6. else x ← right[y]
10 13 18 23
7. if x  NIL 6

8. then p[x] ← p[y] 7

CS 477/677 - Lecture 10 23
 TREE-DELETE(T, z) – cont.
9. if p[y] = NIL 15 y

5 16
10. then root[T] ← x x
3 12 20
11. else if y = left[p[y]]
10 13 18 23
12. then left[p[y]] ← x
6
13. else right[p[y]] ← x 7
14. if y  z
15. then key[z] ← key[y]
16. copy y’s satellite data into z
17. return y
CS 477/677 - Lecture 10 24
 Binary Search Trees - Summary
• Operations on binary search trees:
– SEARCH O(h)
– PREDECESSOR O(h)
– SUCCESOR O(h)
– MINIMUM O(h)
– MAXIMUM O(h)
– INSERT O(h)
– DELETE O(h)
• These operations are fast if the height of the tree
is small – otherwise their performance is similar
to that of a linked list
CS 477/677 - Lecture 10 25
 Red-Black Trees
• “Balanced” binary trees guarantee an O(lgn)
running time on the basic dynamic-set
operations
• Red-black-tree
– Binary tree with an additional attribute for its nodes:
color which can be red or black
– Constrains the way nodes can be colored on any path
from the root to a leaf
• Ensures that no path is more than twice as long as another
 the tree is balanced
– The nodes inherit all the other attributes from the
binary-search trees: key, left, right, p
CS 477/677 - Lecture 10 26
 Red-Black-Trees Properties
1. Every node is either red or black
2. The root is black
3. Every leaf (NIL) is black
4. If a node is red, then both its children are black
• No two red nodes in a row on a simple path from the
root to a leaf
5. For each node, all paths from the node to
descendant leaves contain the same number of
black nodes
CS 477/677 - Lecture 10 27
 Example: RED-BLACK-TREE
26

17 41

NIL NIL
30 47

NIL 38 NIL 50

NIL NIL NIL NIL

• For convenience we use a sentinel NIL[T] to represent all

the NIL nodes at the leafs
– NIL[T] has the same fields as an ordinary node
– Color[NIL[T]] = BLACK
– The other fields may be set to arbitrary values
CS 477/677 - Lecture 10 28
 Black-Height of a Node
h=4
26 bh = 2

h=1 h=3
bh = 1 17 41 bh = 2

NIL NIL h=2 h=2

30 bh = 1 47 bh = 1
h=1
bh = 1
h=1
NIL 38 NIL 50 bh = 1

NIL NIL NIL NIL

• Height of a node: the number of edges in a longest

path to a leaf
• Black-height of a node x: bh(x) is the number of
black nodes (including NIL) on the path from x to leaf, not
counting x CS 477/677 - Lecture 10 29
 Properties of Red-Black-Trees
• Claim
– Any node with height h has black-height ≥ h/2
• Proof
– By property 4, at most h/2 red nodes on the path from
the node to a leaf
– Hence at least h/2 are black
26

17 41

30 47
Property 4: if a node is
red then both its
38 50
children are black
CS 477/677 - Lecture 10 30
 Properties of Red-Black-Trees
Claim
– The subtree rooted at any node x contains at least
2bh(x) - 1 internal nodes
Proof: By induction on height of x
x
Basis: height[x] = 0 
NIL
x is a leaf (NIL[T]) 
bh(x) = 0 
# of internal nodes: 20 - 1 = 0

CS 477/677 - Lecture 10 31
 Properties of Red-Black-Trees
Inductive step:
• Let height(x) = h and bh(x) = b
• Any child y of x has:
– bh (y) = b (if the child is red), or
– bh (y) = b - 1 (if the child is black)

26

17 41

30 47

38 50

CS 477/677 - Lecture 10 32
 Properties of Red-Black-Trees
• Want to prove:
– The subtree rooted at any node x contains at
x
least 2bh(x) - 1 internal nodes
• Internal nodes for each child of x:
l r
2bh(x) - 1 - 1
• The subtree rooted at x contains at least:
(2bh(x) - 1 – 1) + (2bh(x) - 1 – 1) + 1 =
2 · (2bh(x) - 1 - 1) + 1 =
2bh(x) - 1 internal nodes

CS 477/677 - Lecture 10 33
 Properties of Red-Black-Trees
Lemma: A red-black tree with n internal nodes has
height at most 2lg(n + 1). height(root) = h root
bh(root) = b
Proof:
n ≥ 2b - 1 ≥ 2h/2 - 1 l r
number n
of internal since b  h/2
nodes

• Add 1 to both sides and then take logs:

n + 1 ≥ 2b ≥ 2h/2
lg(n + 1) ≥ h/2 
h ≤ 2 lg(n + 1)
CS 477/677 - Lecture 10 34
 Operations on Red-Black-Trees

• The non-modifying binary-search-tree operations

MINIMUM, MAXIMUM, SUCCESSOR,
PREDECESSOR, and SEARCH run in O(h) time
– They take O(lgn) time on red-black trees

• What about TREE-INSERT and TREE-DELETE?

– They will still run on O(lgn)

– We have to guarantee that the modified tree will still be

a red-black tree
CS 477/677 - Lecture 10 35
 INSERT
INSERT: what color to make the new node?
• Red? Let’s insert 35!
– Property 4: if a node is red, then both its children are
black
• Black? Let’s insert 14!
– Property 5: all paths from a node to its leaves contain
the same number of black nodes
26

17 41

30 47

38 50
CS 477/677 - Lecture 10 36
 26
DELETE 17 41

DELETE: what color was the 30 47

node that was removed? Red? 38 50

1. Every node is either red or black OK!
2. The root is black OK!
3. Every leaf (NIL) is black OK!
4. If a node is red, then both its children are black
OK! Does not change OK! Does not create
any black heights two red nodes in a row

5. For each node, all paths from the node to descendant

leaves contain the same number of black nodes

CS 477/677 - Lecture 10 37
 26
DELETE 17 41

DELETE: what color was the 30 47

node that was removed? Black? 38 50

1. Every node is either red or black OK!
2. The root is black Not OK! If removing
the root and the child
3. Every leaf (NIL) is black OK! that replaces it is red
4. If a node is red, then both its children are black
Not OK! Could change the Not OK! Could create
black heights of some nodes two red nodes in a row

5. For each node, all paths from the node to descendant

leaves contain the same number of black nodes

CS 477/677 - Lecture 10 38
 Readings
• Chapter 12

CS 477/677 - Lecture 10 39