2.3.

Big-O Notation Problem Solving with Algorithms and Data

6/11/17,
Str... 2:56 PM

2.3. Big-O Notation

When trying to characterize an algorithms eff iciency in terms of execution time, independent of any
particular program or computer, it is important to quantif y the number of operations or steps that the
algorithm w ill require. If each of these steps is considered to be a basic unit of computation, then the
execution time f or an algorithm can be expressed as the number of steps required to solve the problem.
Deciding on an appropriate basic unit of computation can be a complicated problem and w ill depend on
how the algorithm is implemented.

A good basic unit of computation f or comparing the summation algorithms show n earlier might be to
count the number of assignment statements perf ormed to compute the sum. In the f unction sumOfN , the
number of assignment statements is 1 ( theSum = 0 ) plus the value of n (the number of times w e
perf orm theSum = theSum + i ). We can denote this by a f unction, call it T, w here T (n) = 1 + n.
The parameter n is of ten ref erred to as the size of the problem, and w e can read this as T(n) is the
time it takes to solve a problem of size n, namely 1+n steps.

In the summation f unctions given above, it makes sense to use the number of terms in the summation to
denote the size of the problem. We can then say that the sum of the f irst 100,000 integers is a bigger
instance of the summation problem than the sum of the f irst 1,000. Because of this, it might seem
reasonable that the time required to solve the larger case w ould be greater than f or the smaller case.
Our goal then is to show how the algorithms execution time changes w ith respect to the size of the
problem.

Computer scientists pref er to take this analysis technique one step f urther. It turns out that the exact
number of operations is not as important as determining the most dominant part of the T (n) f unction. In
other w ords, as the problem gets larger, some portion of the T (n) f unction tends to overpow er the
rest. This dominant term is w hat, in the end, is used f or comparison. The orde r of m agnitude f unction
describes the part of T (n) that increases the f astest as the value of n increases. Order of magnitude
is of ten called Big-O notation (f or order) and w ritten as O(f(n)). It provides a usef ul approximation
to the actual number of steps in the computation. The f unction f(n) provides a simple representation of
the dominant part of the original T (n).

In the above example, T (n) = 1 + n. As n gets large, the constant 1 w ill become less and less
signif icant to the f inal result. If w e are looking f or an approximation f or T (n), then w e can drop the 1
and simply say that the running time is O(n) . It is important to note that the 1 is certainly signif icant f or
T (n). How ever, as n gets large, our approximation w ill be just as accurate w ithout it.
As another example, suppose that f or some algorithm, the exact number of steps is
T (n) = 5n2 + 27n + 1005. When n is small, say 1 or 2, the constant 1005 seems to be the dominant
part of the f unction. How ever, as n gets larger, the n2 term becomes the most important. In f act, w hen
n is really large, the other tw o terms become insignif icant in the role that they play in determining the
f inal result. Again, to approximate T (n) as n gets large, w e can ignore the other terms and f ocus on
5n2 . In addition, the coeff icient 5 becomes insignif icant as n gets large. We w ould say then that the
f unction T (n) has an order of magnitude f(n) = n2 , or simply that it is O(n2 ).

Although w e do not see this in the summation example, sometimes the perf ormance of an algorithm
depends on the exact values of the data rather than simply the size of the problem. For these kinds of
algorithms w e need to characterize their perf ormance in terms of best case, w ors t cas e , or ave rage
cas e perf ormance. The w orst case perf ormance ref ers to a particular data set w here the algorithm

1 of 4
2.3. Big-O Notation Problem Solving with Algorithms and Data
6/11/17,
Str... 2:56 PM

perf orms especially poorly. Whereas a diff erent data set f or the exact same algorithm might have
extraordinarily good perf ormance. How ever, in most cases the algorithm perf orms somew here in
betw een these tw o extremes (average case). It is important f or a computer scientist to understand
these distinctions so they are not misled by one particular case.

A number of very common order of magnitude f unctions w ill come up over and over as you study
algorithms. These are show n in Table 1. In order to decide w hich of these f unctions is the dominant part
of any T (n) f unction, w e must see how they compare w ith one another as n gets large.

f(n) Nam e

1 Constant

log n Logarithmic

n Linear

n log n Log Linear

n2 Quadratic

n3 Cubic

2n Exponential

Figure 1 show s graphs of the common f unctions f rom Table 1. Notice that w hen n is small, the
f unctions are not very w ell def ined w ith respect to one another. It is hard to tell w hich is dominant.
How ever, as n grow s, there is a def inite relationship and it is easy to see how they compare w ith one
another.

2 of 4
2.3. Big-O Notation Problem Solving with Algorithms and Data
6/11/17,
Str... 2:56 PM

As a f inal example, suppose that w e have the f ragment of Python code show n in Listing 2. Although
this program does not really do anything, it is instructive to see how w e can take actual code and
analyze perf ormance.

Lis ting 2

a=5
b=6
c=10
for i in range(n):
for j in range(n):
x = i * i
y = j * j
z = i * j
for k in range(n):
w = a*k + 45
v = b*b
d = 33

The number of assignment operations is the sum of f our terms. The f irst term is the constant 3,
representing the three assignment statements at the start of the f ragment. The second term is 3n2 ,
since there are three statements that are perf ormed n2 times due to the nested iteration. The third term
is 2n, tw o statements iterated n times. Finally, the f ourth term is the constant 1, representing the f inal
assignment statement. This gives us T (n) = 3 + 3n2 + 2n + 1 = 3n2 + 2n + 4 . By looking at the

3 of 4
2.3. Big-O Notation Problem Solving with Algorithms and Data
6/11/17,
Str... 2:56 PM

exponents, w e can easily see that the n2 term w ill be dominant and theref ore this f ragment of code is
O(n2 ). Note that all of the other terms as w ell as the coeff icient on the dominant term can be ignored
as n grow s larger.

Figure 2 show s a f ew of the common Big-O f unctions as they compare w ith the T (n) f unction
discussed above. Note that T (n) is initially larger than the cubic f unction. How ever, as n grow s, the
cubic f unction quickly overtakes T (n). It is easy to see that T (n) then f ollow s the quadratic f unction
as n continues to grow .

Se lf Che ck
Write tw o Python f unctions to f ind the minimum number in a list. The f irst f unction should
compare each number to every other number on the list. O(n2 ). The second f unction should be
linear O(n) .

4 of 4