EECC550 - Shaaban
#1 Lec # 3
How can one measure the performance of this machine running this
program?
Intuitively the machine is said to be faster or has better performance
running this program if the total execution time is shorter.
Thus the inverse of the total measured program execution time is a
possible performance measure or metric:
PerformanceA = 1 / Execution TimeA
How to compare performance of different machines?
What factors affect performance? How to improve performance?
EECC550 - Shaaban
#2 Lec # 3
EECC550 - Shaaban
#3 Lec # 3
== Seconds
Seconds
Program
Program
T =
==Instructions
xx Seconds
Instructions xx Cycles
Cycles
Seconds
Program
Instruction
Cycle
Program
Instruction
Cycle
I x
CPI
EECC550 - Shaaban
#4 Lec # 3
x CPI x
EECC550 - Shaaban
#5 Lec # 3
== Seconds
Seconds
Program
Program
==Instructions
xx Seconds
Instructions xx Cycles
Cycles
Seconds
Program
Instruction
Cycle
Program
Instruction
Cycle
10,000,000
10,000,000
.125 seconds
== Seconds
Seconds
Program
Program
==Instructions
xx Seconds
Instructions xx Cycles
Cycles
Seconds
Program
Instruction
Cycle
Program
Instruction
Cycle
I
x CPI
x
C
Instruction Cycles per
Clock Rate
Count
Instruction
Program
Compiler
Instruction Set
Architecture (ISA)
Organization
Technology
EECC550 - Shaaban
#7 Lec # 3
Instruction Count I
Depends on:
Program Used
Compiler
ISA
CPU Organization
CPI
Clock
Cycle
C
Depends on:
CPU Organization
Technology
EECC550 - Shaaban
#8 Lec # 3
== Old
OldExecution
ExecutionTime
Time ==Iold
Iold xx CPI
CPIoldold
New
NewExecution
ExecutionTime
Time Inew
Inew xx CPI
CPInew
new
xx Clock
Clockcycle
cycleoldold
xx Clock
ClockCycle
Cyclenew
new
EECC550 - Shaaban
#9 Lec # 3
Then:
CPI C
i 1
Instruction Count I = C
i
EECC550 - Shaaban
#10 Lec # 3
CPI
1
2
3
CPI CPI i F i
n
i 1
CPIi x Fi
CPI
EECC550 - Shaaban
#12 Lec # 3
50%
20%
10%
20%
1
5
3
2
CPI
CPIi x Fi
.5
1.0
.3
.4
% Time
23% =
45% =
14% =
18% =
.5/2.2
1/2.2
.3/2.2
.4/2.2
Typical Mix
n
CPI CPI i F i
i 1
CPI = .5 x 1 + .2 x 5 + .1 x 3 + .2 x 2 = 2.2
EECC550 - Shaaban
#13 Lec # 3
Metrics of Computer
Performance
Execution time: Target workload,
SPEC95, etc.
Application
Programming
Language
Compiler
ISA
Datapath
Control
Function Units
Transistors Wires Pins
EECC550 - Shaaban
#14 Lec # 3
Microbenchmarks:
Small, specially written programs to isolate a specific aspect of
performance characteristics: Processing: integer, floating point,
local memory, input/output, etc.
EECC550 - Shaaban
#15 Lec # 3
Pros
Types of Benchmarks
Representative
Portable.
Widely used.
Measurements
useful in reality.
Small Kernel
Benchmarks
Microbenchmarks
Cons
Very specific.
Non-portable.
Complex: Difficult
to run, or measure.
Less representative
than actual workload.
Easy to fool by
designing hardware
to run them well.
Peak performance
results may be a long
way from real application
performance
EECC550 - Shaaban
#16 Lec # 3
SPEC92, 1992:
SPECInt92 (6 integer programs) and SPECfp92 (14 floating point programs).
SPEC95, 1995:
SPECint95 (8 integer programs):
go, m88ksim, gcc, compress, li, ijpeg, perl, vortex
SPECfp95 (10 floating-point intensive programs):
tomcatv, swim, su2cor, hydro2d, mgrid, applu, turb3d, apsi, fppp, wave5
Performance relative to a Sun SuperSpark I (50 MHz) which is given a score of SPECint95 = SPECfp95 = 1
EECC550 - Shaaban
#17 Lec # 3
SPEC95 Programs
Benchmark
Integer
Floating
Point
go
m88ksim
gcc
compress
li
ijpeg
perl
vortex
tomcatv
swim
su2cor
hydro2d
mgrid
applu
trub3d
apsi
fpppp
wave5
Description
Artificial intelligence; plays the game of Go
Motorola 88k chip simulator; runs test program
The Gnu C compiler generating SPARC code
Compresses and decompresses file in memory
Lisp interpreter
Graphic compression and decompression
Manipulates strings and prime numbers in the special-purpose programming language Perl
A database program
A mesh generation program
Shallow water model with 513 x 513 grid
quantum physics; Monte Carlo simulation
Astrophysics; Hydrodynamic Naiver Stokes equations
Multigrid solver in 3-D potential field
Parabolic/elliptic partial differential equations
Simulates isotropic, homogeneous turbulence in a cube
Solves problems regarding temperature, wind velocity, and distribution of pollutant
Quantum chemistry
Plasma physics; electromagnetic particle simulation
EECC550 - Shaaban
#18 Lec # 3
EECC550 - Shaaban
#19 Lec # 3
EECC550 - Shaaban
#20 Lec # 3
CINT2000
(Integer)
CFP2000
(Floating
Point)
Benchmark
Language
164.gzip
175.vpr
176.gcc
181.mcf
186.crafty
197.parser
252.eon
253.perlbmk
254.gap
255.vortex
256.bzip2
300.twolf
C
C
C
C
C
C
C++
C
C
C
C
C
Compression
FPGA Circuit Placement and Routing
C Programming Language Compiler
Combinatorial Optimization
Game Playing: Chess
Word Processing
Computer Visualization
PERL Programming Language
Group Theory, Interpreter
Object-oriented Database
Compression
Place and Route Simulator
168.wupwise
171.swim
172.mgrid
173.applu
177.mesa
178.galgel
179.art
183.equake
187.facerec
188.ammp
189.lucas
191.fma3d
200.sixtrack
301.apsi
Fortran 77
Fortran 77
Fortran 77
Fortran 77
C
Fortran 90
C
C
Fortran 90
C
Fortran 90
Fortran 90
Fortran 77
Fortran 77
Source: http://www.spec.org/osg/cpu2000/
Descriptions
EECC550 - Shaaban
#21 Lec # 3
#
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
MHz
1300
2200
2200
1667
1000
1400
1050
1533
750
833
1400
833
600
675
900
552
750
700
800
400
Processor
POWER4
Pentium 4 811
Pentium 4 Xeon
Athlon XP 724
Alpha 21264C
Pentium III 664
UltraSPARC-III Cu
Athlon MP 609
PA-RISC 8700
Alpha 21264B
Athlon
554
Alpha 21264A
MIPS R14000
SPARC64 GP
UltraSPARC-III
PA-RISC 8600
POWER RS64-IV
Pentium III Xeon
Itanium
365
MIPS R12000
int peak
814
790
810
697
679
648
610
587
604
571
495 1533
533
500
478
467
441
439
438
358
353
Top 20 SPECfp2000
int base
MHz
Processor
fp peak
790
1300
POWER4
1169
1000
Alpha 21264C
960
776
788
1050
UltraSPARC-III Cu 827
2200
Pentium 4 Xeon
802
621
2200
Pentium 4
801
833
Alpha 21264B
784
643
537
800
Itanium
701
833
Alpha 21264A
644
571
568
1667
Athlon XP
642
497
750
PA-RISC 8700
581
Athlon MP
547
504
511
600
MIPS R14000
529
483
675
SPARC64 GP
509
449
900
UltraSPARC-III
482
438
1400
Athlon
458
417
1400
Pentium III
456
409
500
PA-RISC 8600
440
431
450
POWER3-II
433
500
Alpha 21264
422
383
328
400
MIPS R12000
407
Source: http://www.aceshardware.com/SPECmine/top.jsp
fp base
1098
701
779
779
701
596
526
499
371
427
426
437
397
426
382
EECC550 - Shaaban
#22 Lec # 3
EECC550 - Shaaban
#23 Lec # 3
CPI
1
2
3
Code from:
Compiler 1
Compiler 2
CPI C
i
For compiler 1:
CPI1 = (5 x 1 + 1 x 2 + 1 x 3) / (5 + 1 + 1) = 10 / 7 = 1.43
MIP1 = 100 / (1.428 x 106) = 70.0
CPU time1 = ((5 + 1 + 1) x 106 x 1.43) / (100 x 106) = 0.10 seconds
For compiler 2:
CPI2 = (10 x 1 + 1 x 2 + 1 x 3) / (10 + 1 + 1) = 15 / 12 = 1.25
MIP2 = 100 / (1.25 x 106) = 80.0
CPU time2 = ((10 + 1 + 1) x 106 x 1.25) / (100 x 106) = 0.15 seconds
EECC550 - Shaaban
#25 Lec # 3
EECC550 - Shaaban
#26 Lec # 3
Amdahls Law:
Performance improvement or speedup due to enhancement E:
Execution Time without E
Speedup(E) = -------------------------------------Execution Time with E
Performance with E
= --------------------------------Performance without E
EECC550 - Shaaban
#27 Lec # 3
Affected fraction: F
Unchanged
F/S
After:
Execution Time with enhancement E:
Execution Time without enhancement E
1
Speedup(E) = ------------------------------------------------------ = -----------------Execution Time with enhancement E
(1 - F) + F/S
EECC550 - Shaaban
#28 Lec # 3
Freq
50%
20%
10%
Cycles
1
5
3
CPI(i)
.5
1.0
.3
% Time
23%
45%
14%
CPI = 2.2
Branch
20%
2
.4
18%
If a CPU design enhancement improves the CPI of load instructions
from 5 to 2, what is the resulting performance improvement from this
enhancement:
Fraction enhanced = F = 45% or .45
Unaffected fraction = 100% - 45% = 55% or .55
Factor of enhancement = 5/2 = 2.5
Using Amdahls Law:
1
1
Speedup(E) = ------------------ = --------------------- =
(1 - F) + F/S
.55 + .45/2.5
1.37
EECC550 - Shaaban
#29 Lec # 3
Freq
50%
20%
10%
Cycles CPI(i)
1
.5
5
1.0
3
.3
% Time
23%
45%
14%
CPI = 2.2
Branch
20%
2
.4
18%
If a CPU design enhancement improves the CPI of load instructions from
5 to 2, what is the resulting performance improvement from this
enhancement:
Old CPI = 2.2
New CPI = .5 x 1 + .2 x 2 + .1 x 3 + .2 x 2 = 1.6
Original Execution Time
Speedup(E) = ----------------------------------New Execution Time
old CPI
= ------------ =
2.2
---------
= 1.37
new CPI
1.6
Which is the same speedup obtained from Amdahls Law in the first solution.
EECC550 - Shaaban
#30 Lec # 3
100
----------------------------------------------------Execution Time with enhancement
= 25 seconds
= 80 seconds / n
= 80/5 = 16
EECC550 - Shaaban
#31 Lec # 3
For the previous example with a program running in 100 seconds on a machine
with multiply operations responsible for 80 seconds of this time. By how
much must the speed of multiplication be improved to make the program five
times faster?
Desired speedup = 5 =
100
----------------------------------------------------Execution Time with enhancement
20 seconds
= 80 seconds / n
EECC550 - Shaaban
#32 Lec # 3
Speedup
Speedup
((1 F ) F )
i
EECC550 - Shaaban
#33 Lec # 3
Percentage1 = F1 = 20%
Speedup2 = S2 = 15
Percentage1 = F2 = 15%
Speedup3 = S3 = 30
Percentage1 = F3 = 10%
While all three enhancements are in place in the new design, each enhancement
affects a different portion of the code and only one enhancement can be used at
a time.
What is the resulting overall speedup?
Speedup
((1 F ) F )
i
EECC550 - Shaaban
#34 Lec # 3
S1 = 10
F1 = .2
S2 = 15
S3 = 30
F2 = .15
F3 = .1
/ 15
/ 10
/ 30
Unchanged
EECC550 - Shaaban
#35 Lec # 3