CSE 431

Computer Architecture
Fall 2005

Lecture 07: Overcoming Data

Hazards
Mary Jane Irwin ( www.cse.psu.edu/~mji )
www.cse.psu.edu/~cg431

[Adapted from Computer Organization and Design,

Patterson & Hennessy, © 2005, UCB]

CSE431 L07 Overcoming Data Hazards.1 Irwin, PSU, 2005

 Review: MIPS Pipeline Data and Control Paths
PCSrc
ID/EX
EX/MEM
Control
IF/ID

Add
Branch MEM/WB
RegWrite Add
4 Shift
left 2
Read Addr 1
Instruction Data
Register Read
Memory Read Addr 2Data 1 Memory MemtoReg
Read ALUSrc
PC

File Address
Read
Address Write Addr ALU
Read Data
Data 2 Write Data
Write Data
ALU
cntrl
MemWrite MemRead
Sign
16 Extend 32 ALUOp

RegDst

CSE431 L07 Overcoming Data Hazards.2 Irwin, PSU, 2005

 Control Settings

EX Stage MEM Stage WB Stage

Reg ALU ALU ALU Brch Mem Mem Reg Mem
Dst Op1 Op0 Src Read Write Write toReg
R 1 1 0 0 0 0 0 1 0
lw 0 0 0 1 0 1 0 1 1
sw X 0 0 1 0 0 1 0 X
beq X 0 1 0 1 0 0 0 X

CSE431 L07 Overcoming Data Hazards.3 Irwin, PSU, 2005

 Review: One Way to “Fix” a Data Hazard

Fix data hazard

by waiting –

ALU
I add $1, IM Reg DM Reg
n stall – but
s impacts CPI
t stall
r.

O stall
r
d

ALU
e sub $4,$1,$5 IM Reg DM Reg
r

ALU
and $6,$7,$1 IM Reg DM Reg

CSE431 L07 Overcoming Data Hazards.4 Irwin, PSU, 2005

 Review: Another Way to “Fix” a Data Hazard
Fix data hazards
by forwarding

ALU
I add $1, IM Reg DM Reg
results as soon as
n
they are available
s
to where they are

ALU
t sub $4,$1,$5 IM Reg DM Reg
r. needed

ALU
IM Reg DM Reg
r and $6,$7,$1
d
e
r

ALU
or $8,$1,$1 IM Reg DM Reg

ALU
sw $4,4($1) IM Reg DM Reg

CSE431 L07 Overcoming Data Hazards.5 Irwin, PSU, 2005

 Data Forwarding (aka Bypassing)
 Take the result from the earliest point that it exists in any of
the pipeline state registers and forward it to the functional
units (e.g., the ALU) that need it that cycle
 For ALU functional unit: the inputs can come from any
pipeline register rather than just from ID/EX by
 adding multiplexors to the inputs of the ALU
 connecting the Rd write data in EX/MEM or MEM/WB to either (or
both) of the EX’s stage Rs and Rt ALU mux inputs
 adding the proper control hardware to control the new muxes
 Other functional units may need similar forwarding logic
(e.g., the DM)
 With forwarding can achieve a CPI of 1 even in the
presence of data dependencies

CSE431 L07 Overcoming Data Hazards.6 Irwin, PSU, 2005

 Data Forwarding Control Conditions
1. EX/MEM hazard:
if (EX/MEM.RegWrite Forwards the
and (EX/MEM.RegisterRd != 0)
and (EX/MEM.RegisterRd = ID/EX.RegisterRs)) result from the
ForwardA = 10 previous instr.
if (EX/MEM.RegWrite to either input
and (EX/MEM.RegisterRd != 0) of the ALU
and (EX/MEM.RegisterRd = ID/EX.RegisterRt))
ForwardB = 10

2. MEM/WB hazard:
if (MEM/WB.RegWrite Forwards the
and (MEM/WB.RegisterRd != 0)
result from the
and (MEM/WB.RegisterRd = ID/EX.RegisterRs))
ForwardA = 01 second
if (MEM/WB.RegWrite previous instr.
and (MEM/WB.RegisterRd != 0) to either input
and (MEM/WB.RegisterRd = ID/EX.RegisterRt)) of the ALU
ForwardB = 01
CSE431 L07 Overcoming Data Hazards.7 Irwin, PSU, 2005
 Forwarding Illustration

ALU
I add $1, IM Reg DM Reg
n
s

ALU
t sub $4,$1,$5 IM Reg DM Reg
r.

ALU
IM Reg DM Reg
r and $6,$7,$1
d
e
r

EX/MEM hazard MEM/WB hazard

forwarding forwarding

CSE431 L07 Overcoming Data Hazards.8 Irwin, PSU, 2005

 Yet Another Complication!
 Another potential data hazard can occur when there is a
conflict between the result of the WB stage instruction
and the MEM stage instruction – which should be
forwarded?

ALU
n
add $1,$1,$2 IM Reg DM Reg

s
t
r. add $1,$1,$3

ALU
IM Reg DM Reg

O
r

ALU
d add $1,$1,$4 IM Reg DM Reg
e
r

CSE431 L07 Overcoming Data Hazards.10 Irwin, PSU, 2005

 Corrected Data Forwarding Control Conditions
2. MEM/WB hazard:
if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd != 0)
and (EX/MEM.RegisterRd != ID/EX.RegisterRs)
and (MEM/WB.RegisterRd = ID/EX.RegisterRs))
ForwardA = 01

if (MEM/WB.RegWrite
and (MEM/WB.RegisterRd != 0)
and (EX/MEM.RegisterRd != ID/EX.RegisterRt)
and (MEM/WB.RegisterRd = ID/EX.RegisterRt))
ForwardB = 01

CSE431 L07 Overcoming Data Hazards.11 Irwin, PSU, 2005

 Datapath with Forwarding Hardware
PCSrc

ID/EX
EX/MEM
Control
IF/ID

Add
Branch MEM/WB
Add
4 Shift
left 2
Read Addr 1
Instruction Data
Register Read
Memory Read Addr 2Data 1 Memory
Read
PC

File Address
Read
Address Write Addr ALU
Read Data
Data 2 Write Data
Write Data
ALU
16 Sign 32 cntrl
Extend

EX/MEM.RegisterRd

ID/EX.RegisterRt
Forward MEM/WB.RegisterRd
ID/EX.RegisterRs Unit

CSE431 L07 Overcoming Data Hazards.13 Irwin, PSU, 2005

 Memory-to-Memory Copies
 For loads immediately followed by stores (memory-to-
memory copies) can avoid a stall by adding forwarding
hardware from the MEM/WB register to the data memory
input.
 Would need to add a Forward Unit and a mux to the memory
access stage

I
n

ALU
s lw $1,4($2) IM Reg DM Reg
t
r.

ALU
O sw $1,4($3) IM Reg DM Reg
r
d
e
r

CSE431 L07 Overcoming Data Hazards.14 Irwin, PSU, 2005

 Forwarding with Load-use Data Hazards

ALU
IM Reg DM Reg
I lw $1,4($2)
n
s

ALU
IM Reg DM Reg
t stall $4,$1,$5
sub
r.

ALU
and $4,$1,$5
sub $6,$1,$7 IM Reg DM Reg
O
r

ALU
d
or $6,$1,$7
and $8,$1,$9 IM Reg DM Reg
e
r

ALU
or
xor $8,$1,$9
$4,$1,$5 IM Reg DM Reg

xor $4,$1,$5

ALU
IM Reg DM

CSE431 L07 Overcoming Data Hazards.16 Irwin, PSU, 2005

 Load-use Hazard Detection Unit
 Need a Hazard detection Unit in the ID stage that inserts
a stall between the load and its use
2. ID Hazard Detection
if (ID/EX.MemRead
and ((ID/EX.RegisterRt = IF/ID.RegisterRs)
or (ID/EX.RegisterRt = IF/ID.RegisterRt)))
stall the pipeline

 The first line tests to see if the instruction now in the EX

stage is a lw; the next two lines check to see if the
destination register of the lw matches either source
register of the instruction in the ID stage (the load-use
instruction)
 After this one cycle stall, the forwarding logic can handle
the remaining data hazards
CSE431 L07 Overcoming Data Hazards.17 Irwin, PSU, 2005
 Stall Hardware
 Along with the Hazard Unit, we have to implement the stall
 Prevent the instructions in the IF and ID stages from
progressing down the pipeline – done by preventing the
PC register and the IF/ID pipeline register from changing
 Hazard detection Unit controls the writing of the PC (PC.write)
and IF/ID (IF/ID.write) registers
 Insert a “bubble” between the lw instruction (in the EX
stage) and the load-use instruction (in the ID stage) (i.e.,
insert a noop in the execution stream)
 Set the control bits in the EX, MEM, and WB control fields of the
ID/EX pipeline register to 0 (noop). The Hazard Unit controls the
mux that chooses between the real control values and the 0’s.
 Let the lw instruction and the instructions after it in the
pipeline (before it in the code) proceed normally down the
pipeline
CSE431 L07 Overcoming Data Hazards.18 Irwin, PSU, 2005
 Adding the Hazard Hardware
PCSrc

ID/EX.MemRead
Hazard ID/EX
Unit EX/MEM
0
IF/ID 1
Control 0
Add
Branch MEM/WB
Add
4 Shift
left 2
Read Addr 1
Instruction Data
Register Read
Memory Read Addr 2Data 1 Memory
Read
PC

File Address
Read
Address Write Addr ALU
Read Data
Data 2 Write Data
Write Data
ALU
16 Sign 32 cntrl
Extend

Forward
Unit
ID/EX.RegisterRt

CSE431 L07 Overcoming Data Hazards.20 Irwin, PSU, 2005

 Next Lecture and Reminders
 Next lecture
 Reducing branch costs
- Reading assignment – PH, Chapter 6.6 and 6.8

 Reminders
Tuesday, October 4
 HW2 due September 29th
 SimpleScalar tutorials scheduled
- Thursday, Sept 22, 5:30 to 6:30 pm in 218 IST

 Evening midterm exam scheduled

- Tuesday, October 18th , 20:15 to 22:15, Location 113 IST

CSE431 L07 Overcoming Data Hazards.21 Irwin, PSU, 2005