110001011010011110100111011011010011110011

The critically missing

VHDL testbench feature
- Finally a structured approach

www.bitvis.no

Your partner for SW and FPGA

What is missing?

- And What is the improvement potential?

110001011010011110100111011011010011110011

Missing:

A good Testbench Architecture and Structure

For Protocol and Control oriented Design in particular
Consequences:

Inefficient testbench development, extensions, modifications

Difficult to reuse TB parts in a project - or to share the TB itself
High risk of missing critical corner cases
Improvement Potential using a well structured approach:

Major time saving (many MWs for medium complexity FPGA)

Significant quality improvement for end product
A new world for overview, maintainability, extendibility and reuse
The critically missing VHDL testbench feature

FPGA functional scenario

110001011010011110100111011011010011110011

Sequential testing

FPGA

Find some bugs?

PIF

uart

SPI

DMA

P3

Intr
ctrl

ETH

Then behaves fine

P2

Parallell operation
Corner case bugs

P1
ETH

The critically missing VHDL testbench feature

Corner case example

TB

Data
In this particular case: Bug in FSM 1
Valid
Data May only be detected if a and bData
in the same cycle:
provider a. Selection Ack
receiver
of data source - e.g.
via register interface
FSM 1 b. Acknowledge from Data Receiver
FSM 2
FSM 1: Data source changes
Jumps to relevant state

Typical corner case:

Wait for Valid

When
available:
Two data
events
may happen at the same time
Set Data & Valid

- But it is unlikely that they actually do so in your TB

Wait
for Ack
- Most
probably your lab test will not detect it
Sample Data & Set Ack
- Real life: Guess what???
Reset Valid

Reset Ack

Will show typical protocol interface bug in tutorial @13:30

Select data source

The critically missing VHDL testbench feature

Typical testbench
110001011010011110100111011011010011110011

Typical testbench
Sequential
Clock
Gen

Testcase
Sequencer

Sequential testing
No bug found
Parallel operation
Corner case bugs

FPGA

PIF

uart

SPI

DMA

P2

P3

Intr
ctrl

P1

ETH

Cycle related corner cases?

Adding
threads

Lab
CCL ?
?

Ad hoc
"structure"

ETH

Structured, but far too complex

Structured
with good overview
The critically missing VHDL testbench feature

Simple problem ad-hoc solution

110001011010011110100111011011010011110011

Example: A regular data stream

E.g. a filter or simple algorithm

Simultaneous access on two interfaces

Contiguous data input & contiguous data output

The critically missing VHDL testbench feature

Regular Data Stream - Example

Single thread.
Dead
simple
example
!!!time
Can
only handle
one interface
at any

110001011010011110100111011011010011110011

p_main (test-sequencer)
BFM

enable_p_apply_data

p_apply_data
BFM
BFM
Input
stimuli

enable_p_fetch_data

DUT
(e.g. Filter)
in

out

BFM
BFM

Exp.
Output

Model
Scoreboard

The critically missing VHDL testbench feature

p_fetch_data

The UART
110001011010011110100111011011010011110011

Still a VERY simple module!

BUT
May have lots of corner cases
often never simulated at all
often never tested in the lab

DUT
(UART)
Clocks
Bus
interface

Other
Ports Ext. I/O
RX + TX

So
Need to control RX, TX and PIF independently
Must be tightly controlled from a sequencer
Must allow full flexibility in data, access times, etc
Will show typical protocol interface bug in tutorial @13:30
The critically missing VHDL testbench feature

FIFO
rx/tx

Wishful thinking for a testbench?

110001011010011110100111011011010011110011

Wouldn't it be nice if we could ...

handle any number of interfaces in a structured manner?

reuse major TB elements between module TBs?
reuse major module TB elements in the FPGA TB?
read the test sequencer almost as simple pseudo code?
recognise the verification spec. in the test sequencer?
understand the sequence of event
- just from looking at the test sequencer

Is this feasible at all?

The critically missing VHDL testbench feature

Scenario: Check data out of UART (DUT) TX

110001011010011110100111011011010011110011

p_main (test-sequencer)
BFM

p_transmit
BFM

trigger_p_receive
num_data_words
p_receive_data
baudrate severity_data_error
baudrate_margin TO_on_ack
baudrate_severity severity_TO
TO_on_receive

DUT (UART)
RX
TX

BFM

Oooops...

ack
busy
data_received

p_receive

Problem
growing for every detected need
As for the Regular
Data is
Stream:
Stumbling
intotoproblems
as
we develop
our tests....
-control
Need
set number
of data
words
Need
Need to
separate
threads
for of
Sequencer
and
interfaces
severity
receiving
wrong
data to TX & RX
Occasionally
p_receive
doesn't
detect
the trigger
"pulse"
-toNeed
tothat
see
actual
data
received
in the
sequencer
Can't
wait
forever
forthe
the
acknowledge

Need
send
to
p_receive
for
checking
data
received
Sequencer
must sometimes
wait untilto
p_receive
is free

Need
acknowledge
from
p_receive
sequencer

Need timeout,
with
error
handling
and
reporting
Receiving
data to
may
take
forever
- e.g.
if bug
in TB or DUT
Need
set
baudrate
Need -busy-indication
from p_receive (or 'completed')
Need -time
for margins
data reception
Needout
to set
for baudrate, and severity
(May need severity for time out for data reception)
- etc, etc .....

The critically missing VHDL testbench feature

10

Problem keeps growing...

110001011010011110100111011011010011110011

More and more control signals must be entered

New issues detected during test case development
Old code lines must be updated
---------------After "finishing" the control and data reception of UART TX

Same issues for control and data to UART RX

Same issues for other modules
Same issues at top level

For every new interface and feature

"old" code may have to be updated
The critically missing VHDL testbench feature

11

Let's Solve the problem

Handles the problem on the sequencer side.
p_main
(test-sequencer)
What
about the executor side?

110001011010011110100111011011010011110011

transmit(x"C1")
BFM

DUT (UART)
RX
TX

TLM

p_transmit

BFM

TLM

forward(UART, receive, x"2A")

pif_write(C_ADDR_TX, receive_uart(x"2A")
x"2A")
receive(UART, x"2A")
pif_read(C_ADDR_RX, x"C1)

BFM
receive(x"2A")

receive(UART,
x"2A", WARNING,
1 ms, FAILURE)

p_receive

Problem
Need BFM-like
specification
solution
BFMs:
-- Signals
Procedural
being set
added,
removed
modified
Valid
minimum
of values
and or
signals

-- Protocol
Signals
between
being modified
Sequencer
or extended
and p_receive / p_transmit
Valid
and hidden
protocol
-- Knowledge
No physical
of
signals,
valid signals
but used
andtoprotocols
control transactions
Hidden
complexity
- Cumbersome
to write
this
code over and over again
Increased
readability
compact
Transaction
leveland
models/commands
Basically
allmodifications
the same problems
BFMs
Allows
painless
of signals,resulting
protocol, in
features

The critically missing VHDL testbench feature

12

The executor side - for Receive

110001011010011110100111011011010011110011

Must detect command from central test sequencer

Then execute BFM receive() towards the DUT
A VHDL process cannot do both at the same time

From UART BFM and VVC template to UART VVC :

May need VHDL Verification Component (VVC)

1 hour

p_main (test-sequencer)

receive(UART, x"2A")

DUT (UART)
RX
TX

receive(x"2A")

The critically missing VHDL testbench feature

13

TLM

transmit(x"C1")

BFM

TLM

p_transmit

receive_uart(x"2A")

receive(UART,
x"2A", WARNING,
1 ms, FAILURE)

p_receive
VVC
for receive()

Hitting the corner cases

Simple test sequencer example:
. . .

write(PIF, C_ADDR_TX, x"2A", "Uart TX");

expect(UART, RX, x"2A, "From DUT TX");

Clock
Generation

DUT
(UART)
Clocks

UART
VVC

transmit(UART, TX, xC1);-- into DUT RX

insert_delay(UART, TX, 2*C_BIT_PERIOD);
transmit(UART, TX, xC2);

Test
Seq.

PIF
VVC

Bus interface

Baudrate
Checker

Other
Ports
Ext. I/O

TX/RX
handling

await_completion(UART);
check(PIF, C_ADDR_RX, xC1", "Uart RX");
check(PIF, C_ADDR_RX, xC2", "Uart RX");
await_completion(PIF);

UVVM & VVCs

BFMs

Bitvis Utility Library

report_simulation_summary;

The critically missing VHDL testbench feature

14

Making a comprehensive test (1)

110001011010011110100111011011010011110011

In central test sequencer

UVVM provides an enhanced version of

OSVVM Random & Coverage

test_uart_transmit_with_delay(<data_tx>, <data_rx>, <delay>);

test_uart_transmit_random_with_skewed_delay(<num_times>);
-- random data and random delay

Simple test sequencer example:

. . .

DUT

Clock
Generation

write(PIF, C_ADDR_TX, x"2A", "Uart TX");

(UART)

expect(UART, RX, x"2A, "From DUT TX");

transmit(UART, TX, xC1);-- into DUT RX

Clocks

insert_delay(UART, TX, 2*C_BIT_PERIOD, "Between 2 transmits");

transmit(UART, TX, xC2, "2nd of two transmits. Delay between");
await_completion(UART);
check(PIF, C_ADDR_RX, xC1", "Uart RX");
check(PIF, C_ADDR_RX, xC2", "Uart RX");
await_completion(PIF);

report_simulation_summary;

The critically missing VHDL testbench feature

UART_
VVC

Test
Seq.

Bus interface

Baudrate
Checker

Other
Ports

TX/RX
handling

Ext. I/O

UVVM & VVCs

15

PIF_
VVC

BFMs

Bitvis Utility Library

Making a comprehensive test (2)

110001011010011110100111011011010011110011

Using VVC local test sequencers

Inside UART_VVC (in executor):
uart_transmit_buffer(<buffer_idx>, <num_bytes>);
Inside PIF_VVC (in executor):
pif_check_buffer(C_ADDR_RX, <buffer_idx>, <num_bytes>);
DUT

Clock
Generation

(UART)

Calls from central sequencer:

Clocks

uart_transmit (UART_VVC,1,TX,
pif_check (PIF_VVC,1,

UART_
VVC

PIF_
VVC

Bus interface

<num_bytes>);
BUFFER, C_TX_BUFFER, <num_bytes>);
Test
Seq.

Baudrate
Checker

Other
Ports

TX/RX
handling

C_ADDR_RX, <num_bytes>)
BUFFER, C_TX_BUFFER, <num_bytes>)
Ext. I/O

Name of procedures and use of parameters is a user choice

UVVM & VVCs

The critically missing VHDL testbench feature

16

BFMs

Bitvis Utility Library

Solving the multi-interface problem

Central Sequencer with TLM and VVCs:

One single brain in the system

Inherent synchronisation between sequencer and VVCs
No process is starting by "pure magic"
Full control of complete env. from the main sequencer
Allows simple queuing of commands
Lego-like VVC connections

Global connections and proper Verbosity control:

Automatic connection between sequencer and VVC
Allows built-in VVCs (+ checkers and monitors)

Excellent transcript of test sequence

Allows selected progress information at all levels

The critically missing VHDL testbench feature

17

Wishful thinking? - Recaptured

110001011010011110100111011011010011110011

Wouldn't it be nice if we could ...

handle any number of interfaces in a structured manner?

reuse major TB elements between module TBs?

reuse major module TB elements in the FPGA TB?
read the test sequencer almost as simple pseudo code?
recognise the verification spec. in the test sequencer?
understand the sequence of event
- just from looking at the test sequencer

The critically missing VHDL testbench feature

18

More info in tutorial

110001011010011110100111011011010011110011

A typical corner case - found everywhere

Verification components
Experience data on making a new VVC

Simulation progress information

Debug friendliness
Low user threshold

Constrained random and functional coverage

The critically missing VHDL testbench feature

19

will be on
freeware
Ballpark UVVM
numbers
benefits
110001011010011110100111011011010011110011

Using a structured TB like UVVM for handling multi-interfaces issues

Savings on a mainly control or protocol oriented design:
a) 2000-hour FPGA Development Project: 100-500 MH
b) 5000-hour FPGA Development Project: 500-1500 MH
Increasing from project to project
Additionally saving a lot of time for SW developers
Significantly improving quality on final product
Drastically improving LCC and TTM

The critically missing VHDL testbench feature

20

The critically missing

VHDL testbench feature

110001011010011110100111011011010011110011

- Finally a structured approach

Thank you

Your partner for Embedded SW and FPGA

www.bitvis.no