PCI Express
S. Simon 1
Overview (1/2)
• computer evolution
• history of the IO Bus
• PCI-SIG
• PCI-Express
• Lanes
• transfer rates
Logo
• PCI Express for Graphics reference: www.pcisig.com
• compatibility
• advantages
• PCI-Express usage
S. Simon 2
Overview (2/2)
• Design and components
• OSI-Modell
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example
S. Simon 3
Rapid computer evolution
• data medium size from kB to TB
reference: http://wikipedia.de
reference: http://verkaufsbilder.pc-interaktiv.de/
reference: http://www.osxbook.com/
S. Simon 4
Rapid computer evolution
• raising processorspeed
reference: bw.fh-deggendorf.de
• multicoreprocessing
• I/O-bus getting more and more a bottleneck
S. Simon 5
History of the IO Bus
• ISA (Industry Standard Architecture)
• developed in 1979
• first use with 8086 and 8088 Intel
processor
• 16,66 MByte/s
3xVLB, 3xISA 16Bit, 1xISA 8Bit
• VLB (Vesa Local Bus) reference: http://dic.academic.ru
• developed in 1992
• based on Intel 80486 CPUs
• 276 MByte/s
• AGP (Accelerated Graphics Port)
• directly connected to North-
Bridge
• independent from PCI
• 266 MByte/s - 2,1 GByte/s
reference: http://ist94.wikispaces.com
S. Simon 6
History of the IO Bus
• PCI (Peripheral Component Interconnect)
• displaced VLB
• PCI 1.0 (1991) – PCI 3.0 (2004)
• 133 MByte/s - 532 MByte/s
• PCI-X (PCI-Extended)
• high performance 64 bit designed for
servers 1xPCI, 2xPCIe 1x, 1xPCIe 15x
reference: weblearn.hs-bremen.de
PCI-X Card
reference: wikipedia.de
S. Simon 7
History of the IO Bus
• PCIe/PCI-E (PCI-Express)
• formerly known as 3GIO
• PCIe 1.0 (2002) – PCI3.0 (2009?)
• 250MByte/s – 32GByte/s
• more lanes -> higher datarate
• 8B10B-coding for clock recovery
• 8B10B code = 20% overhead
PCIe Datarates
reference: wikipedia.de
S. Simon 8
PCI-SIG
Logo
reference:
www.pcisig.com
reference: www.pcisig.com
S. Simon 9
PCI
• parallel bus-system
• used for network cards, sound cards,
modems, USB-Ports,
TV tuner cards and disk controllers
• connected components
have to share the bandwith
reference: weblearn.hs-bremen.de
reference: http://dell.com
S. Simon 10
PCI-Express
• high-speed serial bus-system
• data transfer with lanes
• point to point connection
• communication through switches
• an important characteristic
of the PCI-Express-Standard
is the softwarecompatibility to PCI
• number of required conductors on cards or on the mainboard is
intensively reduced reference: weblearn.hs-bremen.de
reference: dell.com
S. Simon 11
Lanes
• each lane with two unidirectional pairs of lines
• full-duplex
• 250 MByte/s in each direction
• coupling of several lanes
can be increase speed,
e.g. 2 lanes = 500MByte/s
reference: arstechnica.com
S. Simon 12
PCI Express for Graphics
• PEG (PCI Express for Graphics)
• standard PCI-Express-slot max. 25 Watt
• PCIe-x16 slot
• up to 75 Watt
• direct connection to power supply
• 6 pole additionally 75 Watt
• 8 pole additionally 150 Watt
S. Simon 13
Hardware
• easy layout
• serpentine routing uncomplicated, only aline pair of a lane have
to be exactly layouted
meander
• pair of lane may not exceed 50,8 cm reference: www.s-t-e.de
conductor board
reference: www.leiterplatten-print.de
S. Simon 14
PCI/PCI-Express compatibility
• software-compatibility to PCI-standard
• Neither operating system, nor the application has to be
adapted
• configurationspace (initialising- and configurationdata)
• PCI 256 Byte
• PCI-Express 4096 Byte
• PCI and PCI-Express hardware
not compatible
S. Simon 15
PCI-Express Kompatibilität
Cars/Slot PCIe x1 PCIe x4 PCIe x8 PCIe x16
PCI Express x1 Ok Ok Ok Ok
PCI Express x4 - Ok * *
PCI Express x8 - - Ok *
PCI Express x16 - - - Ok
Ok = compatible / - = incompatible / * = not regulated, but possible
PCIe-slots on a mainboard
reference: zone.ni.com
S. Simon 16
PCI-Express-bridge
• in the transitional period PCI-Express-bridges are needed for
reducing the costs
• AGP to PCIe for using available graphic chips on the PCIe
architecture
• PCIe to AGP e.g. gor using new graphic chips with the AGP-
slot
reference: www.scantec.de
S. Simon 17
Advantage PCI-Express
• software compatibility
• scalability
• higher datarates
• lower costs due to serial bus system
• universal use
• I/O- connections
• chip-to-chip-interconnect
• hot plug capable
• quality of service: virtual channels with priorities
• realtime applications
• plug & play
• direct communication between
PCIe components due to switches
reference: www.pcwissen.eu
S. Simon 18
PCI-Express usage
• Mini-PCI-Express
• onboard extensions for notebooks and PCs
• higher bandwith
• lower dimensions
reference: www.wikipedia.de
S. Simon 19
PCI-Express usage
• ExpressCard (NewCard)
• PCMCIA is being replaced
• with the high bandwith new range of applications
(dockingstation etc.)
• tasks:
network,
soundcards,
measuringcards,
SATA-, USB-
and Firewire-adapter
Quelle: www.wikipedia.de
reference: www.wikipedia.de
S. Simon 20
PCI-Express usage
• Solid State Drive (SSD)
• up to Terabyte region
• transferrate up to 750 MByte/s
• by comparison S-ATA up to 300 MByte/s
reference: www.notebookjournal.de
S. Simon 21
Overview
• PCI-Express
• ….
• PCI-Express useage
• Design and components
• OSI-Modell
• Overview
• Comparisons
• Physical Layers
• Phases in detail
• Transactions
• Concept
• Example
S. Simon 22
Design and components (1/2)
• Schematic outline
CPU
• Simple tree like structure
PCIe
• responsible for system End- PCIe
configuration, enumeration of point
PCIe resources, manages Root
Complex Memory
interrupts and errors
(Hub)
PCIe PCIe
PCIe to
PCI
Bridge PCIe
Switch
PCIe
PCIe
End-
…
…
point
Legacy PCIe
End- PCIe
PCI, PCI-X
point
PCIe
Legacy End-
End- point
point
S. Simon 25
OSI-Model (1/3) - PCIe
OSI-Layers PCIe-Layers
Applications
• Very similar layer
Application architecture
• Functionally identical
Presentation Software/OS Layer „Physical Layer“ and “Data
Link Layer“
Session
• „Transaction Layer“
combines transport and
network
Transport • „Application Layer“ implied
by the operating system,
Transport drivers and general
Transaction Layer application software.
Network
Session
Transport
Transport
Network
The „Device“
Data Link
Physical
S. Simon 29
Physical Layers - PCIe (1/3)
• Located exactly where data packages are sent and received
• Works with the hardware components of PCIe-Links (Mechanical
Layer); initializes Links
• Prepares sending data per link or passing them to the Data Link
Layer
• 8-bit/10-bit encoding
• Serial/Parallel conversion
• 2-Sub-Blocks:
Physical Layer
Logic
Electric
• Packet Framing
START
Scrambled/Encoded Transaction Layer Paket END
TLP
OR
START
Scrambled/Encoded Data Link Layer Paket END
DLLP
reference: weblearn.hs-bremen.de
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 35
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 36
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 37
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 38
The Three Layers Architecture of PCIe
• Transaction layer is for beginning the
process of turning request or data from
the device’s core into a PCIe packet
S. Simon 39
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 40
Transaction Layer
Transaction Layer:
• It receives -
• Incoming transaction,
S. Simon 41
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 42
Data-Link Layer
• It checks the correctness of the data
S. Simon 43
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 44
Physical Layer
• sends and receives the data which is sent with
PCIe link
S. Simon 45
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 46
Software Layer
• Has a PCI-compatible configuration mechanism
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 47
Software Layer
• Containing the enhanced
mechanism that increase the
size of available configuration
space
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 48
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 49
Types of Transaction
• Memory Transaction:
• I/O Transactions:
• Targeting the I/O space transfer data to or from I/O mapped location
S. Simon 50
Types of Transaction
• Configuration Transaction:
• Message Transaction:
S. Simon 51
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 52
Packets
• Transaction Layer Packet (TLP)
• Are the means for communicating between
PCI Express devices
• It's build out of Header, Optional
Data Payload, Optional TLP digest
• Generated by Transaction Layer with information
received from its device core
• Data Link Layer Packet (DLLP)
• It´s responsible for the integrity of TLPs
movement, link initialization and power
managment
• Used for passing messages and status
between the Transaction Layer and
the Physical Layer
• Used only for the Local Traffic
S. Simon 53
Packets
S. Simon 54
Index
1. Introduction of PCI Express Protocol
1. Transaction Layer
3. Physical Layer
3. Software Layer
4. Types of Transaction
5. Packets
S. Simon 55
Transaction Layer Packet
• Are the means for communicating
between PCI Express devices
• Header
device core
S. Simon 56
Transaction Layer Packet: Header
TLP
S. Simon 57
Transaction Layer Packet: Header
S. Simon 58
Transaction Layer Packet: Header
S. Simon 59
Transaction Layer Packet: Header
S. Simon 60
Transaction Layer Packet: Header
TLP
S. Simon 61
Transaction Layer Packet: Header
S. Simon 62
Transaction Layer Packet: Header
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
….
and have to be =0
TLP
S. Simon 65
Transaction Layer Packet: Header
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 66
Transaction Layer Packet
• CRC, ECRC and LCRC are for providing a method for PCIe
devices to verify the contents of received packets.
S. Simon 67
Transaction Layer Packet: Procedure
• Through the flowing of the transaction into the layers, each layer adds
on the specific information.
• Transaction layer make a header and adds the data also a optional
ECRC. The data link layer generate the sequence number and LCRC.
The physical layer frames it for transmission to the other device.
S. Simon 68
Transaction Layer Packet: Procedure
S. Simon 69
Index
10. Error Handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 70
Error Handling
PCI express defines two error reporting mechanisms:
-Baseline
Baseline:
Baseline defines the minimum error reporting capabilities required by all PCI
express devices
S. Simon 71
Error Handling
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 72
Error Handling
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 73
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 74
Link Initialization
Training states:
Detect
Polling
S. Simon 75
Link Initialization
Training states:
Polling
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 76
Link Initialization
Training states:
Polling
Is used for:
-Clock synchronization
-Forcing resets
-Link Disabling
-Test modes
-Data scrambling. v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 77
Link Initialization
Training states:
Configuration
Recovery
S. Simon 78
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 79
Link Initialization
Link states
L0
Full on
L0s
S. Simon 80
Link Initialization
Link States
L1
L2
S. Simon 81
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 82
Link Initialization
Other States
Disabled
Hot Reset
(External) Loopback
S. Simon 83
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 84
Flow Control
Flow Control guarantees that transmitters never send Transaction Layer
Packets (TLPs) that the receiver can't accept. This is made possible by Flow
Control Buffers that report their available buffer space to the opposite end of
the link. This Buffers are organized in virtual channels.
Flow Control
Flow Control
S. Simon 85
Flow Control
Each virtual channel has independent flow control and thus maintains
independent flow control pools for these types.
S. Simon 86
Index
10. Error handling
1. Training States
2. Link States
3. Other States
13. Symbols
14. Summary
S. Simon 87
Symbols
8b/10b Data Symbols
The 8b/10b encoding achieves DC-balance. Also it embeds a clock into the
code.
A byte value represented by bits HGFEDCBA is broken into two separate bit
streams, mainly HGF and EDCBA .
H G F
0 0 1
H G F E D C B A
25h D for Data D5.1
0 0 1 0 0 1 0 1
E D C B A
0 0 1 0 1
a b c d e f
D5.x 1 0 1 0 0 1
Nach Tabelle
a b c d e f g h i j
1 0 1 0 0 1 1 0 0 1
Dx.1 g h i j
1 0 0 1
S. Simon 88
Symbols
5b/6b
input RD = −1 RD = +1 input RD = −1 RD = +1
S. Simon 89
Symbols
3b/4b
input RD = −1 RD = +1 input RD = −1 RD = +1
S. Simon 90
Symbols
8b/10b Special Symbols
Special symbols are
coded according to the
same process except
that the prefix K is used
instead of D. There are
only 12 special symbols.
v.g.l:Intel Press, Introduction to PCI Express: A Hardware and Software Developer's Guide
S. Simon 91
Summary
• PCIe is a layered approach of device design, which contains
Transaction, Data-Link and Physical Layer
• TLPs used for transporting the data through the layers between the
PCIe devices
• DLLPs used for the local communication between the Data-Link
layers
• PCIe devices have a complete software compatibility
• Error Handling happens on all layers and is reported by messages
• Link Training and Initialization is based on a state machine
• Flow Control is an advanced approach for link monitoring
S. Simon 92
bibliography
[1] Intel Press, Introduction to PCI Express: A Hardware and Software
Developer's Guide
S. Simon 93