Solving PCI Express Design

Challenges in an SoC ASIC

White Paper

ChipX, Inc.
moreinfo@chipx.com

A B S T R A C T This paper describes the challenges of

developing a highly integrated SoC ASIC with a PCI Express
interface. Tips based on experience are presented for critical
design steps, including sourcing the PHY, selecting the
proper ASIC technology, placement and routing,
characterization, board integration, and test.

March 20, 2008

Doc. 0391-6K-090-B
 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Introduction
Developing a highly integrated SoC ASIC with a PCI Express interface presents many
challenges. The process begins with sourcing a high quality PHY core that will work
reliably in production and a link layer controller that is compatible with and works
seamlessly with the PHY. The next step involves selecting the ideal ASIC technology
from a range of options that span Standard Cell, Embedded Array, and Structured ASIC.
The proper geometry that will deliver the desired performance is part of this selection
criteria. Whether you are doing your own placement and routing or relying on an ASIC
partner, you must ensure that placement is done in an optimal manner that isolates the
core from noise. In addition, the proper EDA tools must be used to check for signal
integrity and best performance. Minimizing risk throughout all project steps should be of
paramount concern, because anything that goes wrong will be very costly in terms of
lost time and expense. Consider whether you need to start early software development.
Or do you need to develop your prototype and system boards? What PCB design
guidelines should you use? Should you do any characterization of the chip and/or PCI
Express interface, or should you rely on your IP and/or ASIC partner? How much testing
is enough?
This paper describes all the PCI Express design challenges in detail and answers the
questions raised. It provides information on symptoms of potential trouble, various ways
to solve frequently seen problems, and ways to minimize risks and costs during the
design and test process.

Technology Selection

Process
One key decision to make early in a silicon development flow involves selecting the
manufacturing process technology. Traditionally, process selection was driven mainly by
the anticipated sales volume. For high volume production, an advanced deep submicron
process is typically preferred. The high NRE cost of using an advanced process is
justified at high volumes, because the process yields a denser die size with a lower
device cost. Selecting a more stable mainstream technology involves lower NRE costs,
but typically leads to larger die size.
When dealing with a complex high speed interface such as PCIe, it is important to
consider additional parameters, such as IP availability, IP integration experience, IP
proven tape outs, and process stability. It is wise to favor the process node in which the
IP was proven.

ASIC Type
The next step involves selecting one of the available ASIC technologies:
 Standard Cell ASIC
 Structured ASIC
 Embedded Array

2 of 11 Doc. 0391-6K-090-B March 20, 2008

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Each architecture has its own advantages and disadvantages when dealing with an
interface as complex as PCIe.
On one hand, Standard Cell ASIC provides the most flexible solution that fits any given
design with maximum silicon utilization. On the other hand, when integrating a PCIe
hard IP into the chip, a Standard Cell implementation requires special attention while
creating a power structure to power the IP and when designing a package supporting
the high speed lanes.
Structured ASIC has a great advantage when integrating PCIe in terms of risks and
reuse—with a restriction that a design has to fit the available array. The array, including
the PCIe, is already implemented following integration guidelines. Moreover, a proven
silicon test chip demonstrating the PCIe performance is typically available. A package
supporting the high speed lanes of the PCIe is already designed, available, and proven.
WIth Structured ASIC, you know what you are getting.
Embedded Array provides the flexibility of the Standard Cell ASIC and the ability to
reuse the same array if the design iterates, with no need to repeat integration efforts.

IP Selection
IP selection, especially regarding the high speed PHY, has long term implications,
because a wrong choice here could translate into some very undesirable
consequences, such as:
 Longer time-to-market of the product —and sometimes, of a whole product family if
it uses the same PHY/IP
 Potential compliance issues that could impact product release
 Potential yield and performance issues.
While selecting the IP (PHY), check the following, as a minimum:
 Maturity of the IP—Was it proven on silicon using the desired manufacturing
process? Was it used by another customer?
 Characterization reports—Read these carefully, and observe the conditions in
which the measurements were taken.
 Compatibility with different controllers, resulting in proved system solutions
 Compliance testing achieved with the specific IP
 Power consumption data for the core part and for the analog part, separately.
 Availability of the PHY I/O simulation model—Request it and apply the specific
system parameters, examining the resulting performance.
 Built-in production testing utilities, especially regarding the high speed part. Check
for BIST circuitry or PRBS generation.
 Silicon area of the IP—The hard IP part should include I/O pads and a size
estimate of complementary soft parts of the IP.

March 20, 2008 Doc. 0391-6K-090-B 3 of 11

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

 Number of metal layers used in the implementation, and width of the top metal
layers. Both must fit with the desired application.
 ESD protection may be part of the I/O structure; or a detailed description of ESD
protection is required.
 I/O pad pitch and structure (inline or staggered)
 DRC clean of the IP according to the targeted manufacturing process, using the
latest rules from the fab
 Check if Library views such as:.lib, .lef, .cdl, are available and meet the target EDA
tool requirements.

Design

Package Design
The package is an important element in the chain of the overall product design, and as
such, it affects the PHY performance. The key factors to be supervised and managed
regarding package design include:
 High speed signal handling—Must be designed to meet the required impedance
(differential 100 Ω)
 The differential skew for each pair of high speed signals
 Noise isolation of the high speed signals (both Tx and Rx)
 PHY supply power, ground inductance, and noise isolation
 Ball mapping.
Ball mapping has a significant impact on the board layout design, and therefore, on the
system level performance of PCI Express electrical properties.
The high speed differential pair routing on the board is affected by the package ball
allocation, therefore, careful mapping provides several benefits:
 Mapping allows routing each pair for easier differential matching and enables
routing that is as straightforward as possible1.
 Mapping prevents Tx signals from potentially harming Rx signals, which can
happen if these signals are routed too closely together. The same applies to clock
signals.
 Mapping helps achieve the capability of placing decoupling capacitors between
power and ground balls in the high speed signals area (improving decoupling
quality without the penalty of having those capacitors block routing of the high
speed signals on the board). This is especially relevant in cases where high speed
signals are routed on the Print Side (PS) of the board. See Figure 1.

1. Receiver response depends, among other parameters, on the differential matching of each pair,
end-to-end, both in length and in interconnect pattern.

4 of 11 Doc. 0391-6K-090-B March 20, 2008

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Figure 1 Routing and Placement of Decoupling Capacitor Pads for High Speed Lanes

Board Design
PCI Express product performance is determined by the combined effects of silicon,
package, and board design.
To provide the highest possible margin of performance, it is wise to focus design
attention on the following areas:
 DC supply scheme, decoupling, and filtering
 Reference Clock signal handling
 Layout
 Ball assignment of the PHY device.
For the DC supply scheme, consider decoupling and filtering requirements of the board
design, in order to provide a stable and clean operating supply to the fast switching
elements on board. Minimize high speed spurs that could interfere with the supply of
sensitive parts, such as the PHY analog supply.
Ensure ample bulk capacitance, combined with high speed decoupling capacitors, per
supply voltage; isolate the sensitive power supply areas from potential noise using
filtered “islands.
The layout design includes the definition of the board stack-up, with special attention to:
 routing the high speed signals, keeping the intra-pair skew low (max of 5 mil)
 routing the reference clock.
Another layout goal is to achieve efficient decoupling placement and connectivity.
Design the ball assignments for the PHY device in advance, to enhance the board
layout (as described in “Package Design” on page 4), by enabling placement of
decoupling capacitors and by enabling efficient routing of the high speed signals and the
reference clock.
One major consideration is controlling the PCB layout to minimize the effect of crosstalk
on performance, because such crosstalk could impact jitter performance of the PCI
Express interface.

March 20, 2008 Doc. 0391-6K-090-B 5 of 11

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Analysis and Validation

Power
PCIe PHY is very sensitive to power noise and mismatch. Thus, when integrating a
PHY, follow these basic rules to avoid silicon noise-related problems:
 Power separation must be implemented according to vendor instructions.
 To protect the PHY from any ESD events, connect the ESD power ring per the IP
vendor design guidelines.
 The PHY ESD scheme must be compatible with the I/O ESD scheme and
integrated in a way that will not negatively affect noise requirements of PHY or any
sensitive I/Os (such as LVDS or DDRI/II interfaces).
 Keep all differential pairs matched, and verify matching using parasitic RC
extraction of signal path.
 All differential pairs should be guarded by the same supply, to ensure the same
effect if any noise is injected into supplies.
 Any additional layout required for integration must be simulated for supply noise
effect and mismatch1 with parasitic RC extraction-based netlist.
 You must shield critical signals in all integration-related layout.
 If staggered I/O is required, consider line inductance and resistance differences
between the inner and outer bonding pad rings—Differential pairs must be on the
same plane. This might require a non-conventional staggering scheme.

Note: Follow these basic rules to significantly reduce the chance of having a signal
integrity problem due to power noise or due to faulty integration of the PHY.

Signal Integrity Analysis

The signal integrity analysis is done in several stages during the overall product design
process—from the initial stage, in which there was no actual design of the package nor
PCB, to the advanced stages, before releasing the package and/or PCB to
manufacturing.
The major goal of signal integrity analysis is to provide the expected end-to-end
(between driver and receiver) electrical performance of the PHY high speed lane.
This analysis, in addition to providing the expected margin of performance, can be
helpful in validating both the package and the PCB design.
The simulated topology for high speed PCI Express signal analysis includes:
 The PHY I/O driver
 Package model

1.
Mismatch can cause jitter on the transmitting eye or receiving problems that are difficult to detect.

6 of 11 Doc. 0391-6K-090-B March 20, 2008

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

 PCB interconnect model

 AC coupling capacitors
 Connector model (when and where applicable)
 Load model and termination.
The simulation results were stored so that the PCI Express signal masks could be
applied. This facilitated the process of analysis, and at the same time provided easy
reference for comparison with the later real-life measurements.
One major challenge in this verification process is accurately modeling the different
elements in the end-to-end path, because this determines the accuracy of the analysis
results. One such example is via modeling of the PCB interconnect.
The signal integrity design verification process also includes crosstalk analysis between
transmit lanes or clock signals and victim signals of the receive lane.

Functional Verification of the PCI Express Operation

The verification of the design before tape-out, to validate compliance and end-to-end
functionality, is always very significant, and in the case of the PCI Express application,
has additional importance regarding protocol functionality.
One major consideration is how to ensure that the verification environment will
accurately reflect correct behavior of the real environment that interacts with the product
at the PCI Express layers level.
A good approach is to use a dedicated verification environment that supports the PCI
Express protocol and that allows the generation of test cases.
Such an environment contributes to the reliability of the verification results, because it
has been used by many different users, thus increasing confidence in the accuracy and
stability of the models, and similarly, confidence in the accuracy of the results.
An indirect resulting benefit is that there is some increased level of interoperability
during verification.
Having a thorough test plan to be carried out in the verification environment is extremely
important. It allows comparison against the compliance checklists, and later on,
comparison against lab scenarios. The test plan must include error scenarios to
extensively verify correct operation.

Post Tape-Out Testing

Typical test, measurement, and verification stages post-silicon include:
 Electrical/Physical layer measurements
 Protocol level tests
 PCIe Plug Fest
 Production testing.

March 20, 2008 Doc. 0391-6K-090-B 7 of 11

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

The Electrical/Physical layer measurements at the board level are done first to check
that the product complies with the PCI Express Specification. The set-up consisted of a
real-time DSO connected via high quality (low loss) 50-ohm coax cable with SMA
connector cables to the standard Compliance Board, which is attached to the DUT.
The DSO samples run through the PCI-SIG SIGtest tool to confirm that the DUT passes
the requirements.
The DSO Data Compliance and Analysis software provides a more in-depth analysis of
the high speed signals.
The next cycle of measurements is focused at fine tuning the PHY performance over
operating conditions.
There is motivation to go beyond the compliance requirements, in order to verify and
ensure that the product will perform in the field reliably and with a large enough margin.
This assurance can be achieved by testing as many product samples as possible, and
for each product sample under test, repeating tests and accumulating statistics during
the measurements to obtain more accurate results (see Figure 2 and Figure 3).
For the protocol level tests, two tools can be used:
 the PCI-SIG PCIECV
 the PCI Express Exerciser and Analyzer
both for the standard required tests, as well as for debugging different cases in which
specific checklist cases are not passing.
The test coverage for protocol level testing is, of course, dictated by the relevant
application (endpoint, root-complex, and so on).
The PCI-SIG PCIECV and the PCI-SIG SIGtest can be downloaded directly from the
PCI-SIG forum website, www.pcisig.com.
Also, specific checklists for the desired testing type are available from the PCI-SIG
forum.
Several dry runs must be made first in the Lab, to ensure that the tests required by Plug
Fest are successfully completed. Those tests cover the electrical quality, protocol level,
and interoperability tests.
The interoperability tests are best carried out on different motherboard systems, and in
each case, being checked using both the PCIECV as well as operating an application to
validate that in each case, the PCI Express entity is correctly identified, configured, and
operates correctly.
For production testing, each vendor has to trade off between the cost of testing and the
coverage of performance that would reflect yield and potential field failures.
Typically, over time (in terms of production tests), the test coverage gets more optimized
in terms of cost performance.

8 of 11 Doc. 0391-6K-090-B March 20, 2008

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Figure 2 Example of Eye Diagram Measurement, Non-transition Eye (Worst Non-transition

Signal Eye)

Figure 3 Example of Eye Diagram Measurement (Worst Transitional Signal Eye)

Table 1 shows a sample Protocol Test Report, performed using the PCI Express
Analyzer/Exerciser. The table indicates the Specification paragraph tested, the
description of the paragraph, and the test result.
Figure 4 shows the typical user interface of the PCI Express Analyzer/Exerciser
application.

March 20, 2008 Doc. 0391-6K-090-B 9 of 11

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Table 1 Sample Protocol Test Report using the PCI Express Analyzer/Exerciser

Test Description Result

TXN.2.2#4 The receiving device must check that the size of the data payload of Pass
a Received TLP, as given by the TLP's Length field, must not exceed
the length specified by the value in the Max_Payload_Size field of the
receiver's Device Control register, taken as an integral number of
DW. If this rule is violated, then the TLP is a Malformed TLP and is
reported (if enabled) as an error associated with the receiving port.

Figure 4 Example of PCI Express Analyzer/Exerciser Typical Application

10 of 11 Doc. 0391-6K-090-B March 20, 2008

 ChipX White Paper
Solving PCI Express Design Challenges in an SoC ASIC

Terms
The following acronyms are used in this paper.

BIST built-in self-test

DRC design rule check
DSO digital storage oscilloscope
DUT device under test
EDA electronic design automation
ESD electrostatic discharge
NRE non-recurring engineering
PCI peripheral component interconnect
PCIe PCI Express
PCIECV PCI Express Configuration Test software from www.pcisig.com
PHY PHYsical layer interface
PRBS pseudo random bit sequence
SMA subminiature version A (coaxial RF connector)
TLP transaction layer packet

ChipX, Incorporated
2323 Owen Street
Santa Clara, CA 95054
Tel: 408-988-2445
800-95-CHIPX
Fax: 408-988-2449
moreinfo@chipx.com
www.ChipX.com

© 2008 ChipX, Incorporated. All rights reserved.

Disclaimer This document is provided for general information only. ChipX makes every effort to improve products for
its customers on an ongoing basis. Specifications are subject to change without notice. Trademarks are property of
their owners. Errors and omissions excluded (E&OE).

March 20, 2008 Doc. 0391-6K-090-B 11 of 11