HP BladeSystem c-Class architecture

technology brief

Abstract.............................................................................................................................................. 3
Evaluating requirements for next-generation blades.................................................................................. 3
HP solution: HP BladeSystem c-Class architecture..................................................................................... 3
HP BladeSystem c7000 product description............................................................................................ 4
General-purpose compute environment................................................................................................... 5
Physically scalable form factors.......................................................................................................... 5
Blade form factors ........................................................................................................................ 5
Interconnect form factors ............................................................................................................... 7
Star topology ............................................................................................................................... 7
NonStop signal midplane provides flexibility ....................................................................................... 8
Physical layer similarities among I/O fabrics ................................................................................... 8
Connectivity between blades and interconnect modules .................................................................. 10
NonStop signal midplane enables modularity.................................................................................... 11
BladeSystem c-Class architecture provides high bandwidth and compute performance............................... 11
Server-class components ................................................................................................................. 11
NonStop signal midplane scalability and reliability............................................................................ 11
Best practices............................................................................................................................. 12
Separate power backplane ......................................................................................................... 12
Channel topology and emphasis settings....................................................................................... 13
Passive midplane........................................................................................................................ 14
Power backplane scalability and reliability........................................................................................ 14
Power and cooling architecture with HP Thermal Logic ........................................................................... 15
Server blades and processors .......................................................................................................... 15
Enclosure ...................................................................................................................................... 16
Meeting data center configurations............................................................................................... 16
High-efficiency voltage conversions .............................................................................................. 16
Dynamic Power Saver Mode........................................................................................................ 16
Active Cool fans......................................................................................................................... 16
Mechanical design features ......................................................................................................... 17
Configuration and management technologies ....................................................................................... 17
Integrated Lights-out technology ....................................................................................................... 18
 Onboard Administrator................................................................................................................... 18
Virtualized network infrastructure with Virtual Connect technology ....................................................... 20
Availability technologies..................................................................................................................... 21
Redundant configurations................................................................................................................ 21
Reliable components....................................................................................................................... 21
Reduced logistical delay time .......................................................................................................... 22
Conclusion........................................................................................................................................ 22
For more information.......................................................................................................................... 24
Appendix: Acronyms in text ................................................................................................................ 25
Call to action .................................................................................................................................... 26
Abstract
This technology brief describes what a general-purpose infrastructure is and how the BladeSystem
c-Class architecture was designed to be a general-purpose, flexible infrastructure. The brief describes
how the BladeSystem c-Class architecture solves some major data center and server blade issues. For
example, it provides ease-of configuration and-management, reduces facilities operating costs, and
improves flexibility and scalability, all while providing high compute performance and availability.
This technology brief describes the rationale behind the BladeSystem c7000 implementation, which is
the first product implementation of the c-Class architecture. To ensure that customers understand the
basic components of the BladeSystem c-Class, the brief gives a short description of the product
implementation and how the components work together. Other technology briefs provide detailed
information about the product implementation. The section titled “For more information” at the end of
this paper lists the URLs for these and other pertinent resources.
It is assumed that the reader is familiar with HP ProLiant server technology and has some knowledge
of general BladeSystem architecture.

Evaluating requirements for next-generation blades

More critically than ever, data center administrators need the ability to use their computing resources
fully, to have agile computing resources that can change and adapt as business needs change, to
have 24/7 availability, and to manage power and cooling costs, even as systems become more
power-hungry and facilities costs rise.
Server blades have solved some data center problems by increasing density, but they also have
introduced other issues. For example, higher density typically means higher power and cooling costs
and the need for purchasing more blade switches to manage in the networking infrastructure.
Administrators installing blade architectures must be able to effectively amortize the cost of the
infrastructure over the number of blades installed.
In evaluating computing trends, HP saw clearly that significant changes affecting I/O, processor, and
memory technologies were on the horizon:
• New serialized I/O technologies to meet demands for greater I/O bandwidths
• More complex processors using multi-core architectures that would impact system sizing
• Higher-power processors and memory formats requiring more power that may cause data center
administrators to rethink how servers are deployed
• Server virtualization tools that would also affect processor, memory, and I/O configurations per
server
Therefore, HP determined that its next-generation blade environment should address as many of these
issues as possible to solve customer needs in the data center.

HP solution: HP BladeSystem c-Class architecture

The HP BladeSystem architecture supports full-featured server blades in a highly dense form factor and
supports new serial I/O technologies. HP took the opportunity in this architecture to make the
compute, network, and storage resources extremely modular and flexible, and to create a general-
purpose, adaptive infrastructure that can accommodate continually changing business needs.
Moreover, the extremely efficient BladeSystem c-Class architecture addresses the growing concern of
balancing compute performance with the power and cooling capacity of the data center. HP
architected an adaptive infrastructure that would meet the data center challenges of efficiently

3
managing resources and reducing personnel costs to install and configure systems, all while
increasing compute performance.
The HP BladeSystem c7000 enclosure, announced in June 2006, is the first enclosure implemented
using the BladeSystem c-Class architecture. The BladeSystem c7000 enclosure is optimized for
enterprise data centers. In the future HP intends to release c-Class enclosure sizes optimized for other
computing environments, such as remote sites or small businesses. The BladeSystem c-Class
architecture supports common form-factor components, so that modules such as server blades,
interconnects, and fans can be used in other c-Class enclosures.

HP BladeSystem c7000 product description

The HP BladeSystem c-Class system includes an enclosure, server blades, storage blades, interconnect
modules (switches and pass-thru modules), a NonStop signal midplane that connects blades to the
interconnect modules, a shared power backplane, power supplies, fans, and enclosure management
controllers (Onboard Administrator modules). The BladeSystem c-Class uses redundant and hot-
pluggable components extensively to provide maximum uptime to the enclosure. Figure 1 shows the
c7000 implementation of the architecture.

Figure 1. HP BladeSystem c7000 enclosure as viewed from the front and the rear

c7000 enclosure - front c7000 enclosure - rear

Half-height Full-height 8 interconnect bays
server blade server blade Storage blade
Single-wide or double-wide

10 U

Insight Display Redundant

fans
Redundant power
supplies
Redundant Redundant
Onboard single phase or
Administrators 3-phase power

This section discusses the components that comprise the BladeSystem c-Class; it does not discuss
details about all the particular products that HP has announced or plans to announce. For specific
product implementation details, the reader should refer to the HP BladeSystem website 1 and the
c-Class technology briefs on the ISS technology page. 2

1
Available at www.hp.com/go/blades
2
Available at www.hp.com/servers/technology

4
The HP BladeSystem c7000 enclosure will hold up to 16 half-height server or storage blades, or up to
eight full-height server blades, or a combination of the two blade form factors. Optional mezzanine
cards within the server blades provide network connectivity to the interconnect modules. The
connections between server blades and the network fabric can be fully redundant. Customers can
install their choice of mezzanine cards and interconnect modules for network fabric connectivity in the
eight interconnect bays at the rear of the enclosure.
The enclosure houses either one or two Onboard Administrator modules. Onboard Administrator
provides intelligence throughout the infrastructure to monitor power and thermal conditions, ensure
hardware configurations are correct, and simplify network configuration. The Insight Display panel on
the front of the enclosure simplifies configuration and maintenance. Customers have the option of
installing a second Onboard Administrator module that acts as a completely redundant controller in
an active-standby mode.
The c7000 enclosure can use either single-phase or three-phase power inputs and can hold up to six
2250 W power supplies. The power supplies connect to a passive power backplane that distributes
the power to all the components in a shared manner.
To cool the enclosure, HP designed a fan known as the Active Cool fan. The c7000 enclosure can
hold up to ten hot-pluggable Active Cool fans. The Active Cool fans are designed for high efficiency
and performance to provide redundant cooling across the enclosure as well as providing ample
capacity for future cooling needs.

General-purpose compute environment

The BladeSystem c-Class is designed as a general-purpose computing environment. The enclosure
itself―with its device bays, interconnect bays, NonStop signal midplane, and Onboard
Administrator―is a general-purpose system that can be configured with many different options of
blades and interconnect devices.
The general-purpose environment is scalable and flexible. The BladeSystem c-Class architecture
provides:
• Physically scalable form factors for blades and interconnects
• Scalable bandwidth and flexible configurations through the high-performance NonStop signal
midplane

Physically scalable form factors

The basic architectural model for the BladeSystem c-Class uses device bays (for server or storage
blades) and interconnect bays (for interconnect modules providing I/O fabric connectivity) that enable
a scale-out or a scale-up architecture.

Blade form factors

Figure 2 shows how a single device bay could accommodate two half-height server blades, stacked in
an over/under configuration in a scale-out configuration, or a full-height, higher-performance blade in
a scale-up configuration.
The ability to use either form factor in the same space enables efficient real estate use. Customers can
fully populate the enclosure with high-performance blades for a backend database or with
mainstream, 2P blades for web or terminal services. Alternatively, customers can populate the
enclosure with some mixture of the two form factors. 3

3
The c7000 enclosure uses a shelf to hold the half-height blades. When the shelf is in place, it spans two device
bays, so there are currently some restrictions on how the enclosure can be configured.

5
Figure 2. After evaluating slim blades (left-) and wide blades (right), HP selected the wide blade form factor to
support cost, reliability, and ease-of-use requirements.

Slim Form Factor Wide Form Factor

Single-Wide Double-Wide Full-Height

Blades Blade Blade

Backplane
connectors on
different PCBs Midplane connectors
on the same printed
circuit board (PCB)

Half-Height
Blades

Room for tall

heat sink
Slanted memory
DIMMs
Vertical memory
DIMMs

There are two general approaches to scaling the device bays: scaling horizontally, by providing bays
for single-wide and double-wide blades, or scaling vertically by providing half-height and full-height
blades (as shown in Figure 2). HP chose to use the stacked configuration that scales vertically and
provides a wider bay for the blades.
The stacked configuration of the device bays offers several advantages:
• Supports commodity performance components for reduced cost, while housing a sufficient number
of blades to amortize the cost of the enclosure infrastructure (such as power supplies and fans that
are shared across all blades within the enclosure).
• Provides simpler connectivity and better reliability to the NonStop signal midplane when expanding
to a full-height blade because the two signal connectors are on the same printed circuit board (PCB)
plane, as shown in Figure 2.
• Enables the use of vertical DIMMs in the server blades for cost-effectiveness.
• Provides improved performance because the vertical DIMM connectors enable better signal
integrity, more room for heat sinks, and allow for better airflow across the DIMMs.

Using vertical DIMM connectors, rather than angled DIMM connectors, provides more DIMM slots per
processor and requires a smaller footprint on the PCB. Having more DIMM slots allows customers to
choose the DIMM capacity that meets their cost/performance requirements. Because higher-capacity
DIMMs typically cost more per GB than lower-capacity DIMMs, customers may find it more cost-
effective to have more slots that can be filled with lower capacity DIMMs. For example, if a customer
requires 16 GB of memory capacity, it is more cost-effective to populate eight slots with lower cost, 2-
GB DIMMs, rather than populating four slots with 4-GB DIMMs. On the other hand, power
consumption can go up with the number of DIMMs installed, so customers should evaluate the cost of
power against the purchase cost of DIMMs.

6
Interconnect form factors
A single interconnect bay can accommodate two smaller interconnect modules in a scale-out
configuration or a larger, higher-bandwidth interconnect module for scale-up performance (Figure 3).
This provides the same efficient use of space as the scale-up/scale-out device bays. Customers can fill
the enclosure as needed in their environments.

Figure 3. HP selected a horizontal interconnect form factor that would provide efficient use of space and
improved performance.

Single-wide interconnect modules

Double-wide Two midplane

interconnect connectors on the same
modules PCB

The use of scalable, side-by-side interconnect modules provides many of the same advantages as the
scalable device bays:
• Simpler connectivity and improved reliability when scaling from a single-wide to a double-wide
module because the two signal connectors are on the same horizontal plane
• Improved signal integrity because the interconnect modules are located in the center of the
enclosure, while the blades are located above and below to provide the shortest possible trace
widths between interconnect modules and blades
• Optimized form factors for supporting the maximum number of interconnect modules

The single-wide form factor in the c7000 enclosure accommodates up to eight typical Gigabit
Ethernet (GbE) or Fibre Channel switches with16 uplink connectors. The double-wide form factor
accommodates up to four 10 GbE and InfiniBand switches with up to 16 uplink connectors.

Star topology
The result of the scalable device bays and scalable interconnect bays is a fan-out, or star, topology
centered around the interconnect modules. The exact star topology will depend upon the customer
configuration. For example, if two single-wide interconnect modules are placed side-by-side as shown
in Figure 4, the architecture is referred to as a dual-star topology: Each blade has redundant
connections to the two interconnect modules. If a double-wide interconnect module is used in place of
two single-wide modules then it is a single star topology that provides more bandwidth to each of the
blades. When using a double-wide module, redundant connections would be configured by placing
another double-wide interconnect module in the enclosure.

7
Figure 4. The scalable device bays and interconnect bays enable redundant star topologies that differ depending
on the customer configuration.

blades blades

Interconnect Module A
Interconnect Interconnect
Module A Module B Interconnect Module B

blades blades

The c7000 enclosure supports multiple dual-star topologies depending on the interconnect modules
installed, for example:
• Quad-dual-star with eight single-wide modules
• Triple-dual-star with two single-wide Ethernet modules, two single-wide Fibre Channel modules and
two double-wide InfiniBand modules
• Dual-dual-star with four double-wide interconnect modules

NonStop signal midplane provides flexibility

Because the BladeSystem c7000 enclosure includes blades, interconnect modules, and a midplane to
connect them, customers might think it is the same basic architecture as the BladeSystem p-Class or as
other blade environments provided by competitors. However, the BladeSystem c-Class provides an
entirely different architecture by using a high-speed, NonStop signal midplane that provides the
flexibility to intermingle blades and interconnect fabrics in many ways to solve a multitude of
application needs.
The NonStop signal midplane is unique because it can use the same physical traces to transmit GbE,
Fibre Channel, InfiniBand, Serial Attached SCSI (SAS), or PCI Express signals. As a result, customers
can fill the interconnect bays with a variety of interconnect modules, depending on their needs: for
example, six Ethernet switches and two Fibre Channel switches; eight Ethernet switches, etc.

Physical layer similarities among I/O fabrics

The NonStop signal midplane can transmit signals from different I/O fabrics because of the physical
layer similarities of those fabrics. Serialized I/O protocols such as GbE, Fibre Channel, SAS, PCI
Express, and InfiniBand are based on a physical layer that uses multiples of four traces with the
SerDes (serializer/deserializer) interface . In addition, the backplane Ethernet standards 4 of
1000-Base-KX, 10G-Base-KX4, and 10G-Base-KR, and a future 8-Gb Fibre Channel standard 5 use a
similar four-trace SerDes interface (see Table 1).

4
IEEE 802.3ap Backplane Ethernet Standard, in development, see www.ieee802.org/3/ap/index.html for more
information.
5
International Committee for Information Technology Standards, see www.t11.org/index.htm and
www.fibrechannel.org/ for more details.

8
Table 1. Physical layer of I/O fabrics and their associated encoded bandwidths

Interconnect Lanes Number Bandwidth per lane Aggregate

of traces (Gigabits per second bandwidth
Gb/s) (Gb/s)

GbE 1x 4 1.2 1.2

(1000-base-KX)

10 GbE (10G-base-KX4) 4x 16 3.125 12.5

10 GbE (10G-base-KR) 1x 4 10 10.3

Fibre Channel 1x 4 1.06, 2.12, 1.06, 2.12,

(1, 2, 4, 8 Gb) 4.2, 8.5 4.2, 8.5

Serial Attached SCSI 1x 4 3 3

InfiniBand 1x – 4x 4 – 16 2.5 2.5 – 10

InfiniBand DDR 1x – 4x 4 – 16 5 5 – 20
InfiniBand QDR 1x – 4x 4 – 16 10 10 – 40

PCI Express 1x – -4x 4 – 16 2.5 2.5 – 10

PCI Express (generation 2) 1x – 4x 4 – 16 5 5 – 20

By taking advantage of the similar four-trace differential SerDes transmit and receive signals, the
signal midplane can support either network-semantic protocols (such as Ethernet, Fibre Channel, and
InfiniBand) or memory-semantic protocols (PCI Express), using the same signal traces. Consolidating
and sharing the traces between different protocols enables an efficient midplane design. Figure 5
illustrates how the physical lanes can be logically overlaid onto sets of four traces. Interfaces such as
GbE (1000-base-KX) or Fibre Channel need only a 1x lane (a single set of four traces). Higher
bandwidth interfaces, such as InfiniBand DDR, will need to use up to four lanes. Therefore, the choice
of network fabrics will dictate whether the interconnect module form factor needs to be single-wide
(for a 1x/2x connection) or double-wide (for a 4x connection).

Figure 5. Traces on the NonStop signal midplane can transmit many different types of signals, depending on
which I/O fabrics are used. The right-hand side of the diagram represents how the signals can be overlaid so
that different protocols can be used (one at a time) on the same traces.

1x 2x
(KX, KR, SAS, (SAS,
Fibre Channel) PCI Express)

1X

Lane-0 2X

Lane-0
Lane-1
4X
Lane-0
Lane-0 Lane-1 4x
Lane-1 Lane-2 (KX4, InfiniBand,
Lane-2 Lane-3 PCI Express)
Lane-3

9
Re-using the traces in this manner avoids the problems of having to replicate traces to support each
type of fabric on the NonStop signal midplane or of having large numbers of signal pins for the
interconnect module connectors. Thus, by overlaying the traces, the interconnect module connectors
are simplified, the midplane enables efficient real estate use, and customers are assured of flexible
connectivity. These benefits are also realized in each server blade design.

Connectivity between blades and interconnect modules

The c-Class server blades use mezzanine cards to connect to various network fabrics. The connections
between the multiple types of mezzanine cards on the server blades are hard-wired through the
NonStop signal midplane to specific interconnect bays (Figure 6).

Note.
See the paper titled “HP BladeSystem c-Class enclosure” for complete
details about how the half-height and full-height blades connect to the
interconnect bays.

Figure 6. Diagram showing how c-Class half-height server blades connect redundantly to the interconnect bays.
The four colors indicate corresponding ports between the server blades and interconnect bays.

c-Class Server Blades c-Class Interconnect Bays

Blade 1/2
1 NIC1
NIC2 1 2
1 3/4
Mezz 1 2

1
2
5/6 3 4
Mezz 2 7/8
3
4

Blade
2 NIC1 1/2
NIC2 5 6
1 3/4
Mezz 1 2

1 5/6 7 8
2
Mezz 2 3 7/8
4

To provide such inherent flexibility of the NonStop signal midplane, the architecture must provide a
mechanism to properly match the mezzanine cards on the server blades with the interconnect
modules. For example, within a given enclosure, all mezzanine cards in the mezzanine 1 connector
of the server blades must support the same type of fabric. If one server blade in the enclosure has a
Fibre Channel card in the mezzanine 1 connector, another server blade cannot have an Ethernet card
in its mezzanine 1 connector.
HP developed the electronic keying mechanism in Onboard Administrator to assist system
administrators in recognizing and correcting potential fabric mismatch conditions as they configure
each enclosure. Before any server blade or interconnect module is powered up, the Onboard
Administrator queries the mezzanine cards and interconnect modules to determine compatibility. If

10
Onboard Administrator detects a configuration problem, it provides a warning with information about
how to correct the problem.

NonStop signal midplane enables modularity

In addition to the flexibility it provides, the architecture of the NonStop signal midplane makes it
possible to develop even more modular components than those in previous generations of blade
systems. New types of components can be implemented in the blade form factor and connected
across the NonStop signal midplane to form different compute structures. For example, HP is
introducing storage blade solutions that communicate across the midplane. Storage blades provide
another option for disk drive capacity rather than standard internal disk drives or drives in a storage
area network (SAN).
Future implementations of the c-Class architecture may allow for greater modularity of other
components, such as local I/O interconnects that span across adjacent blade bays, or local I/O
interconnects that span between blade bays and interconnect bays. These possibilities exist because
the NonStop signal midplane can carry either network-semantic traffic or memory-semantic traffic
using the same sets of traces. By designing the c-Class enclosure to be a system, HP made the
architecture truly adaptive and able to meet the needs of IT applications today and in the future.

BladeSystem c-Class architecture provides high bandwidth

and compute performance
A requirement for any server architecture is that it provides high compute performance and bandwidth
to meet future customer needs. The BladeSystem c-Class enclosure was architected to ensure that it
can support upcoming technologies—in their demand for both bandwidth and power—for at least the
next 5 to 7 years. It provides this through:
• Blade form factors that enable server-class components
• High-bandwidth NonStop signal midplane
• Separate power backplane

Server-class components
To ensure longevity for the c-Class architecture, HP uses a 2-inch blade form factor that allows the use
of server-class, high-performance components. Using a wide blade form factor allowed HP to design
half-height servers supporting the most common server configuration: two processors, eight full-size
DIMM slots with vertical DIMM connectors, two Small Form Factor (SFF) disk drives, and two optional
mezzanine cards. When scaled up to the full-height configuration, the server blades can support
approximately twice the resources of a half-height server blade: for example, four processors, sixteen
full-size DIMM slots, four SFF drives, and three optional mezzanine cards. Future versions of blades
may be able to support even more functionality in the same form factor.

NonStop signal midplane scalability and reliability

The NonStop signal midplane is capable of conducting extremely high signal rates of up to 10 Gb/s
per lane (that is, per set of four differential transmit/receive traces). Therefore, each half-height server
blade has the cross-sectional bandwidth to conduct up to 160 Gb/s per direction. In an enclosure
fully configured with 16 half-height server blades, the aggregate bandwidth is up to 5 Terabits/sec
across the NonStop signal midplane. 6 This is bandwidth between the blade bays and interconnect
bays only. It does not include traffic between interconnect modules or blade-to-blade connections.

6
Aggregate backplane bandwidth calculation: 160 Gb/s x 16 blades x 2 directions = 5.12 Terabits/s

11
To achieve this level of bandwidth between bays, HP had to give special attention to maintaining
signal integrity of the high-speed signals. This was achieved through:
• Using general signal-integrity best practices to minimize end-to-end signal losses across the signal
midplane
• Moving the power into an entirely separate backplane to independently optimize the NonStop
signal midplane
• Providing a means to set optimal signal waveform shapes in the transmitters, depending on the
topology of the end-to-end signal channel

Best practices
Following best practices for signal integrity was important to ensure high-speed connectivity among all
16 blades and 8 interconnect modules. To aid in the design of the c7000 signal midplane, HP
involved the same signal integrity experts that design the HP Superdome computers. Specifically, HP
paid special attention to:
• Controlling the differential impedance along each end-to-end channel on the PCBs and through the
connector stages
• Planning signal pin assignments so that receive signal pins are grouped together while being
isolated from transmit signal pins by a ground plane (see Figure 7).
• Keeping signal traces short to minimize losses
• Routing signals in groups to minimize signal skew
• Reducing the number of through-hole via stubs by carefully selecting the layers to route the traces,
controlling the PCB thickness, and back-drilling long via stubs to minimize signal reflections

Figure 7. To achieve efficient routing across the midplane and to minimize cross-talk, receive signal pins are
separated by a ground plane from the transmit signal pins.

Receive Signal Pins

Interconnect Bay Connector

Transmit Signal Pins

Separate power backplane

Distributing power on the same PCB that includes the signal traces would have greatly increased the
board complexity. Separating the power backplane from the NonStop signal midplane improves the
signal midplane by reducing its PCB thickness, reducing electrical noise (from the power components)
that would affect high-speed signals, and improving the thermal characteristics. These design choices
result in reduced cost, improved performance, and improved reliability.

12
Channel topology and emphasis settings
Even when using best practices, high-speed signals transmitted across multiple connectors and long
PCB traces can significantly degrade due to insertion and reflection losses. Insertion losses, such as
conductor and dielectric material losses, increase at higher frequencies. Reflection losses are due to
impedance discontinuities, primarily at connector stages. To compensate for these losses, a
transmitter’s signal waveform can be shaped by selecting the signal emphasis settings. The goal is to
anticipate the losses in such a way that after the signal travels across the entire channel, the waveform
will still have an adequate wave shape and amplitude for the receiver to successfully detect the
correct signal levels (Figure 8).

Figure 8. Hypothetical example showing how a signal (a) can degrade after traveling through a channel where
the leading portions of the signal waveform are attenuated. If the signal’s trailing bits of the same polarity are
de-emphasized (signal b), the signal quality is improved at the receiver.

However, the emphasis settings of a transmitter can depend on the end-to-end channel topology as
well as the type of component sending the signal. Both can vary in the BladeSystem c-Class because
of the flexible architecture and the use of mezzanine cards and embedded I/O devices such as NICs
(Figure 9). Therefore, HP provided a supplementary method to the electronic keying mechanism with
the Onboard Administrator to ensure proper emphasis settings based on the configuration of the
c-Class enclosure.

13
Figure 9. The electronic keying mechanism in the BladeSystem c-Class identifies the channel topology for each
device and Onboard Administrator ensures that the proper emphasis settings are defined. In this example, the
topology for Device 1 on server blade 1 (a-b-c) is completely different than the topology for device 1 on server
blade 4 (a-d-e).

Server blade-1 Midplane Switch-1 PCB

a b PCB c Switch
e Device
DEV-1

Server blade-4
a d

DEV-1 Enclosure
Manager

Passive midplane
Finally, to provide high reliability, the NonStop signal midplane was designed as a completely
passive board. The PCB consists primarily of traces and connectors. While there are a few
components on the PCB, they are limited to passive devices that are extremely unlikely to fail. The
only active device is the FRU EEPROM, which Onboard Administrator uses to acquire information
such as the midplane serial number. Failure of this device does not affect the signaling functionality of
the NonStop signal midplane. The NonStop signal midplane follows best design practices and is
based on the same type of passive midplane used for decades in fault-tolerant computers such as the
HP NonStop S-series.

Power backplane scalability and reliability

The power backplane is constructed of solid copper plates and integrated power delivery pins to
ensure power distribution with minimum losses (Figure 10). The use of a solid copper plate enables
low voltage drops, high current density, and high reliability. Regardless of how blades may increase
their power demands in the next few years, this power backplane will provide more than enough
amperage.

14
Figure 10. Sketch of the c-Class power backplane showing the power delivery pins

Power
delivery pins
for the fan
modules
Power delivery
pins for the
switch modules
Power
delivery pins
for the server Power feet that
blades attach to the
power supplies
connector board

Power and cooling architecture with HP Thermal Logic

Power conservation and efficient cooling were key design goals for the BladeSystem c-Class. To
achieve these goals, HP consolidated the power and cooling resources, while efficiently sharing and
managing them within the enclosure. HP uses the term Thermal Logic to refer to the mechanical
features and control capabilities throughout the BladeSystem c-Class that enable IT administrators to
optimize their power and thermal environments.
Thermal Logic encompasses technologies at every level of the c-Class architecture: CPU, server
blades, Active Cool fans, and the c-Class enclosure. Through the Onboard Administrator controller, IT
administrators can have access to real-time power and temperature data, allowing them to understand
their current power and cooling environment. Onboard Administrator also allows customers to
dynamically and automatically adjust operating conditions to meet their data center requirements.
This allows customers to maximize performance based on their power and cooling budgets, and
forestall expensive power and cooling upgrades.
The technology brief titled “HP BladeSystem c-Class enclosure” gives additional information about HP
Thermal Logic technologies. It is available on the HP technology website at
www.hp.com/servers/technology.

Server blades and processors

At the CPU level, HP Power Regulator for ProLiant 7 is a ROM-based power management feature of HP
ProLiant servers. Power Regulator technology takes advantage of the power states available on Intel
x86 processors to scale back the power to a processor when it is not needed. 8 Because the c-Class

7
For additional information about Power Regulator for ProLiant, see
http://h18000.www1.hp.com/products/servers/management/ilo/power-regulator.html
8
Power states of AMD x86 processors can be changed manually, but the change is not integrated with Power
Regulator and requires a system reboot.

15
architecture shares power among all blades in an enclosure, HP will be able to take advantage of
Power Regulator technology to balance power loads among the server blades. As processor
technology progresses, HP can recommend that customers use lower-power processor and component
options when and where possible.
A specially designed heat sink for the CPU provides efficient cooling in a smaller space. This allows
the server blades to include full-size, fully-buffered memory modules and hot-plug drives.
Most importantly, c-Class server blades incorporate intelligent management processors (Integrated
Lights-Out 2, or iLO 2, for ProLiant server blades, or Integrity iLO for Integrity server blades) that
provide detailed thermal information for every server blade. This information is forwarded to the
Onboard Administrator and is accessible through the Onboard Administrator web interface.

Enclosure
At the enclosure level, HP provides:
• Power designed to meet data center configurations
• High-efficiency voltage conversions
• Dynamic Power Saver mode to operate power supplies at high efficiencies
• Active Cool Fans that minimize power consumption
• Mechanical design features to optimize airflow

Meeting data center configurations

Rather than design the power budgets for the c-Class architecture based on the anticipated
requirements of server blades, HP designed the c-Class enclosure to conform to typical data center
facility power feeds. Thus, the enclosure is sized not only to amortize the cost of blades across the
infrastructure, but also to support the most blades possible while using the power available today. As
IT facilities managers choose to increase the number of power feeds into their facility, c-Class
enclosures can be added that will fit into those typical power feed budgets. Because the entire
enclosure is sized to meets today’s power infrastructure, there is no need for a separate power
enclosure.
The c-Class architecture is designed to use either single-phase or three-phase power enclosures,
providing customers with the flexibility to choose whichever fits into their data center.

High-efficiency voltage conversions

Moving the power supplies into the enclosure reduced the distance over which power would need to
be distributed. This allowed HP to use an industry-standard 12V infrastructure for the c-Class
BladeSystem. Using a 12V infrastructure eliminates several power-related components and improves
power efficiency on the server blades and infrastructure.

Dynamic Power Saver Mode

Most power supplies operate inefficiently when lightly loaded and more efficiently when heavily
loaded. When enabled, Dynamic Power Savings mode will save power by running the required
power supplies at a higher rate of utilization and putting unneeded power supplies in a standby
mode. When power demand increases, the standby power supplies instantaneously deliver the
required extra power. As a result, the enclosure can operate at optimum efficiency, with no impact on
redundancy. Both efficiency and redundancy are possible because the power supplies are
consolidated and shared across the enclosure.

Active Cool fans

Quite often, small form-factor servers such as blade or 1U servers use very small fans designed to
provide localized cooling in specific areas. Because such fans generate fairly low flow (in cubic feet

16
per minute, or CFM) at medium back pressure, a single server often requires multiple fans to ensure
adequate cooling. Therefore, when many server blades, each with several fans, are housed together
in an enclosure, there is a trade-off between powering the fans and cooling the blades. While this
type of fan has proven to scale well in the BladeSystem p-Class, HP believed that a new design could
better balance the trade-off between power and cooling.
A second solution for cooling is to use larger, blower-style fans that can provide cooling across an
entire enclosure. Such fans are good at generating CFM, but typically also require higher power
input, produce more noise, and must be designed for the highest load in an enclosure. Because these
large fans cool an entire enclosure, a failure of a single fan can leave the enclosure at risk of
overheating before the redundant fan is replaced.
With these two opposing solutions in mind, HP solved these problems by designing the Active Cool
fan and by aggregating the fans to provide redundant cooling across the entire enclosure.
The Active Cool fans are controlled by the Onboard Administrator so that cooling capacity can be
ramped up or down based on the needs of the entire system. Along with optimizing the airflow, this
control algorithm allows the c-Class BladeSystem to optimize the acoustic levels and power
consumption. Because of the mechanical design and the control algorithm, Active Cool fans deliver
better performance—at least three times better than the next best fan in the server industry. As a result
of the Active Cool fan design, the c-Class enclosure supports full-featured servers that are 60 percent
more dense than traditional rack mount servers. Moreover, the Active Cool fans consume only 50
percent of the power typically required and use 30 percent less airflow. By aggregating the cooling
capabilities of a few, high-performance fans, HP was able to reduce the overhead of having many,
localized fans for each server blade—thereby simplifying and reducing the cost of the entire
architecture.
Mechanical design features
The overall mechanical design of the enclosure is a key component of Thermal Logic technologies.
The enclosure uses PARSEC architecture—parallel, redundant, scalable, enclosure-based cooling. In
this context, parallel means that fresh, cool air flows over all the blades (in front of enclosure) and all
the interconnect modules (in the back of the enclosure). Fresh air is pulled into the interconnect bays
through a dedicated side slot in the front of the enclosure. Ducts move the air from the front to the rear
of the enclosure, where it is then pulled into the interconnects and the central plenum, and then
exhausted out the rear of the system.
Each power supply module has its own fan, optimized for the airflow characteristics of the power
supplies. Because the power supplies and facility power connections are in a separate region of the
enclosure, the fans can provide fresh, cool air and clear exhaust paths for the power supply modules
without interfering with the airflow path of the server blades and interconnect modules.
Because the enclosure is designed into four physical cooling zones, the Active Cool fans provide
cooling for their own zone and redundant cooling for the rest of the enclosure. One or more fans can
fail and still leave enough fans to adequately cool the enclosure.
To ensure scalability, HP designed both the fans and the power supplies with enough capacity to
meet the needs of compute, storage, and I/O components well into the future.
HP optimized the cooling capacity across the entire enclosure by optimizing airflow and minimizing
leakage through the use of a relatively airtight central plenum, self-sealing louvers surrounding the
fans, and automatic shut-off doors surrounding the device bays.

Configuration and management technologies

As stated in the introduction, HP developed the c-Class architecture to be an agile computing
infrastructure that can adapt to changes in business needs. Specifically, one of the goals of the
BladeSystem c-Class architecture was to dramatically reduce the amount of time that IT personnel must

17
spend to deploy new systems. Another goal was to provide an intelligent infrastructure that can
provide essential power and cooling information to administrators and help automate the
management of the infrastructure. The BladeSystem c-Class provides such an intelligent infrastructure
through the iLO 2 management processor and Onboard Administrator; it provides an
easy-to-configure system through the unique Insight Display function of the Onboard Administrator.
The BladeSystem c-Class architecture also reduces the complexities of switch management in a blade
environment. While blade environments provide distinct advantages because of their direct backplane
connections between switches and blades (reducing the number of cables, and therefore cost and
complexity), they still present the challenge of managing many additional small switches. HP has
solved this in an innovative way by developing Virtual Connect technology. The Virtual Connect
technology provides a way to virtualize the server I/O connections to the Ethernet or Fibre Channel
networks.
The technology briefs titled “Managing the HP BladeSystem c-Class” and “HP Virtual Connect
technology implementation for the HP BladeSystem c-Class” give detailed information about these
technologies. They are available on the HP technology website at www.hp.com/servers/technology.

Integrated Lights-out technology

Each ProLiant server blade designed for the BladeSystem c-Class includes an iLO 2 management
processor. The iLO 2 processor monitors thermal and operational conditions within each server blade
and forwards this information on to the Onboard Administrator. Regardless of a server blade’s
operating condition, the iLO 2 management processor enables the remote management capabilities
that customers have come to expect from ProLiant servers: access to a remote console, virtual media
access, virtual power button, and system management information such as hardware health, event
logs, and configuration. The iLO 2 device 9 provides a higher-performance remote console (virtual
KVM) as well as virtual media functionality that administrators can access from a web browser,
command line, or script. The virtual KVM uses an architecture that acquires video directly from the
video controller and uses an enhanced compression and refresh technology that reduces the amount
of traffic on the network (thereby improving network efficiency).

Onboard Administrator
Onboard Administrator is a management controller module that resides within the BladeSystem
c-Class enclosure. The Onboard Administrator controller communicates with the iLO 2 management
processors on each server blade to form the core of the management architecture for BladeSystem
c-Class. Customers have the option of installing a second Onboard Administrator board in the c7000
enclosure to act as a completely redundant controller in an active-standby mode. IT technicians and
administrators can access the Onboard Administrator through the c7000 enclosure’s LCD display (the
Insight Display), through a web GUI, or through a command-line interface.
Onboard Administrator collects system parameters related to thermal and power status, system
configuration, and managed network configuration. It manages these variables cohesively and
intelligently so that IT personnel can configure the BladeSystem c-Class and manage it in a fraction of
the time that other solutions require.
Onboard Administrator monitors thermal conditions, power allocations and outputs, hardware
configurations, and management network control capabilities.
If thermal load increases, the Onboard Administrator’s thermal logic feature instructs the fan
controllers to increase fan speeds to accommodate the additional demand.
The Onboard Administrator manages power allocation rules and power capacity limits of various
components. It uses sophisticated power measurement sensors to accurately determine how much

9
iLO 2 is the fourth-generation of lights-out remote management.

18
power is being consumed and how much power is available. Because Onboard Administrator uses
real-time, measured power data instead of maximum power envelopes, customers can deploy as
many servers and interconnects as possible for the available power.
One of the major advantages of the BladeSystem c-Class is its flexibility in allowing customers to
configure the system in virtually any way they desire. To assist in the configuration and setup process
for the IT administrator, the Onboard Administrator verifies four attributes for each blade and
interconnect as they are added to the enclosure: electronic keying of interconnects and mezzanine
cards, power capacity, cooling capacity, and location of components. The electronic keying
mechanism ensures that the interconnects and mezzanine cards are compatible. It also determines the
signal topology and sets appropriate emphasis levels on the transmitters to ensure best signal
reception by the receiver after the signal passes across the high-speed NonStop signal midplane.
Onboard Administrator provides tools to automatically identify and assign IP addresses for the
BladeSystem c-Class components on existing management networks (for components supporting
DHCP). This simplifies and automates the process of configuring the BladeSystem c-Class.
The Insight Display capability (Figure 11) provides quick, onsite access to all the setup, management,
and trouble shooting features of the Onboard Administrator. For example, when the enclosure is
powered up for the first time, the Insight Display launches an installation wizard to guide an IT
technician through the configuration process. After the technician initially configures the enclosure, the
Insight Display provides feedback and advice if there are any installation or configuration errors. In
addition, technicians can access menus that provide information about Enclosure Management,
Power management, and HP BladeSystem Diagnostics. The Insight Display provides a User Note
function that is the electronic equivalent of a sticky note. Administrators can use this function to display
helpful information such as contact phone numbers or other important information. Additionally, the
Insight Display provides a bi-directional chat mode (similar to instant messaging) between the Insight
Display and the web GUI. Therefore, a technician in the data center can communicate instantly with a
remote administrator about what needs to be done.

Figure 11. The main menu on the Insight Display provides technicians with easy access to all the enclosure
settings, configuration, health information, port mapping information, and trouble shooting features for the entire
enclosure.

19
Virtualized network infrastructure with Virtual Connect technology
HP BladeSystem c-Class is designed from the ground up integrating the Virtual Connect technology.
The OnBoard Administrator, the c-Class PCI-Express mezzanine cards, the embedded NICs, and iLO
all provide functionality to support the Virtual Connect technology. The fact that the Virtual Connect
capability is so tightly integrated into the HP BladeSystem c-Class infrastructure is what allows its
functionality to be so effective and seamless.
Virtual Connect implements server-edge virtualization: It puts an abstraction, or virtualization, layer
between the servers and the external networks so that the local area network (LAN) and SAN see a
pool of servers rather than individual servers (see Figure 12). Specific interconnect modules—Virtual
Connect modules—provide the virtualized connections. The virtualization layer establishes a group of
NIC and Fibre Channel addresses for all the server blades in the specified domain and then holds
those addresses constant in software for the entire domain. If any changes need to occur (for instance,
if a server blade needs to be upgraded), the server administrator can swap out the server blade and
the Virtual Connect Manager will manage the physical NIC address changes.

Figure 12. HP Virtual Connect technology provides a virtualization layer that masks the physical mapping of
Ethernet and Fibre Channel ports from the view of the network and storage administrators.

From the network administrator’s perspective, the LAN and SAN connections are established to the
group of servers, and the network administrators see no changes to their networks. This allows server
configurations to be moved, added, or changed without affecting the LAN or SAN. In addition, the
Virtual Connect modules do not participate in network control activities (such as Spanning-Tree
Protocol for Ethernet or FSPF for Fibre Channel) as a switch would, so network administrators need not
be concerned about having extra switches to manage on the edge of their networks.

20
HP Virtual Connect technology provides a simple, easy-to-use tool for managing the connections
between HP BladeSystem c-Class servers and external networks. It cleanly separates server enclosure
administration from LAN and SAN administration, relieving LAN and SAN administrators from server
maintenance. It makes HP BladeSystem c-Class server blades change-ready, so that server
administrators can add, move, or replace those servers without affecting the LANs or SANs.

Availability technologies
The BladeSystem c-Class incorporates layers of availability throughout the architecture to enable the
24/7 infrastructure needed in data centers. The BladeSystem c-Class provides availability through the
use of redundant configurations to eliminate single points of failure and through the architecture that
reduces the risk of component failures and reduces the time required for changes.

Redundant configurations
The BladeSystem c-Class minimizes the chances of a failure by providing redundant modules and
paths.
The c-Class architecture includes redundant power supplies, fans, interconnect modules, and Onboard
Administrator modules. For example, customers have the option of using power supplies in an N+N
redundant configuration or an N+1 configuration. The interconnect modules can be placed side by
side for redundancy, as shown in Figure 6. And the enclosure is capable of supporting either one or
two Onboard Administrator modules in an active-standby configuration.
The architecture provides redundant paths through the use of multiple facility power feeds (single-
phase c7000 enclosures accept up to six IEC C19-C20 power cords, and three-phase c7000
enclosures use dual input power), blade-to-interconnect bay connectivity, and blade-to-enclosure
manager connectivity.
In addition, because all the components are hot-pluggable, administrators are able to return rapidly to
a completely redundant configuration in the event of a failure.

Reliable components
HP took every opportunity in the c-Class architecture to design for reliability, especially for critical
components that can be considered a single point of failure. Some customers might consider the
NonStop signal midplane for the BladeSystem c-Class enclosure to be a single point of failure, since it
is not replicated. However, HP mitigated this risk and made the PCB extremely reliable by:
• Designing the NonStop signal midplane to provide redundant paths between the blades and
interconnect bays
• Eliminating all active components from the PCB that would affect functionality, therefore removing
potential sources of failure
• Removing power from the NonStop signal midplane to reduce board thickness, reduce thermal
stresses, and reduce the risk of any power bus overloads affecting the data signals
• Minimizing the connector count to reduce mechanical alignment issues
• Using mechanically robust midplane connectors that also support 10 Gbps high-speed signals with
minimum crosstalk

The result is a NonStop signal midplane that has an extremely high mean time between failure.
Some customers may choose to have a single Onboard Administrator module rather than two for
redundancy. In this case, the Onboard Administrator can be a single point of failure. In the unlikely
event of an Onboard Administrator failure, server blades and interconnect modules will all continue to

21
operate normally. The module can be removed and replaced without affecting operations of the
server blades and interconnect modules.
Operating temperatures of components can play a significant role in reliability. As the operating
temperature increases beyond specified maximum values, thermal stresses increase, which results in
shortened life spans. The PARSEC design of the BladeSystem c7000 enclosure minimizes the
operating temperature of components by delivering fresh, cool air to all critical components such as
server blades, interconnect modules, and power supplies. The airflow is tightly ducted to make every
gram of airflow count—ensuring the most thermal work out of the least amount of air. The server
blades are designed with ample room for in-take air and heat sinks (both on the CPU and memory
modules). Rather than use the traditional heat sink design for the CPUs, HP designed a copper-finned
heat sink that provides more heat transfer in a smaller package than traditional heat sinks used in 1U
rack-optimized servers.
Finally, Onboard Administrator’s thermal monitoring of the entire system ensures that the Active Cool
fans deliver adequate cooling to the entire enclosure. And, because the fan design uses a high-
performance motor and impeller, it consumes less power and uses less airflow to cool an enclosure
than a traditional fan design would. Its unique fan blade, housing, motor windings, bearings, and
drive circuit means the Active Cool fan provides higher reliability than typical server fans.

Reduced logistical delay time

As with the BladeSystem p-Class and other ProLiant servers, the BladeSystem c-Class was designed
with ease-of-use as a priority. Reducing the amount of time needed to replace, upgrade, and
configure systems is enabled by several important technologies in the BladeSystem c-Class: Onboard
Administrator and the related Insight Display, Virtual Connect technology, and hot-plug devices.
Onboard Administrator and Virtual Connect technologies have already been discussed in the section
titled “Configuration and management technologies." With the intelligence of the Onboard
Administrator and the easy-to-use Insight Display panel, administrators can configure and troubleshoot
their systems in minutes, rather than hours or days. And by adopting Virtual Connect technology,
server administrators can remove administrative burdens from LAN and SAN administrators so that
they are not required to change their network setup every time a configuration change occurs within
the server blade environment. Furthermore, because the network connections are made to a pool of
server blades, it is easy to migrate the network service from a failed server blade to a functional
server blade in a short amount of time.
Finally, the fans, power supplies, interconnect modules, Onboard Administrator modules, server
blades, and storage blades are hot-pluggable, meaning that they can be removed without harming
any other components in the enclosure. For some components, for example, server or storage blades,
an administrator would want to make sure that the blade is powered down before removing it from of
the enclosure. However, there is no harm to the enclosure itself by removing a hot-plug component.

Conclusion
HP designed the BladeSystem c-Class as an architecture that would deliver on the promise of a
modular, adaptive, automated data center. To do this, HP worked very closely with our customers to
understand their requirements and challenges in managing their data centers. By combining this
knowledge with the recognition of emerging industry standards and technologies, architects from
multiple business units within HP collaborated to define the c-Class architecture, enclosure design, and
Thermal Logic cooling technologies.
The-Class architectural model provides scalable device bays and interconnect bays allowing
customers to add the components they need, when they need them. Customers can easily scale the
enclosure from the minimum of one blade to the maximum by adding more fans and power supplies,

22
because there is plenty of power and cooling headroom for future generations of blades. By
designing a unique NonStop signal midplane that can adapt to customer needs and technology
directions over multiple generations, HP has ensured flexibility and a long life for the BladeSystem c-
Class. By consolidating these resources—volume space, power, cooling, and signal traces across the
midplane—the BladeSystem c-Class ensures that resources can be shared efficiently so that the
amount of resources matches what the customer needs. The c-Class architecture is designed for
longevity and to be interoperable with server blades, storage blades, and interconnect modules for
several generations of products. The c-Class enclosure is optimally designed not only for mainstream
enterprise products, but also as a general purpose infrastructure that has the potential to support
workstation blades, storage systems, or NonStop systems in the future.
With the BladeSystem c-Class, HP has delivered even more hardware control, intelligent monitoring,
automation capabilities, and virtualization capabilities than with previous generations of blade
systems. The Onboard Administrator and Insight Display work in conjunction with the intelligent
management processors on each server blade to provide information and control to administrators. In
addition, HP has differentiated the BladeSystem c-Class from its competitors through HP Thermal Logic
that dynamically monitors and controls power and cooling in an extremely cost-effective manner, and
HP Virtual Connect technology that simplifies network management and IT changes by virtualizing
I/O connections.

23
For more information
For additional information, refer to the resources listed below.

Source Hyperlink

HP BladeSystem c-Class documentation http://h71028.www7.hp.com/enterprise/cache/316735-0-0-0-

121.html

HP BladeSystem power sizer www.hp.com/go/bladesystem/powercalculator

HP BladeSystem website www.hp.com/go/bladesystem/

HP Power Regulator for ProLiant http://h18000.www1.hp.com/products/servers/management/ilo/po

wer-regulator.html

HP Technology Briefs: http://h18013.www1.hp.com/products/servers/technology/whitepap

HP BladeSystem c-Class enclosure ers/proliant-servers.html#bl

HP Virtual Connect technology

implementation for the HP BladeSystem c-Class
Managing the HP BladeSystem c-Class
Appendix: Acronyms in text
The following acronyms are used in the text of this document.

Table A-1. Acronyms

Acronym Acronym expansion

CPU Central processing unit

DDR Double data rate

DHCP Dynamic host configuration protocol

DIMM Dual-inline memory module

EEPROM Electrically Erasable Programmable Read-Only Memory

FRU Field replaceable unit

FSPF Fabric Shortest Path First. A routing protocol for Fibre

Channel switches

Gb Gigabit

GB Gigabyte

GUI Graphical user interface

IT Information technology

I/O Input/output. Generally refers to storage or peripheral

devices such as disk drives or network interconnects.

KVM Keyboard, video, and mouse

LCD Liquid crystal display

MB Megabyte

NIC Network interface card

PCI Peripheral component interconnect

PCIe PCI Express

QDR Quad data rate

RAM Random access memory

ROM Read only memory

SAN Storage area network

SCSI Small Computer System Interface

U Unit of measurement for rack-mount equipment (a rack-unit

or U is 1.75 inches or 4.44cm)
 Call to action
Send comments about this paper to TechCom@HP.com.

© 2006 Hewlett-Packard Development Company, L.P. The information contained

herein is subject to change without notice. The only warranties for HP products and
services are set forth in the express warranty statements accompanying such
products and services. Nothing herein should be construed as constituting an
additional warranty. HP shall not be liable for technical or editorial errors or
omissions contained herein.
Intel is atrademark or registered trademark of Intel Corporation or its subsidiaries in
the United States and other countries.
TC061105TB, November 2006