White Paper

10 Gigabit Ethernet Dense Wavelength-Division Multiplexing

Interconnections in Enterprise Campus Networks
This document provides technical description about 10 gigabit ethernet dense wavelength-division multiplexing
(DWDM) pluggable optics and application information for deploying 10 gigabit ethernet dwdm interconnections
in enterprise campus networks
INTRODUCTION
Since the IEEE 802.3ae standard was ratified in 2002, the number of 10 Gigabit Ethernet deployments has grown rapidly. The 10 Gigabit
Ethernet market has grown from US$385 million in 2004 to $885 million in 2005 and has a potential of $3.4 billion in 2009, according to
the DellOro Group.
The current trend in the networking industry is consolidation. Even without specific applications requiring 10 Gbps, bandwidth needs
within the campus network have grown quickly because of network consolidation feeding the demand for higher bandwidth. Today 10
Gigabit Ethernet is deployed through the campus core, the campus distribution, and also the campus access network, to achieve a
reasonable oversubscription ratio, if deploying Gigabit Ethernet to the desktop or to servers. (See Figure 1.)
Figure 1.

10 Gigabit Ethernet Campus Design

To scale the core bandwidth beyond 10 Gigabit Ethernet, a new IEEE standard is needed. But standardization has not been started yet and
is not expected to be ratified within in the next four years. The standard Ethernet service 802.3ad link aggregation can be used to aggregate
up to eight 10 Gigabit Ethernet links into a virtual 80 Gbps connection. DWDM can be used to transport various channels over an optical
infrastructure. DWDM is an optical technique to transmit multiple channels on the same fiber strand. Using the standard ITU 100 GHz grid
in the C-band with the 4-skip-1 channel plan, 32 different channels can be transported over the same fiber, resulting in a fiber transmission
capacity of up to 320 Gbps.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 1 of 14

The goal with convergence is to build network architectures that are simpler to operate with enhanced flexibility to deliver new services
with reduced capital expenditures (CapEx) and operating expenses (OpEx). In order to address the CapEx and OpEx aspects, the trend in
DWDM network architecture is to drive cost elements out of the photonic layer and into the IP routers and Ethernet switches and to
achieve network convergence on the photonic layer.
NETWORK CONVERGENCE ON THE PHOTONIC LAYER
Optoelectronics technologies over the past few years have evolved dramatically. Not only can 10 Gigabit Ethernet optics be packaged
in Small Form-Factor Pluggable (SFP) transceivers such as Xenpak, X2, or XFP, but also even more sophisticated long-reach DWDM
optics can now be accommodated within the size and power constraints of hot pluggable form factors such as the Xenpaks. These technical
advances are opening up a new paradigm to build enterprise core networks using the power of DWDM.
DWDM functions used to be typically carried out in dedicated optical equipment such as transponders, capable of operating the
wavelength conversion from, for example, a traditional 1310 nm laser to a DWDM channel with very tight control over the wavelength
stability. The Cisco Systems DWDM Xenpak enables 10 Gigabit Ethernet over DWDM without the need of dedicated external
equipment. The sophisticated DWDM function is squeezed into a Xenpak module, which is equivalent from a Layer 2 and higher
perspective to any other Xenpak available on Cisco devices.
The capabilities of current hot-pluggable Xenpak transceivers allow the migration of DWDM functions directly into routers and switches,
and customers can experience operational and CapEx savings by removing required costly optical equipment to perform the wavelength
translation just mentioned. IP over DWDM (IPoDWDM) is the Cisco paradigm to design cost-effective, converged, scalable optical
networks based on Cisco Catalyst 6500 Series Switches for Ethernet equipped with DWDM pluggable optics. (See Figure 2.)
Figure 2.

IPoDWDM on Ethernet Switches

Moreover, consolidating the DWDM function in an Ethernet switch also provides significant operational advantages:

Less power and rack space is now required because of elimination of transponders.

Fewer optics and electronics also mean fewer points of failure in the network.

The DWDM layer can be managed by a LAN administrator without the need of specialized personnel to run the optical layer, which can
very well be reduced to pure passive elements to multiplex and demultiplex the Xenpak wavelengths.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 2 of 14

DWDM is the unifying technology that allows consolidation of data center and enterprise campus applications over a common transport
infrastructure. The Cisco DWDM Xenpak pluggables allow enterprise companies to provide scalable and easy-to-deploy 10 Gigabit
Ethernet services in their networks. The Cisco DWDM Xenpak input/output devices are hot-swappable, plug into an Ethernet Xenpak
port of a Cisco switch or router, and are not wavelength-tunable. These DWDM pluggables are delivered with 32 different product
numbers (SKUs) supporting the standard C-band ITU 100-GHz wavelength channel plan, as shown in Figure 3. The Cisco DWDM
Xenpak solution allows the reach of these networks to scale up to over 200 km by means of optical amplifiers known as EDFAs and
dispersion compensation units using single-mode ITU-T G.652 fibers.
Figure 3.

ITU 100 GHz DWDM Channel Plan and 10 GbE DWDM Xenpak Optic

The Cisco DWDM XENPAK supports digital optical monitoring (DOM) capability according to the industry-standard SFF-8724 Multi
Source Agreement (MSA). This feature gives the end user the ability to monitor real-time parameters of the Xenpak, such as optical output
power, optical input power, temperature, laser bias current, and transceiver supply voltage. Digital optical monitoring enables network
administrators to view the optical network physical layer performance through proactive monitoring and management of ports and links
and enables optical management capabilities, without the usage of complex optical tools that are not well known and supported by
enterprise network operators. Digital optical monitoring is supported on DWDM Xenpaks with Cisco IOS Software Release 12.2(18)SXE
on Cisco Catalyst 6500 Series Switches. The real-time digital optical monitoring parameters can be gained using a command-line interface
or by using the SNMP interface.
Figure 4 shows the output of a 10 Gbps DWDM Xenpak device. The measurement has been performed with a single DWDM Xenpak
interconnected with a loopback cable via a 10 dB attenuator. The internally calibrated transmitted power is 0.0 dBm and is above the -1.0
dBm minimum transmitter power specified in the data sheet. The measured received optical input is -10.1 dBm caused by the attenuator
and connector losses. The specified minimum optical input power is -24 dBm, giving a power budget of 23 dB. A more detailed command
line output including the threshold values for temperature, current, voltage and optical TX/RX values is shown by executing the command
show interface transceiver detail. To enable threshold notification for all sensors in the transceivers the command snmp-server enable
trap transceiver has to be made. Enabling transceiver monitoring is done with the command transceiver type all.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 3 of 14

Figure 4.

DOM Output of a DWMD Xenpak Device at ITU Channel 26

Cisco offers two different optical architectures to implement 10 Gigabit Ethernet using Xenpak DWDM pluggables:

Auto-adjusting intelligent DWDM solution based on Cisco ONS 15454 Multiservice Transport Platform (MSTP) with
R-OADM technology

Fixed-channel filter solution based on Cisco ONS 15216 Metropolitan DWDM 100-Ghz Fleylayer Filter Solution

Traditional DWDM solutions are using fixed-channel filters that can add and drop specific channels or bands of wavelengths. The
components of these systems are passive and do not require any electrical connections, enabling a very low-cost optical solution if
combining with DWDM transceiver pluggables. Such a system consists of add-drop modules covering various channel ranges, optical
splitters or combiners to connect add-drop modules, and optical amplifiers to amplify the signal and variable optical attenuators (VOA)
to equalize the optical power of channel groups.
Figure 5 shows the components of an add path, where six 10 Gigabit Ethernet Xenpaks are multiplexed on a single fiber. The system
consists of two 2-channel add modules, one 8-channel add module, a 4:1 combiner, a variable optical attenuator, and an optical amplifier to
move beyond typical campus distances. Additional channels can be added easily without service interruption to the existing 8-channel add
module, but also an additional module can be connected to the free combiner port. The system shown is connected in a parallel layout with
x:1 combiners, but also linear layouts are possible. The layout depends on the balance between the highest degrees of flexibility compared
with the better channel power equalization.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 4 of 14

Figure 5.

Fixed-Filter Based DWDM Components (Add Path)

The Cisco ONS 15216 DWDM Flexlayer solution enables enterprises to deploy low-cost fixed-filter based DWDM networks in the
campus and metro area. The ONS 15216 provides all the components to build a 32 channel fixed-filter DWDM solution, including optical
amplifier, variable optical attenuators, and the support of an optical supervisor channel for management connectivity. (See Figure 6.)
Figure 6.

ONS 15216 DWDM Flexlayer Chassis (4 Module Slots) Filled with a 8-Channel OADM, an Optical Splitter/Coupler, and a
4-Channel VOA

In fixed-filter DWDM solutions, changes to the optical services must be engineered in advance and subsequent deployment most likely
will disrupt existing services. A reconfigurable optical network offers the possibility to grow services between sites with no advanced
engineering or planning. In the past, reconfigurable optical networking technology was too expensive or delicate to be widely deployed.
With newly mature silicon-based integrated Planar-Lightwave-Circuit (PLC) components, reconfigurable optical add/drop multiplexers
(R-OADMs) can be build on waveguides utilizing manufacturing processes from the semiconductor industry allowing easy manufacturing
at low cost. PLC technology offers the integration of all components (switch, multiplexers, de-multiplexers, variable optical attenuators and
photodiodes) into a compact device. Dynamic channel equalization is available through integrated power measurement and control of every
channel. The optical switching time to provide future protection mechanisms is less than 50 msec.
Figure 7 illustrates the 32 channel R-OADM architecture as 4 channel device for simplification. Incoming DWDM signals are split into a
pass-through path and a drop path. With software control each of the 32 channels can be dropped in the de-multiplexers. The pass-through
wavelengths are feeding in multiplexers doing wavelength adding via software control. While using 10 Gigabit Ethernet DWDM
pluggables no transponders are used in this setup. In both East and West directions 2 x 10 Gigabit Ethernet services are terminated, while
the remaining wavelengths are fed through the wavelength selective switch.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 5 of 14

Figure 7.

R-OADM Architecture

The fast and simple provisioning of networks using cost-effective R-OADM technologies has reset the standard of DWDM
transmission in the last years and has been deployed widely in the industry.
The Cisco ONS 15454 Multiservice Transport Platform (MSTP) shown in Figure 8 offers a comprehensive reconfigurable optical
networking solution, including 32 channel R-OADMs available in C and L bands, multi-rate tunable transponders and muxponders
(supporting any service, including Ethernet, TDM, and storage) and intelligent WDM software that can automatically manage and
adjust optical power levels as new services are added or deleted and offer end-to-end management of wavelengths.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 6 of 14

Figure 8.

Cisco ONS 15454 MSTP and R-OADM Cards

Compared with the fixed-filter design, the R-OADM design allows any wavelength to be dropped at any site without affecting any of
the other services delivering the highest flexibility. The in-service growth from 1 to 32 wavelengths and the simplified network design and
provisioning will result in lower operational expense, faster service provisioning, but will need additional CapEx costs, compared with the
low-cost fixed optical filter solution.
Enterprise customers should compare both solutions using fixed filter or R-OADM design, before they decide which technology fits best to
build a converged optical network with 10 Gigabit Ethernet DWDM pluggables.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 7 of 14

10 GIGABIT ETHERNET DWDM DEPLOYMENTS IN CAMPUS NETWORKS

The main driver for DWDM deployment in metropolitan enterprise networks in the past has been data center interconnection, offering
multiple services over rented fibers. In campus enterprise networks, fiber exhaustion can be the driver for deploying DWDM technologies,
but most enterprises have enough fiber available to scale their network needs today. This situation changes dramatically if the campus is
segmented over several areas crossing public ground. Enterprises have to rent fibers from local service providers giving the opportunity
to deploy campus DWDM to save cost, ease the deployment and accelerate the service setup time (see Figure 9).
Figure 9.

Campus Segmentation and Fiber Layout

In this kind of scenario, the fixed-filter based DWDM solution with Cisco ONS 15216 DWDM Flexlayer combined with 10 Gigabit
Ethernet DWDM Xenpaks is offering a very attractive solution.
In the example shown in Figure 10 a full mesh network with 2 x 10 Gigabit Ethernet between each switch on the four campus sites is built.
Standard non-DWDM 10 Gigabit Ethernet Xenpaks are used for connections build over private ground, where enough fibers are available.
Between the campus sites, where rented fiber is used, a fixed-filter DWDM system using 2-channel add-drop-multiplexer is designed to
build the full mesh of 2 x 10 Gigabit Ethernet between each of the switches. Even if one external fiber is broken, all switches can be
reached via not more than one additional switch hop. Taking into account the maximum insertion loss for the 2-channel OADMs of 2.5 dB
for add/drop, the maximum insertion loss of 1.6 dB for pass, an additional optical power margin of 2 dB and the fiber attenuation of 0.25

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 8 of 14

dB/km the total maximum loss between sites is 16 dB. DWDM Xenpaks have an optical power budget of 23 dB and can be used in this
kind of scenario even without amplification.
Figure 10.

Segmented Campus with Fixed-Filter DWDM Solution

Another area for Enterprise DWDM solution is the interconnection of primary and backup data centers to protect information and have
24-hour access to critical application services. Cisco offers different optical solutions allowing these customers to select the transport
servicewhether IP/Ethernet, SONET/SDH, or WDMthat best meets their recovery time and point objectives for business continuity.
Cisco also offers several DWDM platforms for metro networks, such as the Cisco ONS 15216 Flexlayer and Cisco ONS 15454 MSTP.
In addition to transparently supporting various services, benefits of these systems include increased capacity (more services, such as
Gigabit Ethernet and Fiber Channel/FICON on a fiber pair), high-density service aggregation (many services on a single wavelength),
scalability from 2.5- to 10-Gbps wavelengths, and service mixing (different services on the same wavelength).
In this white paper we like to focus on DWDM application in the data center itself. Data centers often have elaborate fire prevention
and fire extinguishing systems. Fire prevention and detection systems are typically zoned, and high-quality fire-doors and other physical
fire-breaks used, so that even if a fire does break out it can be contained and extinguished within a small part of the facility, as illustrated
in Figure 11.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 9 of 14

Figure 11.

Data Center Fire-Break Zones and Backup Data Center

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 10 of 14

When deploying new fibers through fire-breaks, the fire break need to be opened and sealed again within maintenance windows involving
various infrastructure, security and operational teams. This complex workflow may be expensive, depending on the facility conditions and
infrastructure. Network planners and operators like to be more independent from these infrastructure restrictions. Deploying R-OADM
technology in every fire-zone enables the network operators to deploy wavelength services between the fire zones providing in-service
growth, simplified network design and provisioning and fast reaction to service requests. This concept will help to lower operational
expenses and allows the flexibility needed in data center environment.
Figure 12 shows a R-OADM deployment scenario between 4 different fire zones within a data center. In this example 6 wavelengths are
used to interconnect the 4 zones in a full mesh with 2 x 10 Gigabit Ethernet services.
Figure 12.

R-OADM Deployment Between Data Center Fire Zones

Tools like the Cisco MetroPlanner offer the possibility to model and test DWDM optical networks in a graphical environment to validate,
analyze and optimize network designs. After entering the number of network sites, the fiber distances, fiber type and service demands
(e.g. 2 x 10 Gigabit Ethernet from site 1 to site 3) the Cisco MetroPlanner will do a power consumption calculation, design the wavelength
routing and define the required add/drop filters, optical amplifiers and dispersion compensation units to fit the user traffic demands at
minimum costs. The Cisco MetroPlanner will generate reports about optical node shelf configuration, bill of material lists, installation
parameters and the optical results. The tool is able to calculate a network upgrade after freezing existing network parameters to implement
forecasted channels.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 11 of 14

Figure 13 shows a screenshot of the MetroPlanner tool for the network described in Figure 12. The network layout consists of 4 sites
connected with a fiber ring. The distances between the sites are fixed to 1 km and a loss factor of 0.25 db/km is used to calculate the power
budgets. As services between the sites a full mesh with 2 x 10 Gigabit Ethernet is chosen, with an upgrade option of additional 2 x 10
Gigabit Ethernet channels. As client interfaces 10 Gigabit Ethernet DWDM Xenpaks have been selected. The wavelength routing table ?
(6 wavelengths) and the shelf configuration for site 1 are also shown.
Figure 13.

MetroPlanner 7.0 Screenshot

The Cisco ONS 15454 MSTP supports besides Gigabit and 10 Gigabit Ethernet DWDM pluggable interfaces and also a large variety
of transponder line cards and muxponder line cards offering fiber channel, SDH, ESCON and other services, so that the system can be
extended to a multi-service optical network offering additional convergence on the optical layer.
SUMMARY: DEPLOYING IP OVER DWDM IN ENTERPRISE NETWORKS
Deployment of 10 Gigabit Ethernet in campus networks is the only available technology today to serve the demand for higher bandwidth in
consolidated campus networks. An Ethernet standard beyond 10 Gigabit Ethernet is not likely to be ratified within the next 4 years, forcing
enterprise network planners to aggregate 10 Gigabit Ethernet links. To achieve network consolidation on the photonic layer, cost-effective
optical transmission technique is combined with available 10 Gigabit Ethernet DWDM pluggables and optical switching: IP over DWDM.
IPoDWDM is a paradigm for cost effective, scalable, simplified and more efficient optical DWDM networks. This concept derived for
service providers can be extended to enterprise networks, offering the optical layer as convergence layer within the campus infrastructure.

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 12 of 14

DWDM deployments in enterprise campus networks are often driven by fiber exhaustion between campus segments or if fibers passing
public ground have to be rented between the campus areas. A cost-effective solution using 10 Gigabit Ethernet DWDM pluggables and
fixed-channels OADM filters (ONS 15216) offers more flexibility to campus network operators. The ease and speed of service delivery
can be further accelerated using R-OADM technology (ONS 15454), where service installation and provisioning is fast and simplified.
In data centers the segmentation into fire zones often is restricting fast and easy deployment of new services, if fibers are exhausted
between the zones. The R-OADM technique allows data center operators to deploy a scalable optical transport system, with fast and
easy service installation and provisioning and makes them more independent from infrastructure constraints in an environment, where
fast reaction is an obligation.
REFERENCE LIST

Cisco IP over DWDM Solution for IP NGN:

Cisco Optical Solutions Deliver Reliable and Manageable Carrier Services:

http://www.cisco.com/en/US/prod/collateral/optical/ps5724/ps2006/ps5320/product_data_sheet0900aecd803fc52f.html

http://www.cisco.com/en/US/prod/collateral/optical/ps5724/ps2006/ps5320/prod_bulletin0900aecd803fc6e4.html

A Fully Flexible Approach to DWDM: Designing Optical Networks:

http://www.cisco.com/en/US/products/hw/optical/ps1996/ps1999/index.html

http://www.cisco.com/en/US/products/hw/optical/ps1996/products_data_sheet09186a0080179158.html

DWDM Xenpak datasheet

http://www.cisco.com/en/US/products/hw/switches/ps708/products_data_sheet0900aecd801f9333.html

Digital Optical Monitoring (DOM) white paper:

http://www.cisco.com/en/US/prod/collateral/optical/ps5724/ps2006/ps5320/prod_white_paper0900aecd803fc746.shtml

Cisco ONS 15216 Metropolitan DWDM Fleylayer:

http://www.cisco.com/en/US/netsol/ns341/ns121/ns389/networking_solutions_customer_profile0900aecd8040ebb9.html

R-OADM for Cisco ONS 15454 MSTP:

http://www.cisco.com/en/US/products/ps5763/products_white_paper0900aecd80395e03.shtml

http://www.cisco.com/en/US/products/hw/modules/ps5455/products_device_support_table09186a00804b6f90.html

Cisco MetroPlanner DWDM Operations Guide, Release 7.0

http://www.cisco.com/en/US/products/ps6095/products_configuration_guide_book09186a00806c2270.html

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

Page 13 of 14

Printed in USA

All contents are Copyright 19922006 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.

C11-367468-00 10/06

Page 14 of 14