Anda di halaman 1dari 69

Roziitelnost a

vysok dostupnost
v L2 stch
Techtorial

Ji Tesa
Systems Engineer
CCIE #14558
jitesar@cisco.com
Sponsor
Logo
CIscoEXPO

Sponsor

Sponsor

Sponsor

Logo

Logo

Logo

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

2
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

802.1ah Terminologies
BEB: Backbone Edge Bridge encapsulates customer frames for
transmission across backbone.
B-BEB: B type BEB contains a B-component, supports bridging in the
provider backbone based on B-MAC and B-TAG info.
I-BEB: I type BEB contains an I-component for bridging in the customer
space, including customer MAC, service VLAN IDs.
B-TAG: Backbone VLAN Tag an S-TAG used in conjunction with
backbone MAC addresses.
I-TAG: Service Instance Tag - encapsulates customer addresses and
contains the Service Instance identifier (I-SID).
I-SID: Service Instance identifier - A field of the Service Instance tag
which identifies the service instance of the frame.
S-TAG: A field defined in the 802.1ad Q-in-Q encapsulation which
identifies the Service VLAN (S-VLAN).

3
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

802.1ah Terminologies and Interconnections


CE

MPLS
Core

PE
I

Peer
802.1ah
PBBN

B
BEB

IB
BEB

I BEB
B

PE/
BEB

MPLS
Core

I
BEB
I type
Backbone
Edge Bridge
CiscoEXPO

C-Tagged
Interface

S-Tagged
Interface

I-Tagged
Interface

B
BEB
B type
Backbone
Edge Bridge

CE

CE

CE

Q
C

PEB

B-Tagged
Interface

MPLS
Interface

PE/
BEB

PB

PEB

CE

IB type
Backbone
Edge Bridge

MPLS PE and
Backbone
Edge Bridge

Provider
Bridge
(S Bridge)

Provider
Edge Bridge
(C + S) Bridge

802.1Q
C Bridge

Customer
Equipment

Cisco Public

CE

IB
BEB

2010 Cisco Systems, Inc. All rights reserved.

CE

PB

802.1Q
I
BEB

Legend:

I
BEB

B
B

B
BEB

CE

PEB

PB

BCB
(PB)

802.1ad / Q-in-Q
PBN

B
BEB

Hierarchical B
802.1ah
BEB
PBBN

802.1ah
PBBN

IEEE 802.1ah Provider Backbone Bridges


(PBB)
I-TAG: Contains 24 Bits to
Identify a Service Instance
B-DA

B-SA

Second MAC-Header

B-TAG TPDNRI

C-DA

C-SA

C-TAG

B-TAG: Equals S-TAG

L2 PDU

FCS

P802.1ah (Provider Backbone Bridges)


Encapsulation Example

Service Scalability
Define a new Service Instance Identifier24 Bits wide
(taking the place of the former VLAN): I-SID

Domain Isolation, MAC-Address Scalability


Encapsulate Customer MAC-frames at the edge of the network into a Provider MACFrame: New MAC-Header with B-TAG

Backward Compatibility to 802.1ad


Packet header of Provider Backbone Bridges (PBB, P802.1ah)
and Provider Bridges (PB, P802.1ad) look the same
802.1ah assumes existing L2 control plane mechanisms such as spanning tree;
however these are not required

5
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

802.1ah I-TAG
B-DA

I-PCP

C-SA

C-TAG

46
Res2

Ether-Type
(0x88-e7)
Bits

C-DA

3
Res1

12

B-TAG TPDNRI

I-DEI
NCA

Octets

B-SA

L2 PDU

712
I-SID

FCS

1318
C-SA

C-DA

8 7 6 5 4 3 2 1

802.1ah Frame Format Settled


Priority Code Point (I-PCP)
Drop Eligible Indicator (I-DEI)
No Customer Addresses (NCA)
Reserved 1 (Res1)
Reserved 2 (Res2)
Backbone Service Instance Identifier (I-SID)
6
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Ethernet Encapsulation Evolution


C-DA: Customer dest addr
C-SA: Customer src addr
C-TAG: Customer tag
S-TAG: Service tag
B-DA: Backbone dest addr
S-SA: Backbone src addr
I-TAG: Service instance tag
VID: VLAN identifier (part of C-/S-/
B-TAG)
I-SID: Backbone service instance
identifier (part of I-TAG)

Service
Instances
(I-SID)
224=16,777,216
B-DA
Service
Instances
(VID)
12
2 =4,096

B-TAG

C-DA

C-DA

C-DA

C-SA

C-SA

C-DA

C-SA

S-TAG

S-TAG

C-SA

C-TAG

C-TAG

C-TAG

Payload

Payload

Payload

Payload

FCS

FCS

FCS

FCS

CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

B-SA
I-SID

PB: Provider Bridges


PBB: Provider backbone bridges
I-TAG

Service
Instances
(VID)
212=4,096

802.1Q/ad
service
Instances (212)

802.1ah
service
Instances (224)
7

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

8
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

7600 802.1ah Line Card Support


ES+

ES+

UNI

NNI

Ingress
IB-BEB

ES+
or
Any
DFC

UNI

NNI

ES+
or
Any
DFC

BCB

Egress
IB-BEB

802.1ah Imposition/Disposition is done on UNI facing ES+ cards


NNI Facing Line card
1.
2.

CiscoEXPO

Any DFC card


Adds B-VID

2010 Cisco Systems, Inc. All rights reserved.

7600 802.1ah LC Requirements


Ingress LC

Egress LC

Native 802.1ah

ES+

Any DFC card


(Recommend ES+)

802.1ah + MPLS

ES+

ES+, ES20, SIP600, SIP400


(Recommend ES+)

Cisco Public

7600 VLAN Local Significance Support


Interface Types

ES+

ES20

SIP400

67xx

EVC Dot1q

Yes

Yes

Yes

N/A

EVC QinQ

Yes

Yes

Yes

N/A

Sub-interface Dot1q

Yes

No

Yes

No

Sub-interface QinQ

Yes

Yes

Yes

N/A

VLAN Local Significance does means

VLAN is terminated in the NPU => VLAN lookup, rewrites, etc are performed in
NPU

Same VLAN tag can be used on multiple ports

VLAN tag leaving the port is different to VLAN allocated in internal Database
VLAN Local Significance does NOT mean

More than 4000 VLANs are supported for Layer 3 termination


10

CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Flexible Forwarding Model


P2P XCONNECT
EVC to L3/VRF

C-BRIDGE

C
P2P X

C
ONNE

MPLS

L3
T
CON
VFI X

NECT

B-BRIDGE

EFPs

L2 B
R
L2 BRIDGED

L2

L2

IDG
E

EFPs

Local Connect

TRUNK
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

11

Flexible Ethernet Edge Example


Access port
ES

Local
connect

Service instance or
Ethernet Flow Point

CiscoEXPO

AS

core interface, L2
trunk or L3 MPLS

CS

service instance 1 ethernet


encapsulation dot1q 20 second-dot1q 10
rewrite ingress tag pop 1 sym
bridge-domain 10 c-mac

802.1ah (PBB
or .1ah over
VPLS

service instance 2 ethernet


encapsulation dot1q 11-100
rewrite ingress tag push dot1q 101
xconnect 1.1.1.1 1000 en mpls

E-LINE
(VPWS)

service instance 3 ethernet


encapsulation dot1q 101 second-dot1q 10
rewrite ingre tag translate 2-to-1 100
bridge-domain 200
Interface vlan 200
xconnect vfi myvpls

E-LAN (VPLS
or Local
bridging)

service instance 4 ethernet


encapsulation dot1q 102
rewrite ingress tag pop 1
bridge-domain 201
Interface vlan 201
ip address 2.2.2.2 255.255.255.0
ip vrf myvrf

L3 termination

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

12

IEEE 802.1ah Control Plane Model


IB-BEB
I-Component

B-Component

C-MAC Lookup Function


MAC Relay

B-MAC Lookup Function


MAC Relay

EFP

I-EFP

B-EFP

(Physical)

(Virtual)

(Virtual)

Switch
Port
(Physical)

CIP

PIP

CBP

B-MAC Tagging/
I-SID Insertion

B-VLAN Re-write/
I-SID Validation

PBP

Ingress EFP (802.1ah UNI)

MAC Tunnel

Egress switchport (NNI)

int gig1/1

ethernet mac-tunnel virtual abc.com


bridge-domain 100

int gig1/2

service instance 15 ethernet


encapsulation dot1q 9 second-dot1q 8
bridge-domain 10 c-mac

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

service instance 31 ethernet


encapsulation i-sid 10000
bridge-domain 10 c-mac

switchport
switchport mode trunk
switchport allowed vlan 100

13

802.1ah on ES+ NPU Overview


802.1ah is implemented on ES+/7600 for first time on a Cisco
platform
802.1ah utilizes both PFC/DFC ASIC and NPU to perform the 2
required layer2 switching decisions for dot1ah
Dot1ah on ES+/7600 follows the IB Backbone Edge Bridge model
PFC/DFC represents the B-component of the IB-BEB and switches
the packet towards the provider backbone port or NNI
802.1ah (NPU) represents the I-component as well as the provider
instance port (tunnel engine) and switches the packet towards the
customer instance port or UNI
PFD/DFC learns Backbone MAC addresses or B-MACs and floods
on Backbone VLANs or B-VLANs
802.1ah (NPU) learns Customer MAC addresses or C-MACs and
floods on Customer bridge-domains or C-BDs
14
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

15
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

7600 PBB IB-BEB Logical Flow


7600
Ingress

C-MAC1

Egress

ISID-1
B-MAC1

Port,
802.1q or
802.1ad
(QinQ)
EFP
Service
instances

C-MAC2

ISID-2

VLAN local

EFP or switchports
with the B-VLANs

Significance per
Port
C-MAC3

ISID-3

VLAN tag
translation and
manipulation
C-MAC4

B-MAC2

ISID-4

802.1q/qinq/ 802.1ad
PBN

PBBN
AS

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

16

7600 PBB IB-BEB Configuration


ELAN Service Implementation
Step 1
Ingress EFP configuration (UNI)
interface TenGigabitEthernet3/1
dot1q tunneling ethertype 0x88A8
service instance 100 ethernet
description ** UNI EFP - ELAN Service
encapsulation dot1q 100 second-dot1q 1-4094
rewrite ingress tag pop 1 symmetric
service-policy input vz-ingress-policer
service-policy output vz-H-QoS-parent
l2protocol forward
bridge-domain 100 c-mac

interface TenGigabitEthernet3/2
dot1q tunneling ethertype 0x88A8
service instance 100 ethernet
description ** UNI EFP - ELAN Service
encapsulation dot1q 100 second-dot1q 1-4094
rewrite ingress tag pop 1 symmetric
service-policy input vz-ingress-policer
service-policy output vz-H-QoS-parent
l2protocol forward
bridge-domain 100 c-mac

Step 2
Mac-in-Mac tunnel configuration
ethernet mac-tunnel virtual 1
description ** IB-BEB - Mac Tunnel 1
bridge-domain 1000
service instance 1 ethernet
description ** ELAN Service - ISID
encapsulation dot1ah isid 10000
bridge-domain 100 c-mac

Step 3
Egress EFP configuration (NNI)
interface TenGigabitEthernet3/3
dot1q tunneling ethertype 0x88A8
service instance 1 ethernet
description ** B-VLAN - MAC Tunnel 1
encapsulation dot1q 1000
rewrite ingress tag pop 1 symmetric
service-policy output vz-core-queuing
bridge-domain 1000

or
Egress switchport configuration (NNI)
interface TenGigabitEthernet3/3
switchport
switchport mode trunk
switchport allowed vlan 1000

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

17

7600 MPLS + 802.1ah IB-BEB Logical Flow

Ingress
C-MAC1

Egress (ES
+40)

ISID-1

VPWS (P2P) Pseudowire


B-MAC1

C-MAC2

Port,
802.1q or
802.1ad
(QinQ)
EFP
Service
instances

ISID-2

MPLS Interface/
Sub-interfaces
C-MAC3

ISID-3

VLAN tag
translation and
manipulation

C-MAC4

MPLS Transport
Network

(H)-VPLS Pseudowire(s)
B-MAC2
ISID-4

VFI

VPWS (P2P) Pseudowire

802.1q/qinq/ 802.1ad
PBN

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

AS

18

7600 MPLS + PBB IB-BEB Configuration


VPWS + 802.1ah Service Implementation
Step 1
Ingress EFP configuration (UNI)
interface TenGigabitEthernet3/1
dot1q tunneling ethertype 0x88A8
service instance 100 ethernet
description ** UNI EFP VPWS Service
encapsulation dot1q 100 second-dot1q 1-4094
rewrite ingress tag pop 1 symmetric
service-policy input vz-ingress-policer
service-policy output vz-H-QoS-parent
l2protocol forward
bridge-domain 100 c-mac

Step 2
Mac-in-Mac tunnel configuration
ethernet mac-tunnel virtual 1
description ** IB-BEB - Mac Tunnel 1
bridge-domain 1000
service instance 1 ethernet
description ** VPWS Service - ISID
encapsulation dot1ah isid 10000
bridge-domain 100 c-mac

Step 3
VPWS configuration
interface Vlan1000
description ** IB-BEB VPWS Service
xconnect 2.2.2.2 3000 encapsulation mpls

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

19

7600 MPLS + PBB IB-BEB Configuration


VPLS + 802.1ah Service Implementation
Step 1
Ingress EFP configuration (UNI)
interface TenGigabitEthernet3/1
dot1q tunneling ethertype 0x88A8
service instance 100 ethernet
description ** UNI EFP VPLS Service
encapsulation dot1q 100 second-dot1q 1-4094
rewrite ingress tag pop 1 symmetric
service-policy input vz-ingress-policer
service-policy output vz-H-QoS-parent
l2protocol forward
bridge-domain 100 c-mac

interface TenGigabitEthernet3/2
dot1q tunneling ethertype 0x88A8
service instance 100 ethernet
description ** UNI EFP - VPLS Service
encapsulation dot1q 100 second-dot1q 1-4094
rewrite ingress tag pop 1 symmetric
service-policy input vz-ingress-policer
service-policy output vz-H-QoS-parent
l2protocol forward
bridge-domain 100 c-mac

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

Step 2
Mac-in-Mac tunnel configuration
ethernet mac-tunnel virtual 1
description ** IB-BEB - Mac Tunnel 1
bridge-domain 1000
service instance 1 ethernet
description ** VPWS Service - ISID
encapsulation dot1ah isid 10000
bridge-domain 100 c-mac

Step 3
VPLS configuration
l2 vfi Vz-MAC-Tunnel-1 manual
vpn id 3000
neighbor 2.2.2.2 encapsulation mpls
neighbor 3.3.3.3 encapsulation mpls
interface Vlan1000
description ** IB-BEB VPLS Service
xconnect vfi Vz-MAC-Tunnel-1 manual

20

Flexible Ethernet Edge for .1ah

The Cisco implementation will provide for the services mandated by 802.1ah, and will
extend them to support all the following offerings:

S-Tagged Service
Multiplexed: Each S-VID maps to an I-SID. It is possible to retain or pop the STAG. (Retention of S-TAG is an extension of 802.1ah)
Bundled (same as 802.lah): Multiple S-VIDs map to an I-SID. The S-TAG must
be retained

C-Tagged Service (extension of 802.1ah)


Multiplexed: Each C-VID maps to an I-SID. It is possible to retain or pop the CTAG.
Bundled: Multiple C-VIDs map to an I-SID. The C-TAG must be retained.

S/C-Tagged Service (extension of 802.1ah)


Multiplexed: Each S-VID/C-VID pair maps to an I-SID. It is possible to retain or
pop the S-TAG only or both S-TAG/C-TAG pair.
Bundled: Multiple S-VID/C-VID pairs maps to an I-SID. The S-TAG/C-TAG pair
must be retained.

Port Based Service (same as 802.1ah): All frames are mapped to the same I-SID.
All tags, if any, are retained.
21

CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

PBB IB-BEB Packet Flow


CE Side

PBBN Side

S-tagged Service - Multiplexed


C-DA

C-SA S-TAG C-TAG 0x800

Data

or

B-DA

B-SA B-TAG I-TAG C-DA

B-DA

B-SA B-TAG I-TAG

B-DA

B-SA B-TAG I-TAG C-DA

C-DA

C-SA S-TAG C-TAG 0x800


C-SA C-TAG 0x800

Data

Data

S-tagged Service - Bundled


C-DA

C-SA S-TAG C-TAG 0x800

Data

C-SA S-TAG C-TAG 0x800

Data

C-tagged Service - Multiplexed


C-DA

C-SA C-TAG 0x800

Data

or

B-DA

B-SA B-TAG I-TAG

C-DA

C-SA C-TAG 0x800

B-DA

B-SA B-TAG I-TAG

C-DA

C-SA

B-DA

B-SA B-TAG I-TAG

C-DA

C-SA C-TAG 0x800

0x800

Data

Data

C-tagged Service - Bundled


C-DA

C-SA C-TAG 0x800

Data

Data

S/C-tagged Service - Multiplexed


C-DA

C-SA S-TAG C-TAG 0x800

Data

or
or

B-DA

B-SA B-TAG I-TAG C-DA

C-SA S-TAG C-TAG 0x800

B-DA

B-SA B-TAG I-TAG

C-DA

C-SA C-TAG 0x800

B-DA

B-SA B-TAG I-TAG

C-DA

C-SA

B-DA

B-SA B-TAG I-TAG C-DA

B-DA

B-SA B-TAG I-TAG

0x800

Data

Data

Data

S/C-tagged Service - Bundled


C-DA

C-SA S-TAG C-TAG 0x800

Data

C-SA S-TAG C-TAG 0x800

Data

Port Based Service


C-DA

Presentation_ID

C-SA

2006 Cisco Systems, Inc. All rights reserved.

C-DA

C-SA

22

MAC Address Scalability in H-VPLS


H-VPLS

H-VPLS with PBB

IP/MPLS

IP/MPLS

IP/MPLS

c-mac
c-mac
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
c-mac

Presentation_ID

IP/MPLS

c-mac
c-mac
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
:
c-mac

2006 Cisco Systems, Inc. All rights reserved.

c-mac
c-mac
:
:
:
:
:
c-mac

b-mac
b-mac
:
b-mac

b-mac
b-mac
:
b-mac

c-mac
c-mac
:
:
:
:
:
c-mac

No customer MAC addresses


on N-PE nodes
N-PEs only learn backbone
MAC addresses imposed by
U-PEs
23

Deployment Scenario:
H-VPLS extension + 802.1ah
802.1Q
CE

VPWS/
H- VPLS w/
802.1ah

C-VLAN

uPE/
IB-BEB

CE

VPWS

VPWS/
H- VPLS w/
802.1ah 802.1ad/Q-in-Q

VPWS/VPLS

uPE/
IB-BEB

nPE

nPE

VPWS

VPWS

VSI

VPWS/VPLS
IP/MPLS Core

CE
BEB
I-SID

nPE

VSI

nPE

uPE/
IB-BEB

I-SID

E-Line Service

uPE/
IB-BEB

E-LAN Service

S-VLAN

CE

802.1ad/Q-in-Q

VSI

CE

S-VLAN

uPE/
IB-BEB

I-SID

CE

CE

VPWS

CE

802.1ad/ Q-in-Q

S-VLAN

CE

CE
S-VLAN

MPLS
Access

Presentation_ID

Aggregation

2006 Cisco Systems, Inc. All rights reserved.

Core Transport

Aggregation

Access

24

Scalability
Scalability Factor

Scalability Number

Total number of EVCs in the system

32000

Total number of EVCs per linecard

16000

Total number of ISIDs in the system


Total C-MAC addresses per LC

16M
128000 (32000 per NPU)

Total number of EVCs per ISID per NPU

110

Total number of EVCs per ISID for a two port Excalibur

220

Total number of EVCs per ISID for a four port Excalibur

440

Total B-bridge-domains per chassis

4094

Total I-SIDs or MAC-Tunnels

16000

Total entries in a C-MAC table

32000

25
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

26
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Test: Verify PCP Propagation and Queuing Behavior for ELAN


Service Using UNI Ingress 2R3C Policer Marking
IXIA
Sniffer
Ten3/3
L2 Link

Ten3/1

PE1Ten3/2

IB BEB

2R3C Queuing, etc..


Policer

ES2
IXIA
C-DA
C-SA
S-Tag, CoS=1
C-Tag, CoS =2
C payload
FCS

1.

2.

Ten3/4
MPLS Link

On PE1-IB_BEB

Ten1/0/1

IXIA

Ten1/0/0

ES1

ES3
IXIA

Ten3/3
L2 Link

Ten3/1

P-

PE2-

BCB

IB-BEB

Ten3/2

Ten3/4
MPLS Link
B-DA

ES4

B-SA

IXIA

B-Tag,
CoS=3,4,5
I-Tag, CoS=3,4,5
C-DA
C-SA
C-Tag, CoS =2

C-DA
C-SA
S-Tag,
CoS=3,4,5
C-Tag, CoS =2
C payload
FCS

C payload
FCS

Ingress card frames are remarked to CoS 5,4,3 by 2R3C Policer

Egress card frames are sent to separate queues based on Policer marking for Queuing, etc .

Show end-2-end remarked CoS mapping in PBB frame and Egress PE2-IB_BEB

Presentation_ID

Sniffer capture to show COS is mapped from S-TagI-TagB-Tag and back to egress S-Tag
2006 Cisco Systems, Inc. All rights reserved.

27

References
IEEE 802.1ah - Provider Backbone Bridges, Draft 4.2, April 2008.
VPLS Interoperability with Provider Backbone Bridges, draftsajassi-l2vpn-vpls-pbb-interop-04-txt, March 2009.
Extensions to VPLS PE model for Provider Backbone, Bridging
draft-balus-sajassi-l2vpn-pbb-vpls-00.txt, March 2009.
Provider Backbone Bridging and MPLS: Complementary
Technologies for Next-Generation Carrier Ethernet Transport, S.
Salam and A. Sajassi, IEEE Communications Magazine, Vol. 46,
No. 3, March 2008.
The Evolution of Carrier Ethernet Services Requirements and
Deployment Case Studies, L. Fang, N. Bitar, R. Zhang, and M.
Taylor, IEEE Communications Magazine, Vol. 46, No. 3, March
2008.

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

28

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

29
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

L2 Convergence

EVC L2 Convergence - SRD and before


MST (BPDU relay, switchport, EVC)
REP (switchport)
PW Redundancy (MPLS aggregation)
Flex-Link
Etherchannel/LACP (single-homed devices)
Interface-Backup (single-homed devices)

New features in SRE


MST AG simplify MST based deployments
REP support for EVC added
mLACP dual homed LACP

30
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

30

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

31
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Why MST Access Gateway?


Avoid running full Spanning Tree protocol on NPE STP is
challenging to troubleshoot
Terminate multiple Ethernet access rings running MST on NPE without
running full STP
Each ring can have its own independent topology
Isolate topology changes/MAC flushes localized to each ring

Improve scalability
No full STP processes on NPE routers

Maintain existing STP topologies on the access networks


Access nodes just speak regular MSTP/RSTP

Platform Support
ASR9K since FCS
7600 12.2SRE

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

32

MST AG operation
SRE

MST AG ports send preconfigured BPDUs with root or zero cost to root information towards
access network. Access network sees a loop because of root reachability from both NPEs.
Both NPEs can send the same information or arbitrarily can be set as best and second best
bridge via priority or cost setting for load balancing purposes
Root bridge can be one of the NPEs or arbitrarily set non-existent bridge address
MST AG ports are always in Designated state and are forwarding
L2 domain runs regular MST protocol. All convergence operations and port state transitioning
happen in the access network.
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

33

MST AG TCN Propagation

NPEs snoop and relay TCN from BPDU received from access network
NPEs trigger MAC withdrawal to neighbors
TCN is forwarded only to the port within the same MST AG group thus providing L2
domains isolation
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

34

Access Network Failure Scenarios

Failure Scenarios 1,2 and 3 cause a primary data path disruption


UPE-2 BPDU on Atlernate Port is has now the best BPDU port transitions to root port
role and forwarding state and data path is restored
TCN propagates across L2 domain and is relayed between NPEs
NPEs trigger MAC withdrawal

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

35

Root Port Recovery

When root bridge recovers it starts sending best BPDU towards the access network and
convergence to the original path occurs
Sending of the best BPDU has to be delayed to allow core convergence; e.g. if the router was
reloaded
7600 router runs STP state machine on MST AG when the port is coming from down to up state.
The ports is going through LST-LRN-FWD states. To disable this behavior spanning tree port
fast has to be configured on MST AG ports. BPDUs are sent immediately upon port recovery
which can cause traffic black-hole if core has not converged. EEM can be used to delay port-up
event under certain scenarios.
ASR 9K does not run spanning tree state machine and defines a dedicated timer to delay the
best BPDU generation.

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

36

NPE isolation failure scenario

N-PE isolation occurs if all core facing interfaces are not available resulting in VPLS,
Psedowires or L3 connectivity failure
N-PE isolation failure is not propagated into access interfaces therefore STP topology
remains unchanged, this results in traffic blackhole as access network continues
forwarding towards isolated PE
An uplink tracking feature is under consideration for future releases
Current solution is based on EEM when router isolation is discovered the access
interfaces from redundant networks can be shut down which triggers MST convergence.
Upon recovery, timer can be set to delay access links recovery and avoid immediate
BPDU sending to the access network.

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

37

Using EEM for Uplink Tracking


Backbone uplink on NPE-1 is going down
event manager applet Backbone-DN
event syslog pattern "%LINK-3-UPDOWN: Interface GigabitEthernet1/40, changed s"
action 1.0 cli command "enable"
action 1.1 cli command "conf t"
action 1.2 cli command "int g1/31"
action 1.3 cli command "sh

Backbone uplink on NPE-1 is going up


event manager applet Backbone-UP
event syslog pattern "Interface GigabitEthernet1/40, changed state to up"
action 1.0 cli command "enable"
action 1.1 cli command "conf t"
action 1.2 cli command "int g1/31"
action 1.3 cli command "no sh

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

38

Special PW failure scenario

Special PW failure can be result of PE isolation or a miss-configuration


Unlike MST, R-L2GP special PW failure does not cause a loop and therefore does not
cause permanent traffic loss because BPDU forwarding topology remains unchanged
and is not affected by this failure
TCN will not be relayed between the two NPEs MAC flush may not happen in a part of
L2 domain which may cause temporary traffic loss until MAC aging occurs. Bidirectional
traffic will be restored immediately.
MAC withdrawal will still be generated by the NPE receiving TCN
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

39

EVC STP Modes Comparison


STP mode
EVC
Support

RPVST/
PVST
NO
BPDU relay only

VLAN-STP
mapping

Per VLAN

Dynamic
Port State

N/A

Dynamic
BPDU

always FWD

N/A

MST

MST AG

SRD

SRE

Single MST Region

Single MST Region

VLAN to Instance mapping


applies to all ports

VLAN to Instance mapping applies to


all ports

YES
YES

NO
Designated ports, always forwarding

NO
preconfigured BPDUs

YES

TCN
isolation

YES

NO

TCN forwarded between ports within


the same L2GP group

MAC
Withdrawal

NO

YES

YES

Complexity

Medium

High

Low

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

Prone to miss-configuration
40

MST AG configuration steps


1. Configure MST parameters

MST AG reuses global MST configuration template to construct BPDUs. To


insure proper MST function, parameters like name, revision and timers should
match on other bridges.
Note: due to single domain support the same MST parameters will be used on
all MST AG groups. In particular IST to VLAN mapping.

spanning-tree mode mst


spanning-tree mst configuration
name c7600
revision 1
instance 1 vlan 3500-3599
spanning-tree mst hello-time 1
spanning-tree mst forward-time 4
spanning-tree mst max-age 6

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

41

MST AG configuration steps


2. Configure MST AG Pseudo-Information (NPE-1)
spanning-tree pseudo-information transmit 1
remote-id 2
! use the number of pseudo-information of the peer router
mst 0-1 root 24576 001e.f7f6.6040
! root bridge and priority that will be send in BPDU on MST AG ports

3. Assing MST AG Pseudo-Information to a port


interface GigabitEthernet1/32
no ip address
spanning-tree portfast trunk
spanning-tree pseudo-information transmit 1
! the port will send preconf. BPDUs as per MST global and pseudo-inf. gr. 1 conf.
service instance 3500 ethernet
encapsulation dot1q 3500
rewrite ingress tag pop 1 symmetric
bridge-domain 3500

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

42

MST AG configuration steps


Configure Special PW for TCN relay
This is the same configuration step as for MST BPDU relay

interface Vlan1
no ip address
xconnect vfi BPDU
end
NPE-1#sh run | sec BPDU
l2 vfi BPDU manual
vpn id 1
forward permit l2protocol all
neighbor 10.1.1.6 encapsulation mpls

Configure Service Instances and Bridge Domains


Configure all Egde Ports explicitly with portfast feature
avoid LRN/LSTN states when bridge is converging

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

43

MST AG configuration validation


NPE-1# sh spanning-tree mst 1
##### MST1
Bridge
Root

vlans mapped:
3500-3599
address 001e.f7f6.6040 priority
this switch for MST1

Interface
---------------Gi1/1
Gi1/32
PW 10.1.1.6:1

Role
---Desg
Desg
Desg

Sts
--FWD
FWD
FWD

Cost
--------20000
20000
200

32769 (32768 sysid 1)


Prio.Nbr
-------128.1
128.32
128.55

Type
-------------Edge P2p
P2p R-L2GP
P2p R-L2GP

NPE-1#sh spanning-tree pseudo-information 1 configuration


Pseudo id 1, type transmit:
remote_id 2
mst_region_id 0, port_count 1, update_flag 0x0
mrecord 0x1A6BE02C, mrec_count 2:
msti 0: root_id 24576.001e.f7f6.6040, root_cost 0, update_flag 0x0
msti 1: root_id 24577.001e.f7f6.6040, root_cost 0, update_flag 0x0
NPE-1# sh spanning-tree pseudo-information 1 interfaces
Pseudo id 1:
GigabitEthernet1/32

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

44

Other useful commands


on RP
sh spanning-tree mst configuration!
sh spanning-tree details!
sh vlan id 3500!

on SP
deb spanning-tree pseudo-information!
debug spanning-tree bpdu!
deb spanning-tree mstp tc!
deb spanning-tree mstp flush !!

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

45

MST AG Restrictions
Supported on ES+ and ES20
Applicable to EVC with Bridge Domain only
No xconnect, connect or subinterface support

No EVC untagged, priority tagged or default encapsulation support


Native VLAN is used for BPDU forwarding

Single MST region support


All MST AG groups share MST Instance - VLAN mapping, name and
revision

No MST boundary function (for RPVST/PVST/RSTP


interoperability)

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

46

7600 MST AG Scale


Feature

Scale

Comment

STP Regions

All R-L2GP groups have to use common MST


configuration; name, version, timers, IST-VLAN
mapping

MST
instances

64

As above

R-L2GP
groups

256

Ports in
No limit
R-L2GP group

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

All ports in a chassis can be assigned to a


single R-L2GP group
50,000 vport limit per chassis

47

ARS9K MST Access Gateway


Interface gig 0/0/0/10.1 l2
encap untagg
spanning-tree ring-termination ring1
preempt delay { until <hh:mm:ss> |
for <n> { hours |
minutes | seconds } }
interface GigabitEthernet0/0/0/10.1
name cisco
revision 1
bridge-id 0000.0000.0001
instance 0 Im the root
root-id 0000.0000.0001
priority 4096
root-priority 4096
!
instance 1
vlan-ids 101,103,105,107
root-id 0000.0000.0002
priority 8192
root-priority 4096
!
instance 2 Im the root
vlan-ids 102,104,106,108
root-id 0000.0000.0001
priority 4096
root-priority 4096

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

Access switch configuration


interface GigabitEthernet1/1/1
switchport mode trunk
spanning-tree mst 0,2 cost 100000
MST root for
instance 0,2
VFI

VFI

MST root for


instance 1

Access switch configuration


interface GigabitEthernet1/1/1
switchport mode trunk
spanning-tree mst 1 cost 100000

Interface gig 0/0/0/10.1 l2


encap untagg
spanning-tree ring-termination ring1
preempt delay { until <hh:mm:ss> |
for <n> { hours |
minutes | seconds } }
interface GigabitEthernet0/0/0/10.1
name cisco
revision 1
VFI
bridge-id
0000.0000.0002
instance 0
root-id 0000.0000.0001
priority
8192
VFI
root-priority 4096
!
instance 1 Im the root
vlan-ids 101,103,105,107
root-id 0000.0000.0002
priority 4096
root-priority 4096
!
instance 2
vlan-ids 102,104,106,108
root-id 0000.0000.0001
priority 8192
root-priority 4096

48

Conclusions
MST AG provides an appealing option to operate STP
networks to service providers:
Maintaining access networks without modification
Lower maintenance complexity on N-PEs no full
spanning tree support
Lower troubleshooting complexity on the network
STP isolation for L2 aggregation domains separated by
VPLS core
Deterministic root location

Improvements from MST/EVC:


TCN isolation between access domains
More robust implementation, special PW failure does not
cause traffic black-hole
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

49

Agenda
IEEE 802.1ah Technology and Benefits
7600 Platform 802.1ah Architecture
Implementing 802.1ah + VPWS/VPLS Services on 7600
Implementing 802.1ah QoS on 7600

L2 Convergence
Overview and Evolution
MST Access Gateway Concept
mLACP

Conclusions

50
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

mLACP

SRE
Standby POA

DHD

Virtual LACP Peer

Inter-chassis Communication

EtherChannel with mLACP


Active POA

mLACP provides a good mechanism for multi-chassis resiliency


DHD is attached to a group of Points of Attachments which look like a
single node
mLACP appears to DHD as a single 802.3ad LACP
POA work in active/standby mode
ICC exchanges redundancy information between chassis
Links to standby PoA are in hot-standby state
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

51

ICCP Overview
ICCP is implemented according to the standard draft-martini-pwe3iccp-00.txt
ICCP is an extensible Protocol to synchronize event/states between
multiple chassis which are part of the redundant group.
ICCP is a reliable protocol which runs over TCP
ICCP PDUs are exchanged between Peers to keep the application state
consistent across Routers.
Control Messages to setup, notify and exchange heartbeats.
Data Messages to exchange the application state consistent across the
chassis. Ex: LACP Parameters
ICCP failure detection
ICC Heartbeat Slow (~ 30 sec)
/32 Next-hop Tracking Depends on IGP timers
BFD ~50 150 msec

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

52

Pseudowire Redundancy in SRE


VCCV over Primary and Backup PW
Preferential Forwarding status bit according to
draft-ietf-pwe3-redundancy-bit
Upon Receipt of PW switchover status request, The receiver
should clear the preferential status forwarding bit and activate the
PW.
Back up Pseudowire will be preprovisioned in the data plane.But
forwarding is disabled.
Supported with Scale EoMPLS configuration only.
VPLS Redundancy is supported only with mLACP configuration.
Supported on ES40,ES20 only.

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

53

mLACP with two sided VPWS/VPLS


redundancy
MPLS
L2

Standby

PE1

Standby

PW 2

Active

PW 1

Standby

L1

Active

Active
PE3 E Active

L3
PE2

Active

Active

PW 3
DHD1

SRE

Standby

PW 4

L4
PE4

Standby DHD2

Standby

VPWS
Two PEs form one virtual group on each site, one PE is primary the other is backup
PEs send primary/backup information during PW signaling
PW with both sides status <active> are established, others are hot standby
MPLS uplinks, attachment circuits and PW status tracking
Message exchange within virtual group (for mLACP it is ICC) with redundancy
status

VPLS
PW will be active between PEs with active access circuits only
Single active path through VPLS domain between PE virtual group

Similar model applies to REP access

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

54

Pseudowire Redundancy
Two-way

PE1

IP/MPLS

Prim.

Primary
Pseudowire

PE2a
Prim.

CE1

CE2

ICCP

ICCP

LACP
Back.

IP or MPLS

PE2b

Redundant
Pseudowires
ICCP = Inter-Chassis Control Protocol

LACP
Back.

LACP = Link Aggregation Control Protocol

Failures within MPLS network are protected by MPLS FRR


Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy
(Note: both sides of the PW are protected)
Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for
LACP and PW redundancy
Synchronization of state (active/standby) between the ACs and PWs
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

55

Pseudowire Redundancy
Two-way

PE1

IP/MPLS

Prim.

Primary
Pseudowire

PE2a
Prim.

CE1

CE2

ICCP

ICCP

LACP
Back.

IP or MPLS

PE2b

Redundant
Pseudowires
ICCP = Inter-Chassis Control Protocol

LACP
Back.

LACP = Link Aggregation Control Protocol

Failures within MPLS network are protected by MPLS FRR


Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy
(Note: both sides of the PW are protected)
Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for
LACP and PW redundancy
Synchronization of state (active/standby) between the ACs and PWs
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

56

Pseudowire Redundancy
Two-way

PE1

IP/MPLS

Prim.

Primary
Pseudowire

PE2a
Prim.

CE1

CE2

ICCP

ICCP

LACP
Back.

IP or MPLS

PE2b

Redundant
Pseudowires
ICCP = Inter-Chassis Control Protocol

LACP
Back.

LACP = Link Aggregation Control Protocol

Failures within MPLS network are protected by MPLS FRR


Failures of Ethernet Attachment Circuits or PE handled by two-way PW redundancy
(Note: both sides of the PW are protected)
Inter-Chassis Control Protocol (ICCP) for synchronization of redundancy state control for
LACP and PW redundancy
Synchronization of state (active/standby) between the ACs and PWs
Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

57

mLACP CLI
Interchassis Redundancy Group
redundancy
interchassis group 1
member ip 10.12.1.33
! this is IP-address of opposite end of direct link between NPE-1/2
backbone interface GigabitEthernet1/40
mlacp system-priority 100
mlacp node-id 0
! monitor peer [bfd | route-watch]

Pseudowire Class to reflect or decouple AC and PW status


pseudowire-class HS-PW
encapsulation mpls
status peer topology dual-homed
!This command reflects AC circuit status on all PWs

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

58

mLACP CLI (cont)


Port-Channel Definition
interface Port-channel1
description mLACP
no ip address
lacp fast-switchover
lacp max-bundle 1
mlacp lag-priority 100
mlacp interchassis group 1
service instance 3701 ethernet
encapsulation dot1q 3701
xconnect 10.1.1.1 3701 pw-class HS-PW
backup peer 10.1.1.4 3701 pw-class HS-PW

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

59

Platform Specifics for mLACP


mLACP is SSO Aware
mLACP is only supported with EVC configuration.
No support for Subinterfaces, Access subinterfaces, Switchport configurations.

mLACP is only supported with ES20,ES40


mLACP is not supported with EVC Routed Pseudowire
configuration
Exception is inter-POA Routed PW use case for VRRP/HSRP

Number of chassis part of redundancy group is 2


802.1ah supports only 1 member link on the Port-Channel. mLACP
configuration should be active-standby with 1 member link.
ASR9K target for mLACP is in release 4.0

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

60

Attachment Circuit Redundancy Options


summary
STP based solution cant provide sub second convergence time
and is difficult to support
REP is simple, spanning tree free protocol and can coexist with
spanning tree topologies. REP integration with EVC in SRE.
Etherchannel / LACP provides a good link redundancy scheme for
single homed devices, supported with EVC starting from SRC
mLACP will provide good redundancy scheme for dual homed
devices.
Hot-standby PW synchronization with mLACP and REP will be
required

Presentation_ID

2006 Cisco Systems, Inc. All rights reserved.

61

Reference
1.

LDP Specification - RFC3036

2.

Pseudowire Setup and Maintenance Using


the Label Distribution Protocol (LDP) RFC4447

3.

PWE3 - RFC 3985

4.

Inter-Chassis Communication Protocol (ICCP)


to synchronize multi-chassis LACP and PW
redundancy state - draft: pwe3-iccp

5.

Pseudowire Virtual Circuit Connectivity


Verification (VCCV) - RFC5085

6.

Bidirectional Forwarding Detection (BFD) for


the Pseudowire Virtual Circuit Connectivity
Verification (VCCV) draft-ietf-pwe3-vccv-bfd-03

7.

Pseudo Wire (PW) OAM Message Mapping draft-ietf-pwe3-oam-msg-map-09

62
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Questions & Answers

63
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

64
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Backup Slides

CIscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

65

Pseudowire Operation - Creation


IGP = transport infrastructure
Targeted LDP for L2VPN
PW creation:
PW Label Withdrawal
It will result in the Label Mapping Message being advertised only if the
attachment circuit is active

PW Status TLV
Mapping for primary and and backup, but using TLV Status for detection

66
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

PW Status
1. Until the SRC: when the AC associated with a PW is down (or
being held down for PW redundancy) labels advertised to peers
are withdrawn.
2. RFC4447 specifies extensions for LDP which allow PW status to
be carried in notification messages to peers. This diverges LDP
label mappings from the AC status notification and allows labels
to be retained through AC status changes:
- as soon as the xconnect is provisioned,
- and until the xconnect is unprovisioned or AC interface shutdown.
3.

The router can send pseudowire status to a peer router, even when the
attachment circuit is down

interface Loopback0 !
ip address 10.1.1.1 255.255.255.255 !
! !

Router# show mpls l2transport vc detail

pseudowire-class atomstatus !

encapsulation mpls !

Last remote LDP TLV status rcvd: AC DOWN(rx,tx faults)

status !
! !
interface GigabitEthernet10/5 !
xconnect 10.1.1.2 123 pw-class atomstatus !
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

67

Virtual Circuit Connectivity Verification


Pseudowire VCCV
Control channel between a pseudowire's ingress and egress points over
which connectivity verification messages can be sent
Encapsulated using PWE3, follows data paths

Control Channel (CC) Types


in-band, out-of-band,

Connectivity Verification (CV) Types


LSP Ping [RFC4379], ICMP Ping [RFC0792], BFD

Can additionally carry fault detection status between the


endpoints of the PW
Translated into the native OAM status codes used by the native access
technologies
68
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public

Pseudowire - Detection, Notification


1. VCCV-BFD Connectivity Verification
fault detection only
1. LDP status TLV
mechanism for AC and PW status and defect notification
1. PW OAM Message Mapping
specifies the mapping and notification of defect states
between a Pseudo Wire and the Attachment Circuits (AC)
of the end-to-end emulated service
69
CiscoEXPO

2010 Cisco Systems, Inc. All rights reserved.

Cisco Public