Technical Description

Symphony Plus Ethernet Networking

Overview
The Symphony Plus Plant Network, PN800, uses standard IP Ethernet technology for its communication
backbone. PN800 replaces the INFI-NET loop from Symphony Harmony. Considering the fact that the
network topology is specific to each project, this document outlines best practices, recommended
topologies, and other implementation details and information.

Summary
This document is intended to guide plant engineers who are responsible for the safe and secure
implementation of PN800 and the devices connected to it. These engineers are expected to have a basic
familiarity with IP Ethernet networking technology and Symphony Harmony. It is not intended to be a
definitive listing of how to configure Ethernet networks for Symphony Plus. It is intended to serve as a
guide to a qualified industrial network engineer.

Scope
The primary scope of this document covers how to connect PN800 nodes to PN800 such that they can
communicate with each other, and to operator consoles and engineering workstations. It also covers
other Ethernet networks associated with Symphony Plus.

Networking Overview
The Network Stack
IP Ethernet networking can be considered as a stack, a group of protocols which build upon each other.
Each layer does not need to know about any of the other layers – they only need to be able to remove
any additional overhead they add to a communication packet.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 1 of 18

 Figure 1 The Networking Stack

Physical Layer
The physical layer defines the physical media (and its properties) which transmits the data. The media
may be copper (e.g. 100BASE-TX or 1000BASE-TX over CAT5, CAT5E, or CAT6 cable) or fiber optic cable
(100BASE-FX, 100BASE-SX, 1000BASE-SX, et al over single- or multi-mode optical fibers).

The physical layer of all existing Ethernet-connected Symphony Plus modules is 100BASE-TX (using
CAT5E at minimum). However, connections between switches may use copper or fiber optic cable. A
minimum speed of 100Mbps is required; 1000Mbps or faster connectivity is recommended for switch
interconnects.

Data Link Layer

The Data Link Layer handles communication between two directly connected devices (e.g. two devices
connected to the same switch). This is the layer at which Ethernet devices’ MAC addresses are added to
the packet. Switches inspect the MAC address information, and only re-transmit the packet on the port
to which the destination device is connected.

IP Layer
The IP layer handles communication between nodes that may not be directly connected together on the
same layer 2 segment. As implied by the name, the IP layer adds IP addressing and subnet information
to the packet. A source module will compare its IP address and subnet mask to those of the destination,
and if they are on the same IP network, it will transmit the packet to the destination’s MAC address.
Otherwise, the source module will transmit the packet to its default gateway router (using the router’s
MAC address), which will then repeat this process until the destination node is directly connected to a
router, or the packet expires.

IP routing is currently not supported for PN800.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 2 of 18

Transport Layer
The transport layer is responsible for sessions which span multiple messages (for example, a request and
response). Common protocols include Transmission Control Protocol (TCP) or Universal Datagram
Protocol (UDP).

Application Layer
The application layer contains the actual data the running application is trying to communicate (e.g.
process data). The protocol is defined by the individual applications. For example, INFI-NET message
data (exception reports, etc.) is contained at the application layer.

Putting It All Together

Figure 2 Example Ethernet Packet

As illustrated in Figure 2 above, which provides an abstract example of an Ethernet packet, each layer of
the stack encapsulates the higher layer when being transmitted. When the packet is received, each
layer removes its encapsulation before passing the packet to the next layer up, until the application
receives just the data it is expecting, with all the wrappers removed by the lower layers.

Additional Documentation
2VAA002630 S+ Control: SPIEB800 INFI-NET to PN800 Ethernet Bridge User Manual
IEC 62439-3 International Standard, Industrial Communication Networks – High availability
automation networks
Table 1 Additional Documentation

How INFI-NET concepts and terms translate to PN800

The table below lists several terms and definitions used in Symphony Harmony, and how they translate
to Symphony Plus (as used in this document).

INFI-NET Term Symphony Plus Term

Loop or Ring Segment
Table 2 INFI-NET Concepts in PN800

Symphony Plus Ethernet Networks

PN800 Plant Network

PN800 is the network that connects Symphony Plus nodes together. It is functionally equivalent to INFI-
NET loops in Symphony Harmony.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 3 of 18

SOE Time Synchronization Network
Every controller has a SOE Time Synchronization Network port (SynchroNet). SynchroNet is a dedicated,
independent, non-redundant IP Ethernet network whose only purpose is to distribute highly accurate
time to controller nodes for the purpose of time stamping Sequence of Events (SoE) data. Symphony
Plus controllers receive time from the local time master (e.g. a satellite clock) on SynchroNet using
Simple Network Time Protocol (SNTP). A SNTP client in the controller’s firmware connects to and
receives time from a SNTP server. Not all controllers have to be connected to SynchroNet; only those
which need highly accurate time for SoE tracking should be connected.

SynchroNet is not used to provide Symphony Plus system-wide time. Symphony Plus system time
mastership and distribution are managed using a proprietary protocol on PN800, similar to the one used
for Symphony Harmony. However, a dedicated network interface on one (or more) console(s) could be
connected to SynchroNet. The console’s operating system can then be configured to use the same time
master as the controllers, and the console software (Symphony Plus Operations, etc.) can then be
configured to assume the system time master role, and therefore provide similar time for Symphony
Plus system time. Note that Symphony Plus system time will not be as accurate as SynchroNet time in
this scenario, only similar.

There are no special requirements for the network switches on SynchroNet. However, the number of
switches between the time master and destination nodes should be kept to a minimum to minimize
propagation delay (and therefore clock skew).

The figure below shows an example SynchroNet network. It does not show any other networks.

CTB810 CTB811
F F F F
R R R R

1 1
2 2
3 3
4 4
5 5
CTB810 CTB811
6 6
F F F F
7 7
R R R R
8 8
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

1 1 1 1
2 2 2 2
3 3 3 3
4 4 4 4
5 5 5 5
HN800 CW800 HN800 CW800
6 6 6 6
HC800 C P800 HC800 CP800
7 7 7 7
8 8 8 8

L+ L+
L- L-

SA SA
HN800 CW800 HN800 CW800 SB SB

HC800 CP800 HC800 CP800

MB810 COM MB810 COM

L+ L+
L- L-

SA SA
SB SB

MB810 COM MB810 COM

Operations
Console
(optional)

(optional)

Operations
Satellite Clock SynchroNet Switch Server

Figure 3 Example SynchroNet Network

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 4 of 18

Foreign Device Interface
All Symphony Plus controllers have a dedicated, independent, non-redundant Ethernet Foreign Device
Interface, which can be used for Modbus TCP, HGS, or other IP Ethernet-based communications.
Symphony Plus controllers have no special requirements for network switches on this connection. Refer
to the documentation of other devices on this network for any special requirements. Such requirements
are outside the scope of this document.

PN800 Network Redundancy

PN800 uses Parallel Redundancy Protocol (PRP, IEC 62439-3 Clause 4) to provide a redundant
communication network. PRP provides redundancy using a second, completely independent, fault
isolated network. The topology and hardware of the second network should be identical to the first.

This section is intended to provide a brief overview of PRP. Refer to IEC 62439-3 Clause 4 for the
complete PRP standard used on PN800.

PRP Functional Overview

PRP operates at the device level by transmitting effectively identical packets simultaneously on both
networks, with the destination node discarding the duplicate packet when received. This results in zero
failover time when one network fails, since both are always actively communicating. As such, referring
to the two networks as “primary” and “backup” (or using similar terms) is inaccurate. Therefore, we will
use refer to the networks as “A” and “B” (per the IEC standard).

Each PRP-enabled node has two Ethernet ports and a “Link Redundancy Entity” (LRE, either in
software/firmware or hardware) which coordinates redundancy. Logically, the LRE sits between the
physical Ethernet ports and the operating system on the node, such that the higher layers of the stack
are not aware that the connection is redundant. To help facilitate the seamless redundancy, a PRP-
enabled node uses the same MAC address on both networks A and B; but, because of this, the networks
cannot be bridged together.

The figure below shows how the LRE is integrated into the network stack.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 5 of 18

 Figure 4 PRP LRE in the network stack

PRP Device Types

A device that is PRP-enabled and connected to both networks A and B is called a Doubly Attached Node,
or DAN. A Singly Attached Node, or SAN, is a device that is not PRP-enabled, and is therefore attached
to only one of the networks (either A or B). SANs can only communicate with DANs and SANs on the
same network (e.g. a SAN on network A cannot communicate with a SAN on network B).

All Symphony Plus embedded modules which communicate on PN800 (e.g. CP800, PNI800, et al) are
DANs. PCs and servers running Symphony Plus Operations and Engineering software are SANs, but will
not be fault-tolerant if there is a network failure on the network to which they are connected. With
additional network cards and PRP driver software, PCs may connect to PN800 as DANs. Third-party
network drivers are available for PCs which enable them for PRP, but their use is outside the scope of
this document.

Alternately, S+ Operations servers may be installed redundantly, with one server in each redundant pair
on either network A or B.

Topology Restrictions
AT NO TIME SHOULD NETWORKS A AND B BE CONNECTED TOGETHER!

PRP operates at the data link layer (layer 2) of the network stack, and is transparent to devices that are
not PRP-enabled. Because PRP uses layer 2 broadcasts to coordinate network redundancy, it is currently
not possible to extend redundant PN800 using layer 3 (IP) routers.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 6 of 18

Network Security
Do not directly connect PN800 to the public Internet, or to any other public or private network (such
as a corporate Intranet or other office network). The best, most robust, most secure PN800
implementation is in the form of an air-gapped network, where there is no direct electronic connection
to outside networks.

Figure 5 Unacceptable Network Configurations

If it is absolutely necessary to connect between PN800 and other networks, a firewall should be
implemented as a minimum safety and security measure. The firewall should, by default, deny entry to
PN800, and have explicit rules to allow only the necessary traffic to enter. If possible, a secure,
encrypted, authenticated virtual private network (VPN) with limited user access would be a more secure
solution to connect into PN800 from the outside. Connecting between PN800 and other networks, using
a firewall, VPN appliance, or any other device, is neither recommended nor supported, and, therefore, is
outside the scope of this document.

Figure 6 Acceptable, But Not Recommended, Network Connection

Computers running Symphony Plus Operations, Symphony Plus Engineering, or other software which
needs to communicate on PN800 may be multi-homed (i.e., they may have network interfaces on PN800
and network interfaces on another network; e.g. for the purpose of applying security updates to the
operating system and other software from a trusted source on the other network). These computers
must not route communications between PN800 and the other network.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 7 of 18

 Figure 7 Complete Redundant PN800 Network Overview

Directly bridging PN800 to any other network, regardless of the bridging technology, is not
recommended, and the user assumes all responsibility for safety and security. Such a configuration
reduces the overall security of the PN800 network, and increases the risk of security breaches. ABB
strongly advises against such a configuration.

TCP and UDP Ports Used On PN800

If it is necessary to connect PN800 to outside networks, using any method, communication to TCP ports
502, 2500, 2501, and 3000, and UDP ports 123, 2501, and 2502 should be blocked from entering PN800
from other outside networks.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 8 of 18

For optimum security, all incoming traffic should be blocked by default, with only specific paths opened
on an as-needed basis.

Other Security Measures

Implementing other active and/or passive security measures, such as intrusion detection systems, is
outside the scope of this document.

IP Addressing and Subnetting

PN800
In PN800 the two least significant bytes of the IP address are equivalent to the INFI-NET Loop and Node
number. As such, it is recommended to set aside a /16 network segment for PN800 (e.g. 10.127.0.0
through 10.127.255.255). The subnet mask of all PN800 devices should be 255.255.0.0 to allow for
maximum expandability. This creates a single network address space equivalent to INFI-NET, and
maximizes the loop/node address space. A different network mask can be used to create a smaller
network, but because layer 3 routing within PN800 is not supported, the two different PN800 networks
will not be able to communicate with each other.

Unlike INFI-NET, there is no restriction on the assignment of loop numbers. I.e., there is no “central” or
“satellite” loop. All IP addresses are valid, and all network topologies are acceptable.

Though layer 3 routing can be added between PN800 and outside networks, it is neither supported nor
recommended. As such, it is outside the scope of this document.

SynchroNet
Because the SynchroNet time synchronization network is completely independent of PN800, its IP
address space should be similarly isolated. This separation allows controllers in different PN800
segments to be connected to one time master. Therefore, the IP address space should be isolated from
the PN800 address space. (e.g. 10.126.0.0 through 10.126.255.255 with a subnet mask of 255.255.0.0).
Though the IP addresses on SynchroNet can be assigned arbitrarily (i.e. they do not have to correlate to
the PN800 IP addresses in any way), it is recommended to use a similar addressing scheme as PN800
(e.g. the third octet represents the PN800 segment number, and the fourth octet represents the node
number on the segment).

To allow both the primary and backup controllers to maintain accurate time from the time master, they
must have unique IP addresses. The backup’s address is set by the primary (via the redundancy link) by
adding 1 to the least significant octet of the SynchroNet IP address. It is recommended that all primary
modules use an even value in that octet (and allow the backup to assume an odd number).

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 9 of 18

Equipment Guide
ABB does not recommend any particular network switch or router vendor, as ABB’s Symphony Plus
product line adheres to standards-based Ethernet protocols. However, several vendors’ equipment has
been used during product development and testing, including Cisco, Linksys, Dell, NetGear, D-Link,
3COM, HP, MOXA, Contemporary Controls CTRLink, Advantech, and others. At minimum, switches
should be capable of 100Mbps Full-Duplex operation.

Types of Ethernet Devices

Switches
A switch is a device which allows two or more devices to communicate with each other over Ethernet. It
inspects each communication packet, and re-transmits the packet (unmodified) on the port to which the
destination device is connected. This means that when a packet is received, the switch inspects the
packet and reads the destination MAC address. It will then re-transmit the packet on the interface
which is connected to the destination device (and no others).

Some managed switches are capable of performing basic IP routing. These are called Layer 3 Switches.
Use of any layer 3 routing is not supported on PN800.

Routers
A router allows an IP Ethernet network to be broken down into smaller Subnets to reduce load on
switches, and reduce the size of MAC address tables in switches. Due to the method with which PRP
coordinates redundancy, installing layer 3 routers between segments of PN800 is not supported.

Managed vs. Unmanaged Switches

PN800 is designed to be fully usable on networks employing only unmanaged switches (i.e. no
functionality relies on the capabilities of managed network switches, such as VLAN tagging, trunking, link
aggregation, etc.). This simplicity reduces the number of possible misconfigurations and security
vulnerabilities on PN800, and keeps the Symphony Plus system cost-effective.

If managed switches are desired (e.g. to collect statistics or other data using protocols such as SNMP),
care must be taken to ensure that switch failures do not affect both networks A and B (e.g. simultaneous
spanning tree recalculations). Such configurations are outside the scope of this document.

Additional care must be taken to ensure that switches with built-in redundancy do not adversely affect
PN800 operation. For example, certain Hirschmann switches have additional redundancy features
which must be disabled for operation on PN800.

Network Cabling
Electrically shielded copper Ethernet cables should be used when wiring PN800 and SynchroNet. Refer
to documentation for other devices connected to the Ethernet Foreign Device Interface to determine if
shielded cables can be used.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 10 of 18

Ethernet cables should also be color-coded to clearly identify a cable’s purpose. Section 4.2.1.3 of the
PRP standard (IEC62439-3) defines red for network A cables, and blue for network B cables. This
document uses green lines to signify cables for SynchroNet.

Additionally, both ends of the cables should be clearly labeled with the network identifier (A or B) and
the location of the other end (e.g. a switch identifier and port number).

INFI-NET – PN800 Bridge IEB800

The INFI-NET – PN800 Bridge (IEB800) allows existing Symphony Harmony installations to expand by
adding a Symphony Plus PN800 segment. When combining PN800 segments with an existing Symphony
Harmony system, all rules for configuring INFI-NET must be followed, including:

 Loop/segment numbers must adhere to INFI-NET requirements (numbered between 1 and 250,
with loop/segment 1 being the central network), and be unique in the entire control system
(e.g. there cannot be both a PN800 subnet 45.xxx and INFI-NET loop 45).
 Node addresses must conform to INFI-NET requirements (numbered between 1 and 250).
 There must be a single “central” network, either a PN800 segment or an INFI-NET loop.
 There must be one IEB800 between the central loop/segment and each satellite loop/segment.

If using INFI-NET as the central loop, the IP address space for the individual PN800 segments (which are
no longer directly interconnected) may be reduced. For example, for a PN800 segment that is satellite
to a central INFI-NET loop, the subnet mask may be changed to 255.255.255.0, and default gateway for
the nodes on that PN800 segment is the IP address of the IEB800. If a PN800 segment is used as the
central network, the IP address space should remain unchanged.

Refer to the IEB manual for additional implementation details.

SynchroNet Time Masters

ABB does not recommend any particular time master. During product development and testing, a
Symmetricom SyncServer S350 with a GPS antenna was used. But, any time master supporting SNTP can
be used.

Recommended Physical Topologies

Thanks to the efficiencies of network switches compared to hubs, individual packets are only
transmitted on the necessary switch ports in order for them to get to their destination (i.e. individual
modules should only see traffic that is destined for them). All modules will still receive broadcast
messages, and subscribed modules will receive multicast messages.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 11 of 18

Single Segment

Figure 8 Single Network Segment Topology

In this topology, there is only one network switch on each of the redundant networks, with all PN800
nodes connected to the single switch. It can seamlessly be expanded into a larger two- or multiple-
segment topology. In this topology, an IEB800 can be connected to a spare port on the segment
switches.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 12 of 18

Two Segments
CTB810 CTB811
F F F F
R R R R

1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

HN800 CW800 HN800 CW800

HC800 CP800 HC800 CP800

L+ L+
L- L-
SA SA
SB SB

MB810 COM MB810 COM

PN800 “A” PN800 “B”

Segment 1 Segment 1
F
R

1
2
3
4
5
6
7
8

PNI800

L+
L-
CTB810 CTB811
F F SA
R R SB

1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

HN800 CW800 HN800 CW800

HC800 CP800

L+
L-
SA
SB

MB810 COM

F F
R R

1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

F
PNI800 PNI800 R

1
2
L+ L+ 3
L- L- 4
SA SA 5
SB SB 6
7
8

PNI800

L+
L-
SA
SB

CTB810 CTB811
F F
R R

1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

HN800 CW800 HN800 CW800

HC800 CP800

L+
L-
SA
SB

MB810 COM

PN800 “A”
PN800 “B”
Segment 2
CTB810 CTB811
F F F F
R R R R

1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
Segment 2
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8

HN800 CW800 HN800 CW800

HC800 CP800 HC800 CP800

L+ L+
L- L-
SA SA
SB SB

MB810 COM MB810 COM

Figure 9 Two Segment Topology

In this topology, there are two redundant network switches, one for each redundant segment. Each
network switch is connected to the PN800 nodes assigned to that segment. This ensures that only the
traffic which needs to cross to the other segment goes across the link between the switches. In this
topology, an IEB800 could be connected to either segment switch (preferably the one which will be

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 13 of 18

communicating with the INFI-NET loop more than the other). Alternatively, the Several Segment
topology could be employed to add an IEB800.

Several Segments

In this topology, there is a core switch which is connected to segment switches, with one segment
switch per segment. The segment switches are then connected to PN800 nodes (e.g. CP800, PNI800) on
their respective segments. In this topology, an IEB800 may be connected to the core switch. Refer to
the IEB800 manual for additional information and configuration restrictions.

Expanding INFI-NET
In this scenario, a customer has an existing Symphony Harmony system, which is being expanded (using
the IEB800) to add a PN800 segment.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 14 of 18

 Figure 10 Expanding Symphony Harmony with Symphony Plus

Other Topologies
Many other network topology designs are possible. It is possible to sub-segment PN800 segments, to
further minimize excess traffic passing through and between network switches. However, care must be
taken, as every switch through which a packet passes will add additional latency. It may be better for
performance to replace a smaller switch with a larger one.

Using managed switches allows further redundancy to be added by utilizing Spanning Tree Protocol (or
similar protocols) to create multiple paths from source to destination nodes. Because of PN800’s
network redundancy, a single switch failure will not cause any disruption to the process, as one network
will still be operating while the network with the failure re-calculates its spanning tree. However,
implementing a network using such technologies is outside the scope of this document.

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 15 of 18

Example Installation
Below is an example installation, using four redundant HPC800s, three PNI800s (two for S+ Operations
and one for S+ Engineering), two (redundant) S+ Operations servers, two S+ Operations consoles, a
stand-alone S+ Engineering workstation, and a satellite clock SNTP server for SynchroNet.

Figure 11 Example Installation

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 16 of 18

The table below lists IP addresses for all devices on all networks in the above diagram. PN800,
SynchroNet, and OpsNet subnet masks are 255.255.0.0. Addressing the Management Network and
Corporate Intranet are beyond the scope of this document.

Device PN800 SynchroNet OpsNet

HPC1 10.127.1.10 10.126.1.10 N/A
10.126.1.11
HPC2 10.127.1.12 10.126.1.12 N/A
10.126.1.13
HPC3 10.127.1.14 10.126.1.14 N/A
10.126.1.15
HPC4 10.127.1.16 10.126.1.16 N/A
10.126.1.17
PNI1 10.127.1.101 N/A N/A
PNI2 10.127.1.102 N/A N/A
PNI3 10.127.1.103 N/A N/A
S+ Operations Server 1 10.127.1.200 10.126.1.200 10.125.1.200
S+ Operations Server 2 10.127.1.201 10.126.1.201 10.125.1.201
S+ Operations Console 1 N/A N/A 10.125.1.202
S+ Operations Console 2 N/A N/A 10.125.1.203
S+ Engineering 10.127.1.111 N/A N/A
Workstation
SNTP Server N/A 10.126.1.2 N/A
Table 3 IP Addresses for Example Installation

Remote Network Connections

IP Ethernet networking technology provides many options for connecting to remote sites (e.g. VPN,
point-to-point or mesh wireless, et al). The ideal PN800 network connection to a remote site is a
dedicated, hard-wired, direct link, which does not utilize any equipment or services owned or controlled
by a third-party (e.g. an Internet service provider). Configurations which extend any Symphony Plus
communication network using a third-party connection are neither recommended nor supported.

Revision
Revision Page (P) Description Date Changed by
Chap. (C) & Dept.
A All Initial 4Apr2013 C. Marks,
PSPG

ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 17 of 18

 ABB Inc.

Doc Id: 2VAA002993.docx Rev. A Date: 4 April 2013 Page 18 of 18