Published 3Q 2010
Commissioned by iDirect
Bob Gohn
Senior Analyst
Clint Wheelock
Managing Director
Smart Grid Network Technologies
Section 1
INTRODUCTION
Electricity is the lifeblood of the modern world, and will become even more so as society
strives to replace fossil fuels. Yet the grid that delivers this resource has seen relatively few
breakthrough innovations, especially when compared with the communications
technologies now seen as the newest “utilities” of modern life. Though it would be unfair to
characterize the current electrical grid as “dumb,” new economic, technical, environmental,
and political challenges are driving the integration of computing and communications into
the grid in a new way. The goal is clear: dramatically improve the reliability, security,
environmental impact, and operating cost of the grid’s generation, transmission,
distribution, and customer-consumption systems. An architected infusion of networking
technology into the grid, where each of the key components within the grid is
interconnected through a robust communications network, is the so-called “smart grid.”
This paper explores the communications requirements of the emerging smart grid, and
reviews the various networking technologies offered for these applications. Against this
background, a closer examination of IP satellite communications is offered, as this
technology has experienced significant innovation in recent years. Even utilities already
using satellite in their communications mix may be surprised at the new capabilities and
cost points currently available. While satellite communications has long been used for
remote connectivity in utility networks, its use is quietly expanding among major carriers,
enterprise networks, emergency communications, and commercial airlines. Smart grid
communications architects should understand how satellite may fit within their specific
requirements.
2
Smart Grid Network Technologies
Section 2
SMART GRID APPLICATIONS AND
COMMUNICATION REQUIREMENTS
2.1 The Smart Grid
The smart grid is often described as the merging of the “power network” with a modern
“communications network.” While this may be a simplistic description, this merger requires
the integration of embedded computing and communications intelligence throughout the
power system, from generation, transmission, distribution, and consumers (residential,
commercial, and industrial), with the aim of creating a more flexible, efficient, and reliable
power grid.
Utility Operating Cost Reduction – Automating the remote monitoring and reading of
grid devices (meters, substation equipment, etc.) eliminates the need for personnel to
make physical visits (“truck rolls”), thus reducing operating costs. The power network
can be actively managed to deliver greater reliability and efficiency, lowering overall
costs.
Grid Reliability and Security – Communications enables real-time data gathering and
control of the grid, allowing faster response to potential and actual outages. Networked
devices in the field may also act as the backbone of automation technologies that allow
automatic recovery from field failures.
Energy Efficiency and Management – Through proactive management of grid
devices, significant energy efficiency gains can be achieved, including active
optimization of distribution feeder voltage levels, or perhaps more importantly, direct
consumer engagement of electricity usage patterns through smart meters.
Renewable and Distributed Generation – The grid is rapidly evolving from one-way
“pipes” from generation source to consumers, toward multi-way power flows from
home- or business-based renewable and/or distributed generation (solar panels, wind,
combined heat/power generators, etc.) sources. Advanced communications are
required for the utility to accommodate and manage these resources.
There are many other factors driving smart grid deployment, including energy security and
carbon emission reductions. As such, transformation of the electrical grid becomes a
national priority for countries around the world.
3
Smart Grid Network Technologies
Smart Metering and AMI – providing frequent and granular consumption and power
quality measurements and 2-way communications and control between the utility and
the customer premise. This includes the possibility of a customer premise “home area
network” (HAN).
Substation Automation – enabling remote monitoring and control of transmission and
distribution substations throughout the grid, including evolving SCADA technologies,
protection switching, and physical security monitoring.
Distribution Automation – delivering a range of advanced services within distribution
feeder circuits, including automatic restoration, dynamic reconfiguration, dynamic
volt/VAR controls, distributed generation and electric vehicle support.
Each of these is described in more detail in the appendix.
4
Smart Grid Network Technologies
The table below compares the relative communications requirements for the three primary
Smart grid applications discussed above.
5
Smart Grid Network Technologies
Section 3
SMART GRID COMMUNICATIONS TECHNOLOGIES
3.1 Smart Grid Communications Architecture
There are many interconnected and communicating systems within the current grid.
However, most of these systems have been built as vertically integrated “system silos”
focused on singular applications. Examples include substation SCADA systems, meter
reading (or AMI) systems, autonomous distribution automation systems, enterprise
communications, and mobile workforce radio systems.
An important attribute of the “smart grid” concept is shifting this paradigm toward a well
engineered, generic communications network infrastructure supporting multiple
applications together. This offers significant benefits in operating and management costs
as well as long-term cross-utilization of connected resources between grid applications,
such as the use of advanced metering systems by outage management systems. In many
ways, this follows the evolution of the separate voice, data, and video telecom networks of
20 years ago in the “Internet phenomenon” that has revolutionized personal and corporate
communications.
The smart grid is about more than just adding communications to the grid. It concerns how,
architecturally, communications are incorporated, and the new applications that can result.
SA-WAN
Enterprise
Facility
WB WB SR GR SS LAN
RTU, IEDs
Enterprise LAN NI GR NI GR Major
Substation
HAN
Customer
HN Premises
6
Smart Grid Network Technologies
The model portrays a four-tier network hierarchy, which may or may not correspond to any
given physical implementation. The major functional areas are:
Just as the telecom industry wrestled with the shift from purpose-built networks to open
data networks, the utility industry is grappling with adoption of the Internet Protocol (IP)
suite within the smart grid. The widespread use of IP is at the heart of the NIST efforts, as
IP can be the common denominator facilitating interoperability between fragmented
automation systems in the utility network.
7
Smart Grid Network Technologies
Leased Lines and Broadband - The utility industry has made extensive use of leased
telecom lines within their own private networks. Today, these are used to connect early
substation automation systems (and other independently “siloed” systems) to the utility
central control center. While many other industries have moved much of this connectivity
(usually for Internet data and/or voice access) to newer xDSL or cable technologies,
utilities have been slower to do so due to a lack of bandwidth demand by existing SCADA
applications, lack of broadband availability at remote grid locations, and fear of hidden
dependencies by older SCADA implementations on serial communications.
Despite the relatively good reliability delivered by traditional leased lines, many utilities are
increasingly wary of using public infrastructure for critical grid communications. Utilities are
concerned that the public network might be unreliable when needed most, such as during
storm outages or other natural disasters. Also, many carriers want to discontinue traditional
leased line services as they move toward packet-based infrastructures, leaving utilities to
find alternatives or pay significant increases to already high rental fees.
8
Smart Grid Network Technologies
Private Fiber - In some cases, utilities have deployed their own fiber cables within their
own power line rights-of-way, and can leverage these for their own private networks. This
is often used to connect major substations, with dedicated fiber used for the most sensitive
protection switching applications, and dedicated SONET equipment used for backbone
communications. Fiber offers very high performance, reliability (if used with resilient
technologies such as SONET), and security. However, it requires clear right-of-way access
to install, and can be very expensive, especially over longer distances. Hence fiber is not
practical for many more remote substations or in most distribution automation applications.
Narrowband Power Line Communications (PLC) - As the name suggests, PLC provides
connectivity using existing power lines as the communications medium. There are many
different types of systems (and terminologies) associated with power line communications,
and nearly as many standards. Performance ranges from a few bits/sec to as high as 100
kbps. This technology is most often used to connect smart meters on the low-voltage side
of distribution transformers, where an aggregation node uplinks to the utility network using
a different networking technology. PLC systems are more popular in Europe and parts of
Asia, which average more than 50 meters per distribution transformer, as compared to
North America, which averages less than five meters per distribution transformer. PLC is
less likely to be used for distribution and substation automation applications due to
difficulties optimizing the technology for the range of medium-voltage electrical lines, as
well as concerns over losing communications when needed most, such as when a power
line fails.
Broadband over Power Line (BPL) - BPL systems generally are PLC systems supporting
data rates over 1 Mbps. There is a long-running standards war over BPL technology for in-
home use, and systems leveraging the distribution grid have long been discussed (and
provisionally deployed) as another means of providing alternative “last-mile” broadband
Internet and voice services. As these efforts have faded, due to both technical issues
(including radio interference) and business model issues, BPL is seeing some new interest
as a Smart grid technology. One potential application is as a fiber alternative between
transmission substations, using the high-voltage transmission lines, which offer a relatively
clean BPL communications medium. However, technology for this type of application is still
in early stages of development.
9
Smart Grid Network Technologies
RF Mesh RF Star
WAN
WAN Base
Concentrator
Station
Meters
Meters
The protocols used in RF mesh NAN implementation remain essentially proprietary, though
many vendors have adopted IP at the networking layer. NIST has identified this as an
important standards gap, and the IEEE is working on an extension to the IEEE 802.15.4
specification to create a standard for this application.
Most North American systems today operate in the 900 MHz unlicensed spectrum bands,
with link speeds ranging from 19.2 kbps to as high as 150-250 kbps. As with any radio
technology, the range for reliable communications is influenced by many factors, including
operating frequency, radio design, and deployment terrain. Given all these variables, users
can expect node-to-node communications to range from 100+ meters to several
kilometers. With a mesh topology, a denser mesh (number of nodes in given area) affords
a greater the opportunity for strong communications coverage. Of course, as with any
unlicensed frequency band use, the potential for interference from other sources is a
concern.
These RF mesh networks offer good flexibility and attractive cost points for smart metering
and some distribution automation applications. There are challenges, however, as the
mesh topology inherently creates a lack of predictability in key throughput and latency
performance characteristics. Effective bandwidth is usually a fraction of the offered link
speed and can change as the mesh topology reconfigures, altering the number of “hops”
between any two nodes. This is usually not an issue for smart meter functions, but can be
a concern using the same mesh network for distribution automation or grid control
functions that require guaranteed latencies.
10
Smart Grid Network Technologies
Microwave links may use licensed or unlicensed spectrum. The latter spectrum has lower
costs, but risks possible interference. Unlicensed frequencies range from 2.4 GHz (same
as Wi-Fi, ZigBee, and Bluetooth) to as high as 60 GHz-70 GHz. The higher frequencies
tend to have less potential interference, but also significantly reduced range. The lower
frequency bands (2.4 GHz and 5.8 GHz) can achieve distances up to 20 miles.
Licensed microwave links require an FCC license, use of better (and more expensive)
antennas to focus transmissions more effectively, and incur higher installation costs.
However, they generally offer greater reliability due to a guarantee of no interference, and
depending on equipment, can offer higher data rates.
Licensed spectrum can be expensive and difficult to obtain. Therefore, vendors of these
private systems usually acquire and manage the rights to this spectrum themselves,
allowing the utility customers to use it without the associated initial costs and regulatory
hassles. Additionally, the frequencies used are usually in the 400 MHz and 900 MHz
bands, though may be higher, with associated trade-offs in range (lower is usually better)
and bandwidth (higher is usually better). Typical data rates – shared among potentially
thousands of end nodes per tower – range from 4.8-9.6 kbps for traditional Multiple
Address System (MAS) radios to as much as 300+ kbps for proprietary AMI-optimized
systems.
These systems are typically configured so that each device has access to two or more
gateways for redundancy, which are located on towers with antennas at heights of 200 to
650 feet, often sharing space with cellular infrastructure. Alternatively, gateways may be
located on rooftops and other urban locations. The process and costs of obtaining tower
access can be a significant undertaking in the deployment of these systems, as well as
assuring that reasonable line-of-sight is possible to each end node.
11
Smart Grid Network Technologies
driving another look at this opportunity. Meanwhile, utilities are more interested in cellular
as a communications option because of the low capital investment, variable cost structure
of purchasing connectivity by the megabyte, and the low cost, standardized modules and
chips available for a wide variety of equipment and devices.
The cost of utilizing cellular network connectivity for smart meters still remains higher than
with proprietary RF mesh networks, but cellular is widely used for AMI backhaul
connectivity. In Europe, the lower cost and better coverage of cellular infrastructure is
enabling deeper use in the smart grid.
While the efforts of public carriers to address smart grid applications is gaining some
attention, most utilities remain very cautious about using public wireless infrastructure for
critical communications, fearing for the system’s reliability or severe bandwidth contention
during the moments most needed, such during some sort of public emergency or natural
disaster. Full coverage over a utility’s full service footprint is also a concern.
The emergence of 4G technologies (WiMAX and LTE) will allow public wireless carriers to
offer more robust bandwidth and security guarantees to utilities, however this is unlikely to
see much adoption by utilities until the 4G build-outs are further along, and may not fully
address utilities’ general concerns with public network reliability.
Today, satellite communications has recently undergone a major transformation and has
evolved in improving performance reliability and reducing costs. Satellite networks are now
two-way communications systems, built on IP, with broadband data rates. Next-generation
coding standards have made satellite more reliable and cost efficient. Further, satellite
networking hardware has been engineered to meet next-generation carrier standards,
integrating well with terrestrial wireless and wireline communications.
Broad geographic coverage, including areas where standard wired and wireless
technologies cannot reach.
Flexible data rate performance, ranging from 16 kbps suitable for basic SCADA
connectivity to speeds of 1 Mbps and above in support of voice, video and general data
applications. Performance is often further enhanced by bandwidth optimization
technologies for IP communication, such as User Datagram Protocol (UDP) header
compression.
Highly reliable connectivity, suitable for day-to-day operation or as a backup to
terrestrial systems during disaster recovery situations.
12
Smart Grid Network Technologies
In addition to supporting sites such as substations and power generation plants, satellite is
an increasingly viable option for other smart grid applications, including:
13
Smart Grid Network Technologies
(Source: iDirect)
14
Smart Grid Network Technologies
Also note that any evaluation of cost must include the cost of infrastructure acquisition,
communications media acquisition, and operating cost. More bandwidth generally costs
more, but in a manner that may not be immediately obvious. For example, depending on
higher bandwidth, modes of WiMAX might mean a shorter design range, incurring more
tower-based base stations and higher total systems costs. Other technologies incur higher
costs as distance between nodes increases, such as private fiber, where the largest costs
may come from the fiber installation itself, or leased lines, whose monthly charges may be
proportional to distance. These can be difficult to generalize.
15
Smart Grid Network Technologies
Private
1M WiMAX Broadband
BPL
100K 2G/3G
Cellular
RF Mesh
10K
RF Star
DA-WAN Leased Line
1K N-PLC
AMI - NAN
100
Private
This chart also illustrates the range of performance / cost points required for each of the
major applications considered (AMI, Distribution Automation, Substation Automation, etc.).
For AMI technology, SCEs SmartConnect program includes the rollout of over five million
smart meters based on Itron’s OpenWay platform with 900 MHz mesh radio NAN and
ZigBee HAN. SCE currently is leveraging 3G public cellular backhaul for the AMI network,
with about 300 meters per AMI concentrator node, which allows capacity headroom for
additional functionality over time.
Virtually all of SCEs 900+ substations have some level of automation in place, a program
started over 15 years ago. While SCE is expanding the use of fiber to connect these
substations (in addition to existing leased lines and microwave links), SCE has also made
extensive use of satellite links for as much as 30% of these substations. Satellite provides
reliable connectivity to SCE’s many remote substations locations. SCE is rolling out
technology upgrades to their satellite network to enable Voice over IP, general data
(Internet), and perhaps video surveillance applications, in addition to SCADA, at these
substations.
Approximately 41%of SCE’s distribution circuits are currently automated, which may
include automation of tie switches, reclosers, fault indicators and capacitor banks. Most
16
Smart Grid Network Technologies
communications use a 900 MHz RF mesh network based on the Landis+Gyr (formerly
Cellnet) UtiliNet radio system. Over 40,000 radio nodes are deployed, and are currently
maintained separately from the AMI network system.
SCE was awarded ARRA stimulus funding for its Irvine Smart Grid Demonstration (SGD)
project, targeted as a “deep vertical dive” into end-to-end smart grid technologies that
leverage SCE’s experience in smart metering, distribution automation (via an early “Avanti
distribution circuit” project), and synchrophasor development. This project will evaluate a
variety of communications technologies, including WiMAX for some distribution and
substation automation, AMI backhaul, and perhaps even into the home connectivity.
Expanded use of SCE satellite network is also being evaluated for AMI backhaul and
distribution automation applications, particularly in regions where other wireless coverage
is weak or not feasible.
17
Smart Grid Network Technologies
Section 4
CONCLUSIONS AND RECOMMENDATIONS
The evolution of the smart grid will continue to leverage a wide variety of different
communications technologies, though as emerging standards take hold, the number of
proprietary systems will likely decrease. However, utility preference for their own private
networks is likely to continue until public carriers can deliver the coverage, service level
guarantees, and cost points required.
Smart grid networking choices are likely to converge around four primary technology
groupings:
1) Private, purpose-built mesh or star RF networks (mostly in North America) and PLC
networks (mostly in Europe), aimed at smart meter connectivity at the neighborhood
level (AMI-NAN). Proprietary today, these will eventually use emerging standards.
Utilities planning integrated smart grid network infrastructures need to clearly identify and
balance the relative cost, performance (both for bandwidth and latency), coverage,
reliability, and security requirements against the variety of choices available. Standard, IP-
based, and adaptable technologies should be given a preferential place in any evaluation.
Modern satellite communications options should be evaluated as part of smart grid
communications architecture.
18
Smart Grid Network Technologies
Section 5
APPENDIX
SMART GRID APPLICATIONS AND USE CASES
5.1 Smart Grid Applications and Use Cases
The basic electrical grid structure can be divided into the following major components:
A) Generation – any type of major generation plant: coal, hydro, gas, etc. Facilities may
be owned by a vertically integrated utility or separate generation companies.
Distribution substations, located throughout a utility’s service territory, that convert the
sub-transmission voltage levels to lower voltages used for distribution feeder circuits
into the service neighborhoods.
Distribution feeder circuits, including various distributed devices (switches, reclosers,
sectionalizers, capacitors, etc.) to control and condition power distribution.
Distribution transformers, located on poles, ground-level, or underground, that convert
the distribution feeder voltage to the service voltage (110/220V in NA, serving 3-5
homes, or 220/480V in Europe, serving ~50 homes).
D) Residential and Commercial Consumers – Residential homes or businesses, each
equipped with an individual meter(s).
19
Smart Grid Network Technologies
Smart meters provide the information necessary to offer consumers incentives to conserve
and/or shift their power use, and even support their own distributed power generation.
Substations vary in size and complexity based on the type (transmission or distribution),
purpose (residential or industrial), location (above ground in a dedicated yard,
underground, or if very small, pole mounted), and service territory (number of distribution
feeders supported, etc.). While many substations are manually configured and controlled,
increasingly, these systems are being automated with networking technology.
The communications within the SCADA system vary considerably and are evolving. Initially
based on proprietary protocols over serial lines, most systems today leverage an Ethernet
Local Area Network (LAN) within the substation.
The wide area network connections also vary considerably, with many larger substations
20
Smart Grid Network Technologies
Basic distribution automation consists of equipping the various distribution feeder devices
with communications for remote sensing and control. The devices may have inherent
control and communications capability, and/or use “bolt-on” controls (such as remotely
activated motors to crank manual switches, etc.) to implement the desired automation.
1
As presented to Utah Public Utilities Commission Smart grid Workshop, May 2009, by Rob Pratt Pacific Northwest National
Laboratory.
21
Smart Grid Network Technologies
General Voice and Data Communications - Like any organization, utilities maintain
standard voice, data, and increasingly, video communications networks. These
networks have similar requirements to any other enterprise with multiple sites (customer
offices, remote service facilities, etc.), but uniquely, these often are extended to remote
substation and distribution sites throughout the service territory.
Outage and Fault Management Systems - IT systems that leverage data from field
networks to manage outage situations.
Asset Management and Tracking - Internal systems to indentify and manage the
huge number of physical assets deployed throughout the grid system.
Workforce Automation and Communications - Utilities have large, mobile,
workforces with strong needs for central dispatch and coordination. This is required for
safety, as well as efficient work management.
22
Smart Grid Network Technologies
Section 6
ABOUT THE SPONSOR
iDirect is a world leader in satellite-based IP communications technology. Our products
and services help to transform the way the world gets and stays connected. With more
than 15 years of global satellite communications experience, we serve customers in over
50 countries through a diverse network of service provider partners, including seven of the
World Teleport Association’s Global Top Ten operators. Headquartered in Herndon,
Virginia, we have offices in Europe, Middle East, Africa and Latin America.
We partner with utilities to provide customized communication solutions to help meet smart
grid objectives. Through our technology, utility companies gain a secure and more reliable
network solution to monitor SCADA (Supervisory Control and Data Acquisition) devices,
with the ability to extend broadband applications such as video surveillance, VoIP and
corporate data access to remote substations. iDirect’s technology can also be leveraged by
utility companies to support the need to backhaul smart meter data from aggregation sites
and manage green energy sites. And it is also relied on by utilities to provide back-up
communications in the wake of disasters and communication circuit failures.
23
Smart Grid Network Technologies
Section 7
TABLE OF CONTENTS
Section 1 ...................................................................................................................................................... 2
Introduction ................................................................................................................................................. 2
Section 2 ...................................................................................................................................................... 3
Smart Grid Applications and Communication Requirements ............................................................... 3
2.1 The Smart Grid .............................................................................................................................. 3
2.1.1 Smart Grid Communications Drivers ....................................................................................... 3
2.1.2 Smart Grid Applications ........................................................................................................... 4
2.2 Smart Grid Communications Requirements ................................................................................. 4
Section 3 ...................................................................................................................................................... 6
Smart Grid Communications Technologies ............................................................................................. 6
3.1 Smart Grid Communications Architecture ..................................................................................... 6
3.1.1 An Example Architectural Model ............................................................................................. 6
3.1.2 Smart Grid Communication Standards ................................................................................... 7
3.2 Survey of Smart Grid Networking Technologies ........................................................................... 8
3.2.1 Wired Technologies ................................................................................................................. 8
3.2.2 Terrestrial Wireless Technologies ........................................................................................... 9
3.2.3 Satellite Communications ...................................................................................................... 12
3.2.4 Summary Comparisons ......................................................................................................... 15
3.3 Smart Grid Networking Example – Southern California Edison .................................................. 16
Section 4 .................................................................................................................................................... 18
Conclusions and Recommendations ...................................................................................................... 18
Section 5 .................................................................................................................................................... 19
Appendix Smart Grid Applications and Use Cases ............................................................................... 19
5.1 Smart Grid Applications and Use Cases ..................................................................................... 19
5.1.1 Smart Metering and AMI ....................................................................................................... 19
5.1.2 Substation Automation .......................................................................................................... 20
5.1.3 Distribution Automation ......................................................................................................... 21
5.1.4 Related Grid Communications Applications .......................................................................... 22
Section 6 .................................................................................................................................................... 23
About the Sponsor .................................................................................................................................... 23
Section 7 .................................................................................................................................................... 24
Table of Contents ...................................................................................................................................... 24
Section 8 .................................................................................................................................................... 25
List Of Figures, Tables, and Charts ........................................................................................................ 25
24
Smart Grid Network Technologies
Section 8
LIST OF FIGURES, TABLES, AND CHARTS
25
Smart Grid Network Technologies
Published 3Q 2010
This publication is provided by Pike Research LLC (“Pike”). This publication may be used only as
expressly permitted by license from Pike and may not otherwise be reproduced, recorded, photocopied,
distributed, displayed, modified, extracted, accessed or used without the express written permission of
Pike. Notwithstanding the foregoing, Pike makes no claim to any Government data and other data
obtained from public sources found in this publication (whether or not the owners of such data are noted
in this publication). If you do not have a license from Pike covering this publication, please refrain from
accessing or using this publication. Please contact Pike to obtain a license to this publication.
26