Tool for systemic risk analysis and secure mediation of data exchanged across linked CI information infrastructures

QoS of SCADA system interconnecting a Power grid and a Telco network

Michele Minichino michele.minichino@enea.it, ENEA

Brussels, 14 September 2009

ICT-SEC 225353 MICIE (1)

QoS of SCADA system interconnecting a Power grid and a Telco network ACK: contribute of WP Interdependency Analysis and Modeling participants
CRPHT CRAT ROMA3 ENEA IEC ITRUST MULT FCTUC UNIBRAD

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (2)

QoS of SCADA system interconnecting a Power grid and a Telco network

Talk contents
Motivation of the research: lesson learned by IRRIIS project
MICIE: Prediction of risk of loss/degradation of quality of services of CI operators (i.e. SCADA and NMS operators)
Reference scenario and service oriented approach Fault Isolation and System Restoration (FISR) service Risk of loss/degradation of FISR FISR models Indicators of risk of loss/degradation of FISR

MICIE: Models for the online risk prediction tool Discussion

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (3)

IRRIIS project - scenario of failure propagation from Telco Network to ACEA MV Power Grid A mini black on the Telecom Italia PoP node in Rome
Flooding of a Telecom Italia major telecommunication node has occurred in Rome on January the 2nd 2004. Part of wired and wireless services tilted (a mini black out for Italian Telco infrastructure) causing problems and delays in different infrastructures, including
Fiumicino airport (stop of check-in, ticketing services and of luggage acceptance and switching), ANSI print agency, post offices and banks, ACEA power distribution and the communication network (GARR), connecting the main Italian research institutions.

The mini black out occurred to the Telecom Italia major node in Rome, thePoP of Laurentina -Inviolatella, on Tor Pagnotta street

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (4)

IRRIIS project - Laurentina Inviolatella node

Green arrow indicates the area where the Telecom centre is located
Luxemburg, 20 May 2010 ICT-SEC 225353 MICIE (5)

IRRIIS project - Telco blackout impacted on services of SCADA operator of ACEA power grid
ACEA SCADA has two main Control Centres: Flaminia Control Centre that is unmanned;
receive/send data and control commands from a first part of the cabins of Rome electrical distribution network;

Ostiense Control Centre that is manned;

receive/send data and control commands from a second part of the cabins;

all the tele-measures, commands and alarms managed by Flaminia Control Centre are dispatched to Ostiense Control Centre using two redundant TELCO communication links at 2Mbits/sec;
One is the main link; the other one is a backup link that is always in stand-by position; such links were expected to be located on two different geographical paths;

both links were out of service during the Telco blackout

as a consequence no alarms, signals on the status of power distribution network and commands where exchangeable between the unmanned centre and the manned one. in this situation SCADA operator completely lose the visibility and controllability of all the remote substations managed by the unmanned Flaminia Control Centre.

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (6)

IRRIIS project - Loss of services of SCADA operator on failure of SCADA communication links

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (7)

IRRIIS project - SCADA interconnecting power grid and telco network

Power grid: a portion of the HV (High Voltage) grid at 150 kV and the backbone of the MV (Medium Voltage) grid at 20 kV.
Each node represents a primary substation (Pi, large rectangle), in case of HV network, or a secondary substation (Mi, small rectangle), in case of MV network. Nodes, named Ei, represent the substations of the national power transmission grid. They feed the power distribution grid. The physical link between any two nodes is an electrical trunk

SCADA system
A Main SCADA Control Centre (MSC) directly controls and supervises the portion of the power grid. A Disaster Recovery SCADA centre (DRS), directly controls and supervises a complementary portion of the power distribution grid. two types of Remote Terminal Units (RTUs), which interface the SCADA with power distribution grid: HV RTUs, located at HV substations, and MV RTUs, located at MV substations.

Telco network
Default Proprietary Network of SCADA Public Switched Telephone network (MSC and DRS are connected, via firewalls, by two redundant, public, high speed Telco links) Global System Mobile connections

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (8)

IRRIIS project - Portion of grid directly observed by SCADA operator (feeding the flooded Telco node)

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (9)

IRRIIS project - SCADA system and its mapping on the whole power grid

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (10)

QoS of SCADA system interconnecting a Power grid and a Telco network: Framework

MICIE main product

MICIE will design and implement a so-called "MICIE alerting system"
MICIE alerting system will support the CI operators by means of an on line risk prediction tool that provides them a real time risk level making use of CI models CI operators are currently assumed to be SCADA and NMS operators

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (11)

MICIE: How can models predict the risk of loss/degradation the QoS of SCADA and NMS operators?

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (12)

QoS of SCADA system interconnecting a Power grid and a Telco network Quality of services of SCADA and NMS operators
How can models predict the risk of loss/degradation the QoS of SCADA and NMS operators with the final aim to improve the quality of power to grid customers?
Reference scenario and service oriented approach Fault Isolation and System Restoration (FISR) service Risk of loss/degradation of FISR FISR models Indicators of risk of loss/degradation of FISR FISR models for the online risk prediction tool

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (13)

MICIE project

Reference scenario and service oriented approach

Reference Scenario consists in identification of services, sequences of adverse events that could impair the quality of such services (i.e. in terms of continuity, readiness, performances, time response) the set of interconnected networks supporting such services (in terms of topologies, essential systems (i.e. Telco emergency power supply, cooling systems))

interconnections among networks and systems

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (14)

Understanding risk of loss/degradation of (SCADA and NMS operators) services due to interdependencies A recursive approach

methodology scenarios

tools

models

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (15)

Reference scenario & service oriented approach

Interconnected networks
MICIE Reference scenario currently includes the following subset of interconnected networks/CIs:
E CI, Electrical CI: a portion of the electrical 22 KV grid and of 161 KV transmission lines C CI, Communication: a portion of communication transmission equipments.
It transfers information and data from Remote Terminal Units and control centres of SCADA and Network Management System for the control and the management of the CIs ( it does not include SCADA and NMS systems)

ICT CI, SCADA system for 22KV grid and NMS system for control and management of fibre optic grid It also includes all the Automatic systems on substations that are included in scenarios
ICT-SEC 225353 MICIE (16)

Luxemburg, 20 May 2010

Reference scenario and service oriented approach

E CI Electrical 22 KV grid portion

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (17)

Reference scenario and service oriented approach

E CI Electrical 22 KV grid portion
(interconnected with C CI and ITC CI)

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (18)

Reference scenario and service oriented approach

CCI Communication portion and NMS

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (19)

Reference scenario and service oriented approach

SCADA and interconnections (C CI and NMS)

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (20)

Reference scenario and service oriented approach

Events impact (through interconnected CI) on energy supplied to MV grid customers

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (21)

Reference scenario and service oriented approach: services identification

A first set of services have been identified:

trying to reveal interdependencies and thus opening the way to cascading failures and escalation effects;
Mutual interactions of services; what a service requires (or it is supposed to require) in order to be properly supplied, in terms of ancillary services and company policies and strategies.

services can be lost (ON/OFF) or can degradate (Quality of Service)

services loss or degradation can propagate impacting on the final (end) user with diverse severities.

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (22)

Fault Isolation & System Restoration (FISR) service

Currently we are focusing on the service Fault Isolation and System Restoration performed by SCADA operator by means of SCADA control centre of the MV power distribution network
Outages in MV power distribution network, need to be automatically detected, isolated and the network has to be restored to power its end users again.

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (23)

Risk of loss/degradation of FISR service

The quality of FISR service affects the quality of power supply in terms of SAIDI SAIFI CAIFI The degradation/loss of FISR service performed by SCADA operator, is critical because it is strictly correlated to the quality of power supplied to customers. A timely actuation of FISR service, consequential to a permanent failure of the grid, reduces the outage duration

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (24)

FISR models and tools(tools) [online/offline]

Reliability of Interconnected networks FISR dependability (WNRA reliability analyzer) [online]

FISR performance and rerouting (NS2 simulator) [offline] FISR worst case measures in presence of hacker attacks (MILP algorithm) [online]

Bayesian Belief Networks (GENIE) [online] Holistic Reductionistic models (CISIA extension) [online] Deterministic and Agent Based simulation (RAO) [online] Raw data models of operational status (algorithm) [online]
Luxemburg, 20 May 2010 ICT-SEC 225353 MICIE (25)

WP2000 Interconnected networks supporting FISR

MV power grid
N.O.

N.O. N.O.

Protection breaker

Tie switch

N.O.

Tie switch

Load Electrical junction Remotly controlled switch MV substation N.O. = Normally Open

Protection breaker
Luxemburg, 20 May 2010 ICT-SEC 225353 MICIE (26)

WP2000 Interconnected networks supporting FISR

SCADA system
SCADA implements FISR on Power Grid by monitoring/ controlling/ reconfiguring the grid (measures/ switches/RTUs)

RTU SCADA Control Centre Ethernet Bus

Gateway
FIU MOSCAD

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (27)

WP2000 Interconnected networks supporting FISR

Telco network
Hierarchical structure

Backbone (Point of Presence) Transit Exchange (TeX) Local Access (LeX)

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (28)

WP2000 Interconnected networks supporting FISR

Power grid,SCADA system, Telco network

INTERCONNECTIONS

SCADA and Telco

Telco and HV grid RTUs, SCADA and Telco devices energised by Power grid by means of emergency power supply systems see D2.2.1
Luxemburg, 20 May 2010 ICT-SEC 225353 MICIE (29)

Indicators of risk of loss/degradation of FISR

Performances of FISR (NS2 models)
Dynamical path between SCADA control centre and RTUs Throughput of nodes of Telco network Round Trip time between SCADA control centre and RTUs FISR response time: the time between the occurrence of loss of power supply to customers (due to a grid failure) and the restoration of power supply to customers. outage duration % of affected customers

Dependability of FISR
Connectivity between SCADA control centre and RTUs: minpaths and mincuts (WNRA models) Reliability and availability between SCADA control centre and RTUs (WNRA models) Probability of loss of a service on occurrence of specific events (BBN models) Reliability indices of power grid: SAIDI, SAIFI, CAIDI (RAO simulator)

FISR operativity level (CISIA)

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (30)

MICIE: Models for the online risk prediction tool

at the state of the art, no single technique has the modelling and the analytical power to cope with a meaningful and quantitative evaluation of degradation/loss of services performed by SCADA or NMS operators at regional/national level. the aim of risk prediction tool should be a meaningful, on line and possibly quantitative evaluation of the risk of degradation/loss of services performed by SCADA or NMS operators As a consequence, a successful development of the risk prediction tool should carefully evaluate all the formalisms and models and QoS indicators investigated and computed within WP2000 and should integrate the most adequate ones, according to the requirements of the on line risk prediction tool

Luxemburg, 20 May 2010

ICT-SEC 225353 MICIE (31)