Improving Metrics in Cyber Resiliency

Improving Metrics
in Cyber Resiliency
(With ETIF/ETIT)
Improving Metrics in Cyber Resiliency (ETIF/ETIT) © Copyright 2017, Cloud Security Alliance. All rights reserved 1
© 2017 Cloud Security Alliance – All Rights Reserved
All rights reserved. You may download, store, display on your computer, view, print, and link to
the Improving Metrics in Cyber Resiliency (ETIF/ETIT) white paper at https://cloudsecurityalliance.
org/download/improving-metrics-in-cyber-reciliency subject to the following: (a) the Report may
be used solely for your personal, informational, non-commercial use; (b) the Report may not be
modified or altered in any way;(c) the Report may not be redistributed; and (d) the trademark,
copyright or other notices may not be removed. You may quote portions of the Report as
permitted by the Fair Use provisions of the United States Copyright Act, provided that you
attribute the portions to the Improving Metrics in Cyber Resiliency (ETIF/ETIT) white paper.
Acknowledgements
Lead Authors
Dr. Senthil Arul
Dr. Shimon Modi
Contributors
Josep Bardallo
Ramon Codina
Bernd Jaeger
Courtney Keogh
Paul Lanois
Daniel Miller
Michael Roza
Duncan Sparrell
John Yeoh
Stephen Lumpe (design)
Table of Contents
Improving Cyber Resiliency by Defining, Measuring and Reducing Elapsed Time to Identify
Failure (ETIF) and Elapsed Time to Identify Threat (ETIT)
Abstract
Introduction
Modified Cyber Resiliency Model
Conclusions and future work
References
Improving Cyber Resiliency by Defining,
Measuring and Reducing Elapsed Time to
Identify Failure (ETIF) and Elapsed Time
to Identify Threat (ETIT)
Abstract
Cyber resiliency is important as it gives us “the ability to prepare and plan for, absorb, recover
from, or more successfully adapt to actual or potential adverse effects.” Despite billions of
dollars being spent on cybersecurity, information systems data breaches are increasing year
after year. To reverse this trend, it is essential to develop metrics and processes to measure (1)
threats before they become cyberattacks, (2) recovery of lost functionality after a cyberattack.
This paper introduces two essential metrics: Elapsed Time to Identify Failure (ETIF) and Elapsed
Time to Identify Threat (ETIT). Measuring them and developing processes to lower the values of
ETIF and ETIT would improve the resiliency of an information system. The paper also discusses
challenges associated with measuring ETIF and ETIT and proposes that the measurement and
reporting of ETIF and ETIT could be transferred from companies whose systems encounter
cyberattacks to companies who are in the Intrusion Detection System (IDS) space. Transferring
the responsibility for measuring and detecting cyber intrusions to the IDS community would
bring superior algorithms that are needed to detect anomalies and improve cyber resiliency.
Introduction
Preventing cyberattacks is an important objective. However, achieving such a lofty objective
is a challenge. Case in point, cyberattacks, data breaches, and spending on cybersecurity are
increasing year over year. According to ITRC survey study1, a 27.5% increase in cyberattacks
occurred in 2014 compared to 2013. These cyberattacks resulted in the exposure of over
80 million components of personally identifiable information (PII) including social security
numbers, driver’s license numbers, medical records, etc.1. In 2014, McAfee2 estimated that
the annual cost of cybercrime could be as high as $575 billion globally. Target Corporation’s
(NYSE:TGT) data breach in 2014 resulted in $1 billion in losses and the company incurred $88
million in breach related expenses3. According to Ponemon Institute4, the average cost of a
data breach for an American company is $5.4 million and the average per capita cost is $188.
In the USA, spending on cybersecurity reached more than $70 billion in 20145. The Yahoo data
breach, which is considered to be the largest discovered in history with over 500 million users
affected, has reduced the acquisition price by $350 million.6
The flexibility, scalability, availability, lower cost, etc., are driving the corporate information
infrastructure to migrate to the cloud from local servers and the operational assets information
such as financial, human resources, production, logistics, etc., are being stored remotely. The
cloud services could introduce new cyberattack vectors due to (1) lack of direct control of
cloud service providers on the shared infrastructure and dependency on third party developed
capabilities7, (2) ease in procuring and accessing cloud services allows nefarious users to hack
into data of other users in a multi-tenant cloud architecture8, and (3) Man In Cloud Attack –
involves the theft of user tokens which cloud platforms use to verify individual devices without
requiring logins during each update and sync9., etc. These new attack vectors would not only
compromise the information assets but also result in a detrimental impact to the operations
of the whole enterprise. Therefore, the discussions presented in this paper on improving
information system resiliency could be extended to the operational systems’ resiliency as well.
The cyber ecosystem is non-discriminant in the sense that good and harmful information
coexists harmoniously. The internet and transmission protocols by which the information
travels within the cyber ecosystem is also non-discriminant in the sense that good and
harmful information is transmitted across
cyberspace without discrimination as to IT/OT Infrastructure
priority or hierarchy. Thus, the opportunity
Information Information Information
to discriminate between good and harmful System System System
Application 1 Application 2 Application 3
information would not occur in the cyber
ecosystem. Discrimination must occur within Layered Security Layered Security Layered Security
one’s own Information Technology/Operational

Technology (IT/OT) infrastructure, and must
Layered security such as Intrusion Detection,
occur in the security infrastructure layers. Authentication, and Malware Detection.
As shown in Figure 1, the first opportunity

to discriminate between good and harmful
information occurs at the security layers
between the cyber ecosystem and IT/OT
infrastructure. The second opportunity occurs Cyber Ecosystem
at the security infrastructure layers part of the
information system.
If a cyberattack is identified and captured at

Figure 1
the security layers in the IT/OT infrastructure, Depiction of various layered security in
the IT/OT infrastructure
then the information system is protected.
However, if a cyberattack is identified by the security layers residing in the information system
(i.e., harmful information already past the IT/OT infrastructure), then the information system
is compromised. The loss of function resulting from a cyberattack depends on the kind of
function that the attack targets (logistics, tactics, etc.) and on the type of information that is
available and exposed (trade secrets, personally identifiable information, etc.). For example, a
2014 cyberattack on Sony Pictures (NYSE:SNE)6 disabled computers, leaked upcoming movies,
and completely paralyzed the operation for a short period of time whereas a cyberattack7 on
the Pentagon’s Joint Forces e-mail system resulted in the shutdown of the e-mail system for a
short period of time but only locally, without affecting rest of the Pentagon’s e-mail systems.
The National Academy of Sciences8 (NAS) defines resiliency as “the ability to prepare and
plan for, absorb, recover from, or more successfully adapt to actual or potential adverse
effects”. Bruneau et al.9 developed a framework to quantitatively assess the resiliency of a
community after an earthquake by measuring Quality of Infrastructure, Q(t), over a period of
time. The value of Q(t) ranges from 0% to 100% where 0% means no service available and
100% means no degradation
in service. At time t0, an
earthquake event occurred 100
and dropped Q(t) from 100%
Quality of Service, Q(t)
to 50% instantaneously. It took
Q’(t)
time t1 to fully recover, Figure
2. This approach is based
on the notion that Quality
of Infrastructure, Q(t), of a
community is affected after 0.0
an earthquake and it takes t0 Time, t t1
a certain amount of time to Figure 2

fully recover from it. Zobel et Resiliency of community after an earthquake (adopted from Bruneau et al.)
al.10 applied the resiliency of

community after an earthquake framework to quantify cyber resiliency after a cyberattack.
They analyzed many types of cyberattacks: (1) slow-onset single event, (2) sudden-onset single
event (3) slow-onset multi event, (4) sudden-onset multi event. An information system performs
many functions; they were all combined into a single function defined as Quality of Service, Q(t). The
functional value for Q(t) ranged between 0 and 1; value of zero (0) represents unable to perform
the intended function and value of one (1) represents fully performing the intended functions.
This paper expands the cyber resiliency model presented by Zobel et al. 10. It introduces two
new variables, namely Elapsed Time to Identify Failure (ETIF) and Elapsed Time to Identify
Threat (ETIT) and qualitatively presents a compelling case that the measurement and
publication of ETIF and ETIT would spur innovation in the IDS space, and aid in the overall
improvement of the resiliency of information systems.
Modified Cyber Resiliency Model
Figure 3a graphically represents the Zobel et al.10 characterization of a resiliency profile
for a slow-onset single-event cyberattack. In this case, the loss of quality of service occurs
gradually over time, a virus gradually propagating across the system unnoticed. The start of the
cyberattack in the timescale is represented as t0start, the end of the cyberattack is represented
as t0end, and the corresponding of loss of functionality is Q’(t). The time to recover the full
functionality from t0start is T. The triangular area with the base as T and the height as Q’(t)
represent the loss of resiliency. The smaller the area under the resiliency curve the system
is more resilient. That is, the cyberattack did not impact the functional performance, or the
system recovered quickly and/or a combination of both.
1.0 1.0
Loss of Reliliency n
ctio
Lo
un
ss
of F
of
ry
ove
Fu
Rec
nc
tio
n
Q’(t) Q’(t)
ETIF
0.0 0.0
t0start t0end ts ti te
Time, t Time, t
Figure 3a Figure 3b
Figure 3a represents the original cyber resiliency model for slow-onset single-event (adopted from Zobel
et al. slow-onset multi-event resiliency profile) and Figure 3b represents modified cyber resiliency model.
Please note - ETIF and Recovery time may vary
Figure 3b represents the modified cyber resiliency model. In a typical cyberattack, the failure
is identified by studying the anomaly/degradation of intended functions at the IDS layer.
Therefore, the modified resiliency model starts with the time at which the failure was identified
ti (in Zobel et al. model is t0end) and the corresponding loss of function is Q’(t). Immediately
after the identification of a failure, an investigation would start to identify the root causes of
failure and to implement recovery actions to restore full functionality. The investigation would
identify the time at which the cyberattack started, ts (in Zobel et al.’s model is t0start). The line
connecting ti to ts represent the loss of function. The duration between the start of the failure
and the identification of the failure is defined as the Elapsed Time to Identify Failure (ETIF). The
recovery actions would fix the failure, necessary to achieve full functionality. The time at which
full functionality is achieved is represented as te. The line connecting ti and te represents the
recovery function. Therefore, the modified resiliency model includes ETIF and the slopes of loss
and recovery of functions, Figure 3b.
Cyberattacks occur when the information system vulnerabilities are exposed and exploited.
For a given information system, these vulnerabilities are embedded into the architecture of the
software and the hardware and characterize the attack surface15.
As noted earlier, resilience of a system is characterized by its ability to achieve its intended
functions despite disruptions. From that perspective, the bounding function on how a system
is attacked and recovers is dependent on the attack surface. Thus, the functions representing
resiliency, in this case ETIF, are bounded by the attack surface. These functions can be linear or
non-linear; for illustrative purposes, the graphical information presented in this paper show linear
relationships. However, discussions are certainly valid for both linear and non-linear functions.
A E C G
100
D
Q”(t)
ETIF”
ETIF’ B
Q’(t)
ETIF’” F
Q*(t)
0.0
ts ti” ti’ te” t e’ te’”
Time, t
Figure 4
Illustrating the role of ETIF in cyber resiliency
ETIF measures the loss of resiliency of an information system. If ETIF moves from ETIF’ to ETIF’’,
Figure 4, then the loss of resiliency area would reduce from triangle ∆ABC to triangle ∆ADE,
the loss of quality of service, Q(t), would reduce from Q’(t) to Q’’(t), and the recovery time (te)
would reduce from te’ to te’’. Extending this argument, the loss of resiliency would reduce to
zero when ETIF approaches zero. That is, the start of the cyberattack (t0) and the identification
of the cyberattack (ti) occurred at the same time (i.e., the cyberattack was nullified at the IT/OT
infrastructure.)
If ETIF moves from ETIF’ to ETIF’’’, Figure 4, then the loss of resiliency area would increase from
∆ABC to triangle ∆AFG, the loss of quality of service, Q(t), would increase from Q’(t) to Q’’’(t), and
the recovery time (te) would increase from te’ to te’’’. However, if ETIF continue to increase, at
some point the loss of resiliency reaches a level beyond which the system is not recoverable.
This condition would be reached either when the quality of service, (Q(t), reached a point of
no recovery, Q*(t), or when the cost of recovery is more than that to build a new information
system, Figure 4. An example of this condition would be a security breach at a power
generation company causing an electric generator to have a catastrophic failure resulting in a
total loss of functionality. In this case recovery may not be possible.
Discussion
Table 1 shows some of the recent cyberattacks and the ETIF for each of the attacks. The ETIF
ranged from as low as three (3) hours (University of Maryland) to as long as one (1) year (Sony
Pictures & the US Government’s Office of Personnel Management). The actual cyberattack date
and time, as mentioned before, is determined by forensic analysis. The forensic analysis process
is not standard across all industries and as such the data in the public domain is not easily
comparable. As of today, there is no agreed definition of what activities would characterize the
start of a cyberattack. For example, in some cases, certain viruses are dormant in the system
before becoming malignant and in other cases there are multiple attacks over a period of time.
Corporations are concerned about confidentiality, competitive pressures, litigation, image, etc.,
and are therefore reluctant to disclose the specific details of a cyberattack including the date of
the attack and the date of the identification. Given the lack of standardization in publicly available
data on start of a cyberattack, ts, and on the identification of a cyberattack ti, it is currently difficult
to consistently and specifically calculate ETIF and hence cyber resiliency.
Table 1
Calculated ETIF based on recent cyberattacks
Company/ Cyberattack Elapsed Time to

No Cyberattack Date
Organization Identified Date Identify Failure (ETIF)
1 Premera Blue Cross11 May-14 29-Jan-15 > 7 months
2 Anthem Blue Cross12 Dec-14 29-Jan-15 > 1 month
3 Sony Pictures6 as early as Nov 2013 24-Nov-14 1 year
4 Staples13 as early as July 20, 2014 20-Oct-14 > 3 months
5 Home Depot14 as early as April 2014 2-Sep-14 > 4 month
6 JP Morgan15 as early as mid-June 2014 Mid-August 2014 > 2 months
7 Community Health16 Apr-14 1-Jul-14 3 months
8 Toys R Us17 28-Jan-15 30-Jan-15 3 days
9 University of Maryland18 17-Feb-15 17-Feb-15 3 hours
10 Office of Personnel May-14 1-May-15 1 Year

Management (OPM)19
If skilled companies in the IDS calculate and report ETIF instead of the corporations
experiencing the cyberattack , it would enable standardization of the forensic process and
development of the tools necessary to clearly define, measure and calculate the start of a
cyberattack. By reporting ETIF instead of the start of cyberattack, ts, and the identification
of cyberattack, ti, corporations may feel less exposed to litigation and negative publicity.
Publishing ETIF and reducing the value of ETIF could spur competition in IDS space to
develop advanced algorithms to identify anomalies in the quality of service data. However,
for ETIF to be meaningful and to be an effective metric for improving cyber resiliency, clear
definitions for ti and te, and agreed upon processes and tools need to be established to
measure ti and te.
The discussion until now has been focused on the loss of resiliency of an information system.
However, an ideal state would be to have no loss of resiliency. That is, the threat of a failure is
identified early enough before it becomes an attack. Such an outcome is possible if there was
a mechanism to disseminate information about the adverse events and the threats among
various entities on a real-time basis. Ideally, if an entity that experienced failure could share the
system’s vulnerability with other entities, analysis of such shared failure event would be critical
in detecting the presence of a threat or recover quickly from the failure. We define this metric
as Elapsed Time to Identify Threat (ETIT). The effectiveness of such a metric can be measured
by its ability to move ti (time to identify failure) closer to ts (start time of the failure). ETIT is
critical in changing limits on the loss and recovery functions and thus impacting the quality of
service. If there is an ability for early identification of the threat that is causing the failure, then
the overall time to recovery and hence the loss of resiliency could be reduced.
It is conceivable that different information systems could have the same loss of resiliency
(i.e., same area of the triangle) but with different slopes for loss and recovery functions. This
is illustrated in Figure 5 where System 1 and System 2 have the same loss of resiliency but
System 1 has much steeper loss and recovery functions and it fully recovers to perform
its full intended function much sooner than System 2. From a user/customer perspective,
System 1 would be preferred in the sense that it recovered to perform its fully intended
function in a much shorter duration than System 2. Therefore, in addition to the loss of
resiliency, the slope of the loss and recovery functions would become important. The steeper
the slope for the loss and recovery functions and shorter the duration between ts and ti (i.e.,
ETIF), the smaller the loss of resiliency.
100
System 2
System 1
0.0
Time, t
Figure 5
Comparing two different information systems with the same loss of resiliency with
different Elapsed Time to Identify Failure (ETIF), loss and recovery functions
Conclusions and Future Work
A modified information system resiliency model was presented and introduced two new
variables: Elapsed Time to Identify Failure (ETIF) and Elapsed Time to Identify Threat (ETIT). The
model qualitatively demonstrated that the lower the ETIF, the smaller the loss of resiliency for
an information system. Challenges in calculating ETIF in recent cyber attacks showed difficulty
in measuring the start of the cyberattack (ts) and the identification of cyberattack (ti).
By transferring the responsibility of calculation and publication of ETIF from the companies that
experience cyberattacks to the companies that are in IDS space could bring standardization
and continuous improvement in measuring and reporting ETIF. In addition, it could encourage
competition in the IDS space and would bring innovative and superior algorithms to find
anomalies in the data quality thus improving cyber resiliency.
With the growth in cloud computing, information about the operational assets is being stored
away from the local servers. Although the benefits of cloud computing are obvious, such
decoupling could result in poor operational resiliency if the information asset is compromised.
Therefore, to keep the operational resiliency unaffected, it is essential to bolster information
asset resiliency in the cloud. Hence, a technical framework, along with appropriate regulatory
framework, needs to be created to enable the measurement and reporting of ETIF and ETIT.
The Cybersecurity Act of 201516 passed by the US Congress, aims at setting a regulatory
framework for inter-government and private company to government exchanges. NIST has
released a special publication outlining the building blocks of an incident exchange program.17
Future work would entail architecting a framework that allows a set of cooperative systems to
aggregate, summarize, and utilize ETIF and ETIT to improve resiliency.
References
1. http://www.idtheftcenter.org/ITRC-Surveys-Studies/2014databreaches.html, 2014
2. “Net losses: Estimating the global cost of cybercrime Economic impact of cybercrime II”,
Center for strategic and international studies, June 2014
3. Perlroth, N., Harris, E. A., “Cyberattack insurance a challenge for business”, http://www.
nytimes.com/2014/06/09/business/cyberattack-insurance-a-challenge-for-business.html?_
r=0 , June 2014
4. “2013 cost of data breach study: global analysis”, Research sponsored by Symantec and
independently conducted by Ponemon Institute LLC, May 2013
5. http://www.gartner.com/newsroom/id/2828722, August, 2014
6. http://money.cnn.com/2017/02/21/technology/yahoo-verizon-deal/
7. https://blogs.microsoft.com/microsoftsecure/2013/01/17/cloud-services-building-
resiliency-and-business-continuity/
8. http://www.levelcloud.net/why-levelcloud/cloud-education-center/advantages-and-
disadvantages-of-cloud-computing/
9. https://www.incapsula.com/blog/top-10-cloud-security-concerns.html
10. https://en.wikipedia.org/wiki/Sony_Pictures_Entertainment_hack
11. http://www.cnn.com/2015/08/05/politics/joint-staff-email-hack-vulnerability/
12. “Disaster resiliency: A national imperative”, The National Academies Press, 2012
13. Bruneau M., et al., “A framework to quantitatively assess and enhance the seismic resiliency
of communities”, Earthquake Spectra, 19(4), 733-752, 2003
14. Zobel C.W., et al. “Quantifying cyberinfrastructure resilience against multi-event attacks”,
Decision Sciences, Volume 43, No. 4, 687-709, August, 2012
15. Mandhata, P., Wing, J., “Measuring a System’s Attack Surface”, CMU-DS-04-102, January
2014
16. https://www.congress.gov/bill/114th-congress/senate-bill/754
17. Johnson, C, et al., “Guide to cyber threat information sharing”, NIST special publication (800-
150), April, 2016.
18. http://www.npr.org/sections/alltechconsidered/2015/03/18/393868160/premera-blue-
cross-cyberattack-exposed-millions-of-customer-records
19. https://www.anthemfacts.com/
20. http://www.technobuffalo.com/2014/12/22/staples-1-16-million-credit-cards-affected-in-
cyber-attacks/
21. http://www.huffingtonpost.com/2014/09/18/home-depot-hack_n_5845378.html
22. http://www.wsj.com/articles/j-p-morgan-says-about-76-million-households-affected-by-
cyber-breach-1412283372
23. http://www.forbes.com/sites/danmunro/2014/08/18/cyber-attack-nets-4-5-million-records-
from-large-hospital-system/
24. http://www.scmagazine.com/attacks-attempt-to-access-rewardsrus-accounts/
article/401160/
25. http://www.diamondbackonline.com/news/umd-social-security-numbers-compromised/
article_b8236dea-99b6-11e3-92eb-0017a43b2370.html
26. http://www.politico.com/story/2015/07/federal-government-cyber-attack-breach-21-million-
people-affect-119918

Improving Metrics in Cyber Resiliency

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Improving Metrics in Cyber Resiliency

Diunggah oleh

Hak Cipta:

Format Tersedia

Improving Metrics

one’s own Information Technology/Operational

As shown in Figure 1, the first opportunity

If a cyberattack is identified and captured at

a certain amount of time to Figure 2

al.10 applied the resiliency of

Quality of Service, Q(t)

Company/ Cyberattack Elapsed Time to

1 Premera Blue Cross11 May-14 29-Jan-15 > 7 months

2 Anthem Blue Cross12 Dec-14 29-Jan-15 > 1 month

3 Sony Pictures6 as early as Nov 2013 24-Nov-14 1 year

4 Staples13 as early as July 20, 2014 20-Oct-14 > 3 months

5 Home Depot14 as early as April 2014 2-Sep-14 > 4 month

6 JP Morgan15 as early as mid-June 2014 Mid-August 2014 > 2 months

7 Community Health16 Apr-14 1-Jul-14 3 months

8 Toys R Us17 28-Jan-15 30-Jan-15 3 days

9 University of Maryland18 17-Feb-15 17-Feb-15 3 hours

10 Office of Personnel May-14 1-May-15 1 Year

Anda mungkin juga menyukai