An Approach to Carry Out Consistency Analysis on

Requirements
Validating and Tracking Requirements through a Configuration Structure
Padmalata Nistala

Priyanka Kumari

Business Systems and Cybernetics Center

Tata Consultancy Services
Hyderabad, India
Nistala.padma@tcs.com

Business Systems and Cybernetics Center

Tata Consultancy Services
Hyderabad, India
Priyanka.kumari@tcs.com

AbstractRequirements management and traceability have

always been one of grand challenges in software development
area. Studies reveal that 30- 40% of software defects can be
traced to gaps or errors in requirements Although several models
and techniques have been defined to optimize the requirements
process, ensuring alignment and consistency of elicited
requirements continues to be a challenge. All software
engineering standards and methodologies recognize the
importance of maintaining relationships among the software
elements for traceability. We have leveraged the structured
relationships among the requirement elements to come up with
an approach to systematically carry out consistency analysis of
requirements for software systems.
The framework has multiple models: a multi layered
requirement model, a configuration structure to link and track
the requirement items, a consistency analysis method to identify
the inconsistencies in the requirements and a consistency index
computation to indicate the level of consistency in overall
requirements of the software system. This approach helps to
validate the requirements from both completeness and
correctness perspectives and also check their consistency in
forward and backward directions. The paper outlines the
framework, describes the encompassing models and the
implementation details from pilot of the framework to an
industry case study along with results.
Index
TermsRequirement
alignment,
requirement
configuration structure, requirement consistency analysis,
requirement inconsistency matrix, requirement consistency index

I. INTRODUCTION
According to ISO/IEC 15288 systems definition [1], A
system is a combination of interacting elements organized to
achieve one or more stated purposes. As the software system
transforms from a set of needs to a final product through
various forms, at each stage there is a high possibility of loss of
quality due to defects in transformation of configuration items
from previous phase. Thus, an understanding of system model
[1] is important not only to understand what is being
developed but also to understand how it is being developed.
Though a lot of traceability tools and research is trying to
overcome the problem in software industry, managing the
requirements to perform traceability continues to be a primary
issue. Kannenberg and Saiedian in their article [2] reflects the
problem of requirement source traceability without which it is
difficult to determine the incorporation of requirement into

c 2013 IEEE
978-1-4673-5765-4/13/$31.00

system and thus unable to trace whether system [3] is being

developed according to the requirement or not.
J. Dick [4] suggested solution for the stated problem by
maintaining requirements linking from one level to another
which helps in impact analysis and derivation of a need.
Management and traceability of the requirements should help
in achieving the product to meets the stated requirements. The
concept of traceability [5] is not only limited to tracing the
requirement but also on the alignment of customers goals and
requirements to system/software specification which will help
in achieving the stated objective. Tomoyuki [7] has proposed
traceability concept to maintain and demonstrate the
relationships between requirements and software artifacts such
as design models, source code, and test cases.
Standards like SWEBOK [7], ISO/IEC [1], PDTR 30103[8]
have defined the importance of relationships between different
configuration items but they have not defined any
layout/framework on achieving traceability or alignment
through structured relationship between configuration items.
Few book on requirements engineering [3] focuses on how
high level requirements getting transformed into low-level
requirements depending upon the relationships between layers
of information. The established Demings principle of process
step and product quality [9] also supports correlation between
process step and product quality. These established concepts
requires to creating a relational structure to check the quality at
each process step. Though these literatures emphasize on some
kind of relationship but they do not exactly focus defining the
structure and related necessary granularity while forming the
structure. They lack a robust mechanism for creating a
structured relationship between the artifacts.
In other disciplines such as manufacturing and systems
engineering we can find models, well established principles
and practices for formulating and tracing the product parts and
composition. The modern architecture strongly believes in
strong relationship between form or structure [10, 11] of the
product to the purpose or function. Similarly, the concept of
Product Breakdown Structure (PBS) [12, 13, 14] in
manufacturing industry is the breakdown of a product into its
required components. This helps in providing a clear picture of
the product and helps in handling the complex parts with easy

320

RE 2013, Rio de Janeiro, Brasil

Industry Track

visuals prints of the components. The relationships and

traceability of product parts is visually transparent.
Various traceability tools presently used in software
industry [3,15] maintain document level linkages focusing on
completeness. Few other tools [15, 16] support some kind of
relation but do not have the adequate granularity and holistic
coverage of the relations.
The approach of the proposed framework is to perform
consistency analysis to correctly and completely track the flow
of requirements with various layers. Here we propose to
leverage the relationships among the requirements
configuration items and propose a model for requirements
alignment and traceability. The research problems taken are
To define guidelines for identification of requirement
elements or configuration items for software systems
To define a framework for requirements tracking through a
requirement configuration structure using the relationships
among requirement configuration items
To conduct consistency analysis on the overall system in
terms of completeness and correctness and identify the
gaps at each phase
To apply this framework for an industry case of software
system and illustrate the applicability of the framework
and the no of inconsistencies in requirements found
through this model.
In the subsequent sections, we describe the proposed
framework - the requirement configuration structure, its
application for requirements traceability of a software
application in a vertical business unit, the result achieved,
followed by the conclusion.
II. FRAMEWORK
Here in this framework, the complete value chain of
software requirements process is considered in the form of key
configuration layers. The complete view of framework with its
encompassing layers, configuration items and their structural
relationships is depicted in Fig 1.
The key components of the framework are:

Figure 1.

a multi layered requirement model to ensure the

alignment of requirements right from the business
goals to the software specifications. It comprises of
four key players: business layer, process layer,
requirements layer and specification layer.
a configuration structure to link and track the
requirement items for each layer. This provides a
visual representation of the requirements flow across
various layers.
a consistency analysis method to identify the
inconsistencies in the requirements structure. A
configuration inconsistency matrix is populated with
missing linkages of requirements in both forward and
backward directions.
a consistency index computation to indicate the level of
consistency in overall requirements of the software
system.

A. Layers of the Model

In order to realize the value from the software systems, the
context and business perspective has to be integrated to the
software requirements [17]. Also, to address the leakage issues/
gaps at requirement phase, we initiated the requirement
configuration with the business layer. The key configuration
layers in the framework are FOUR.
1) Business Layer: Here the broad business purpose of the
software system as outlined by the customer is captured in the
form of business goals and objectives. This layer brings the big
picture of the business - the purpose of the software system and
is typically stated by the customer in the RFP (request for
proposal).
2) Process Layer: The key business processes to realize the
goals and objectives are also identified here. This is a crucial
layer and it is required to identify the end to end chain of
business processes to realize the business goals and objectives.
3) Requirements Layer: This layer identifies the key
business requirements of the system and is typically the starting
point for many software systems.

Requirement Configuration Structure Framework

321

4) Requirement Specification Layer: This layer describes

the analyzed requirements in the form of specifications. The
requirement specification could be in any notation depending
on the life cycle process e.g. SRS for structured model; Use
cases for OO model and User stories for agile model.
Each layer will have a set of defined configuration items. As
a guideline, we have listed the key requirement configuration
items widely followed in software development.
Business Layer: Business Goals and/or Objectives.
Process Layer: Business Process (Id + Name) and Sub
Process.
Requirement
Layer:
Business
Requirements
(Requirements group + Requirements Id + Requirements
Description).
Specification Layer: Use Case (Id + Name).
Depending on the context of the client, the life cycle model
and its architecture, the individual configuration items can be
tailored within each layer.
B. Configuration Structure
This composition of requirement configuration items for
each of these layers along with their relationships helps in
creating the requirement configuration structure (RCS). It
provides guidance in identifying and linking configuration/
components item at the business, process and requirement
layers. Fig-1 depicts the layout of the model Requirement
Configuration Structure.
The configuration items at one layer are linked to
corresponding items at another layer to create requirement
structure of the software product/application under
development. The framework suggests compartmentalizing
requirements to the processes it serves and similarly to all other
elements of the configuration item.
C. Consistency Analysis Method
The consistency or track of each relationship from the
above created structure is validated on two dimensions. The
relationship consistency analysis involves:
Completeness analysis Completeness property is the
degree to which the set of functions covers all the specified
tasks and user objectives [18]. Completeness analysis of
RCS involves checking whether for each configuration
item in a layer, are there corresponding configuration
item(s) in forward and backward directions and overall
whether the coverage of the configuration items across the
structure is complete. This analysis helps to establish the
end to end track of requirements in RCS.
Correctness analysis Correctness property is the degree
to which a system or component is free from faults in its
specification, design, and implementation [19].
Correctness analysis of RCS involves checking for faults
or errors in transformation between any two configuration
items in the structure. It validates for each relationship of
RCS, do the next configuration item correctly achieves the
objectives of the previous configuration item or not. This
analysis is to validate the correctness of requirement items
across layers in RCS.

ISO/IEC/IEEE 24765 defines consistency as the degree

of uniformity, standardization, and freedom from
contradiction among the documents or parts of a system or
component. In our framework, we extend this definition
and propose a requirement to be consistent when both
completeness and correctness properties across its path in
the structure are true and maintained. The requirement is
inconsistent if one or more gaps found in RCS either due
to completeness or correctness issues.

The inconsistencies found in analysis are populated into a

configuration inconsistency matrix (CIM). The template of
CIM is given in Table-1. X and Y denote the configuration
items among which there is an inconsistency in relationship;
the direction is forward/backward; inconsistency type is
Missing for a completeness gap or Erroneous for a
correctness gap; analysis describes the gap and count is
summation of the inconsistent configuration items.
TABLE 1. Configuration Inconsistency Matrix
Process

Process for which the related inconsistency exist

Config Item X

X is a configuration item in the requirement

configuration structure

Config Item Y

Y is a configuration item in the requirement

configuration structure

Direction

Relationship: Forward/ Backward

Inconsistency Type

Type of relationship inconsistency:

Missing/ Erroneous

Analysis

Analysis description

Count

No of inconsistencies in the relationship

CIM provides a tool to consolidate all the inconsistencies in

the structure defined above and analyse the data. The numbers
of inconsistencies as found from the CIM are plotted against
the business process as inconsistency graphs. This
information readily gives indication on the completeness and
correctness of requirements realization and gaps in
transformation at each layer. These graphs visually depict the
inconsistency in quality for the requirements.
D. Requirement Consistency Index
Requirement Consistency Index (RCI) quantifies the
percentage consistency of the requirements. In the consistency
analysis, a requirement is considered consistent when both
completeness and correctness across its path in the structure are
maintained and inconsistent if one or more gaps found in
analysis. As an initial measurement, RCI considers each
consistency issue with equally weightage resulting in an
average value for the metric.
Requirement Consistency Index = Percentage consistency
of the requirements.
RCI =A/(B+C)
where,
A =Number of Consistent Requirements.
This is computed as Xi where i denote a requirement
varying from 1 to n; Xi is consistency value of the

322

Figure 2. Requirement configuration structure for the project

requirements.
B = Total number of elicited requirements
C = Number of missing requirements identified
III. CASE STUDY
This framework has been piloted for a critical business
system for a large insurance client of TCS. The framework was
implemented to carry out the due diligence exercise for
requirements, find out the inconsistencies and validate the
alignment and traceability of elicited requirements.
The actual implementation of this requirement consistency
analysis was carried by two researchers who were also
involved in designing the framework with the knowledge help
of business analyst. The implementation period was two
months which resulted in the formation of requirement
configuration structure and the consistency index which were
also validated by the SMEs.
The pilot application was an eRegistration System which
provides different methods of automatically registering external
customers and maintaining and accessing their accounts. The
existing registration process is enhanced with implementation
of security framework for few processes. The new application
also integrates the Management Information (MI) report
creation. The following section details the framework
deployment for the case.
A. Layers of the Framework
The first step is identifying the layers and configuration
items. The framework implementation for the registration
system is illustrated in Fig 2 and described below for each layer.

1) Business Layer: The key configuration items in this

layer are business objectives. The inputs for this layer have
primarily been taken from the delivery proposal report, process
diagrams/flow charts. Customer has identified three high level
business objectives: customer should be automatically
registered though automatic registration and enhancing the
security framework for the existing registration service and to
enhance the interface of registration service with other
connected application.
2) Process Layer: The above objectives were transformed
into seven end to end business processes (P1 to P7). Automatic
registration while customer is buying a product; registering the
employee of the organization where the insurance client have
enrolled their insurance products; registering customer through
internal staffs; enhancing registration process with the security
framework and password reset functionality and providing
management information are few of the identified business
processes. Sub-process: Sub-processes are identified by the
decomposition of the main processes.
3) Requirements Layer: Requirements layer has been
developed with the business system analyst group. The above
business processes (P1-P7) were linked to application
requirements taken from requirements catalogue which is a
client provided artifact. The requirement is defined with
specific requirement group and ID.
4) Requirement Specification Layer: In this layer, use case
documents were provided by customer. Configuration items
(Use Case) for this layer were properly linked to the
corresponding process and requirements.

323

Table 2: Configuration Inconsistency Matrix for the project

Process

P2:Bulk
Registration

C.Item X

C.Item Y

Process2-1: Bulk Registration

for product customer

Requirements

Direction

Type

Forward

Missing

Analysis

1)
2)

P2:Bulk
Registration
P2:Bulk
Registration

Process2-1: Bulk Registration

for product customer
Requirements:9

Requirements

Forward

Missing

1)

Use Case

Forward

Missing

1)

P3:Consumer
Self Elected
Registration
P5:Login and
manage account

Requirements

Use Case: Self

Registration

Backward

Missing

1)

Process5-1: Consumer Login/

Logout

Requirements

Forward

Erroneous

1)

B. Configuration Structure
The above figure-2, illustrates the requirement
configuration structure (RCS) created for eRegistration
System linking all the configuration items, taking an
illustration of bulk registration process to fulfill the objective
- to provide automatic registration facility to the customer.
The process is divided into two sub-processes bulk
registration for product customer and bulk registration for
workplace customer All the requirements and use cases are
correctly mapped to these sub-processes as shown in figure-2.
The first row in figure shows the missing use case when going
forward from requirements to specification layer. Thus, it
identifies the completeness issue with use case.
Similarly, the sub-process (P2-1) has a requirement on
generation of action code but the listed requirements are
insufficient and do not adequately satisfy the objectives of
activation code generation process. This is a correctness
related problem with requirements.
C. Consistency Analysis Method
Based on the RCS structure created, each relationship is
analyzed for consistency in terms of completeness and
correctness as described in the framework. The inconsistencies
found in RCS are logged into configuration inconsistency
matrix (CIM). The table-2 shows a view of the configuration
inconsistency matrix (CIM). The analysis is carried out through
tracking the flow and validating the information. The
inconsistencies listed in CIM have been validated with the
SMEs.
D. Requirement Consistency Index
Figure-3 depicts the summary of the requirement
inconsistencies as analyzed from configuration inconsistency
matrix (CIM). It also segregates the data based on correctness
and completeness related problems. It can be seen from the
graph that the number of inconsistencies originated from
requirements are very high.

Count

Requirements missing regarding security

admin tool main menu for Bulk
Registration as part of tool.
Requirements missing regarding bulk
registration file upload.
Requirements not adequately defined
regarding generation of activation code.
Use Case missing for confirmation and
generation of activation code, bulk
register file upload, output generation.
Password creation and confirnation
requirements missing for self registration

Requirements errorneous in nature

requirements states that customer has to
answer two security question whereas
customer has to asnwer three security
questions.

1
1
1

Figure 3. Requirement Inconsistency Graph

The identified seven processes have severe gaps in realizing

the objectives of the eRegistration software system.
Requirements for key processes like bulk registration and auto
registration were completely missing.
Similarly many use cases were missing and are not
specified.
Total no. of requirements for the case is 50 and the No of
consistent requirements 24.
Applying the formula, the
requirement consistency index was calculated as 48%.
TABLE 3: Requirement Inconsistency Index for the project
Requirement Consistency Index (RCI)

48%

The low value of index readily gives an indication on the

quality of the overall requirements and the leakage issues
within requirements. It validates the statistics that the
requirements are poorly managed in software industry. The
gaps in requirements are hardest to find but could be unearthed
in this framework through establishment of linkages among the
requirement elements.

324

The deployment of this framework helped in identifying

many configuration items which were missing and required for
realizing the business needs into software system.

[3]

IV. CONCLUSION

[5]

The case study results show that significant requirement

inconsistencies can be found through the requirement
configuration structure framework. Requirement consistency
analysis brings out not only the missing configuration items
from completeness analysis but also erroneous configuration
items in the RCS from the correctness analysis. Both forward
and backward traceability for requirements are covered. The
consistency track of complete requirements value chain from
business objectives, business processes and requirements until
the use cases can be maintained, validated and analyzed
through this framework. Computation of Requirement
consistency index for overall requirements provides a
quantifiable measure on the consistency of requirements and
acts as a requirement quality indicator. This method will
benefit the projects to have a structured approach towards
requirements management through the configuration structure
and can go a long way in containing the requirement defects
and improving phase containment.
Future work involves considering the weightage of
requirements into metrics computation, further variations in
requirement scenarios such as change scenarios, reverse
engineering scenarios etc. There are plans to customize the
internal tooling platform include computation of these metrics
and implement for multiple projects.

[6]

[4]

[7]

[8]

[9]

[10]
[11]

[12]

[13]
[14]

[15]

ACKNOWLEDMENT

The authors would like to acknowledge the management of

Tata Consultancy Services for providing an opportunity to
design and validate the framework. We would like to thank Mr
MGPL Narayana, Vice President, TCS for his encouragement
to write this paper, Mrs Lakshmi Suchetha for showing the
interest for the implementation of the framework.

[16]
[17]

[18]

REFERENCES
Systems Engineering Guide for ISO/IEC (System Life Cycle
Processes) 15288 , July 2002 .
[2] A. Kannenberg, H. Saiedian, Why Software Requirements
Traceability Remains a Challenge, 14 Crosstalk - Journal of
Defense
Software
Engineering,
July/August
2009,
www.stsc.hill.af.mil
[1]

[19]

325

E. Hull, K. Jackson, J. Dick, Requirements Engineering,2nd

ed., Springer, 2009, ch-9
J. Dick, Rich Traceability, Proc. 1st International Workshop
on Requirements Traceability, Edinburgh, Sept 2002
J. Cleland-Huang, O. Gotel, A. Zisman, Software and Systems
Traceability 1st ed., Springer, 2012
T. Arao, E. Goto, T. Nagata, Business Process Oriented
Requirements Engineering Process Proc. 13th IEEE
International Conference on Requirements Engineering (RE05),
IEEE computer society, Dec. 2005,
IEEE Standard Committee, Guide to the Software Engineering
Body of Knowledge (SWEBOK), ver 3,IEEE Computer
Society, 2004
ISO/IEC PDTR 30103: Software and systems engineering
Lifecycle processes Framework for product quality
achievement
William W Scherkenbach, Demings Road to continual
improvement, SPC Press Inc., Knoxville, Tennessee (615 584
5005) 1991, p26-38
Louis H. Sullivan,The Tall Office Building Artistically
Considered", Lippincott's Magazine, ver 57, March 1896
S. Dionidis, "A Tale of Four Kernels", Proc. 30th International
Conference on Software Engineering (ICSE 08). Leipzig,
Germany: Association for Computing Machinery, May 2008,
pp. 381390
D. Svensson and J. Malmqvist, Strategies for Product Structure
Management at Manufacturing Firms. Journal of Computing
and Information Science in Engineering, 2002, pp-50-58
Product
Breakdown
Structure
:
http://productbreakdownstructure.com/
Product
Breakdown
Structure
http://www.professionalpractice.asme.org/ProductMgmt/System
s/Product_Breakdown_Structure.cfm
A. Castor, R. Pinto, C. Silva, and J. Castro,Towards
Requirement Traceability in TROPOS , Centro de Informtica,
Universidade Federal de Pernambuco, Brazil , 2004
P. Zielczynski, Requirements Management Using IBM
Rational RequisitePro, IBM press , 2007
Padmalata Nistala, Supriya Kummamuru & MGPL Narayana
An Approach to Understand and Elicit Requirements Using
Systemic Models: Ensuring a Connect From Problem Context to
Requirements CSER 2013,Conference on Systems Engineering
2013, Georgia Institute of Technology, Atlanta, Mar 2013
ISO/IEC 25010:2011 Systems and software engineering-Systems and software Quality Requirements and Evaluation
(SQuaRE)--System and software quality models,
ISO/IEC/IEEE 24765:2010 Systems and software engineering-Vocabulary