METRICS

Nagaraj Yekkala
 Topics Covered

Topic #1 : Overview of Metrics

• Aim
• Metrics ?
• Metrics Classification
Topic # 2 : What affects software
Topic # 3 : Quality Model
Topic # 4: Metrics cont….
Topic # 5 : Translating Business needs
Topic # 6 : Frameworks
Topic # 7 : Process Improvement
 Aim

While metrics are important means to quantify performance of a process, we

often define metrics to comply with process standards, overlooking actual alignment
with business factors. We may put substantial effort and discipline in getting metrics
deployed, but we might not be using the ones that really matter. How can we define
metrics that remain anchored to business needs? In other words, how can we translate
business requirements to the right set of metrics? Once the right metrics are defined,
how can we monitor and benchmark it to control performance and make
breakthrough improvements?
 Metrics ?

Metrics are measurements, collections of data about project activities, resources and
deliverables. Metrics can be used to help estimate projects, measure project progress and
performance, and quantify product attributes.

Software metrics can be classified into three categories:

• Product metrics (size, complexity, performance)

• Process metrics (used to improve development and maintenance)

• Project metrics (cost, schedule, productivity)

 Metrics Classification

Software metrics are numerical data related to software development. Metrics strongly support
software project management activities. They relate to the four functions of management as
follows:

1. Planning –
• Metrics serve as a basis of cost estimating, training planning, resource planning,
scheduling, and budgeting.

2. Organizing
Size and schedule metrics influence a project's organization.

3. Controlling
Metrics are used to status and track software development activities for compliance to
plans.

4. Improving
Metrics are used as a tool for process improvement and to identify where improvement
efforts should be concentrated and measure the effects of process improvement efforts.
 What affects software?

People

Process Technology
 Quality Model

product

operation revision transition

reliability efficiency usability maintainability testability portability reusability

Metrics
 Metrics cont….
Gathering Data

Gather both quantitative and qualitative data about organizations. Use commonly
accepted practices for collecting and interpreting qualitative and quantitative data.
Employ the technique of ethnological study to software organizations. Ethnography
consists of looking at what people do, what they say, and the tension between what they
really do and what they ought to do, as well as what they make artifacts

Quantitative Data
Gather up as much quantitative data as possible. These are hard facts about an
organization. I gather data at the project and application levels and it is rolled up to the
organizational level. The following is a list of some examples are provided for the
quantitative data to be gathered :

• Project Size (measure in function points)

• Application Size (measured in function points)
• Use Cases
• Hours
• Defects
• Staff
• Salary Figures
• Test Cases
• Testing Problems
• Production Defects
• Project Duration
• Lines of Code
• Pages of Documentation
 Metrics cont….
Metrics and Ratios
Develop a set of metrics (ratios) using the quantitative data. One of the metrics to calculate is organizational cost per function
point. This metric is based upon the total organizational cost divided by the total number of function points. Also calculate
project cost per function point or hours per function point. Often, there is a large gap between project costs per function point
and organizational cost per function point.
Too often the only costs that get reported and recorded are programming costs not total organizational costs. Actual programming
costs represent between ten and twenty percent of total software development costs.
Qualitative Data
The qualitative scores are based upon actual artifacts where possible. It is important to have supporting detail especially
artifacts. When working with a company do not makes requests similar to, “Rate your project management skills on a
scale of 1 to 5” because everyone knows the right answer. Instead, rely on artifacts where possible. If a company is a five in
project planning, it will have project plans to examine.
Gather Qualitative Data
• Conduct formal interviews
• Talk to people, and ask a lot of questions
Qualitative Scores And Productivity
There is a strong correlation between high qualitative scores and productivity. It is not by accident that use the word correlate
and not the word causes. Correlate and causation are not the same thing at all. It is similar to figuring out that wealthy people
listen to classical music and concluding that if you listen to classical music, you too will be wealthy. Listening to classical music
is a characteristic of the wealthy; it is not what made them wealthy.
Project Success and Project Failures
Large projects with high qualitative scores in the area of project management skills were more likely to be successful than
projects with low scores. Those large software projects with more than 10,000 function points are more likely to be successful when
they brought in skilled project managers from the outside.
 Translating business needs

Conceptual Level
Establish measurement
goals aligned with
business needs i.e. what
we want to achieve

Operation Level
Generate a set of
questions that can be
used to determine
achievement of goals

Quantitative Level
Establish the set of data,
objective or subjective,
that will answer each
question in a
quantitative way
 Translating business needs

Goal Approach Question Metric

•Estimating effort
•How good is
Improve based on requirements Effort
effort
Estimation •Planning based on Slippage
estimation ?
estimation

•Detecting defects as •What are Defect

Minimize early as possible current defect Density
Defects •Minimize acceptance levels ?
testing defects

•Detecting defects as
Reduce Costs early as possible •What is Rework
•Minimize acceptance Rework Effort ? Effort
testing defects
 A Goal-Based Software Measurement Framework

 Classifying software measures

 Three types of software entities to measure

• Processes – collections of software related activities

• Products

• Resources – entities required by a process activity

 Within each class, we have

• Internal attributes – measured purely in terms of the entity itself

• External attributes – measured with respect to how entity relates to its environment. Behavior
of the entity is important

 Managers want to be able to measure and predict external attributes

• However, external attributes are more difficult to measure than internal ones, and are
measured late in the development process

• Desire is to predict external attributes in terms of more easily-measured internal attributes

 A Goal-Based Software Measurement Framework

 Determining what to measure

 Measurement is useful only if it helps understand the underlying process or one of

its resultant products

 Goal-Question-Metric (GQM) has been proven to be effective in selecting and

implementing metrics

• List the major goals of the development project

• Derive from each goal the questions that must be answered to determine if

• goals are being met

• Decide what must be measured in order to be able to answer the questions

adequately

13
 A Goal-Based Software Measurement Framework

• GQM example – goal is to evaluate effectiveness of coding standard

Goal Goal

Questions Who is using standard? What is coder productivity? What is code quality?

Proportion of coders Experience of coders Code size Effort Errors

Metrics • Using standard • With standard (lines of code,
• Using language • With language function
• With environment, etc. points, etc)

14
 Process Improvement
Quality improvement starts with identifying the right metrics. We apply techniques such as control
charts to improve the stability of the process. Thereafter, we embark on a continuous improvement journey by
increasing the capability of the process.

Business
objectives

Process
Piloting
Sub-process Change
Selection Project -Level
Change

Org
Removal Remove Change Process/
Special Common Technology
Measure
Causes Causes Change

NO

NO

Statistical
Process Process
Analysis Capable ?
Stable ? Yes Yes
(OPP)