Group Homework 1

Problem 1 (4 points): Assume that ACME Software Inc. decides to initiate a measurement effort inside the
software development team. They decide that their software testing (QA) processes are fairly robust and a
great starting point.

What are some measures and metrics that they could use? The business goals are:

 Increase productivity (produce more code)

 Increase quality (fewer problems)
 Reduce rework

Put a simple plan explaining which measures you would choose and how they would help the organization
achieve their business goals.

Answer:

Identifying Process Issues

1. Clarifying business goals.

The business goals are defined for this problem. The business goals are to increase productivity,
increase quality and reduce rework.
2.Identifying critical processes.
The processes related to these business goals are creating the requirement documents, creating the use
cases, designing the software architecture, developing code based on the proposed architecture, creating
the test strategy and test cases, executing test cases, closing the defects and updating the application
units.
3. Objectives for each critical process.
Critical Process Objectives
Creating the requirement documents.  People involved in the process of creating
the requirement documentation should have
adequate training and experience.
 The amount of effort that goes in creating
the requirement document should not be
much.
 The time taken by the requirements team
should be as minimum as possible.
Designing the software architecture  People associated should have adequate
training and experience.
 Amount of time and effort that goes in
creating the software architecture should be
proportionate to the complexity of the
software.
 Software developers should have adequate
level of training and experience to
understand the proposed software
architecture.
Developing code  Software developers involved in the project
should have adequate level of experience
working such projects.

Designing software test strategy and test cases.
Executing test cases.
Closing the defects and updating the application
units.
4. Potential problem area associated with the processes.
Process
Creating the requirement documents.
Designing the software architecture
Developing code
 Team should try to meet set deadlines.
 Development team should be provided with
adequate hardware to create the software
and later test it.
 The environment should be supportive of
producing code.
 The developer and testers should have
adequate experience and training.
 Test reports created should have adequate
information about the severity of the test,
origin of the error, and the type of error that
is occurring.
 The developer and testers should have
adequate experience and training.
 Test automation tools should be used.
 Results from all the test cases should be
recorded along with the person who
executed the test.
 All the changes should be approved from
the project manager.
 All the changes should be tracked using
change management software.
 All the changes to the software should be
performed in timely manner.
 All the changes that are introduced to the
code should be properly documented, with
details about the origin of the change, effort
that it took to make the change, and the
person who is assigned to make the change.
Potential problem
 Lack of experience and training of the
requirement documentation team.
 The customer is not involved or is not
interested in spending enough time to
provide adequate requirements.
 Requirement documents are not stored
properly and not accessible to other
software teams.
 Lack of experience and knowledge of the
software architecture designing team.
 Software developers don’t understand the
software architecture.
 Software developers don’t have experience
with the tools and technologies that are
required to develop the software.
 The environment in which the development
team is working is not adequate.
 There is lack of support from the higher
management.
 Designing software test strategy and test cases.
Executing test cases.
Closing the defects and updating the application
units.
 Lack of experience and training of the
software testing team.
 Lack of experience and training of the
software testing team.
 The test documents don’t have proper
information and are ambiguous.
 No test automation tools were used.
 Changes are not being approved by the
project manager on time.
 No change management software is being
used.
 Changes are not being documented
properly.

Business Goals and related measurable attributes.

Business Goals Project Issue Process Issue Measurable Product

and Process Attributes
Increase productivity  Schedule  Requirement  Developers
(produce more code)  Progress documentation. Experience and
 Unfavourable training.
environment.  Level of test and
 Test automation. requirement
 Timely change automation.
approval.  Quality of test and
requirement
documentation.
 Changes approved.
 Time to market.
 Effort
Increase quality (fewer  Product  Testing  Testers Experience.
problems) performance documentation.  Changes committed.
 Product correctness  Level of automation.
 Product reliability  Quality of testing
documentation.
 Audits performed

Reduce rework  Product correctness.  Development  Quality of

 Product validation. standard. requirement
 Cost. documents.
 Changes committed.
 Developer and tester
experience.
 Audits performed.

Based on the above table the following metrics should be measured:

Number of bugs, time to run a set of test cases in suite, defects created, defects resolved, rework rate,
milestones achieved, number of revisions.
Problem 2 (2 points): Section 3.5 of the textbook’s Chapter 3 talks about tools for understanding your
data. They are mostly charts that offer a visual representation of your data that helps understanding and
interpreting the results of measurement activities.
For the chosen metrics in Problem 1 above, simulate a measurement with fictitious data (feel free to use
real life data if you have any) and decide which charts (Section 3.5 of the textbook’s Chapter 3) you would
use to represent the data points you chose and why. Draw the charts as well.

Answer:

Number of revisions per date. To represent this data, we are using the bar chart because we are
representing relative numbers against another relative number that is the date. As the project nears its
completion, the number of revisions per day reduce.

Date Revisions
30 January 2018 420
31 January 2018 360
01 February 2018 270
02 February 2018 330
03 February 2018 70
04 February 2018 5
05 February 2018 350
06 February 2018 90

Revisions

450
400
350
300
250
200
150
100
50
0

Created defects vs Resolved defects. This measure shows the efficiency of the testing team. The more
efficient the testing team is, the higher the reliability and quality of the software product delivered.
For this data we have selected the scattered data plot chart since we are showing the association of the
two data. This can help us in predicting the testing efficiency of the team in future projects.

Created Resolved
Date Defects Defects
30 January
2018 280 244
 31 January
2018 280 264
01 February
2018 280 272
02 February
2018 250 244
03 February
2018 250 237
04 February
2018 250 246
05 February
2018 200 192
06 February
2018 200 190

Created vs Resolved Defects

300

250

200

150

100

50

0
Tuesday,Wednesday,
January 30,
Thursday,
January
2018 31,
February
Friday,
2018February
Saturday,
1, 2018 2,February
Sunday,
2018 February
3,
Monday,
2018 4,February
Tuesday,
2018 5, February
2018 6, 2018

Created Defects Resolved Defects

Problem 3 (2 pts): A commonly used software quality measure in industry is the number of known errors
per thousand lines of product source code. Compare the usefulness of this measure for developers and
users. What are the possible problems with relying on this measure as the sole expression of software
quality?

Answer:

Developer User
Effectiveness of inspection and testing. Reliability and usability of the product.

Is the product ready to be released? Stability of the product.

Predict the reliability of the software. Level of experience of the developers.

Complexity of the software. Security of the software.

Level of experience of the development team.

Amount of rework required.

Number of errors per thousand lines of product source code is not the best measurement as the sole
expression of software quality since the number of errors per KLOC would vary for different developers.
Less experienced developers would have more errors per KLOC as compared to more experienced
developers. Also, this measure depends on the effectiveness of inspection and testing team. If there are less
number of errors per KLOC then it might even mean that the testing team is not efficient in finding errors
and not that the software code is correct. This metric might even change because of the complexity of the
software code. Also, this measure is relevant and relies on historical data, meaning that until we have this
measure for another program it is difficult to measure the quality of the present software.

Problem 4 (2 pts): What different product and/or process attributes might the following measure?

1. the number of faults found in program P using a set of test data created specifically for P?
2. the number of faults found in program P using a standard in-house set of test data?
3. the number of faults found in program P by programmer A for one hour.

Answer:

1. Efficiency of test planning team and reliability of the product, stability of the product, cost effectiveness
of the test data for P, complexity of P.

2. Efficiency of the testing process and the testing team, stability of the product.

3. Efficiency, level of experience and the amount of effort required of the programmer A