2162CIT – Software Quality Principles

Exam Questions, 2005

Question 1
a) Identify and describe the Process Attributes that characterise the Established Process, as
defined in ISO/IEC 15504-2. (4 marks)
2 marks for each PA – 1/2 mark for the name, 1 1/2 marks for the description.
Process Definition Attribute
The process definition attribute is a measure of the extent to which a standard process
is maintained to support the deployment of the defined process.
As a result of full achievement of this attribute:
–a standard process, including appropriate tailoring guidelines, is defined that
describes the fundamental elements that must be incorporated into a defined process;
–the sequence and interaction of the standard process with other processes is
determined ;
–required competencies and roles for performing a process are identified as part of the
standard process;
–required infrastructure and work environment for performing a process are identified
as part of the standard process;
–suitablemethods for monitoring the effectiveness and suitability of the process are
determined.
Process Deployment Attribute
The process deployment attribute is a measure of the extent to which the standard
process is effectively deployed as a defined process to achieve its process outcomes.
As a result of full achievement of this attribute:
–a defined process is deployed based upon an appropriately selected and/or tailored
standard process;
–the required roles, responsibilities and authorities for performing the defined process
are assigned and communicated;
–the personnel performing the defined process are competent on the basis of
appropriate education, training, skills and experience;
–the required resources and information necessary for performing the defined process
are made available, allocated and used;
–the required infrastructure and work environment for performing the defined process
are made available, managed and maintained;
–appropriate data are collected and analysed as a basis for understanding the behaviour
of, and to demonstrate the suitability and effectiveness of the process, and to evaluate
where continuous improvement of the process can be made.
b) Identify the four primary phases of the Goal / Question / Metric approach to process
improvement, and describe how measurement can be used to help drive improvement.
(3 marks)
1/2 marks for each –
Planning
Definition
Data Collection
Interpretation
1 marks for discussion on measurement and improvement.
c) List and describe three ways in which Process Improvement can yield measurable benefits
for a systems engineering organization. (3 marks)
Three of the following:
Reduces Development and Maintenance Costs.
• The cost of implementing software improvement methods are heavily outweighed by
– The cost savings from reduced development costs, and
– The cost savings resulting from less rework.
• The major reduction of development costs can be attributed to improved software
productivity.
Improves Customer Satisfaction.
• Typical software development organizations release products with 15% of the defects
remaining for the customer to find.
– No customer is happy with that many problems.
• Some software process improvement methods reduce post-release defects to near zero.
• Improving customer satisfaction is shown to result in repeat customer business and an
improved company image.
Reduces Cycle Time.
• Improvement efforts can reduce typical schedule lengths by 30% to 40%.
– This may allow organizations to beat the competition in getting product to the
field.
– It may result in more product purchased earlier than projected.
– It may result in schedule related bonuses for early delivery.
• Combining improved schedules with higher quality - getting better products out
sooner -is a winning combination as far as our customers are concerned.
Increases Profitability.
• The return on investment for software improvement is very high.
– Many organizations have reported a 7:1 ROI.
• This high ROI is achieved by reducing development costs, rework costs, and turnover
costs.
 – Penalties turn into bonuses.
– Product sales increase from higher quality software.
– Repeat business increases.
• Furthermore, a risk analysis of doing software improvements versus not performing
the improvements highly favors performing the improvements.
Improves Professional Staff.
• Software process improvement improves employee morale, and increases the
confidence of developers.
• It results in:
– less overtime.
– less crisis.
– less employee turnover.
– an improved competitive edge.
• The reduction in employee turnover costs and retraining costs could pay for the
improvement costs alone.
1/2 mark for identifying the outcome; 1 1/2 marks for description.
Question 2
The following extract is drawn from ISO/IEC 16085: System and Software Engineering - Life
Cycle Processes – Risk Management. From this extract:
(1) Identify (by clause number and paragraph) the requirements that would have to be met in
order to claim conformance. (2 marks)

There are ten requirements – in Para 3 of 5.1.3; two in Para 1 of 5.1.3.1; one in Para 1 and one
in Para 3 of 5.1.3.2; and one in Para 1, one in Para 2 and three in Para 3 of 5.1.3.3. Mark out
of 2 based on the proportion of the requirements correctly identified (see highlighting below).
(2) For each requirement, evaluate the extent to which objective evidence could be provided
that the requirement has been met. (3 marks)

Risk analysis shall be performed continuously throughout the life cycle:

Can be evaluated objectively, based on testimony and some work products.
Risks shall be identified in the categories included in the risk management context:
Can be evaluated objectively, provided the risk categories have been identified.
Changes in the risk management context shall also be identified:
Can be evaluated objectively, but will probably depend to a significant extent on
testimony.
The probability of occurrence and consequences of each risk identified shall be estimated:
Can be evaluated objectively in terms of whether probability and consequence have been
estimated, but the accuracy of any estimates will be subjective. Basic work products will
support performance, reinforced by testimony.
The scale(s) used for estimating risk probability and consequences shall be used consistently.
Can be evaluated only with some degree of subjective judgement – evaluation of
"consistency of approach" requires judgement.
Each risk shall be evaluated against its risk thresholds.
Can be evaluated objectively, provided the risk threshold is properly documented.
For each risk that is above its risk threshold, recommended treatment strategies shall be
defined and documented
Can be evaluated objectively.
Measures indicating the effectiveness of the treatment alternatives shall also be defined.
Can be evaluated objectively.
The risks, their recommended treatments, and measures of risk treatment effectiveness shall
be communicated to the stakeholders
Can be evaluated objectively.
Mark out of 3, depending on the completeness of the analysis; note that only one requirement
cannot be fully evaluated objectively. Students who get this correct should get better marks.
(3) List the types of evidence that would be available, and the types of judgement that would
have to be made in order to determine conformance. (10 marks)

Work products that will be important:

Risk management plan – does it demonstrate full life cycle coverage; does it establish
scales for probability/ consequence
Risk repository – demonstrates risk categories, should show probability/
consequences. Records should help to establish continuous analysis across the life
cycle. The risk thresholds should also be identifiable here.
Risk mitigation plans – document proposed treatments (as set out in the Standard).
Measurement repository – should identify and record data collected on risk measures.
Meeting minutes and reports – should record communications with stakeholders on
risk issues.
Testimony from project personnel will be important in establishing identification of
changes in the risk management context.
Note that the specific names of the work products / evidence are not really important – the list
here is indicative.
Mark out of 5 depending on the comprehensiveness of coverage. Are the sources of evidence
reasonably complete? Are they credible as sources of correct information? The use of
testimony from performers as a source of evidence is important and should be recorded for
full marks.
5.1.3 Perform risk analysis

The purposes of the “perform risk analysis” activity are to

a) Identify the initiating events, hazards, threats, or situations that create risks 
b) Estimate   the   probability   of   occurrence,   the   consequences   for   each   risk,   and   the 
expected timing of the risk
c) Evaluate each risk or defined combination of risks against its applicable threshold, 
generate   alternatives   to   treat   risks   above   their   risk   thresholds,   and   make 
recommendations for treatment based on a priority order

Risk analysis shall be performed continuously throughout the life cycle.

The “perform risk analysis” activity consists of the tasks listed in 5.1.3.1 through 5.1.3.3.

5.1.3.1 Risk identification

Risks  shall  be   identified   in   the   categories   included   in   the   risk   management   context. 
Changes   in   the   risk   management   context,   e.g.,   additional   risk   due   to   changes   in   the 
assumptions, shall also be identified.

Various approaches to identifying risks should be used. These approaches may include the 
use of risk questionnaires, taxonomies, brainstorming, scenario analysis, lessons learned, 
and   prototyping   or   other   knowledge   acquisition   approaches.   Repeatable   identification 
processes may be used to aid in the capture of lessons learned. Where possible, events, 
hazards, threats, or situations that can create risks should be identified to aid future risk 
treatment. Risks not identified are implicitly accepted.

Risk categories should be used consistently for effective communication to stakeholders. 
Risks that are related may be combined for ease of analysis, monitoring, and treatment. 
System   or   software   anomalies,   reports   on   measures,   and   other   indicators   should   be 
continuously reviewed as sources for risks.

5.1.3.2 Risk estimation

The probability of occurrence and consequences of each risk identified shall be estimated. 

Estimates may be either quantitative or qualitative. The stakeholders should define which 
risks   will   be   evaluated   using   a   qualitative   scale   and   which   will   be   evaluated   using   a 
quantitative scale. 

The   scale(s)   used   for   estimating   risk   probability   and   consequences  shall  be   used 
consistently. The descriptive and measurement uncertainty inherent in the scale used should 
be described in the risk management plan. The level of confidence in a risk’s estimate 
should be captured in its risk state.

5.1.3.3 Risk evaluation

Each   risk  shall  be   evaluated   against   its   risk   thresholds.   Risks   should   be   evaluated 
independently, in combination, and along with their interactions with system and enterprise 
risks.   Risks   should   be   evaluated   against   the   project   risk   threshold   to   assure   that   a 
combination of risks, while below their individual thresholds, does not unacceptably place 
the project as a whole at risk. Different techniques may be used to evaluate the risks, such 
as   decision   trees,   scenario   planning,   game   theory,   probabilistic   analysis,   and   linear 
programming.

Risks  shall  be   placed   in   a   priority   ordering—the   ordering   criteria   determined   by   the 

stakeholders. Priority may be based upon when the risk is anticipated to become a problem, 
the risk exposure, risk­related measures, or some other consistent criteria.

Various   treatment   alternatives   to   addressing   risk   should   be   considered   to   reduce   or 

eliminate   risks.   For   each   risk   that   is   above   its   risk   threshold,   recommended   treatment 
strategies such as eliminating the risk, reducing its probability of occurrence or severity of 
consequence, or accepting the risk shall be defined and documented in a risk action request 
such as that found in Annex B. Contingency plans should be developed for all risks above 
their thresholds. Measures indicating the effectiveness of the treatment alternatives  shall 
also be defined. The risks, their recommended treatments, and measures of risk treatment 
effectiveness  shall  be   communicated   to   the   stakeholders   for   approval,   rejection,   or 
modification.