CMMI Appraisals Testers

Journey as an Appraisal Team

Member (ATM)
Posted In | ISO standards, Quality assurance | Last Updated:
"November 3, 2015"

Personal Note: Its a great opportunity for a QA/Tester to

be a CMMI Appraisal Team Member (ATM). ATM gets
extensive exposure of a complete CMMI model. If your
organization is going for appraisal and you get an
opportunity to be an ATM, grab it. Its an achievement you
will like to mention in your resume.
To Readers: Appraisal is complex and time taking
process. I have tried to describe it as simple as possible.
This article will give you brief idea about CMMI Appraisal.
Its not about CMMI model but it will tell what happens
during appraisal! There may be understanding problems
initially (If you are not familiar with CMMI terminology)
until you arrive at an example below, Please keep reading
until then.
Also read => CMM levels for QA Processes
What is CMMI Appraisal?

An appraisal is an activity that helps us to identify

strengths and weaknesses of our organizations processes

and verify how closely the processes relate to CMMI best

practices.
Flash Back: When any org decides to go for a maturity
level appraisal (Commonly known as CMMI Appraisal) they
contact an authorized organization from Software
Engineering Institute (SEI) Partner network to do the
appraisal. Authorized agency will appoint consultant to
help/guide org to understand CMMI, creating documents
and organization policies etc. This is a long process. It
starts 1 year or 6-7 months (Depends maturity level of
organizational process) before the final appraisal. After
preparation, LA (Lead Appraiser) will be invited to do the
appraisal.

Preparation at organization level:

Before CMMI implementation, organization has to:
Identify staff to attend Intro to CMMI course
Create SEPT or SEPG (Software Engineering Process
Team/Group) (Responsibility of SEPT/SEPG explained
at the end of this article.)
Create organization policies.
Employ CMMI consultants
ATM Journey: After Intro to CMMI course you will have
Appraisal Team Member (ATM) training. LA provides
training of SCAMPI Appraisal Method Training to ATM.
After training, an exam would be held and you have to
pass. A reference document called Method Definition
Document (MDD) will be provided to all the ATMs, this
document describes the SCAMPI. (Topics of Intro to CMMI
Training course is covered at the end of this article)

Now what is SCAMPI, who is LA and what

ATM does:
SCAMPI: Standard CMMI Appraisal Method
for ProcessImprovement.
Lead Appraiser (LA): Software Engineering Institute
(SEI) Authorized SCAMPI Lead Appraiser. (Commonly
known as LA. The LA announces the final findings report
and signs the Maturity Level certificate. The word
certificate isnt generally used; SEI discourages the word
certification and instead it refers to a company as having
achieved CMMI Level 5).
What ATM does?
ATMs support the Lead Appraisers in collecting and
validating evidences for Document Review, Interview of
FAR (Functional Area Representative) members and
preparation of other important artifacts. (More about ATM
responsibility mentioned at the end of this article.)
Little more about CMMI: Capability Maturity Model
Integration is a process improvement model developed by
the Software Engineering Institute (SEI), Carnegie Mellon
University.
CMMI is comprised of a set of Process Areas. Each
Process Area is intended to be adapted to the culture and
behaviors of your own company.
There are 22 Process Area in
CMMI.

[Source: wikipedia]
Within the Process Areas in the CMMI, there are multiple
Specific Goals (SGs) and Specific Practices (SPs). These
practices define the expected behaviors of projects and
organizations.

Image: Process Area components

SG: A specific goal describes the unique characteristics
that must be present to satisfy the process area.
GG: Generic goals are called generic because the same
goal statement applies to multiple process areas.
Every ATM will be responsible for Process Area allocated by
LA to him/her. Suppose there are 22 ATMs, every ATM will
have 1 process areas of all the projects selected for
Appraisal. Generally LA forms small group of ATMs.
Suppose 20 ATMS are there LA will create 10 groups (each
group with two ATMs) and each group will be responsible
for a few PA of all the projects.
Example:
Let me give an example to make you understand Specific
Goal and Specific Practices and what is ATMs role in
appraisal process.
Take example of Process Area, Validation. (keeping in
mind most of the reader here are Tester/QA)
Validation (VAL) is an Engineering process area at
Maturity Level 3.

Purpose of this Process Area: The purpose of Validation is

to demonstrate that a product or product component fulfills
its intended use when placed in its intended environment.
There are two SG (Specific Goals) in this Process
Area.
1. SG 1 Prepare for Validation
2. SG 2 Validate Product or Product Components
Lets see Specific Practices by Goal. Specific Practices
in SG1 mentioned below.
SG 1 Prepare for Validation
SP 1.1 Select products and product components to be
validated and validation methods to be used.
SP 1.2 Establish and maintain the environment needed
to support validation.
SP 1.3 Establish and maintain procedures and criteria
for validation.
It is not possible to elaborate all Specific Practices here; I
will consider SP1.1 to give you an idea of appraisal process.
SP 1.1 Select products and product components to be
validated and validation methods to be used: STP
(Software Test Plan) is the most suitable artifact for
this SP. All Features/Component/Capability needs to be
tested and the method is captured in this document.

ATM responsibilities:
First ATM will verify if the STP is approved.
Approval mail or any supporting artifact for approval is
checked for. E.g. affirmation from approver in MoM. If
its in MoM, approver presence in that meeting will be

checked in attendance sheet on that particular date for

this meeting.
Review plan present in the PMP Annexure? Was Review
done? Who reviews? Review date! Whether review
points are captured and closed in review log? Closure
date?
If the review is conducted, there should be reviewchecklist.
Was STP baseline? All review points were closed before
the baseline! What is baseline date?
Whether validation method mentioned in the STP?
What is reference document for preparing STP?
Requirement capture document (RCD) or Software
Requirement Specification (SRS) or any other
reference doc; suppose RCD is reference document,
ATM checks Bi-Directional Traceability Matrix (BTM) for
proper mapping of RCD to STP. All requirements
should be covered properly in STP for validation.
Guideline for preparing STP present? Does
organization have standard template for STP? What is
version number of STP? (Guideline and template
creation comes under different PA; I have included this
point just for reference.)
Verifying Date plays important role! Any document will be
baseline after review, so baseline date will come after
review date. Similarly date in checklist, review date, review
point closure date will be compared; date on respective
documents should appear in correct sequence.
-----------I now hope you must be having some clarity on ATM
responsibility. Cross checking all documents prepared
during Software Development Life Cycle, dependency on

each other followed by cross verification, covers the core

responsibility of an ATM. Other than this ATM participates in
the interview of FAR members. I shall cover this topic in
future articles.
Now few more examples:
These are generic examples to give more insight on
appraisal process.
Lets see a scenario: While going through a weekly status
meeting MoM (Minutes of meeting) of a particular date ATM
ensures on verifying two points.
1. Risk discussed in the team meeting.
2. One new member in the team and PM ensuring on
subsequent training.
1) Risk : Risks are covered in the PA called Risk
Management(RSKM).
ATM will check if this risk is mentioned in Risk Log? If
present in the Risk Log, there will be Risk Mitigation Plan or
Contingency Plan? If Mitigation/ Contingency Plan are
present, there may be some action item against the Plan.
Now if Action item is present, would Monitoring be needed?
Monitoring comes under Project Monitoring and Control
(PMC). ATM must have an idea as to what to check and
where to check.
This example shows how process areas are connected to
each other and how many artifacts ATM will go through to
make sure all practices are followed properly.
2) Training
All training related activity comes under Process Area called
Organizational Training (OT).

Project Manager sends a request to HR department

regarding training. HR will find an Internal/External trainer
for the resource and conduct training. This is the general
scenario, what else happens in OT?
Lets see how OT works, suppose this financial year
organization has allocated XX amount of money for
training. Now HR will identify training needs, will collect the
data from managers, resources and will prioritize it, along
with this HR will also identify organizations future needs.
For instance, if organization decides to start working on
communication projects along with software, either HR will
hire new Resource/Managers/TL or will identify within
organization to work on this new domain.
ATM Verification Areas:
Identification of Training needs by HR
Identification of Trainee
Identification of Trainer
Intimation to the manager and trainee to block their
calendar and subsequent affirmation in advance.
Date of request to Admin and IT team for
Infrastructure (Training room, Laptop, Software, food,
vehicle etc.)
Attendance sheet maintained?
After training feedback was taken from trainee?
HR updated Org Skill matrix after training?
If some training is pending from previous year, a plan
is made to execute the same in current year and
reason for HR not conducting the training last year.
Etc.
For sure this is not complete list of activity for
Organizational Training; other activities are also
there, these are just quotes from my memory.

 Responsibility of SEPT/SEPG (Software

Engineering Process Team/Group)

Study organization processes

Write process standards

Develop SDLC stages and details

Create meeting procedures

Create estimation procedures

Create planning procedures

Create Quality Assurance procedures

Create Configuration Management Procedures

Intro to CMMI Training course covers following
topics:
Introduction to the Model

Model-based process improvement

Overview of CMMI components

Institutionalization of Processes across the

Organization

Process areas of CMMI Model

Structure for the continuous and staged

representations
ATM responsibility

Each ATM is responsible for understanding the

CMMI model, understanding the organization
context, and understanding and applying the
SCAMPI methodology to evaluate implementation.

Each ATM is responsible for participating in data

review, participating in interviews (e.g. identifying
interview questions, taking notes, asking questions,
tagging the answers to the model practices),
participating in all group discussions and consensus
and participating in creating the final findings report.

Each Appraisal Team Member will review and

evaluate objective evidence of practice
implementation and participate with the other
Appraisal Team Members (including the Lead) in
coming to consensus on how adequately practices
are implemented at the organizational level and
whether or not goals, practices, and maturity levels
are satisfied.
Appraisal Validity: The findings and results from a
SCAMPI A Appraisal are valid for a period of not more than
three years from delivery of the appraisal findings. That
means after every three years organization have to go for
appraisal.

Conclusion:
Now see the difference in a Testers activity from Pre CMMI
to Post CMMI.
Pre CMMI: Dev team sends a mail All defects closed,
Defect sheet attached and requests for testing.
Post CMMI:

Tester checks for code baseline followed by upload

in repository.

CR raised for code change and accepted by

SEPT/SEPG?

Availability of Updated peer test report?

Correct Version number offered for testing?

The above are the prerequisites for testing.