How to Audit Risk-Based Thinking

by Christopher Paris | Apr 14, 2015 | Guidance Document |

The future of how certification bodies (CBs) will audit the new ISO 9001:2015 risk based
thinking language is already setting, like a wet clay in the oven. In short, they are going to
default to traditional risk management techniques and impose FMEA on their clients. It wont
matter if you are making tanks for the military or muffins in a bakery, you are going to be doing
FMEA because they auditors dont know any other way. That contradicts what ISO 9001s
authors intended, but its what is going to happen.
But if the CBs are wrong, then what is the right way to audit something like RBT, which doesnt
actually have any requirements? How do you audit thinking? Some have argued its impossible,
but its not; its just challenging. Lets take a look.
The Audit Process
First, lets understand what auditors are supposed to do. They are supposed to gather objective
evidence to assess the conformity of a company against ISO 9001 requirements. That means
there needs to be two inputs into the audit process: requirements and evidence.
Lets take that in reverse order and understand what objective evidence is. In short, objective
evidence is evidence that is gathered independent of the auditors opinions and biases, and
which can be confirmed at a later date by any third party. For decades, auditors have been
trained to believe that the only acceptable forms of evidence are documents and records. This is
not true. Other forms of objective evidence include:
Direct observation of work by multiple parties
Direct observation of work by a single party (when a job is only performed by one person)
Gathering of intellectual evidence (i.e., conversations)
Sounds, smells, tactile evidence
and so on.
When auditing RBT it will be imperative for CB auditors to avoid the reliance on documents,
records and procedures since the standard specifically does NOT require them. Auditors who
demand to see such documentation because I cant audit otherwise should be shown the door.
And I mean that in the Old Western saloon bar fight way.
So the second input is the requirement itself. For RBT, as I said, there are no firm requirements
for documents, records, processes or resources. One need merely think about risk when
crafting and managing a QMS. The idea behind this approach was to, according to one TC 176
representative, address risk according to the context of the organization.
Some organizations might be required to take a heavy, formal approach in order to provide the
necessary level of confidence in their ability to provide consistent conforming product. In the
automotive context, design and process FMEA would be expected, and possibly other risk-based
things like sampling criteria etc. In the Food context we have HACCP; and so on. Clearly though,
it wouldnt be appropriate for a small mom & pop store selling innocuous hardware products to
have to go through a full FMEA. So we came up with the risk-based thinking phrase as a way of
diluting the push for out-and-out risk management.

OK, thats simple enough. But what does ISO 9001 actually require? From this standpoint, the
9001:2015 DIS calls out the need to assess risks in the following areas:
Determining the context of the organization
Determining the processes needed for the QMS
Risks associated with assuring product or service conformity
Post-delivery risks
Theres also a general theme running that risks should be considered throughout the QMS
regardless of whether its specifically called out in a clause or not. It appears, then, that we have
very loose, vague language defining how RBT is to be implemented, and (again) thats by TC
176s design. The intent was for the company to determine the level of rigor to be used.
 So that leaves us with the need to find hard evidence of a soft intangible. Impossible? Only if
you are one of those badly trained auditors who probably should have retired 10 years ago.
The Good
Instead, this is a situation that requires gathering of intellectual evidence, typically through
conversations. Auditors, we find, actually dont know how to document a conversation, so here is
what an audit report might look like:
Re: risk based thinking, held interviews with Bob J. the VP Engineering and Jim S. the President.
Management indicated that during the development of the QMS risks specific to customer
product requirements, local labor force availability and previous issues with utilities were taken
into account. This, they reported, resulted in the current process set and related objectives. For
example, the Maintenance process was formerly embedded in production, but now is a
standalone process in order to better manage utilities.

In that example, the names of the people interviewed (Bob J. and Jim S.) are the objective
evidence, since they can be confirmed later by a third party. The rest of the notes are the
supporting evidence to show what they said, and to give some idea that it was acted upon.
Theres no risk registry, no procedure, no records to prove it. Nothing that approaches any
common approach to risk management. And none would be required, yet the company
complies with risk-based thinking just fine.
The Bad
So what might a nonconformance look like in such a scenario? In real life we can expect to see
all sorts of nightmarish NCs written up, as auditors go nuts trying to invent risks from thin air and
then play gotcha! with the client by writing them up for not thinking of them; or worse, for not
completing an FMEA on each risk. None of that is required, but some evidence or risk-based
thinking must be present, or a nonconformity can be issued. Heres an example:
Re: risk based thinking, held interviews with John D. the CEO and Fester B. the President. There
was little understanding of risk and management admitted it did not consider risk when
developing the QMS. The management could not name any risks it might face, nor any actions it
took to address those risks.

What we see is that you have to be pretty ignorant of risk to get such a finding. Which is also by
TC 176s design; they didnt intend for companies to be written up for having a different view of
risk than their auditors, but they did intend a complete absence of any risk-based thinking to be
noncompliant. Thats good news for users, since you dont have to do too much to comply with
RBT, especially since TC 176 says its been implicit in ISO 9001 all along. But for companies
that ignore it entirely, blow it off, or fail to execute something, it will be a nonconformity. And it
should be.
The Ugly
So, to recap, heres how it should work:
1. Determine how the company has interpreted the requirements for risk-based thinking.
2. Determine how the company has implemented risk-based thinking
3. Conduct interviews with key management to confirm; capture names and discussions as
evidence.
4. If available, capture documents and records to support. If not, stop at # 3.
This idea of auditing intangibles may be frustrating, and yes, ISO 9001s risk-based thinking is a
mess. But its not un-auditable, and auditing it doesnt require imposing specific solutions on
clients simply because an auditor lacks the imagination to audit something other than a
document or record.
And the thing is, auditors have already been doing this, without a peep. In the 2000/2008
versions of 9001, the standard required the management to show commitment and a customer
focus, neither of which are tangible things. Auditors typically relied on documentation to check
these off, and if the words were on paper, that satisfied them. They were idiots, of course, and
should instead have done the same thing I am suggesting here: have conversations with
management and key staff, and document that as the evidence.
So we find that auditing RBT isnt the great mystery that many are claiming. If auditors can stop
trying to be consultants, stick to the rules, and understand that they should stop their fixation on
auditing documents, we might see some benefit from this after all.
At least when RBT hits the fan, we know who to blame.