The First

International Proficiency Testing Conference

Sinaia, Romnia
11
th
13
th
October, 2007


1

ISO/IEC 17043: A NEW INTERNATIONAL STANDARD FOR
PROFICIENCY TESTING


Daniel W. Tholen, MS
American Association for Laboratory Accreditation (A2LA)
823 Webster Street, Traverse City, Michigan, 49686,USA
dantholen@charter.net


Abstract

A new requirements standard for proficiency testing is being developed by ISO, and
will be completed in the next 2-3 years. This paper describes the origin of the work
item from ILAC, the various sections of the document, and the major points of interest
for proficiency testing providers and accreditation bodies. Areas of current
disagreement are also discussed. Attendees of the First International Proficiency
Testing Conference are requested to comment on the proposed revision.


Key words

Conformity assessment, proficiency testing, international standard, accreditation


1 INTRODUCTION

In 1984 the International Organization for Standardization (ISO) Conformity
Assessment Committee (CASCO) and the International Electrotechnical Commission
(IEC) published the first edition of ISO/IEC Guide 43. This document provided
guidance on the development and operation laboratory proficiency testing (PT) for a
relatively new field of activity. It contained very basic guidance, and little attention to
the use of the outcomes by laboratory accreditation bodies. These issues, plus the
growing use of PT by laboratories and accreditation bodies over a 10 year period, led
the International Laboratory Accreditation Cooperation (ILAC) to propose a revision of
Guide 43, with the addition of new guidance for the use of the results of proficiency
testing. CASCO published ISO/IEC Guide 43, Parts 1 and 2 [1] in 1997.




Daniel W. Tholen: MS : ISO/IEC 17043: A new international standard for proficiency testing

2
Guide 43 provided guidance in five areas, namely:

to distinguish between use of interlaboratory comparisons for PT and for other
purposes (Introduction to Part 1);
the development and operation of PT schemes (Part 1);
the selection and use of schemes by laboratory accreditation bodies (Part 2);
guidance on statistical methods (Annex A); and
guidelines for development of a quality manual for the operation of PT
schemes (Annex B).

The availability and use of PT continued to increase, and Guide 43 was successfully
used throughout the world to harmonize the development and operation of PT
schemes, and in the use of the results for laboratory accreditation. The statistical
annex led to the development of ISO 13528 [2] by the ISO Technical Committee on
the application of statistical methods (TC69). This work started in 1998 and was
finally approved in 2005. It also led, in 1999, to the development of ILAC Guide 13
(G13) [3], published in 2000. ILAC G13 contained the technical guidelines from
Guide 43-1 expressed as requirements, and included the quality management
system requirements from the ISO/IEC Guide 25, then the requirements document
for laboratory accreditation. G13 quickly became the basis for selection and
recognition of PT schemes, by accreditation bodies throughout ILAC.

The continued increase in use of PT led, in 2005, to a recommendation from the
EEE-PT group (a joint effort by the European Cooperation for Accreditation,
Eurachem, and Eurolab) for an urgent revision of Guide 43. This recommendation
was endorsed by ILAC and in February 2006 ILAC proposed a Work Item for
CASCO, and it was approved by a large majority of CASCO members.

At the time of the first and second editions of Guide 43, CASCO did not write
requirements documents. This has changed, and the approved work item is for a
technical and management requirements, using G13 as the base, along with other
parts of Guide 43. This document, to be identified as ISO/IEC 17043, is now under
development by CASCO Working Group 28 (WG28).

At the same time as the work proposal to CASCO, ILAC started revising G13. This
was to make the management system requirements consistent with ISO 17025:2005
[4], to reword ambiguous requirements, and to remove some that were unnecessary.
There were several suggestions for new requirements in G13, but ILAC choose to
refer these suggestions to CASCO WG28. The revised G13:2007 has been
approved and is available at no charge from the ILAC website. Since G13 has
management systems requirements that are consistent with ISO/IEC 17025, PT
providers accredited to this document are considered to be in conformity with the
requirements of ISO 9001:2000 [5].


2 CURRENT PROJECT STATUS

There has been widespread international interest in this project; WG28 currently has
57 experts, appointed by 30 CASCO Participating (P) member bodies and 4 liaison
organizations. The Terms of Reference and current membership list are attached.

Daniel W. Tholen: MS : ISO/IEC 17043: A new international standard for proficiency testing

3
WG28 has held two meetings in Geneva, the first one in December 2006 and the
second in May 2007. The next meeting is scheduled for J anuary 2008, also in
Geneva. The WG is currently reviewing Working Draft 3 (WD3), which will be revised
into WD4 prior to the J anuary meeting.

CASCO policies call for the work to stay within the WG until the experts are satisfied,
and the document can be balloted as a Committee Draft (CD). This will be a vote by
the 72 CASCO P members (national standards bodies) and 9 liaison organizations;
although all 104 CASCO members can comment Following this, the WG will meet to
discuss comments and (if the ballot was successful) they will submit the revised
document to the 107 ISO P members for ballot as a Draft International Standard
(DIS) (156 ISO members can comment). The committee will meet again to discuss
comments, and then (if the ISO ballot was successful), send the revised document to
the ISO membership as a Final Draft International Standard (FDIS). If this ballot is
successful, the document will become an International Standard.

In summary, there will be at least 4 more draft documents (WD4, CD, DIS, FDIS) and
at least more 3 meetings of WG28. There will be at least one ballot by the CASCO
membership and two ballots by the ISO membership. This process will take 2 to 3
more years so there is a lot of work left to do, and several opportunities for
revisions. However, ISO members who are not members of CASCO can make
formal comment only at the DIS stage (or at FDIS, if voting negative), through the
liaison organizations, or through one of several national mirror committees.


3 CONTENT OF WD 17043

3.1 Scope

The Scope of the document from the latest working draft is as follows:

This International Standard specifies general requirements for the competence of
providers of proficiency testing schemes and for the development and operation of
proficiency testing schemes. These general requirements may be used as a basis for
specific technical requirements for particular fields of application.

There is general agreement among the WG concerning this Scope, but, as discussed
later, there are several members if ILAC that would like for 17043 to be more broadly
applicable for other types of interlaboratory comparisons, and for PT for inspection
bodies, in addition to laboratories. This will be discussed at the next WG28 meeting.

3.2 Definitions

The first order of business of WG28 was to carefully define terms. Progress was not
possible until the WG agreed on what is proficiency testing and other essential
terms. The agreed terms are very similar to Guide 43 and G13:

proficiency testing
determination of laboratory performance by means of interlaboratory comparisons.


Daniel W. Tholen: MS : ISO/IEC 17043: A new international standard for proficiency testing

4
interlaboratory comparisons
organization, performance and evaluation of tests on the same or similar test items
by two or more laboratories in accordance with predetermined conditions

participant
a laboratory that receives proficiency test items and submits results for review by the
proficiency testing scheme provider

proficiency testing provider
an organisation, which undertakes the design and conduct of a proficiency testing
scheme and takes responsibility for the evaluation.

proficiency testing scheme
proficiency testing designed and operated in a specified area of testing,
measurement, calibration or inspection
There are many notes and other relevant definitions, and the document will reference
terms in VIM:2007 [6], ISO 17000:2004 [7], and ISO 3534:2006 [8].

3.3 Structure and Guide 43-2

The WG agreed to incorporate much of Guide 43-2 as an informative Annex, rather
than as requirements. Guide 43-2 is for use by laboratory accreditation bodies to
assist with the selection of PT providers and the use of the results. At the time of its
publication, this document was the only such guidance available. Since that time
however ILAC has developed ILAC P9 [9] (currently under revision), which is specific
for the needs of accreditors of testing and calibration laboratories. However PT is
also being used in other types of organizations, such as inspection bodies, and
recognition of PT providers might be through other means, such as certification or
peer evaluation. Therefore the relevant parts of Guide 43-2 will be placed in Annex C
and they will be quite general regarding uses of PT data for accreditation or other
uses. There is agreement that the requirements in 17043 will be consistent with ISO
17011 [10].

By ISO and CASCO Directives, the new document will follow CASCO style and
content rules. This means that the Management System Requirements will conform
with ISO PAS 17005 [11] and ISO/IEC17021 [12]. This will place the Technical
Requirements in the document before the Management System Requirements, to
emphasize the importance of technical competence.

3.4 Other annexes

There will be three informative Annexes in 17043. Annex C is described above.
Section 4 from Guide 43-1 (Types of Proficiency Testing) will become Annex A in
17043 and will be very similar to the current text, but it will expand the description of
PT for medical laboratories, and the differences between External Quality
Assessment (EQA) and PT. For the purposes of 17043, EQA will be defined
generally as a synonym for PT.

Annex B will discuss Statistical Methods. Currently, this Annex is similar to Annex A
in G13:2007, but edited by ISO TC69 Subcommittee 6 at the TC69 meeting in J une

Daniel W. Tholen: MS : ISO/IEC 17043: A new international standard for proficiency testing

5
2007. It references two important documents on statistical methods for PT ISO
13528 and the IUPAC Harmonized Protocol for Proficiency Testing [13]. The new
Annex B will also provide guidance on issues such as determination of the assigned
value (the correct answer), homogeneity and stability testing, and evaluation of
performance. The Annex will provide further guidance for the evaluation of qualitative
and ordinal analytes; to the extent those analytes are evaluated with statistical
methods.

3.5 Technical requirements

There will be several important changes to the Technical Requirements. These are
not yet final, but they will probably include the following:
PT providers will be required to - at a minimum - design and plan the PT
scheme and to make the determination of performance. The provider may
subcontract anything else, including manufacture of PT items, storage and
shipping, and data analysis.
Criteria for sufficient homogeneity and stability of PT test items will be
determined by the impact on the evaluation of performance. That is, it is
acceptable for samples to not be statistically homogeneous or stable, as long
as there is no significant effect on the performance evaluation.
PT providers will be required to estimate, but not necessarily report, the
uncertainty of assigned values. This estimate may be related to the
acceptance of the assigned value (to detect, for example, unexpected
bimodality) or the determination of performance.
There will be fewer soft requirements, such as where appropriate and
should, than are currently in G13. Those clauses are being converted into
introductory text or notes.


4 UNRESOLVED ISSUES

While (so far) no major points of controversy have come up, several issues are not
yet resolved:

Several ILAC members would like the scope of 17043 to include other types of
interlaboratory comparisons, such as method evaluations and key
comparisons between national measurement institutes. There also is interest
in expanding the use of PT for inspection bodies.
There is a question of how to use the measurement uncertainty of participant
results. According to ISO/IEC 17025, accredited laboratories must estimate
the uncertainty in their measurements. This uncertainty is commonly used in
the evaluation of calibration PT, but not for testing. PT providers for testing
have the common experience that measurement uncertainty is not estimated
consistently in many laboratories, despite the requirements of ISO/IEC 17025.
There is some discussion of whether medical PT (or EQA) needs separate
requirements, considering that ISO 15189 [14] calls for PT to include pre-
analytical and post-analytical sources of laboratory error.
There is a proposal concerning requirements for traceability of the assigned
value to international or national references where possible. This has
different feasibility and importance in calibration than in testing.

Daniel W. Tholen: MS : ISO/IEC 17043: A new international standard for proficiency testing

6
There is a concern that by changing a guidance document into a requirements
standard, important information may be lost. This is especially important for
developing countries and for areas of application where PT is not common.
The informative annexes may need further revision to prevent this.
There is an additional concern that a requirements document could make it
more difficult to develop PT schemes. Since the greatest barriers for
expanding use of PT are availability and affordability, there is interest in
whether some requirements are less essential than others.


5 CONCLUSION

ISO/IEC 17043 should be completed in two or three years. There have been a lot of
changes from current standards and guidelines, but there remains a lot of work to do.
Interested parties have an opportunity to participate through their ISO and CASCO
national member bodies or liaison organizations.


6. REFERENCES

[1] ISO/IEC Guide 43:1997, Proficiency testing by interlaboratory comparisons
Part 1: Development and operation of proficiency testing schemes and Part 2:
Part 2: Selection and use of proficiency testing schemes by laboratory
accreditation bodies
[2] ILAC G13:2007 Guidelines for the Requirements for the Competence of
Providers of Proficiency Testing Schemes (2
nd
edition)
[3] ISO 13528:2005 Statistical Methods for use in proficiency testing by
interlaboratory comparison
[4] ISO/IEC 17025:2005 General requirements for the competence of testing and
calibration laboratories
[5] ISO VIM:2007 International vocabulary of basic and general terms in metrology
T
[6] ISO/IEC 17000:2004 Conformity assessment Vocabulary and general
principles
[7] ISO 3534:2006 Statistics Vocabulary and symbols, Part 1: General statistical
terms and terms used in probability and Part 2: Applied statistics
[8] ISO/IEC 9001:2000 Quality management systems - Requirements
[9] ILAC P9: 2005 Policy on Participation in National and International PT Activities
[10] ISO/IEC 17011:2004 Conformity assessment General requirements for
accreditation bodies accrediting conformity assessment bodies
[11] ISO PAS 17005:2007 Conformity assessment Use of management systems
in conformity assessment Principles and requirements
[12] ISO/IEC17021:2006 Conformity assessment Requirements for bodies
providing audit and certification of management systems
[13] The International Harmonised Protocol for the Proficiency Testing of Analytical
Chemistry Laboratories, Pure and Applied Chemistry, 78 (1), 145-196 (2006)
[14] ISO 15189:2006 Medical laboratories Particular requirements for quality and
competence