See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/318990775

A Virtual CMM Inspection Tool for Capturing Planning Strategies

Conference Paper · August 2017

DOI: 10.1115/DETC2017-67519

CITATIONS READS

2 110

5 authors, including:

Dimitrios Anagnostakis James Ritchie

Heriot-Watt University Heriot-Watt University
7 PUBLICATIONS   38 CITATIONS    152 PUBLICATIONS   1,286 CITATIONS   

SEE PROFILE SEE PROFILE

T. Lim
Heriot-Watt University
127 PUBLICATIONS   1,263 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

BEACONING View project

Games for Quantum Research View project

All content following this page was uploaded by Dimitrios Anagnostakis on 08 August 2017.

The user has requested enhancement of the downloaded file.

 Proceedings of the ASME 2017 International Design Engineering Technical Conferences and
Computers and Information in Engineering Conference
A VIRTUAL CMM INSPECTION TOOL FOR CAPTURING PLANNING STRATEGIES
Dimitrios Anagnostakis James Ritchie
Heriot-Watt University Heriot-Watt University
Edinburgh, UK Edinburgh, UK
Raymond Sung
Renishaw plc
Edinburgh, UK
ABSTRACT
One of the most challenging tasks throughout the
development and manufacturing of a product is the capturing and
formalization of engineering knowledge and expertise. In the
past, many researchers have successfully proposed different
techniques for capturing knowledge during the design, process
and assembly planning of a product. However, few efforts have
focused on applying knowledge capture to the task of product
verification for Coordinate Measuring Machine (CMM)
inspection; most of these are manual, obtrusive for the user and
time consuming since the main sources of knowledge come from
documentation such as handbooks, guides or interview
transcripts. This paper describes a tool for the automated logging
of a planner’s actions while carrying out an inspection planning
task in a virtual CMM measurement environment. The tool
involves a combination of 3D motion tracking and a post-
processor to decipher the context strategy in the form of an
inspection plan. Various representations of a captured strategy
will benefit CMM operators by providing them a tool for:
understanding planning strategies, better training methods for
inexperienced users and producing more efficient part programs
in a shorter time.
1. INTRODUCTION
Coordinate Measuring Machines (CMM) have a twofold
purpose: i) testing a product’s conformance to design and quality
specifications and ii) controlling manufacturing processes.
While CMMs enable automated precision engineering, a
bottleneck exists with respect to planning measurement
strategies and the eventual programming of CMMs. Not only do
complex parts require very good understanding of the critical
engineering requirements but also it is time consuming to plan
and execute a measurement for a complex component [1].
1 Copyright © 2017 ASME
IDETC/CIE 2017
August 6-9, 2017, Cleveland, Ohio, USA
DETC2017-67519
Theodore Lim
Heriot-Watt University
Edinburgh, UK
Richard Dewar
Renishaw plc
Edinburgh, UK
CMM inspection planning has attracted the interest of many
researchers who have developed algorithms and tools for the
automated generation of inspection plans (Computer Aided
Inspection Planning - CAIP) [2]. However, the solutions lack the
ability to capture, formalize and reuse CMM planning expertise
[3] to enrich or be integrated in a CAIP system. Consequently,
the opportunity to create a knowledge base for CMM part
programming is lost. By not having this expertise stored and
accessible for reuse, a repetitive creation of new inspection plans
is required each time a new component is under verification
slowing down the product life cycle pipeline.
To overcome this bottleneck, a virtual inspection planning
tool is proposed for the rapid generation of CMM planning
strategies, which also automatically captures and presents the
embedded process expertise in the form of inspection plans.
2. BACKGROUND
In the last two decades, virtual environments and
technologies (VR) have been widely used in the area of product
engineering [4]. The major body of literature related to VR
systems concerns engineering design [5,6] as well as the
application areas of manufacturing [7] and assembly planning
[8]. The work reported herein focuses on inspection planning for
checking the conformance of a workpiece with regards to design
intent using CMM measurement.
CMM INSPECTION PLANNING KNOWLEDGE
CAPTURE
A series of knowledge based inspection planning systems
has been proposed [2,3]. However, it is not explicitly reported
how the knowledge embedded in these systems was captured and
what are the sources. With regards to extracting knowledge from
CMM planning activities and tasks very few studies have been
carried out.
A method proposed by Barreiro et al [9,10] for inspection
planning knowledge capture from technical documentation aims
to develop high level diagrammatic representations of a
knowledge base. The elicitation approach utilized was a data
mining package for analyzing the documents.
Martinez et al [11,12] focused on the extraction and
modelling of knowledge related to planning decisions such as
best probe selection (length, type of sensor, configuration, etc.)
identified mainly from interviews, guides and handbooks. In this
work, it is not clearly stated the methodology of knowledge
elicitation process.
Sung et al [13] reported that manual knowledge capture
techniques (e.g. interviewing an expert, observation, task
interruption and questioning, post task review, structured
questionnaires, etc.) are very time consuming, require a lot of
work to analyze the recorded data while the process is usually
less effective as the participant’s creativity is interrupted.
Therefore, it is necessary to develop new techniques, less
obtrusive for quick and effective capturing of user activity to
facilitate potentially the generation of engineering knowledge.
UNOBTRUSIVE USER LOGGING AND KNOWLEDGE
CAPTURE IN ENGINEERING
Ritchie et al [14] showed the importance of non-intrusive
user logging in capturing expert knowledge and generating
instruction plans for engineering tasks such as assembly and
process planning.
A system [15] for automated capture of design knowledge
through a CAD application conducts user activity logging and
storing data in XML structures. By post processing the data files,
user activity can be analyzed for potential development of
different knowledge representations.
A haptic virtual process planning system built by Fletcher et
al [16] is capable of generating time-estimated process plans.
The user is logged while performs a machining task by moving
the haptic stylus to simulate the moves of a machining tool; the
operator’s expertise is embedded in the generated plan and
represented in form of instructions for each step of the process.
The use of a virtual reality cable harness design system [17]
was demonstrated by a novel non-intrusive method for
knowledge and information capture with user logging. The
captured expertise was formalized in multiple representations
such as: IDEF0 and DRed diagrams, plain English syntax
instructions, annotated video and combinations of them.
These studies demonstrate that 3D virtual environments in
conjunction with user logging can effectively form a framework
for developing non-intrusive and knowledge capture tools.
VIRTUAL CMM INSPECTION PLANNING
With regard to VR systems related to offline inspection
planning, one of the first attempts is reported by Stouffer and
Horst [18]. This work mostly discusses the interface
development for moving the virtual CMM inspection probe
using a real controller. Another early work by Calonego et al.
2 Copyright © 2017 ASME
[19] focused on describing the basic structure of a virtual CMM
application without further details related to the inspection
planning task.
Young and Chen [20] suggested a haptic virtual CMM to
simulate a measurement task. The system can calculate collision
free inspection paths, after the user has input the probing points
using the haptic device. Wang et al. [21] extended the previous
work with a focus on the precise accessibility analysis. The
virtual CMM proposed by Hu et al. [22] simulates the use of a
real machine with an additional capability of calculating
associated uncertainties.
Zhao [23] developed a VR-based CMM system to calculate
collision free inspection paths. Optimization of the travel
distance is achieved by incorporating a Travelling Salesman
Problem (TSP) algorithm. The CMM is operated using a
keyboard not allowing realistic simulation of the actual planning
task. A novel Virtual CMM [24] was designed based on
augmented virtuality which is controlled either by a marker-less
gesture recognition algorithm or using two joysticks. The
proposed VCMM helps in familiarization with such an
environment by eliminating any associated risks and damages
resulting from careless movements. A virtual training
environment was described by Neamtu et al [25] using some
modules of a CAD/CAM software for simulating the CMM
inspection operation.
The above systems could potentially be utilized for
capturing user rationale during CMM planning operations and
eventually extracting strategies generated during the task.
However, there is neither evidence this direction has been
considered nor any activity logging technique for capturing
expertise and decision making of users.
MOTION CAPTURE APPLICATIONS
Motion Capture (MoCap) is a well-established method to
track users and record task activities. In this section, relevant
cases from the field of engineering are briefly described showing
the effectiveness and extended utilization of the approach.
Motion tracking was employed by Ma et al. [26] to record
activity in a workplace. The data collected was analyzed to
evaluate the efficiency of a worker in performing manual
handling tasks such as general movements, grasping objects and
button pressing. The framework proposed includes a simulator
for replicating the captured motions in a digital environment
aiming to assess the workload and improve human’s body
postures during each task. Qiu et al [27] used motion capture to
record and model human performance in the assembly of car
engine’s connecting rod caps. Shanmin et al [28] built a system
consisting of optical positional trackers, data gloves and a
gyroscope for capturing and simulating human activity during a
maintenance task in the interior of an aircraft. A low-cost motion
capture system was developed using multiple Wii remote
controllers [29]. This application was tested in assembly tasks.
The collected data were then pushed to a 3D environment to
generate simulations of the assembly processes.
A cyber-physical system was proposed by Sivanathan et al
[30] to support the training of workers in a construction
environment. The task of climbing a ladder was used as case
study for this work. The data of human motion during the task
are collected using an optical motion tracking setup and then
assessed in terms of correctness of the task.
A review by Field et al [31] presented some further case
studies related to the use of motion capture systems in robotics
showing the applicability of this technique in programming of
robot, activity recognition and humanoid design.
The above studies have shown the feasibility of motion
tracking for logging user activity in different tasks. The rich data
collected can benefit modelling, simulation and eventually study
of human motion. Since this technology has been applied to very
complex studies such as human body motion modelling and
analysis, it is evident that for simpler cases the MoCap would be
effective to use.

3. EXPERIMENTAL SETUP AND PROCEDURE

The system developed aims to provide the domain of
computer aided metrology with a tool for rapid generation of
CMM inspection plans and the automatic capture of human
metrology expertise. Having unobtrusively logged user and
extracted the sequence of activities, the planned measurement
strategy can be stored, processed and analyzed to help gain a
better understanding on how CMM programmers create part
programs. Figure 1, Framework for CMM inspection planning capture.

APPARATUS The motion capture system consists of eight infrared (IR)

In this study, instead of developing a virtual model of a cameras. The IR light is reflected by a set of passive markers
CMM, a physical setup similar to routing was used to simulate attached to a stylus. The markers, placed on positions where they
virtually the operation of an inspection machine. A single stylus, are always visible, are detected by the cameras and respective
hand held and moved by the user, replicates the motion of the 3D coordinates are calculated by the associated operating
machine’s inspection stylus. Previous research [32] has proven software. With the 3D positional data provided, the position of
that manual tasks such as assembly planning are performed in the stylus tip can be calculated at all times. A third marker using
shorter times in the real world compared to a virtual environment an infrared LED is activated using a clicker device on the stylus
due to less flexibility of hand movement. That is, if a haptic (Figure 2 & Figure 3).
device was used for this experimentation, much time would be
spent in changing the view of the CAD model on the screen
causing interruptions to user’s creativity. Moreover, by using the
physical setup proposed, the real interaction of a participant with
the component is maintained; necessary for a realistic replication
of the actual inspection machine operation. The advantage of the
suggested setup using physical components allows an easy and
under real conditions virtualization of the CMM inspection
planning process. In addition, a user conducting the trial is not
urged to spend any time until becomes familiar with the
infrastructure, as happens with the use of immersive
environments, focusing only on the goal of the task to perform.
The experimental setup was completed with the use of a
motion tracking system to capture the movements of the stylus.
The data from the capture system are then post processed for Figure 2, Details of stylus. Figure 3, Experimental setup.
visualizing the inspection 3D trajectory and producing a report
with timestamps and descriptions showing the user’s sequence Additionally, for capturing inspection planning strategies,
of activity. The framework of the developed methodology is an analogue tablet (paper printed) with sub-activities in each
shown in Figure 1. block is used (Figure 4). In much the same manner, as interactive
pen display or tablet styluses the planner interacts by touching

3 Copyright © 2017 ASME

the stylus tip to an appropriate cell for each of the sub activities.
The options are categorized based on the feature-to-probe, i.e.
feature used for part alignment, feature used as a datum, feature
to inspect; then different tolerance selections are provided. The
tablet’s position is calibrated such that a specific range of 3D
coordinates represents each cell. In this way, the captured
sequence of activities can be easily identified within the stylus
motion trajectory.
Figure 4, Analogue tablet and options.
The 3D positional data captured by the motion tracking system
are stored in a CSV format file. The data represents the
coordinates of each tracked marker with the associated
timestamps. The interactive data interface is shown in Figure 5.
Figure 5, Interface of motion capture software.
EXPERIMENTAL PROCEDURE
The experimental setup was validated through several trials
using four components as shown in Fig. 6. The trials involved
the following key inspection planning tasks: part alignment,
datum definition and feature inspection for checking tolerance.
The purpose of these trials was to determine the feasibility of the
proposed setup; therefore, each trial was carried out for
inspecting just one feature and associated tolerance for each
component since the method for checking multiple tolerances is
the same.
4 Copyright © 2017 ASME
Figure 6, Components used in trials.
The steps followed in each scenario were the same as
required in a session for planning a CMM measurement strategy:
 Selection of stylus.
 Alignment of part on table.
 Probe of features for tolerance evaluation.
Regarding the selection of stylus, currently in the trials only
one stylus was involved. In future work this setting will be
changed to allow different styli configurations.
The part alignment sub-activity was pre-defined for the
beginning of each new strategy. The part alignment datum is
defined by selecting a part feature by touching with the stylus tip
followed by the selection of the relevant “datum button” on the
analogue tablet. Depending on the measurement strategy, one or
more alignment datums, or just plain points around a part, may
be selected. As the planning session continues, the user should
select the type of tolerance and then if required configure a
datum. In this case, the datum feature is selected on tablet and
probed first and then the feature type being evaluated (called the
inspection feature).
To detect probe points a specific process must be followed.
The stylus is moved above the selected inspection feature and
before each touch, the planner presses and holds the clicker on
the stylus until a contact occurs, then it is released. With this
process the infrared LED is activated so it can be tracked by the
motion capture system and used as an indication of an inspection
point. Additionally, the starting point for each contact segment
along with the approach direction are identified.
DATA PROCESSING AND STRATEGY CAPTURE
Data recorded from each trial session are exported in CSV
format and post processed to generate the final inspection plans
and inspection trajectory representations. An STL representation
of the object being measured can be used to virtually visualize
the workflow of the inspection strategy and aid the mapping
process.
The data input file consists of the 3D coordinates of stylus
tip and an index for detecting if the LED marker is active or not
with timestamps. As explained earlier the LED marker is
activated when the clicker is pressed by the user when initially
approaching a feature to probe and released at the end of the inspection plan such information allows the planning strategy to
touch. With this rule the tactical activity of probing contacts be comprehensively described.
points can be easily recognized within the rich data set.
Furthermore, the approach directions and actual points of contact
can be shown.
The use of an analogue tablet was introduced so that
strategic planning activity can be identified. The location of the
tablet in the motion capture volume was calibrated in advance.
The 3D coordinates range in the X, Y and Z directions for each
“button” of the tablet are identified through this process, forming
a 3D volume and embedded in the post-processor. In this way,
whenever the stylus tip is inside this volume the related sub-
activity indication is returned.
All these data are eventually exported in 3D graphs showing
the total motion trajectory, the inspection points with associated
start points and associated approach directions as well as the
measurement strategy depicted in an inspection plan. The results
from the four components’ trials are presented. The current work
was carried out as a pilot study of the system and comprised the
observation of only one participant who had been trained on
CMM practice and machine operation but had limited experience Figure 8, Inspection, starting points and approach directions.
in applying this. The aim of the associated trials being to test the
basic capability and practicability of the system for automated The measurement strategy is stored and represented in the
knowledge capture and planning. form of an inspection plan. Tables 1-3 present sections of the
Trial 1 inspection plan for evaluating a hole’s true position tolerance.
The first trial involved a component of relatively simple Table 1 lists the probing strategy for the alignment of the part on
geometry for testing the central bore’s position. Figure 7 shows the table using 6 touch points. Table 2 indicates the measurement
the complete motion paths of the inspection strategy. However, sequence and choice of datum feature to be used as a reference
this includes moves outside the planning path, such as the of the hole’s position. The last part of the inspection plan (Table
touches on the analogue tablet for selecting the different options. 3) lists the points probed for measuring the actual feature.

Table 1, Part alignment inspection details.

Instruction Time (sec) Description / XYZ-coord (mm)
'part align feature' 21.73 'user' 'selection' 'on tablet'
1 24.60 54.13 13.75 19.5
Instruction
2 Time (sec) Description
25.93 53.71 / XYZ-coord
57.44 (mm)
19.17
'part align3 feature' 21.73
27.40 'user' 'selection'
11.65 41.69 'on tablet'
19.54
1
4 24.60
29.40 54.13
58.13 13.75
-1.94 19.5
8.5
2
5 25.93
30.93 53.71
14.64 57.44
-1.79 19.17
10.87
3
6 27.40
32.80 11.65
-0.86 41.69
43.75 19.54
8.74
4
'true position' 29.40
37.13 58.13 -1.94
'user' 'selection' 'on8.5
tablet'
'datum 5feature' 30.93 14.64
40.40 features -1.79
'user'details.
'selection' 10.87
'on tablet'
Table 2, Tolerance
6 and datum
32.80 -0.86 43.75 8.74
7 42.87 55.72 11.91 19.32
'true position'
8 37.13
44.20 'user'
56.5 'selection'
62.49 'on tablet'
19.13
'datum9feature' 40.40
45.60 'user' 'selection'
20.44 67.07 'on tablet'
19.31
7
10 42.87
47.07 55.72
10.83 11.91
21.23 19.32
19.74
Figure 7, 3D trajectory of user activity of Trial 1. 'datum8feature' 44.20
50.00 56.5 'selection'
'user' 62.49 19.13
'on tablet'
9
11 45.60
52.53 20.44
66.01 67.07
-1.79 19.31
12.04
A second graph is produced showing only the sequence of 10
12 47.07
54.27 10.83
5.8 21.23
-2.48 19.74
12.92
inspection points numbered and colored with red dots (Fig. 8). 'datum13 feature' 50.00
55.87 'user'
7.89 'selection'
-3.26 'on tablet'
5.43
In addition, the respective start points (green dots) highlighted 11
14 52.53
57.40 66.01
66.84 -1.79
-2.13 12.04
2.9
with a yellow vector connecting the two points which represents 'datum12 feature' 54.27
59.07 5.8 'selection'
'user' -2.48 12.92
'on tablet'
13
15 55.87
61.73 7.89
-1.63 -3.26
7.47 5.43
14.9
the approach direction. By detecting each starting point the 14 57.40 66.84 -2.13 2.9
identification of a safety off-set plane can be defined to avoid 16 63.47 -1.14 71.72 12.13
'datum17 feature' 59.07
65.67 'user'
-1.68 'selection'
71.17 'on tablet'
4.8
collisions with other features close to the region of activity. The 15
18 61.73
67.40 -1.63
-2.45 7.47
7.55 14.9
8.02
information provided by this graph explicitly shows the 'inspection16 feature' 63.47
69.47 -1.14 'selection'
'user' 71.72 12.13
'on tablet'
chronology of probed points. When used in conjunction with an 17
19 65.67
73.33 -1.68
37.04 71.17
21.6 4.8
12.26
18
20 67.40
74.67 -2.45
58.6 7.55
41.96 8.02
13.99
'inspection21 feature' 69.47
75.93 'user' 'selection'
37.17 55.29 'on tablet'
12.31
19
22 73.33
77.27 37.04
18.92 21.6
43 12.26
14.6
20
23 74.67
78.80 58.6
34.57 41.96
18.37 13.99
13.48
5 21
24 75.93
80.13 Copyright © 2017 ASME
22
25 77.27
81.33 18.92
38.56 43
55.96 14.6
6.45
23
26 78.80
82.73 34.57
16.32 18.37
38.4 13.48
7.63
24 80.13 51.72 36.22 9.32
 11 52.53 66.01 -1.79 12.04 2 12.64 -39.81 -0.01 10.74
12 54.27 5.8 -2.48 12.92 3 14.27 -21.67 -19.29 10.83
13 55.87 7.89 -3.26 5.43 4 16.82 25.94 -14.08 10.8
14 57.40 66.84 -2.13 2.9 5 18.18 36.37 3.84 10.64
'datum feature' 59.07 'user' 'selection' 'on tablet' 6 19.55 21.11 19.61 10.5
15 61.73 -1.63 7.47 14.9 'part align feature' 22.09 'user' 'selection' 'on tablet'
16 63.47 -1.14 71.72 12.13 7 26.27 -20.08 6.83 4.18
17 65.67 -1.68 71.17 4.8 8 27.36 -30.72 -0.77 7.42
Table 3, Details
18 of inspection
67.40feature.
-2.45 7.55 8.02 Table 4, Inspection
9 plan28.27
details of-21.09
Trial 2. -11.03 6.89
'inspection feature' 69.47 'user' 'selection' 'on tablet' 'part align feature' 31.55 'user' 'selection' 'on tablet'
19 73.33 37.04 21.6 12.26 10 34.18 22.91 9.94 3.89
20 74.67 58.6 41.96 13.99 11 35.27 32.12 1.06 2.88
21 75.93 37.17 55.29 12.31 12 36.27 21.62 -10.39 6.91
22 77.27 18.92 43 14.6 'true position' 42.36 'user' 'selection' 'on tablet'
23 78.80 34.57 18.37 13.48 'datum feature' 46.09 'user' 'selection' 'on tablet'
24 80.13 51.72 36.22 9.32 13 49.36 -24.08 16.91 10.98
25 81.33 38.56 55.96 6.45 14 50.91 -21.9 -18.37 10.82
26 82.73 16.32 38.4 7.63 15 52.36 27.81 16.65 10.34
16 53.64 27.81 -12.22 10.37
Trial 2
'datum feature' 56.09 'user' 'selection' 'on tablet'
In the second trial, another simple component was used to 17 59.64 -19.71 9.13 6.11
for evaluating the tolerance of a hole’s true position. The same 18 60.45 -30.02 0.48 5.93
common steps were executed as previously related to part 19 61.27 -20.7 -9.83 7.16
alignment, datum feature selection and geometrical inspection 20 62.00 -10.67 0.85 5.33
characteristics. In Figures 9-10 the whole motion path, 'inspection feature' 65.09 'user' 'selection' 'on tablet'
inspection and starting points and a part of the final inspection 21 68.09 24.91 10.57 6.52
plan (Table 4) with timestamps are presented. 22 68.91 32.06 -0.95 7.18
23 69.73 24.12 -9.13 9.17
24 70.55 12.3 -2.29 7.67
25 71.55 24.16 9.57 -0.07
26 72.27 31.61 3.04 0.35
27 73.09 20.83 -9.59 3.12
28 73.82 11.5 0.49 3.09

Having extracted the plans from different measurement

strategies for the same type of tolerance (true position), a
comparison can be made related to the number and order of
datum features defined as well as the number of points used for
evaluating the specific tolerance. In this fashion, similarities can
be detected through different approaches for the scope of
creating common rules of practice and potential development of
optimal solutions based on intuitive knowledge and expertise
captured.

Trial 3
Figure 9, 3D trajectory of user activity of Trial 2. A third trial was conducted for planning cylindricity
tolerance measuring. The task carried out following the typical
steps for CMM inspection planning. The planning activity is
shown in Fig. 11 and Fig. 12 using the different formats as in the
previous trials.

Figure . 10, Inspection, starting points and approach directions . Figure 11, 3D trajectory of user activity of Trial 3.

6 Copyright © 2017 ASME

 Trial 4
A more complex geometry was chosen for the last trial. The
purpose of this was to plan an inspection for evaluating the
position of a bore. The different representations (Fig. 13-14)
produced by the post-processor are provided.

Figure 12, Inspection, starting points and approach directions.

The related inspection plan generated during the third trial Figure 13, 3D trajectory of user activity of Trial 4.
is shown in Table 5. By observing the inspection plan, it can be
seen that after the selection of under test tolerance the user
moved directly to probe the inspection feature without defining
at first a datum feature as a reference for the measurement.
Usually datum features are annotated on the design drawing and
depending on the kind of tolerance may or may not be directly
associated with a feature. However, a CMM operator might not
follow this recommendation by changing the datum or even not
selecting one at all. In this way, these automatically generated
and fully understandable inspection plans straightforwardly
reveal the sequence of inspected features followed by the human
planner whilst facilitating directly the identification of
differences.

Table 5, Inspection plan details of Trial 3.

Instruction Time (sec) Description / XYZ-coord (mm)
'part align feature' 9.00 'user' 'selection' 'on tablet'
1 12.00 -17.03 0.08 18.63
2 14.42 -4.58 -19.64 18.42
3 16.75 20.26 -0.67 18.43
'part align feature' 19.08 'user' 'selection' 'on tablet'
4 21.00 -12.13 -2.35 30.97
5 22.83 3.46 -15.77 31.72 Figure 14, Inspection, starting points and approach directions.
6 24.17 11.56 -3.63 32.34
'cylindricity' 28.75 'user' 'selection' 'on tablet' Table 6 presents the inspection plan for the last trial. As in
'inspection feature' 30.67 'user' 'selection' 'on tablet' the Trials 1 and 2, the scenario involves the measurement of an
7 33.25 -9.76 4.61 33.8 internal cylindrical feature’s position. With the aim of comparing
8 34.58 -8.71 -11.82 32.33 different planning strategies for the similar geometries, it can be
9 35.83 11.57 -7.61 33.7 observed that in each strategy a different number of datums was
10 37.25 10.24 3.54 36.03 used for testing the various features’ conformance. Therefore,
11 39.25 12.03 0.25 25.01 another piece of information and knowledge resulting from the
12 40.83 9.04 -11.53 25.44
use of these types of inspection plans are uncovered; that is, they
13 42.50 -10.23 -11.13 27.59
14 44.08 -10.91 2.75 26.86
can indicate the relationships between a feature’s geometry and
the measurement strategy.

7 Copyright © 2017 ASME

 4 17.36 -17.14 -55.33 33.23
5 18.91 24.57 -55.62 31.29
6 21.45 20.79 -53.91 20.86
7 23.27 -23.49 -54.58 21.62
'part align feature' 25.55 'user' 'selection' 'on tablet'
8 29.09 -14.6 20.38 34.43
9 30.36 -33.08 17.53 32.92
10 31.91 -37.02 -17.36 36.34
Table 6, Inspection
11 plan details
33.45 of T-12.69
rial 4. -21.21 32.98 the current stage the trials conducted, involved just one non-
'true position' 37.36 'user' 'selection' 'on tablet' expert participant. In future, multiple evaluation trials with a
'datum feature' 41.55 'user' 'selection' 'on tablet' larger number of expert participants will be carried out.
12 44.64 43.86 -27.36 39.45 Finally, it is apparent that the system developed can
13 46.18 41.03 30.14 39.5 contribute to low cost and risk free CMM planning but, also,
14 47.82 -24.8 30.82 39.82
under real conditions can potentially capture knowledge to
15 49.55 -34.92 -37.69 40.08
'datum feature' 53.18 'user' 'selection' 'on tablet'
facilitate the training of inexperienced CMM operators.
16 56.18 -18.1 -55.15 31.32
17 57.82 24.69 -53.9 29.29 5. FUTURE WORK
18 59.91 -17.98 -56.18 17.02
19 61.55 25.87 -53.93 22.04 At present, the current tool is limited to include only
'inspection feature' 66.09 'user' 'selection' 'on tablet' commonly used forms of tolerances. This can be extended to
20 70.00 -12.85 22.8 35.42 further types by adding more tolerancing indications onto the
21 71.55 -22.31 21.32 32.84 analogue tablet. Additionally, the output does not provide
22 73.09 -44.47 0.73 38.45 information about the stylus orientation for each inspection
23 74.82 -27.02 -23.13 35.62 point. Another limitation is the use of just one stylus
24 77.00 -15.4 23.76 23.32
configuration. Therefore, future research will focus on
25 78.55 -39.1 17.77 28.71
26 80.27 -37.35 -16.77 26.17
overcoming these limitations.
27 82.09 -23.07 -24.74 28.39 Another future enhancement will be the addition of
synchronized user voice recording, which will add significant
4. DISCUSSION value and granularity to the quality of the final outputs by
providing reasoning for each activity further deepening the
To address the fundamental gap in the literature regarding understanding of how a measurement strategy is planned.
the lack of a quick method for capturing inspection planning Further representations of a captured strategy will also be
strategies, this research approach was tested using single user included, such as IDEF and DRed diagrams, revealing the
four test trials on four different components. The results showed rationale behind a planning strategy.
that this approach can effectively capture and represent the As mentioned previously, the current trials were carried out
strategies and decision making used by the user during each trial. by a single limited experience user to test the basic operation and
This suggests that a motion capture technique, alongside 3D feasibility of the CMM knowledge capture system. Now that it
visualization, can be used to elicit and rapidly understand CMM has been shown to capture and represent CMM inspection
planning knowledge from human experts. A significant knowledge quickly and effectively as well as facilitate the
advantage of the current method is that by using a physical setup interrogation and understanding of planning strategies, future
to virtualize CMM inspection planning, time is not spent in trials will involve multiple participants and comparisons of
familiarizing and interacting with the environment as happens, practices between novice and experienced operators. It will also
for instance, with the use of a haptic virtual CMM model; indeed, be tested as a training aid for inexperienced users.
the real interaction of a user with the physical part is maintained
allowing the efficient and effective generation of measurement 6. CONCLUSION
strategies. User logging within motion capture has the capability of
The automatic generation of inspection plans directly from providing an intuitive and quick method of capturing data for
the captured data provides the field of computer-aided metrology analysis and investigation of engineering expertise. The work
with a new and powerful capability to rapidly generate outputs presented within this paper demonstrates the use of a non-
beneficial for CMM operators without the need for them or intrusive quick knowledge capture tool for CMM measurement
others to somehow document their planning strategy while strategies. In parallel, the system can automatically generate
conducting the actual programming task. Potentially the data inspection plans by processing the data captured as well as
used for producing the inspection plans can be embedded in a virtual geometry representations of the planning methods on 3D
product lifecycle management package for direct creation of models for further interrogation. The physical setup designed
design linked CMM part programs. minimizes user interface interactions, uninterruptedly
With regards to a contribution to the field of inspection maintaining the intuitive nature of the performed task. The
planning, the present research can be employed for direct different representations of the captured strategy will facilitate
comparison and study of different approaches facilitating the the effective study of the logged user’s planning activity
recognition of common practices in the measurement of similar contributing to the further understanding CMM inspection
components, similar geometrical tolerances or how a strategy planning strategies.
changes at a strategic and tactical level when for example the
size of a component changes.
The scope of the work presented is solely to demonstrate the
capabilities of the proposed system and show how the results can
lead to a better understanding of CMM planning strategies. At

8 Copyright © 2017 ASME

ACKNOWLEDGMENTS
This work is partially funded under the European H2020
Programme BEACONING project, Grant Agreement nr.
687676.
REFERENCES
[1] Hocken RJ, Pereira PH. Coordinate measuring machines
and systems. vol. 20. 2011. doi:10.1016/S0263-
2241(97)00037-7.
[2] Zhao F, Xu X, Xie SQ. Computer-Aided Inspection
Planning-The state of the art. Computers in Industry
2009;60:453–66. doi:10.1016/j.compind.2009.02.002.
[3] Anagnostakis D, Ritchie J, Lim T, Sivanathan A, Dewar
R, Sung R, et al. Knowledge Capture in CMM Inspection
Planning: Barriers and Challenges. Procedia CIRP
2016;52:216–21. doi:10.1016/j.procir.2016.07.045.
[4] Ritchie J. Virtual Environments — the Eco-Friendly
Approach to Product Design? In: Talabu D, Roche T,
editors. Product Engineering, Netherlands: Springer;
2004, p. 77–88.
[5] Jimeno A, Puerta A. State of the art of the virtual reality
applied to design and manufacturing processes.
International Journal of Advanced Manufacturing
Technology 2007;33:866–74. doi:10.1007/s00170-006-
0534-2.
[6] Choi S, Jung K, Noh SD. Virtual reality applications in
manufacturing industries: Past research, present
findings, and future directions. Concurrent Engineering
2015. doi:10.1177/1063293X14568814.
[7] Abdul Kadir A, Xu X, Hämmerle E. Virtual machine
tools and virtual machining-A technological review.
Robotics and Computer-Integrated Manufacturing
2011;27:494–508. doi:10.1016/j.rcim.2010.10.003.
[8] Seth A, Vance JM, Oliver JH. Virtual reality for
assembly methods prototyping: a review. Virtual Reality
2011;15:5–20. doi:10.1007/s10055-009-0153-y.
[9] Barreiro J, Martinez S, Cuesta E, Alvarez B, Fernandez
P. High level diagrams for identification of knowledge
as a basis for a KBE implementation of inspection
planning process. III Manufacturing Engineering
Society International Congress, 2009, p. 272–8.
[10] Barreiro J, Martinez S, Cuesta E, Alvarez B. Conceptual
principles and ontology for a KBE implementation in
inspection planning. International Journal of
Mechatronics and Manufacturing Systems 2010;3:451.
doi:10.1504/IJMMS.2010.036069.
[11] Martínez S, Barreiro J, Cuesta E, Álvarez BJ, González
D. Methodology for identifying and representing
knowledge in the scope of CMM inspection resource
selection. AIP Conference Proceedings 2012;1431:250–
7. doi:10.1063/1.4707572.
[12] Martínez-Pellitero S, Barreiro J, Cuesta E, Fernández-
Abia AI. KBE rules oriented to resources management
in coordinates inspection by contact. Journal of
Manufacturing Systems 2015;37:149–63.
doi:10.1016/j.jmsy.2015.07.005.
9 Copyright © 2017 ASME
[13] Sung RCW, Ritchie JM, Lim T, Liu Y, Kosmadoudi
ZOE. the Automated Generation of Engineering
Knowledge Using a Digital Engineering Tool: an
Industrial Evaluation Case Study. International Journal
of Innovation and Technology Management
2012;9:1271001. doi:10.1142/S0219877012710010.
[14] Ritchie JM, Dewar RG, Robinson G, Simmons JEL, Ng
FM. The role of non-intrusive operator logging to
support the analysis and generation of product
engineering data using immersive VR. Virtual and
Physical Prototyping 2006;1:117–34.
doi:10.1080/17452750600763947.
[15] Rea HJ, Ritchie JM, Corney JR. Automating Digital
Capture of Engineering Knowledge. Knowledge
Management 2007.
[16] Fletcher C, Ritchie J, Lim T, Sung R. The development
of an integrated haptic VR machining environment for
the automatic generation of process plans. Computers in
Industry 2013;64:1045–60.
doi:10.1016/j.compind.2013.07.005.
[17] Ritchie JM, Sung RCW, Lim T. Using VR as a Tool for
the Automated Capture and Formalisation of
Engineering Knowledge. ASME 2011 World
Conference on Innovative Virtual Reality, 2011, p. 235–
42. doi:10.1115/WINVR2011-5547.
[18] Stouffer K, Horst J. Controller Driven VRML Animation
of the Next Generation Inspection System ( NGIS ) Real-
time Controller. Intelligent Systems and Smart
Manufacturing 2001. doi:10.1117/12.417245.
[19] Calonego N, Kirner C, Kirner T, Abackerli AJ.
Implementation of a Virtual Environment for Interacting
with a Coordinate Measuring Machine. VECIMS 2004 -
IEEE International Conference on Virtual Environments,
Human Computer Interface and Measurement Systems,
Boston: 2004.
[20] Yang Z, Chen Y. Inspection path generation in haptic
virtual CMM. Computer-Aided Design and Applications
2005;2:273–82. doi:10.1080/16864360.2005.10738375.
[21] Wang Y, Chen Y, Zhang W, Liu D, Zhang R.
Accessibility analysis for CMM inspection planning by
means of haptic device and STL representation. 2009
IEEE International Conference on Virtual Environments,
Human-Computer Interfaces, and Measurements
Systems, VECIMS 2009 - Proceedings 2009:174–8.
doi:10.1109/VECIMS.2009.5068888.
[22] Hu Y, Yang Q, Wei P. Development of a novel virtual
coordinate measuring machine. 2009 IEEE
Intrumentation and Measurement Technology
Conference 2009:230–3.
doi:10.1109/IMTC.2009.5168449.
[23] Zhao L. Development of a VR – based CMM System for
Industry Training and CMM Path Planning. University
of Manitoba, 2012.
[24] Wang Y, Guo X, Sun K, Chen Y. Study on Virtual
Coordinate Measuring Machine Based on Augmented
Virtuality. IEEE International Conference on Virtual
 Environments Human-Computer Interfaces and
Measurement Systems 2012.
[25] Neamţu C, Hurgoiu D, Popescu S, Dragomir M, Osanna
H. Training in coordinate measurement using 3D virtual
instruments. Measurement: Journal of the International
Measurement Confederation 2012;45:2346–58.
doi:10.1016/j.measurement.2011.09.026.
[26] Ma L, Zhang W, Fu H-Z, Guo Y, Chablat D, Bennis F. A
Framework for Interactive Work design based on motion
tracking, simulation and analysis. Human Factors and
Ergonomics in Manufacturing 2010;20:339–52.
[27] Qiu S, Fan X, Wu D, He Q, Zhou D. Virtual human
modeling for interactive assembly and disassembly
operation in virtual reality environment. International
Journal of Advanced Manufacturing Technology
2013;69:2355–72. doi:10.1007/s00170-013-5207-3.
[28] Shanmin C, Tao N, Ke W. Motion Control of Virtual
Human Based on Optical Motion Capture in Immersive
Virtual Maintenance System. 2011 International
Conference on Virtual Reality and Visualization
10
View publication stats
2011:52–6. doi:10.1109/ICVRV.2011.24.
[29] Zhu W, Vader AM, Chadda A, Leu MC, Liu XF, Vance
JB. Wii remote-based low-cost motion capture for
automated assembly simulation. Virtual Reality
2013;17:125–36. doi:10.1007/s10055-011-0204-z.
[30] Sivanathan A, Abdel-Wahab M, Bosche F, Lim T.
Towards a Cyber-Physical Gaming System for Training
in the Construction and Engineering Industry. Volume
1B: 34th Computers and Information in Engineering
Conference, ASME; 2014, p. V01BT02A034.
doi:10.1115/DETC2014-34930.
[31] Field M, Stirling D, Naghdy F, Pan Z. Motion capture in
robotics review. 2009 IEEE International Conference on
Control and Automation, ICCA 2009 2009:1697–702.
doi:10.1109/ICCA.2009.5410185.
[32] Sung R, Ritchie J, Lim T, Medellin H. Assembly
planning and motion study using virtual reality.
ASME/AFM World Conference on Innovative Virtual
Reality 2009, Chalon-Sur-Saône, France: 2009, p. 31–8.
doi:10.1115/WINVR2009-713.
Copyright © 2017 ASME