Applied Ergonomics xxx (2010) 1e8

Contents lists available at ScienceDirect

Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo

Extending hierarchical task analysis to identify cognitive demands

and information design requirements
Denham L. Phipps a, *, George H. Meakin a, b, Paul C.W. Beatty a
a
School of Medicine, University of Manchester, Stopford Building, Oxford Road, Manchester M13 9PT, United Kingdom
b
University Department of Anaesthesia, Central Manchester and Manchester Childrens Hospitals NHS Trust, Royal Manchester Childrens Hospital,
Oxford Road, Manchester M13 9WL, United Kingdom

a r t i c l e i n f o a b s t r a c t

Article history: While hierarchical task analysis (HTA) is well established as a general task analysis method, there
Received 25 March 2010 appears a need to make more explicit both the cognitive elements of a task and design requirements that
Accepted 25 November 2010 arise from an analysis. One way of achieving this is to make use of extensions to the standard HTA. The
aim of the current study is to evaluate the use of two such extensions e the sub-goal template (SGT) and
Keywords: the skillseruleseknowledge (SRK) framework e to analyse the cognitive activity that takes place during
Task analysis
the planning and delivery of anaesthesia. In quantitative terms, the two methods were found to have
Cognition
relatively poor inter-rater reliability; however, qualitative evidence suggests that the two methods were
Human error
Information design
nevertheless of value in generating insights about anaesthetists information handling and cognitive
Anaesthesia performance. Implications for the use of an extended HTA to analyse work systems are discussed.
2010 Elsevier Ltd and The Ergonomics Society. All rights reserved.

1. Introduction analysis, whichever method is used, needs to capture both human

1.1. Background and study aim
A long-standing challenge for task analysts has been to capture
both the cognitive and the behavioural aspects of task activity. In
addition, systems engineers are increasingly calling on task analysis
to assist them in the development of user requirements for design
(Stanton, 2004; Putkonen and Hyrkknen, 2007). There has been
some debate about the extent to which hierarchical task analysis
(HTA), one of the most widely-used methods, can meet these
objectives (Annett and Stanton, 2000). It is particularly useful as
a general task analysis method because it provides a exible,
exhaustive and systematic means of identifying the behaviours that
occur during a task (Patrick, 1992). However, while Annett (2004)
argues that HTA incorporates cognition as well as behaviour,
Shepherd (2000) identied a need to consider how to represent
tasks in which the cognitive rather than the behavioural aspects
play the key role. Meanwhile, Ormerod and Shepherd (2004) have
argued that when informing the design of interactive systems, task
* Corresponding author. Present address: School of Pharmacy and Pharmaceu-
tical Sciences, University of Manchester, Stopford Building, Oxford Road, Man-
chester M13 9PT, United Kingdom. Tel.: 44 (0) 161 275 2437.
E-mail addresses: Denham.Phipps@manchester.ac.uk (D.L. Phipps), George.
Meakin@manchester.ac.uk (G.H. Meakin), Paul.Beatty@manchester.ac.uk (P.C.W.
Beatty).
factors (for example training, expertise, and workload) and opera-
tional constraints (aspects of system usage).
In order to address the concerns raised by cognitive tasks and
by design applications, some authors have proposed extensions to
the standard HTA. The aim of the current study is to evaluate two
such extensions to HTA, namely the sub-goal template and the
skillseruleseknowledge framework, when applied to a task anal-
ysis of anaesthetic practice.
1.2. Sub-goal template (SGT: Shepherd, 1993; Ormerod and
Shepherd, 2004)
The SGT was designed to produce requirements specications
from a task analysis (Richardson et al., 1998). It is intended to be
carried out as part of an HTA; after the initial task decomposition,
SGT provides a set of task elements to be appended onto any
subtasks that involve an interaction between the user and a system.
These task elements categorise the type of interaction that occurs
during a given task step, from which the information requirements
e the data or information that needs to be presented to the user e
can be inferred (see Table 1).
The authors of the sub-goal template have demonstrated its use
in plant process control (Ormerod et al., 1998) and railway super-
visory control (Ormerod and Shepherd, 2004). Ormerod et al.
(1998) evaluated the usability of SGT with novice task analysis,
and found that it was particularly helpful when analysing relatively

0003-6870/$ e see front matter 2010 Elsevier Ltd and The Ergonomics Society. All rights reserved.
doi:10.1016/j.apergo.2010.11.009

Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
2 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8

Table 1 for which there is no obvious response). Each of these levels

Task elements for SGT (adapted from Ormerod and Shepherd, 2004). involves progressively more cognitive effort, and is invoked if the
Sub-goal template Task elements Context person is unable to perform at the lower level. For example,
Act A1 Activate Make subunit operational: switch from a machine operator performing routine tasks might perform at the
off to on skill-based level with little conscious control; however, if a problem
A2 Adjust Regulate the rate of operation of a unit occurs (for example, the machine malfunctions) then the operator
maintaining on state
may initially attempt to apply a known rule, perhaps in the form of
A3 Deactivate Make subunit nonoperational: switch
from on to off established emergency procedures or an informally acquired rule
of thumb for dealing with the situation. Should this not be
Exchange E1 Enter Record a value in a specic location successful, or there is no known rule to apply, then the operator
E2 Extract Obtain a value of a specied parameter would then have to engage in more deliberate problem solving at
the knowledge-based level, which would involve drawing upon
Navigate N1 Locate Find the location of a target value
relevant knowledge to formulate a novel solution. The SRK model
or control
N2 Move Go to a given location and search it has been widely applied to the study of human performance and
N3 Explore Browse through a set of locations error; to take just one application, Reasons (1990) Generic Error
and values Modelling Framework (GEMS) framework links the levels of
performance to error types and error-provoking factors, as shown
Monitor M1 Monitor Routinely compare system state
in Table 2. A particular benet of GEMS is that it is able to account
to detect against target state to determine
deviance need for action for cognitive errors, particularly at the knowledge-based level
M2 Monitor Compare system state against (Kirwan, 1992a). However, there is some debate about the utility of
to anticipate target state to determine readiness SRK and GEMS; Kirwan (1992a,b) argues that while they are useful
for known action conceptual models, it is not clear how well they lend themselves to
M3 Monitor Routinely compare rate of change
the identication and remediation of specic errors. Walsh and
transition during state transition
Beatty (2002) and Luczak et al. (2003), though, suggest that SRK
may inform the design of user interfaces that optimise human
performance during a given task; this view is echoed by the
complex process control tasks. However, the SGT is still being proponents of ecological interface design (Rasmussen et al., 1994;
rened; the latest version incorporates changes that are intended Vicente, 1999). Either way, there are again few eld studies
to make it applicable in settings other than process control, as well examining the use of SRK within a task analysis, although Hobbs
as improving its general usability (Ormerod and Shepherd, 2004). and Williamson (2002) suggest that this would be a useful
The SGT appears to be a promising method for elucidating the strategy given their use of SRK to retrospectively classify mainte-
cognitive aspects of a task during an HTA, but there as yet few eld nance errors.
studies using the technique.
1.4. An evaluative or an analytical approach?
1.3. SkillseRuleseKnowledge framework (SRK: Rasmussen, 1983)
Annett (2002) argues that ergonomics methods can be broadly
According to the SRK framework, human performance during classied into two types. Analytical methods focus on under-
a task can be classied using one or more of three levels: skill- standing the mechanisms and processes of a work system. Evalu-
based (associated with routine, learned actions); rule-based ative methods, meanwhile, are concerned with measurement of
(associated with rule- or heuristic-driven responses to problems); specied parameters within the system. In Annetts classication,
and knowledge-based (associated with reasoning about problems task analysis, with its emphasis on the modelling of work activity, is

Table 2
Error modes and error-provoking factors in GEMS, with examples from anaesthetic practice (adapted from Reason, 1987, 1990).

Level Error types Error-provoking factors Example

Skill Skill-based slips: Recency and frequency of previous use Accidentally inserting an endotracheal
i) Inattention Environmental cues tube into the patients oesophagus
(omitted behaviour); Shared schema properties Forgetting to apply a blood pressure
ii) Overattention Concurrent plans during induction
(mistimed behaviour).
Skill-based lapses

Rule Rule-based mistakes:
i) Misapplication of good rules;
ii) Application of bad rules.
Mind set (its always worked before) Using the incorrect type of tube for intubation
Availability (rst come best preferred)
Matching bias (like relates to like)
Oversimplication (e.g. halo effect)
Overcondence (Im sure Im right)

Knowledge Knowledge-based mistakes Limitations in human problem-solving Failing to recognise the anaesthetic implications
ability, for example: of a given situation and taking a sub-optimal
i) Selectivity; course of action as a result
ii) Working memory overload;
iii) Availability (out of sight, out of mind);
iv) Problems with causality and complexity;
v) Memory cueing/reasoning by analogy;
vi) Cognitive biases (e.g. conrmatory bias);
vii) Incomplete or incorrect mental model.

Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8
considered to be characteristic of analytical techniques. The
implication of this view is that, according to Annett, the value of
task analysis is better understood in terms of its ability to provide
useful insights about the object of concern rather than its confor-
mity to the psychometric standards of reliability and validity. This
view, however, is in contrast to received wisdom in the assessment
of ergonomics methods, which assumes that all methods are
evaluative and so subject to psychometric criteria (e.g. Stanton and
Young, 1999, 2003). Against such criteria, though, task analysis is
typically (and paradoxically) found to be of high predictive validity
but low reliability. It would appear possible, then, to evaluate task
analysis in at least one of two ways: in terms of its consistency and
accuracy in revealing the true nature of a work system; or in
terms of its ability to inform enquiry about the system. For the
purposes of the current study, both approaches will be used and the
insights gained from each compared.
1.5. The anaesthetic context
The current study applies SGT and SRK to the examination of
anaesthetic practice. Anaesthesia has long been recognised as
a particularly high risk area of medicine, and as such has received
considerable attention in the human factors literature (Gaba, 2000).
One of the demands facing anaesthetists is to assimilate different
items of data and make decisions, possibly under dynamic and time
pressured conditions (Gaba, 1994). In addition, with respect to the
SRK model, Walsh and Beatty (2002) suggest that human errors
arise in the use of medical monitoring equipment because the actor
is working at an inappropriate level of performance. For example,
he or she fails to recognise that the situation has deviated from
a normal routine (which means a need to move from skill-based to
rule-based behaviour) or that the situation is sufciently novel as to
require knowledge-based behaviour (cf. Rasmussen et al., 1981,
cited in Kirwan, 1992b). Sanderson (2006) highlights the need to
take into account anaesthetists information requirements when
considering how future monitoring technology should be designed.
2. Methods
In a previous study, the authors conducted a hierarchical task
analysis of the preparation for anaesthesia, and delivery of, anaes-
thesia for surgical operations (Phipps et al., 2008). These two sets of
activities will hereafter be referred to as pre-operative tasks and
peri-operative tasks respectively. The present study evaluates the
use of SRK and SGT by novice participants (the authors themselves)
to extend the HTA generated in the previous study.
2.1. Sub-goal template analysis
The procedure for applying SGT was taken from the guidelines
provided by Ormerod and Shepherd (2004). These suggest the
following steps for applying SGT to an HTA:
1. At the point that a subtask involves an information-handling
operation (that is, receiving information, evaluating informa-
tion and acting on information) designate this subtask as an
IHO and decompose it into relevant subtasks (e.g. obtain
current state of system; compare current with target states;
make adjustment);
2. Write the plan using one of the following sequence elements:
a Fixed sequence (S1: Do X);
b Contingent sequence (S2: If a then do X; If not a then do
Y);
c Parallel sequence (S3: Do together X; Y);
d Free sequence (S4: In any order do X; Y);
Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
3
3. For each of the subtasks that comprise an IHO, assign a task
element from those listed in Table 1;
4. Determine the information requirements that are associated
with this task element, and how they are fullled in practice.
Because the current study involved a retrospective application
of SGT to an existing HTA, the procedure differs slightly from
Ormerod and Shepherds guidelines, which assume that the HTA
has yet to be completed. For the current study, DLP reviewed the
existing HTA and identied the points during the task hierarchy at
which the subtasks would have been designated as IHOs; these
were then rewritten as such using the procedure described earlier
e that is, decomposing into subtasks, writing plans and assigning
task elements. Following the assignation of task elements, GHM
was invited to comment on the data that would be used by
anaesthetists to full the information requirements in practice.
PCWB subsequently assigned task elements to the subtasks sepa-
rately from DLP, in order to provide data for inter-rater reliability
analysis.
2.2. SkillseRuleseKnowledge analysis
Each task step at the lowest level of the hierarchy was classied
according to whether it was primarily skill-based, rule-based or
knowledge-based. As a guide to making the classications, a deci-
sion tree from Embrey (1986), shown in Fig. 1, was used. This was
originally developed in the context of nuclear process control, but
the version shown here has been adapted to use terminology that is
more relevant to anaesthesia. As a general guideline for the appli-
cation of SRK in this study:
 Simple psychomotor tasks, such as operating switches, are
usually designated as skill-based;
 Monitoring and administering are usually designated as rule-
based unless no decisions are required about settings or critical
values (for example, administer 100% oxygen);
 Diagnosis and planning are usually designated as rule-based,
unless there are no rules of thumb that can be applied, in
which case they become knowledge-based.
The classication was performed initially by DLP and GHM.
Again, researcher PCWB then carried out a classication separately
from DLP and GHM.
For reasons of brevity, only a selection of the SGT and SRK output
will be shown here by way of illustration. However, the full output
is available on request from the authors.
3. Calculation
Agreement between the two raters (DLP and PCWB) on the
assignation of SGT and SRK categories to each task step was
measured using Cohens Kappa statistic, as described in Robson
(2002). Kappa (K) is calculated using the formula:
K po  pc =1  pc
where po is the proportion of agreement found and pc is the
proportion of agreement expected by chance. In order to interpret
K, the authors followed the suggestion of Bakeman and Gottman
(1986) that K should ideally be 0.7 or more, and denitely not
less than 0.4, if it is to be taken to indicate a good level of
agreement rather than one that is merely better than chance.
When using the SRK framework, it is possible to assign more
than one category to the same task step (see the Discussion section
of this paper). Where two categories have been used by both raters
4 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8

Does the task No Does the task No Does the task No

require manipulative require simple involve diagnosis or Rule based
hand skills? pattern recognition? decision making?

Yes Yes
Yes
No
Is the anaesthetist Rule based Skill based
trained in the task?
Can this be
Yes expressed as simple No
Knowledge
rules of the form If b as e d
Is the task content No A then B?
or the order of Rule based
execution stable?
Yes
Yes

Does the Does the diagnosis /

anaesthetist perform No decision require the No
Rule based Rule based
the task frequently, use of abstract
e.g. daily? knowledge (e.g.
pharmacology)?
Yes
Yes

Skill based Knowledge

ba s ed

Fig. 1. Decision tree for allocating a level of performance to each task step (adapted from Embrey, 1986).

and both are consistent, this has been treated as a single instance of
agreement. If one of the two categories matches one used by the
other rater, this was also treated as a single instance of agreement.
This is because SRK is considered to be hierarchical, such that rule-
based behaviour incorporates skill-based behaviour, and knowl-
edge-based behaviour incorporates rule-based and skill-based
behaviour (Hobbs and Williamson, 2002). It should be noted,
though, that the resulting Kappa values for SRK are more lenient
than would be the case if the categories were more strictly adhered
to (that is, for example, designating a task step as rule-based or
skill-based was considered to be different from designating the task
step as rule-based only).
- M1 (Monitor to detect a deviance) versus M2 (Monitor to
anticipate a cue) [N 11].
4.1.2. SkillseRuleseKnowledge taxonomy
For the pre-operative task steps, the SRK ratings had a kappa
value of 0.273, while the value obtained from the peri-operative task
steps was 0.377. Combining the two sets of task steps (N 166) gave
a kappa value of 0.385. The most frequent misclassications were:
- Skill-based versus rule-based [N 33];
- Rule-based versus knowledge-based [N 21].

4. Results 4.2. Qualitative interpretation of the results

4.1. Inter-rater reliability 4.2.1. Pre-operative care e sub-goal template (Table 3)

4.1.1. Sub-goal template
For the pre-operative task steps, a kappa value of 0.017 was
obtained from the sub-goal template, while the ratings for the peri-
operative task steps achieved a kappa value of 0.228. The kappa
value for both sets of tasks combined (N 144) was 0.211. The most
frequent misclassications were:
- A1 (Activate a unit) versus E2 (Extract a value) [N 27];
- A1 (Activate a unit) versus A2 (Adjust an active unit) [N 13];
As pre-operative care consists largely of information gathering,
most of the task steps were designated as extract (obtain
a specic value) or explore (browse through a set of values).
Hence, many of the questions that were raised during this part of
the analysis concerned the sources of data that were used by the
anaesthetist and how they facilitated the development of the
anaesthetic plan. For example, in order to execute task step 1.1.1
(Identify type of surgery), the anaesthetist would look at the
operating list in the rst instance, but if there is any doubt, consult
the patient, consent form, or the surgeon. Meanwhile, task steps 3.1

Table 3
Exemplar SRK and SGT analyses of pre-operative task steps.

Task step SRK level SGT element Information exchange

1.1.1 Identify type of surgery Skill E2: Extract Information is on the operating list, held in the theatre (lists organised by day and theatre).
Check with patients/parents or look at paperwork.
1.2.1 Determine age Skill E2: Extract Age e ask patient, check medical records or operating list for DOB. Weight e doesnt appear
and weight on the operating list. Patient is weighed at check-in.
1.2.4 Previous anaesthetics Rule E2: Extract Ask patient or patients parents. Check medical records e look for red ash on notes.
3.1 Choose anaesthetic drugs Rule/Knowledge N3: Explore General principle: patient requires hypnotic, analgesic, muscle relaxant and anti-emetic
(if patient requests it).
3.2 Choose delivery method Rule N3: Explore Dictated by choice of drug at step 3.1: for example, propofol is IV, while N2O and
sevouorane are inhaled.
3.3 Decide on post-operative care Rule/Knowledge N3: Explore Post-operative strategy aims to achieve: local and syetmic analgesia; anti-emesis;
disposal (ward, HDU, or ICU).

Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8
(Choose anaesthetic drugs) and 3.2 (Choose delivery method)
cover the anaesthetic plan. For each of these task steps, a general
principle was identied e for example, drugs are chosen to provide
hypnosis and analgesia as a minimum, with muscle relaxation and/
or anti-emesis if required. Hence, the SGT was able to elicit details
about both anaesthetists expertise and their information-seeking
behaviour. Some data may be particularly important if the corre-
sponding task step has a high probability or cost of error. For
example, in the previous study, task step 1.2.4 (Ask about previous
anaesthetics and family history of complications) was rated as
having a potentially high cost if a risk factor was missed. This means
that, in whatever format medical records are presented, it is
important that any previously identied issues are obvious to the
anaesthetist when he or she searches the records (for example, by
providing an indicator that is equivalent to the red ash on
existing records). The SGT also led to an identication of potential
difculties in gathering information e for example, task step 1.1.1
may be affected by inaccurate information on the operating list,
while in task step 1.2.1 (Determine patients age and weight), the
location of the relevant information varies between patients.
4.2.2. Pre-operative care e SkillseRuleseKnowledge (Table 3)
In terms of the SRK taxonomy, the anaesthetist performs at all
three levels (skill, rules, and knowledge) during pre-operative
planning. The skill-based parts of the task are those involving the
gathering of basic information (such as task steps 1.1.1 and 1.2.1).
During the actual planning of the anaesthetic (for example, task
step 3.2) the anaesthetist operates mainly at a rule-based level;
usually, the options are limited enough and the anaesthetist suf-
ciently knowledgeable that he or she can apply a heuristic approach
to decision making. However, some task steps (for example, task
step 3.1) were designated as being either rule-based or knowledge-
based. These dual designations highlight that a challenge for the
anaesthetist is to recognise when the rules of thumb are inappro-
priate for a particular case and a more deliberate approach is
required, involving assimilation of further background details
about the case at hand, or of what is commonly referred to as
principle knowledge e that is, rst principles of physiology or
pharmacology (e.g. Rasmussen, 1993). Some of this principle
knowledge might be obtained through SGT, as illustrated in task
step 3.3 (Identify post-operative care requirements).
4.2.3. Peri-operative care e sub-goal template (Table 4)
The analysis of peri-operative care included preliminary
equipment checks, induction of anaesthesia, maintenance of
anaesthesia and post-operative transfer to the recovery room.
Many of the task steps associated with equipment checks were
designated as activate (make subunit operational), adjust
Table 4
Exemplar SRK and SGT analyses of peri-operative task steps.
Task step SRK level SGT element
1.5.5 Check the cylinder pressure Skill E2: Extract/
M2: Monitor
1.8.1 Inspect the breathing system Rule E2: Extract
1.9.3 Select mode of operation Skill A2: Adjust
7.7.2 Monitor O2 concentration Rule M1: Monitor
7.8 Adjust agent concentration Skill/Rule A2: Adjust
13.1.6 Monitor gas traces Rule M1: Monitor
15.7.2 Inform recovery nurse Skill E1: Enter
Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
5
(regulate the rate or mode of an operational subunit), monitor
(compare system state against target state to anticipate a cue for
action) or extract. For these task steps, using the sub-goal
template led to the elicitation of details about how the operator
carries out machine checks. For example, task step 1.5.5 (Check the
pipeline pressure gauge) requires knowledge of the vapour pres-
sure for individual gases as well as the machine stepdown pressure,
while task step 1.8.1 (Check functionality of monitor) requires
knowledge of the monitor readings expected when sensors are
disconnected or manipulated. Therefore, future anaesthetic
machines may assist the operator by providing prompts and rele-
vant data (actual and target values) for each of the checks.
During induction and maintenance of anaesthesia, many of the
task steps were designated as activate and monitor (compare
system state against target state to detect deviance). Again, this led
to an elicitation of anaesthetists information requirements and
how these should be met by the data presented on monitors. Task
step 7.7.2 (Monitor inspired oxygen concentration) requires the
anaesthetist to ensure that the value remains within a desired
range; the monitor should therefore be designed to present the
data in such a way that he or she can make this comparison easily,
for example by using a moving-pointer xed-scale analogue
display. Task step 13.1.6 (Monitor respiration and respiratory gas
traces) appears to involve keeping track of more than one data
item. This raises the possibility that, for this task step, an object
display such as those suggested by Green et al. (1996) may be
particularly useful. Task step 7.8 (Adjust anaesthetic concentra-
tion) reveals the presence of a hierarchical control relationship.
Here, the anaesthetist wishes to control the patients physiological
state, but is unable to do this directly; rather, he or she controls the
concentration of anaesthetic gas, which affects the gas delivered to
the patient, which in turn affects the patients state, which is then
indicated to the anaesthetist via the anaesthetic monitor. There
may be a need to consider the control dynamics when designing
the interface for an anaesthetic machine, for example by including
control aids or predictive displays (Wickens, 1992).
Finally, some task steps were designated as enter (record
a value in a specic location). One such task step is 15.7.2 (Inform
recovery nurse of post-operative care plan), which occurs during
the post-operative transfer. This leads to a consideration of what
information needs to be provided by the anaesthetist, and whether
there are any ways of prompting this information (for example, by
protocols or by the design of documentation). Incidentally, while
the focus of the current study was on the technical aspects of
anaesthetic delivery, task steps that involve interprofessional
collaboration, such as 15.7.2, may also require a consideration of
non-technical issues such as the relationship between anaes-
thetists and nurses (e.g. Smith et al., 2008).
Information exchange
Target value depends on state of cylinder and saturated vapour pressure for individual gases,
but generally the machine stepdown pressure is 400 kPa.
Visual inspection. Based on anaesthetists knowledge about different types
(e.g. Mapleson A, Mapleson D, Open).
General states: circle mode and open mode. There may be additional machine states
(e.g. ventilation, pressure control).
FiO2 reading on the monitor e should be between 0.3 (30%) and 1.0 (100%).
Anaesthetist may use a proxy indicator of patient response (e.g. blood pressure) as a
basis to set the vaporiser e hence a hierarchical control relationship [vaporiser setting >
concentration > gases > patient response].
ETCO2 3.5  8 kPa, usually 5.2 approx. N2O concentration maximum of 0.7 (70%).
FiO2 at least 0.3 (30%).
Patients details, surgery done, drugs administered, uids administered, special requirements
(e.g. ICU, HDU).
6 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8
4.2.4. Peri-operative care e SkillseRuleseKnowledge (Table 4)
Many of the task steps during the equipment check are skill-
based. However, some of the steps e for example, 1.8.1 (Conrm
that the breathing system tubing is in the correct conguration) e
were designated as rule-based. As noted in Phipps et al. (2008), this
task step is vulnerable to a rule-based error: allowing an inappro-
priate tubing conguration to be used. This could arise for one of
the reasons suggested in Table 2. One design solution could be to
have the user of an anaesthetic machine conrm his or her choice of
tubing conguration before each use of the machine. Task step 1.9.3
(Select mode of operation) was designated as skill-based. Here,
a skill-based error can arise on some types of anaesthetic machine
where it is not immediately obvious to the anaesthetist whether
the machine is set to circle or open mode. In such cases it is
possible that the anaesthetist fails to notice that the machine is in
the wrong mode. One way to prevent this error occurring is to
provide the anaesthetist with a cue to check the machine setting;
another is to eliminate the behaviour completely by making the
machine operate in one mode only.
During the maintenance of anaesthesia, the anaesthetist works
at both skill-based and rule-based levels, and some task steps
(for example, 7.8: Adjust anaesthetic concentration) were desig-
nated as being both skill-based and rule-based. The challenge for
the anaesthetist at such task steps may be, as suggested earlier by
Walsh and Beatty (2002), to realise when it is inappropriate to
continue with an automatic behaviour and consider whether
a different approach needs to be taken. One error that is prone to
occur in such circumstances is stereotype xation, in which the
anaesthetist fails to recognise a deviation from a desirable range of
signals that would require a change of behaviour (Rasmussen et al.,
1981, cited in Kirwan, 1992b).
5. Discussion
As with other empirical evaluations of task analysis methods
(e.g. Stanton and Young, 1999), the ndings of this study present
something of a paradox. On one hand, the qualitative ndings
suggest that incorporating SRK and SGT into a hierarchical task
analysis produces useful insights into the nature of a data handling
task. On the other hand the quantitative results suggest concerns
about the reliability of these techniques; specically, in this case, the
extent to which different raters provide consistent classications.
To take the quantitative issues into consideration rst: on the
face of things, it would appear that the poor reliability of these
methods undermines their validity; in classical test theory terms, if
a test cannot be demonstrate to measure anything at all, then it
cannot be demonstrated to measure the correct thing. Hence, the
combination of HTA, SRK and SGT is not well-suited for use as an
evaluative method; in effect, each time it is used, it will capture
some characteristics of the task, but will not, it seems, consistently
capture all of them. As suggested in the introduction section, this
echoes the ndings of other task analyses and human performance
taxonomies (Stanton and Young, 1999). For example, Olsen and
Shorrock (2010) also found a low level of inter-rater agreement
when assessing the reliability of the human factors analysis and
classication system (HFACS). They suggest that this lack of reli-
ability precludes a valid assessment of error-provoking factors
using HFACS, and so could potentially mislead its users in their
selection of appropriate interventions. A similar argument, it
seems, could be advanced for SGT and SRK on the basis of the
current study.
Some methodological points, though, need to be made in
respect of this argument. Firstly, the authors, while familiar with
the general principles of task analysis and the SRK framework, had
limited experience in the application of these methods in tandem,
Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
and even less with the SGT. It is possible that, with further expe-
rience, the authors ratings may have become more consistent with
each other. That said, as a number of authors have pointed out
(Lyons, 2009; Olsen and Shorrock, 2010), it may well be important
that laypersons or analysts with limited expertise are able to use
these methods effectively. The authors level of experience (familiar
with HTA and with the theoretical background to SRK and SGT, but
having little practice experience of the latter) may therefore reect
their real-world use more faithfully than studies in which only the
methods originator uses it, and intra- rather than inter-rater reli-
ability is assessed.
The second point is that, as described earlier, the SGT in
particular was developed for use in industrial process control;
perhaps, then, the ndings say more about the transferability of
SGT to the medical domain than they do about its general utility. It
might be insightful to consider whether particular categories
within the SGT and SRK frameworks are more reliable than others,
perhaps because of their being easier to understand or apply.
Within the current design, this could be measured using weighted
Kappa (Cohen, 1968), although some thought would be required
about how weights should be assigned to the different categories
within each framework. For example, should those pairings which
are most likely to be misclassied have the largest weightings, or
those pairings whose misclassication would have the greatest
practical impact?
So far, the discussion has dealt with SRK and SGT as evaluative
methods, and the reliability values thereby generated. However,
treating them as analytical methods would shift the focus onto the
qualitative ndings from the current study. From this perspective,
the two methods can be seen to have some value in understanding
the process of anaesthesia delivery; regardless of the consistency
with which it was done, the exercise of applying the categories led
the raters to consider the way that information is handled by
anaesthetists and task characteristics that could affect human
performance. Hence, combining these two frameworks within
a hierarchical task analysis appears to provide a systematic means of
evaluating a work activity. A particular benet of using HTA as the
starting point is that it tends to provide an exhaustive and rigorous
description of a tasks structure, which can serve as a platform for
applying SGT and SRK (Ainsworth and Marshall, 2000).
It would be useful at this point to consider reliability and validity
in the context of what SGT and SRK are designed to provide to
analysis. The SGT focuses on describing how the operator uses
information, rather than on the nature of the information itself. The
corollary of this observation is that the information to be handled is
the same regardless of the SGT category that is applied to it; it could
therefore be argued that the consequence of a misclassication is
not that the task description is invalid, but that it suggests a sub-
optimal design solution.
Meanwhile, the SRK framework is intended as a means of clas-
sifying task behaviour into three general modes, potentially with
distinct psychological determinants, rather than as a psychological
account of task behaviour in its own right (Rasmussen, 1982, 1983;
Sanderson and Harwood, 1988). Unlike theories of skill acquisition
(e.g. Patrick, 1992), SRK does not assume that task behaviour
progresses from knowledge to rules to skills as the operator
becomes more expert at the task, merely that a given task could be
executed at any of the three levels. They are not mutually exclusive
and, while most task steps can be considered to operate primarily
(or optimally) at one level, there are some which can be assigned to
more than one level, as occurred in the current study (Vicente,
1999; Hobbs and Williamson, 2002).
As with SGT, and to revisit the argument made earlier in this
paper, the impact of misclassifying an SRK level depends on
whether the analysis is being treated in itself as a revelation of the
 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8 7

systems true nature or as a set of hypotheses about it. Potentially, for example observational data, critical incident reports or evalua-
then, the users of an analysis could be drawn into an inappropriate tions of design prototypes based on the output.
solution, as Olsen and Shorrock (2010) suggested; alternatively,
though, they could be prompted to consider and test the assump- Acknowledgements
tions that inform their choice of solutions, either at the point of
formulating them or during a subsequent design evaluation. This study was funded by the Royal Manchester Childrens
Despite the apparent benets of using SRK and SGT within a task Hospital Anaesthetic Research Fund. The authors would like to thank
analysis though, there are also some caveats that should be taken Prof. Neville Stanton, Dr David Embrey, Dr Andrew Shepherd and an
into account. As mentioned previously, a drawback of SRK and anonymous reviewer for their comments on earlier versions of this
GEMS is the lack of a clear mapping between the level of perfor- paper, which have been incorporated into the current version.
mance and observed errors; a given error can be produced by more
than one of the internal mechanisms represented by the level of
Appendix A. Confusion matrices
performance. This limits its usefulness for human reliability anal-
ysis, although it appears to be of use to human factors researchers
A1. Pre-operative care: SkillseRuleseKnowledge.
and designers (Walsh and Beatty, 2002; N.A. Stanton, personal
communication, 22 Dec 2006). A potential line of development for
human reliability applications, then, would be to generate an Observer 2
evidence base upon which to suggest links between the level of
Observer 1 Skill Rule Knowledge Total
performance and particular error types, either from eld data or
Skill 3 1 4
from a simulator study. One issue that might need exploration, for
Rule 2 6 4 12
example, is the level of the HTA at which it is most appropriate to Knowledge 1 1 2
assign a level of performance e should this be done at the level of Total 5 8 5 18
specic tasks, or at a more strategic level?
Similarly, and as alluded to previously, it is not entirely clear how
the SGT translates to information handling in a medical environment. A2. Pre-operative care: Sub-goal template.
It would appear from the current study that it is of some use in
capturing the information demands, although the process control
terminology might not map very well onto medical tasks. Again, more Observer 2
empirical work to establish the link between the SGT and actual task Observer 1 E1 E2 N1 N2 N3 M1 M2 M3 Total
activity would be useful, and consideration might need to be given as E1 1 1
to whether it can and should be adapted for use in healthcare. E2 2 2 1 1 4 7 1 18
N1
N2
N3 1 2 1 4
6. Conclusions Total 2 2 1 2 4 7 4 1 23

Lyons (2009) highlights the need for more evidence-based

guidance on the selection and use of human reliability analysis A3. Peri-operative care: SkilleRuleseKnowledge.
methods. In order to provide such guidance, she recommends
further investigation of both the measurement characteristics
Observer 2
(reliability and validity) and the practical application of the various
methods available. The current study has investigated the use of the Observer 1 Skill Rule Knowledge Total
SRK and SGT frameworks in tandem with HTA to analyse a task Skill 76 23 2 101
activity. In doing so, it has highlighted the importance of the analyst Rule 7 23 16 46
Knowledge 1 1
understanding the purpose to which the frameworks are being put Total 83 46 19 148
and the extent to which that purpose can be met. Specically, it is
important to be clear whether the methods are being used in an
analytical or an evaluative manner. A4. Peri-operative care: Sub-goal template.
The results of the current study suggest that SRK and SGT (and
other taxonomy-based methods, such as HFACS described earlier)
lend themselves better to use as analytical rather than evaluative Observer 2
methods. This is because they appear to have utility as a process for
Observer 1 A1 A2 A3 E1 E2 N1 N2 M1 M2 M3 Total
structuring enquiry about a task, but poor reliability in the assign-
A1 15 12 2 29
ment of the task to specic categories. Their output, therefore, A2 1 7 2 10
should be treated as a set of hypotheses about the nature of the task A3 6 6
and which design solutions, if any, are likely to be appropriate. E1 1 4 3 8
Where users expect the output to describe a literal truth about the E2 25 6 2 2 1 1 37
N1 1 1
task without further verication, or wish to compare a series of
M1 1 5 11 7 24
analyses of the same task, the methods described here would be less M2 3 1 2 2 2 10
suitable. It is recommended that future work be carried out to Total 44 26 7 6 6 1 7 6 13 9 125
understand further how SRK and SGT can be applied to task analysis.
The authors have identied two key lines of investigation. The rst is
factors that might inuence the reliability of these methods; for References
example, the level of analyst expertise, the type of task, or the
Ainsworth, E., Marshall, E., 2000. Issues of quality and practicability in task analysis:
categories used during the task. The second is the link between the preliminary results from two surveys. In: Annett, J., Stanton, N.A. (Eds.), Task
output of these methods and other sources of data about the task, Analysis. Taylor & Francis, London, pp. 79e89.

Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
8 D.L. Phipps et al. / Applied Ergonomics xxx (2010) 1e8
Annett, J., Stanton, N.A., 2000. Task Analysis. Taylor & Francis, London.
Annett, J., 2002. A note on the validity and reliability of ergonomics methods. Theor.
Issues Ergon. Sci. 3, 228e232.
Annett, J., 2004. Hierarchical task analysis. In: Diaper, D., Stanton, N.A. (Eds.), The
Handbook of Task Analysis in Human Computer Interaction. Lawrence Erlbaum
Associates, London, pp. 67e82.
Bakeman, R., Gottman, J.M., 1986. Observing Interaction: An Introduction to
Sequential Analysis. Cambridge University Press, Cambridge.
Cohen, J., 1968. Weighted kappa: nominal scale agreement with provision for scaled
disagreement or partial credit. Psychol. Bull. 70, 213e220.
Embrey, D.E., 1986. SHERPA: A Systematic Human Error Reduction and Prediction
Approach Paper presented at the International Topical Meeting on Advances in
Nuclear Power Systems, Knoxville, TN.
Gaba, D.M., 1994. Human error in dynamic medical domains. In: Bogner, M.S. (Ed.),
Human Error in Medicine. Lawrence Erlbaum Associates, Hillsdale NJ, pp.197e224.
Gaba, D.M., 2000. Anaesthesiology as a model for patient safety in health care. Br.
Med. J. 320, 785e788.
Green, C.A., Logie, R.H., Gilhooly, K.J., Ross, D.G., Ronald, A., 1996. Aberdeen poly-
gons: computer displays of physiological proles for intensive care. Ergonomics
39, 412e428.
Hobbs, A., Williamson, A., 2002. Skills, rules and knowledge in aircraft mainte-
nance: errors in context. Ergonomics 45, 290e308.
Kirwan, B., 1992a. Human error identication in human reliability assessment, part
1: overview of approaches. Appl. Ergon. 23, 299e318.
Kirwan, B., 1992b. Human error identication in human reliability assessment, part
2: detailed comparison of techniques. Appl. Ergon. 23, 371e381.
Luczak, H., Reuth, R., Schmidt, L., 2003. Development of error-compensating UI for
autonomous production cells. Ergonomics 46, 19e40.
Lyons, M., 2009. Towards a framework to select techniques for error prediction:
supporting novice users in the healthcare sector. Appl. Ergon. 40, 379e395.
Olsen, N.S., Shorrock, S.T., 2010. Evaluation of the HFACS-ADF safety classication
system: inter-coder consensus and intra-coder consistency. Accid. Anal. Prev.
42, 437e444.
Ormerod, T.C., Shepherd, A., 2004. Using task analysis for information requirements
specication: the sub-goal template method. In: Diaper, D., Stanton, N.A. (Eds.),
The Handbook of Task Analysis in Human Computer Interaction. Lawrence
Erlbaum Associates, London, pp. 347e365.
Ormerod, T.C., Richardson, J., Shepherd, A., 1998. Enhancing the usability of a task
analysis method: a notation and environment for requirements specication.
Ergonomics 41, 1642e1663.
Patrick, J., 1992. Training: Research and Practice. Academic Press, London.
Phipps, D., Meakin, G.H., Beatty, P.C.W., Nsoedo, C., Parker, D., 2008. Human factors in
anaesthetic practice: insights from a task analysis. Br. J. Anaesth. 100, 333e343.
Putkonen, A., Hyrkknen, U., 2007. Ergonomists and usability engineers
encounter test method dilemmas with virtual work environments. In:
Please cite this article in press as: Phipps, D.L., et al., Extending hierarchical task analysis to identify cognitive demands and information design
requirements, Applied Ergonomics (2010), doi:10.1016/j.apergo.2010.11.009
Harris, D. (Ed.), Engineering Psychology and Cognitive Ergonomics. Springer-
Verlag, Berlin, pp. 147e156.
Rasmussen, J., Pejtersen, A.M., Goodstein, L.P., 1994. Cognitive Systems Engineering.
John Wiley & Sons, Chichester.
Rasmussen, J., 1982. Human errors: a taxonomy for describing human malfunction
in industrial installations. J. Occup. Accid. 4, 311e333.
Rasmussen, J., 1983. Skills, rules, and knowledge: signals, signs and symbols, and
other distinctions in human performance models. IEEE Trans. Syst. Man Cybern
SMC-13, 257e266.
Rasmussen, J., 1993. Diagnostic reasoning in action. IEEE Trans. Syst. Man. Cybern.
23, 981e992.
Reason, J., 1990. Human Error. Cambridge University Press, Cambridge.
Reason, J., 1987. Generic Error Modelling System: A cognitive framework for
locating common human error forms. In: Rasmussen, J., Duncan, K.,
Leplat, J. (Eds.), New Technology and Human Error. John Wiley & Sons Ltd,
pp. 63e83.
Richardson, J., Ormerod, T.C., Shepherd, A., 1998. The role of task analysis in
capturing requirements for interface design. Interact. Comput. 9, 367e384.
Robson, C., 2002. Real World Research, second ed. Oxford, Blackwell.
Sanderson, P.M., Harwood, K., 1988. The skills, rules and knowledge classica-
tion: a discussion of its emergence and nature. In: Goodstein, L.P., Ander-
sen, H.B., Olsen, S.E. (Eds.), Tasks, Errors and Mental Models. Taylor & Francis,
London, pp. 21e31.
Sanderson, P.M., 2006. The multimodal world of medical displays. Appl. Ergon. 37,
501e512.
Shepherd, A., 1993. An approach to information requirements specication for
process control tasks. Ergonomics 36, 1425e1437.
Shepherd, A., 2000. HTA as a framework for task analysis. In: Annett, J., Stanton, N.A.
(Eds.), Task Analysis. Taylor & Francis, London, pp. 9e24.
Smith, A.F., Pope, C., Goodwin, D., Mort, M., 2008. Interprofessional handover and
patient safety in anaesthesia: observational study of handovers in the recovery
room. Br. J. Anaesth. 101, 332e337.
Stanton, N.A., Young, M.S., 1999. A Guide to Methodology in Ergonomics. Taylor &
Francis, London.
Stanton, N.A., Young, M.S., 2003. Giving ergonomics away? The application of
ergonomics methods by novices. Appl. Ergon. 34, 479e490.
Stanton, N.A., 2004. The psychology of task analysis today. In: Diaper, D.,
Stanton, N.A. (Eds.), The Handbook of Task Analysis in HumaneComputer
Interaction. Lawrence Erlbaum Associates, London, pp. 569e584.
Vicente, K.J., 1999. Cognitive Work Analysis: Toward Safe, Productive and Healthy
Computer-based Work. Lawrence Erlbaum Associates, Mahwah NJ.
Walsh, T., Beatty, P.C.W., 2002. Human factors error and patient monitoring. Physiol.
Meas. 23, 111e132.
Wickens, C.D., 1992. Engineering Psychology and Human Performance, second ed.
HarperCollins, New York.