International Journal of Health Care Quality Assurance

Does sustained involvement in a quality network lead to improved performance?

Jane Solomon Crispin Day Adrian Worrall Peter Thompson
Article information:
To cite this document:
Jane Solomon Crispin Day Adrian Worrall Peter Thompson , (2015),"Does sustained involvement
in a quality network lead to improved performance?", International Journal of Health Care Quality
Assurance, Vol. 28 Iss 3 pp. 228 - 233
Permanent link to this document:
http://dx.doi.org/10.1108/IJHCQA-05-2013-0054
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

Downloaded on: 02 September 2016, At: 19:45 (PT)

References: this document contains references to 8 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 217 times since 2015*
Users who downloaded this article also downloaded:
(2015),"International Benchmarking and Best Practice Management: In Search of Health Care
and Hospital Excellence", Advances in Health Care Management, Vol. 17 pp. 223-252 http://
dx.doi.org/10.1108/S1474-823120140000017014
(2004),"Paying the price: the future for Europe's media sector", foresight, Vol. 6 Iss 5 pp. 292-301
http://dx.doi.org/10.1108/14636680410562990
(1998),"Finite element analysis of transient natural convection in an odd-shaped enclosure",
International Journal of Numerical Methods for Heat & Fluid Flow, Vol. 8 Iss 2 pp. 199-216 http://
dx.doi.org/10.1108/09615539810201839

Access to this document was granted through an Emerald subscription provided by emerald-
srm:394461 []
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.

*Related content and download information correct at time of download.


IJHCQA
28,3
228
Received 8 May 2013
Revised 4 September 2013
Accepted 25 November 2013
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)
International Journal of Health
Care Quality Assurance
Vol. 28 No. 3, 2015
pp. 228-233
© Emerald Group Publishing Limited
0952-6862
DOI 10.1108/IJHCQA-05-2013-0054
The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com/0952-6862.htm
Does sustained involvement
in a quality network lead to
improved performance?
Jane Solomon, Crispin Day, Adrian Worrall and Peter Thompson
College Centre for Quality Improvement,
Royal College of Psychiatrists, London, UK
Abstract
Purpose – The purpose of this paper is to explore the effects of prolonged investment in one quality
improvement method, which are uncertain. The authors aim to examine the extent to which sustained
involvement in a quality network over five years led to improved performance against standards, and
whether improvement was achieved in areas where service staff could exercise direct control.
Design/methodology/approach – A prospective cohort design was used to examine data from
48 UK inpatient child and adolescent mental health units between 2005/2006 and 2009/2010, which had
been Quality Network for In-patient CAMHS members for two years. These were selected to remove
the initial marked increase in compliance identified in an earlier study. The main outcome measure was
compliance with organisation process standards.
Findings – Units meeting “excellent” quality status across all standards rose from seven
(14.6 per cent) to 18 (37.5 per cent). Standards for Environment and Facilities and Access, Admission
and Discharge improved the most. Units meeting the “excellent” quality status for criteria over which
staff had direct control criteria rose from 17 (35.4 per cent) to 29 (60.4 per cent) over the five-year period.
The unit modal quality status categorisation for criteria where staff had no direct control in 2005/2006
was “poor” (n ¼ 25; 52.1 per cent) but had progressed to “good” in 2009/2010 (n ¼ 24; 50.0 per cent).
Originality/value – The authors provide evidence that sustained investment in one QI method raises
service compliance against standards. Trends showed improvement for direct control standards from
“good” to “excellent” levels and improvement for no direct control from “poor” to “good”.
Keywords CAMHS, Quality improvement networks
Paper type Research paper
Introduction
Since the 1980s, the UK healthcare system has seen many quality improvement (QI)
methods introduced (Walshe, 2009). These range from those such as clinical audit,
which follow a cyclical approach to QI and which measure service performance against
recognised standards; to Lean and Six Sigma methods, which examine processes and
try to minimise errors or deficits. There is some evidence that QI method effectiveness
depends on the context in which it is used and how it is implemented, rather than the
specific method chosen (Walshe and Freeman, 2002). It is suggested that more can be
gained by adopting and sticking to a QI method, developing skills and experience in
its use and building up engagement, commitment and organisational capacity in its
application (Walshe, 2009). We know, for example, that several clinical audits may be
needed to achieve improvements and these may take several years. Quality
improvement initiatives and clinical audit in particular are often criticised for being
abandoned too early and before the desired gain is achieved. It makes sense to persist
until improvements are evident, but this too may not be enough; measurement and
communication must be continued to sustain gains (NHS Institute for Innovation and
Improvement, 2011). These arguments are compelling, but little research has been
 conducted into QI initiatives’ longer term effects. It is also believed that organisational Involvement
change occurs owing to a combination of internal and external factors (Greenhalgh in a quality
et al., 2004) and that there needs to be organisational improvement to achieve better
compliance against standards. Two main internal influences are the clinical team’s
network
readiness to change and attitudes towards adopting innovation. It is believed that
standards dependent on internal control are more likely to improve. Our paper,
therefore, has two aims, to: examine if sustained involvement in a quality network leads 229
to improved performance; and investigate areas in which improvement has and has not
been achieved and the reasons for these outcomes.

Research question and method

The Quality Network for Inpatient Child and Adolescent Mental Health Services (QNIC)
was established in 2001 to improve service quality. The QNIC project began with
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

35 unit members (36 per cent of all UK units), which increased to 100 units in 2010
(96 per cent of all UK units). The QNIC approach applies standards in annual self- and
external peer-review. Standards address eight practice areas: environment and
facilities: staffing and training; access, admission and discharge; care and treatment;
information, consent and confidentiality; young people’s rights and safeguarding
children; clinical governance; and commissioning. There are 51 standards and 306
associated criteria. The self-review requires CAMHS inpatient teams to rate themselves
against each standard. Peer review is followed by a one day visit by an external
multidisciplinary team (three to four true peers). The aim is to obtain a comprehensive
view of practice within each unit and validate the unit’s self-review ratings. Meetings
are held with senior clinical and managerial staff, ward staff, service users and
parents/carers to explore practice. The findings from self and peer review components
are collated into a report that unit staff can use for planning.

Evaluating sustained involvement

A prospective cohort design was used to examine data relating to UK CAMHS
inpatient units for the five-year cycle between 2005/2006 and 2009/2010, which had
been members for the previous two years. In the 2005/2006 cycle, 59 from 84 units
had been QNIC members for two or more years. These were selected to establish
sustained investment effects and to enable the first two years’ data to be removed.
This enabled us to focus on sustained involvement after the initial marked increase in
compliance noted in an earlier study (Landon, 2011). Units included NHS and
independent provision. A total of 34 standards relating to 84 criteria were stable across
the five years. The remaining 17 standards and 217 criteria were subject to significant
revision or were developed during this period and were excluded. To analyse unit
performance against the criteria, all criteria rated as partly met were marked down,
so that there was a two-item rating scale, either met or not met. Each criterion was
categorised independently for direct unit control/no direct unit control by two senior
QNIC staff, with coding differences reconciled by negotiation. Direct unit control was
defined as: no additional financial, environmental, staffing or other resources required;
and no co-operation or contribution from external agencies required; e.g., community
CAMHS teams. In total, 50 criteria were coded as direct unit control (e.g. Criterion
16.2: the inpatient unit has written admission criteria) and 34 no direct unit control
(e.g. Criterion 3.3: all young people have the choice of a single bedroom). A unit’s quality
status was categorised as “excellent” if staff met 85 per cent or more criteria, a “good”
unit was 65 and 84 per cent and a “poor” unit below 64 per cent. On a normal
 IJHCQA distribution curve, the cut off for an “excellent” unit roughly equates to one standard
28,3 deviation above or below the mean and a “good” unit roughly equates to between
0.5 and 1 standard deviation.

Results
Does sustained involvement lead to improved compliance with standards?
230 Of the 59 units, 11 were excluded, either because they had closed or did not participate
in each of the interim years. Units meeting “excellent” quality status across all
standards more than doubled over the five-year period, rising from seven (14.6 per cent)
to 18 (37.5 per cent), (Table I). Over the same period, units rated “poor” halved, from
11 units (22.9 per cent) to six (12.5 per cent). Overall, “good” units also fell, which was
mainly accounted for by the 11 (22.9 per cent) units improved to “excellent” status in
2009/2010 (Table II). In comparison with 2005/2006, 14 units (29.2 per cent) progressed
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

to “excellent” quality status. Seven units (14.6 per cent) deteriorated in status, three
(6.3 per cent) from “excellent” to “good” and four (8.3 per cent) from “good” to “poor”.
Among units rated “poor” in 2005/2006, only two (4.2 per cent) remained in the “poor”
category in 2009/2010; the remaining nine units improved to “excellent” (n ¼ 3; 6.3
per cent) and to “good” (n ¼ 6; 12.5 per cent).
For each standards section there were more “excellent” units in 2009/2010 compared to
2005/2006 (Table I). “Poor” units fell over five years in all sections, apart from young
people’s rights and safeguarding children, which saw five “poor” units (10.4 per cent) at
each time point and included one unit remaining “poor” at both time points. The standards
relating to information, consent and confidentiality had 40 units (83.3 per cent) achieving
an “excellent” quality status in 2009/2010, and environment and facilities saw over a
300 per cent increase in “excellent” units over five years, from units (10.5 per cent) to
21 units (43.8 per cent). There were smaller increases in a unit’s quality status for
standards relating to staffing and training, care and treatment, and clinical governance.
How does a unit’s quality status change where there is direct unit control compared to
no direct control?
Units meeting “excellent” quality status for direct control criteria rose from 17 (35.4 per
cent) to 29 (60.4 per cent) over five years (Table II). Units meeting “good” quality status
fell from 29 (60.4 per cent) to 14 (29.2 per cent) because 16 units improved to “excellent”
quality status (Table III). However, “poor” quality status units rose from two (4.2
per cent) to five (10.4 per cent) over the same period, which included three previously

2005/2006 2009/2010
Poor Good Excellent Poor Good Excellent

Environment and facilities 25 (52.1%) 18 (37.5%) 5 (10.5%) 11 (22.9%) 16 (33.3%) 21 (43.8%)

Staffing and training 13 (27.1%) 25 (52.1%) 10 (20.8%) 9 (18.8%) 23 (47.9%) 16 (33.3%)
Access, admission and
discharge 20 (41.7%) 17 (35.4%) 11 (22.9%) 13 (27.1%) 11 (22.9%) 24 (50.0%)
Care and treatment 10 (20.8%) 25 (52.1%) 13 (27.1%) 6 (12.5%) 25 (52.1%) 17 (35.4%)
Information, consent and
confidentiality 4 (8.3%) 5 (10.4%) 39 (81.3%) 2 (4.2%) 6 (12.5%) 40 (83.3%)
Table I. Young people’s rights and
Quality status: safeguarding children 5 (10.4%) 18 (37.5%) 25 (52.1%) 5 (10.4%) 7 (14.6%) 36 (75.0%)
2005/2006 and Clinical governance 19 (39.6%) 13 (27.1%) 16 (33.3%) 11 (22.9%) 16 (33.3%) 21 (43.8%)
2009/2010 Overall 11 (22.9%) 30 (62.5%) 7 (14.6%) 6 (12.5%) 24 (50.0%) 18 (37.5%)
 Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

Status
2005/2006 Poor Good Excellent
Status Improved to Improved to Deteriorated to Improved to Deteriorated to Deteriorated to
2009/2010 Maintained good excellent poor Maintained excellent poor good Maintained

Direct
control 1 (2.1%) 0 (0%) 1 (2.1%) 3 (6.3%) 10 (20.8%) 16 (33.3%) 1 (2.1%) 4 (8.3%) 12 (25.0%)
No direct
control 11 (22.9%) 10 (20.8%) 4 (8.3%) 2 (4.2%) 13 (27.1%) 5 (10.4%) 0 (0%) 1 (2.1%) 2 (4.2%)
Total 2 (4.2%) 6 (12.5%) 3 (6.3%) 4 (8.3%) 15 (31.3%) 11 (22.9%) 0 (0%) 3 (6.3%) 4 (8.3%)
Involvement

2009/2010
network
in a quality

2005/2006 to
Quality status –
change from
Table II.
231
 IJHCQA “good” units and one “excellent” unit. In total, 17 units (35.4 per cent) improved their
28,3 quality status on direct unit control criteria.
The modal quality status categorisation for no direct unit control criteria in
2005/2006 was “poor” (n ¼ 25; 52.1 per cent) but had progressed to “good” in 2009/2010
(n ¼ 24; 50.0 per cent), (Table III). Over this period, “excellent” status units on no direct
control criteria increased from three (6.3 per cent) to 11 (22.9 per cent) and “poor” status
232 units declined from 25 (52.1 per cent) to 13 (27.1 per cent)(Table III). In total, 19 units (39.6
per cent) improved their quality status on no direct unit control criteria, ten (20.8 per
cent) improved from “poor” to “good”, four (8.3 per cent) from “poor” to “excellent” and
five (10.4 per cent) from “good” to “excellent” (Table II). At the same time, quality status
in three units deteriorated; two (4.2 per cent) previously “good” units deteriorated to
“poor” and one (2.1 per cent) previously “excellent” unit deteriorated to “good” status.
Among 25 units starting in the “poor” quality status category in 2005/2006,
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

11 (22.9 per cent) remained in this category over five years. In the remaining 14 units,
ten (20.8 per cent) improved, moving into the “good” category and four (8.3 per cent)
progressed from “poor” to “excellent” quality status.

Conclusions and recommendations

We provide preliminary evidence that sustained QNIC investment improves service
compliance against the network’s standards. Overall, units reaching “excellent” rose
from 14.6 to 37.5 per cent; whereas “poor” units decreased from 22.9 to 12.5 per cent.
The largest change in service compliance was seen in the QNIC standards relating to
environment and facilities, and access, admission and discharge. In contrast, there was
a disappointing increase in compliance with standards that cover care and treatment.
This raises questions about the extent to which staff are able to implement clinically
effective processes and practice such as evidence-based interventions and clinical
outcome measurement. Over the study’s five-year period, there were policy drivers
at national and local levels in the UK to provide age appropriate environments
(National Institute for Mental Health in England, 2009) and how emergency admissions
were accepted (Department of Health, 2004). These policy priorities may have helped to
encourage and sustain improvement in these particular service standards (World Health
Organisation, 2003; Greenhalgh et al., 2004). At the start of the five-year period, there
were few units with “poor” compliance in relation to direct control standards, whereas,
over half had “poor” compliance with no direct control standards. Improvement was seen
over the five-year period in both categories, with trends for direct control standards
improving from “good” to “excellent” levels and no direct control improving from “poor”
to “good”. These trends may reflect internal drivers and control in service improvement
(Greenhalgh et al., 2004), although interpreting these results is difficult because starting
points for modal unit status differed in 2005/2006.
Our findings should be interpreted cautiously because the study has methodological
limitations. Absent controlled comparisons makes it difficult to know whether the
increase in standards compliance and unit quality status was due to QNIC

2005/2006 2009/2010
Table III. Poor Good Excellent Poor Good Excellent
Quality status
and direct unit Direct unit control 2 (4.2%) 29 (60.4%) 17 (35.4%) 5 (10.4%) 14 (29.2%) 29 (60.4%)
control criteria No direct unit control 25 (52.1%) 20 (41.7%) 3 (6.3%) 13 (27.1%) 24 (50.0%) 11 (22.9%)
 participation. Furthermore, although the QNIC network system required independent Involvement
peer review, our study relied on unit level self-report data, which was subject to only in a quality
partial verification at peer review. Whilst bearing these limitations in mind, the study
provides evidence to show that using a robust, well organised and nationally available
network
QI method can deliver increased compliance with service standards. Given the
improvements with compliance that were observed, some questions can be raised about
potential ceiling effects when significant units reach excellent compliance levels. 233
In accordance with recommendations from the literature (World Health Organisation,
2003), QNIC has developed an accreditation system that will involve greater
independent scrutiny. Our study has identified units that have been able to maintain
their “good” or “excellent” service standards compliance over the five years. Perhaps
more importantly, the study has also identified units that have been able to progress
from “poor” to “good”, or even “excellent”. We also identified large increases in
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)

compliance with standards in environment and facilities, access, admission and

discharge. Further work is required to understand more about how unit staff achieved
success, which will particularly valuable as it provides further understanding of how
staff can improve service quality.

References
Department of Health (2004), National Service Framework for Children, Young People and
Maternity Services – Standard 9: The Mental Health and Psychological Well-being of
Children and Young People, Department of Health, London.
Greenhalgh, T., Robert, G., Bate, P., Kyriakidou, O., Macfarlane, F. and Peacock, R. (2004), How to
Spread Good Ideas: A Systematic Review of the Literature on Diffusion, Dissemination and
Sustainability of Innovations in Health Service Delivery and Organisation, National
Coordinating Centre for the Service Delivery and Organisation, London.
Landon, G. (2011), “Quality improvement in inpatient CAMHS 2001-10”, poster presentation,
International Forum in Quality and Safety in Healthcare, Amsterdam.
National Institute for Mental Health in England (2009), The Legal Aspects of the Care and
Treatment of Children and Young People with Mental Disorder: A Guide for Professionals,
Department of Health, London.
NHS Institute for Innovation and Improvement (2011), NHS Sustainability Guide, NHS Institute
for Innovation and Improvement, Coventry, available at: www.institute.nhs.uk/
sustainability_model/general/welcome_to_sustainability.html (accessed 3 November 2014).
Walshe, K. (2009), “Pseudoinnovation: the development and spread of healthcare quality
improvement methodologies”, International Journal of Quality in Health Care, Vol. 21 No. 3,
pp. 153-159.
Walshe, K. and Freeman, T. (2002), “Effectiveness of quality improvement: learning from
evaluations”, Quality and Safety in Health Care, Vol. 11 No. 2, pp. 85-87.
World Health Organisation (2003), Quality Improvement for Mental Health, Mental Health Policy
and Service Guidance Package, World Health Organisation, Geneva.

Corresponding author
Dr Adrian Worrall can be contacted at: aworrall@rcpsych.ac.uk

For instructions on how to order reprints of this article, please visit our website:
www.emeraldgrouppublishing.com/licensing/reprints.htm
Or contact us for further details: permissions@emeraldinsight.com
 This article has been cited by:

1. Girish Karunakaran Nair, Nidhi Choudhary. 2016. The Impact of Service Quality on Business
Performance in Qatar-Based Hotels: An Empirical Study. The Journal of Hospitality Financial
Management 24:1, 47-67. [CrossRef]
Downloaded by Universiti Putra Malaysia At 19:45 02 September 2016 (PT)