LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40

J Nurs Care Qual

Vol. 00, No. 00, pp. 17
Copyright c 2016 Wolters Kluwer Health, Inc. All rights reserved.

A Measure of Lean Quality

Improvement for Hospital Staff
Nurses
Sheila Serr Roszell, PhD, RN, BC; Mary R. Lynn, PhD, RN

In this study, a measure of Lean management was developed and tested. Items were identified
using the Delphi technique with literature review and expert responses. Twenty-five nurses pilot-
tested the instrument and then 212 nurses in 5 hospitals completed the instrument, and their
responses were subjected to exploratory factor analysis. The 75-item instrument includes 12 factors
describing Lean management conceptualization. Reliability and validity are acceptable for a new
instrument. Key words: instrument development, Lean management, process improvement,
quality improvement, quality management

T HERE is a growing body of evidence that

the tools and philosophy of Lean man-
agement improve quality, safety, cost, and
delivery of care in health care systems. The
Institute of Medicine recommends that hos-
pitals optimize operations by continuously
working to reduce waste, streamline care de-
livery, and focus on activities that improve
patient health to achieve the best care at
lower cost.1 The Joint Commission recom-
Author Affiliations: Department of Performance
Improvement, University of North Carolina Health
Care System, Chapel Hill (Dr Roszell); and Nursing
Health Care Systems, University of North Carolina
School of Nursing, Chapel Hill (Dr Lynn).
The study was partially funded by a dissertation grant
from Alpha Alpha Chapter of Sigma Theta Tau Interna-
tional.
The authors declare no conflicts of interest.
Supplemental digital content is available for this article.
Direct URL citations appear in the printed text and are
provided in the HTML and PDF versions of this article
on the journals Web site (www.jncqjournal.com).
Correspondence: Sheila Serr Roszell, PhD, RN, BC, De-
partment of Performance Improvement, University of
North Carolina Health Care System, Chapel Hill, NC
27514 (sroszell@unc.edu).
Accepted for publication: February 15, 2016
Published ahead of print:
DOI: 10.1097/NCQ.0000000000000185
mends Robust Process Improvement methods
and tools to improve the quality and safety
of health care. These tools include Lean Six
Sigma (LSS), change management methodolo-
gies, and tools for high reliability.2 Chassin
and colleagues3,4 defined a series of incremen-
tal changes that hospitals should undertake to
become high-reliability organizations includ-
ing the deployment of LSS process improve-
ment tools. Reducing waste provides more
time for patient care, creating an environment
that improves the quality of care.
Determining whether care is improved by
Lean management has been difficult, in part,
because there is no measure of the degree of
Lean management presence in a hospital, on a
nursing unit, or in the actions taken by the in-
dividual nurse. This makes it difficult, if not im-
possible, to show a relationship between the
presence of Lean management and health care
outcomes. This study, therefore, developed
and tested such an instrument, the Frontline
Improvement Thinking (FIT) measure, which
is designed to assess nurses perception of
the degree of implementation of Lean man-
agement in health care organizations, units,
and their individual practices.
BACKGROUND
In a systematic review of the literature,
Albliwi et al5 found a significant need for a
1

Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40
2 JOURNAL OF NURSING CARE QUALITY/00 2016
measurement system for LSS performance;
the authors concluded that most organiza-
tions failure is due to the lack of an LSS
maturity model. In other systematic reviews,
DelliFraine et al6 and Brackett et al7 ex-
amined the Lean management research to
determine whether it supported LSS as an
evidence-based management approach. The
researchers concluded that there is little solid
evidence of the effectiveness of LSS; cur-
rent trends in evidence-based management
are based on conceptual arguments rather
than empirical research. Furthermore, Groene
et al8 noted the absence of well-established in-
struments measuring implementation of hos-
pital quality management systems and called
for further research. They also noted that
few instruments link quality management con-
structs to outcomes.8 Measuring the presence
of Lean management in the workplace and
linking it to hospital outcomes would provide
evidence that Lean management has an im-
pact on outcomes.
To find out ultimately whether Lean man-
agement is related to enhanced care, a mea-
sure is needed to capture the variation that
may be responsible for better or worse out-
comes. Consequently, the FIT measure at-
tempts to identify the variation in nurses Lean
knowledge and to quantify Lean quality im-
provement (QI) efforts. The 75-item FIT in-
strument includes items in 3 dimensions: the
organization, patient care unit, and individual
nurse thinking. The FIT instrument includes
12 factors: Learning Organization, Front-
line Involvement, Manager Support, Men-
toring, Patient-Centered Focus, Visual Man-
agement, and an individual nurse who Takes
Action, Solves Problems, Reduces Variation,
Uses Tools, Finds Opportunity, and has a
Mind-set for Change.
METHODS
This study was carried out in 3 phases: (1)
domain identification, assessment of content
validity, response formatting, and item gener-
ation; (2) peer review and pilot testing; and
(3) field testing, factor analysis, and analysis
Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
of the instruments psychometric properties.
The first 2 phases are described here and the
third phase in the Results section.
Phase 1
Domains were identified through a litera-
ture review of Lean concepts, philosophy,
and tools; characteristics of a Lean unit;
and analyses of measures used in industries
other than health care. Next, consultation was
sought from Lean experts using the Delphi
method, which incorporates multiple itera-
tions (rounds) to develop and refine consen-
sus on a specific topic.9,10
Ten experts (Delphi panelists) were se-
lected for their knowledge of Lean manage-
ment in health care. They included QI leaders
who had initiated and sustained Lean manage-
ment programs, nurse directors of Lean man-
agement programs, and Lean management
health care consultants. Each of the Delphis
3 rounds used an electronic link to send the
panelists instructions and a survey; their re-
sponses were returned for further comment
and refining.
The first round included 4 open-ended
questions about Lean management, such as:
How would a nurses actions on an ad-
vanced Lean patient care unit differ from
those on a unit that has no Lean patient care in-
fluences? The panelists responses were syn-
thesized into 121 ideas along with 76 ideas
gleaned from the review of the literature.
These 197 ideas were written as items and
presented in a structured questionnaire for
the second Delphi round. Panelists rated each
item on a scale of 1 to 5 for relevance us-
ing 3 anchors, not relevant to Lean, moder-
ately relevant to Lean, and fully relevant to
Lean, with arrows (= >) at 2 and 4, to indi-
cate progressively greater relevance to Lean.
To analyze the responses, items were ranked
in descending order by the median of the ex-
perts responses. Experts also were asked for
comments on the items. The suggestions were
used to reword items or give ideas back to the
experts for more feedback in the third round.
The panelists further refined the list in
the third round. Because there are 3 levels
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40
A Measure of Lean Quality Improvement for Hospital Staff Nurses
at which Lean thinking has an impactthe
organization, unit, and individual nursethe
ideas were organized into sets, one for each
level. The panelists assessed the content of the
items and evaluated the items for content va-
lidity using Lynns11 4-point content validity
index. This index requires items to achieve
an agreement of 0.80; thus, 8 of 10 experts
needed to rank an item with a response of 3
or 4, or the item was revised or eliminated. Af-
ter the third and final round, there were 118
items ready to be administered to staff nurses
in phase 2.
Phase 2
The aims of phase 2 were both to as-
sess content validity and to again ensure that
the instrument was readable, clear, and un-
derstandable. The authors university insti-
tutional review board approved the study.
Nurses (n = 25) from local hospitals that
used Lean methods as improvement methods
were asked to evaluate the items. They gave
feedback about relevance to their practice,
adequacy of the items in capturing the com-
ponents of Lean management on the clinical
unit, and appearance of the electronic instru-
ment, including its readability and formatting.
Using a 5-point scale, nurses indicated how
much each item described the Lean improve-
ment in their work situation.
Readability of the instrument was assessed
using software found at the Readability For-
mulas webpage.12 The measure was at an
eighth-grade reading level. At this point, the
measure was named Frontline Improvement
Thinking (FIT).
Phase 3
Phase 3, field testing, served as the basis for
the psychometric analysis of the FIT. The goal
was to obtain a minimum sample size of 300
participants, as recommended by Pett et al.13
Hospitals that reported using Lean manage-
ment were identified. Recruitment letters and
e-mails were sent to the nursing chief nursing
officer or nurse researchers at each hospital.
If there was a response to the recruitment ma-
terials, the respondee was asked to distribute
an electronic link to the FIT instrument to
Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
3
staff nurses through a listserv or e-mail group.
Nurses who agreed to participate followed an
electronic link to a protected survey Web site.
Test-retest reliability was assessed using a sam-
ple (n = 35) of nurses who indicated willing-
ness to participate twice. These nurses were
contacted 2 weeks after their first FIT com-
pletion; an anonymous code was used to link
the 2 administrations.14
RESULTS
Acquiring the sample for field testing was
more difficult than expected. More than 500
nurses responded, but only 5 hospitals had
an adequate response rate. While the sample
of 212 responses fell short of the initial goal
of 300 subjects, analyses of the sampling ad-
equacy for the data were good, and thus it
was deemed that a sufficient number of com-
pleted responses had been received for initial
analysis of the FIT. Table 1 describes charac-
teristics of the nurses who completed the FIT
instrument.
Psychometric analysis
The organization, unit, and individual
factors, items, and loading are reported in
Supplemental Digital Content, Tables 1, 2,
and 3 (available at: http://links.lww.com/
JNCQ/A252, http://links.lww.com/JNCQ/
A253, and http://links.lww.com/JNCQ/A254,
respectively). Principal axis factoring was
chosen as an extraction method to avoid over-
estimating the number of factors or the items
loading on factors.15 Data were analyzed
using an oblique rotation, which maximized
loadings for correlated factors. The pattern
matrix was examined for factor/item load-
ings. Interpretability was determined by
looking for common themes suggested by
the review of the Lean and QI literature.
Although Costello and Osbourne15 suggest
loadings of 0.35 with at least 0.15 difference
in cross-loadings, each FIT factor had higher
loadings; 0.40 was used to select the final
items. The result was a 12-factor, 75-item
instrument. Table 2 provides a summary of
psychometric data for all factors.
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40

4 JOURNAL OF NURSING CARE QUALITY/00 2016

Table 1. Demographic Data of Nurses and Their Units

Individual
Nurse Lean Methods
Experience Used on
Worked as With Lean or Hospital
Nurse Nurse, Mean QI, Mean Work Unit,
Hospital Responses, n (SD), y (SD), y Mean (SD), y

Hospital A 10 13 (1.4) 3 (0) 3 (0)

Hospital B 24 18.5 (13) 7.2 (8.5) 6.3 (8.5)
Hospital C 12 13.6 (14.8) 3 (3.0) 7.8 (5.6)
Hospital D 20 15 (13.4) 5.8 (9.2) 7.3 (4.7)
Hospital E 146 12.7 (9.9) 6.2 (8.1) 4.1 (5.3)
Total 212

Abbreviation: QI, quality improvement.

Organization Despite low reliability estimates (R = 0.61),

The first 12 items in the measure address
the organization. Factor 1 was named Learn-
ing Organization: it includes 5 items such
as Everyone in the organization is encour-
aged to get training on improving processes.
internal consistency was high ( = .87), and
the content of the items fits together well.
Factor 2, Frontline Involvement, includes 5
items such as Leaders of the organization in-
clude frontline staff like me, when needed,

Table 2. Summary of Psychometric Data for All Factors

Test-Retest
Estimates
Number Interitem
FIT Section of Items Factor Name Cronbach Correlation Pearson r

Organization 5 Learning .87 0.44-0.72 0.28 0.61

Organization
5 Frontline .79 0.30-0.65 0.37 0.68
Involvement
Unit 9 Manager Support .94 0.41-0.76 0.31 0.70
7 Mentoring .90 0.46-0.72 0.28 0.66
11 Patient-Centered .88 0.26-0.63 0.30 0.53
Focus
2 Visual Management .86 0.76 0.50 0.77
Individual 9 Takes Action .88 0.26-0.65 0.33 0.35
12 Solves Problems .91 0.25-0.66 0.33 0.46
2 Reduces Variation .94 0.89 0.17 0.28
3 Uses Tools .75 0.42-0.60 0.17 0.38
2 Finds Opportunities .66 0.50 0.27 0.38
8 Mind-set for Change .84 0.26-0.59 0.32 0.38
Total 75

Abbreviation: FIT, Frontline Improvement Thinking.

Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40
A Measure of Lean Quality Improvement for Hospital Staff Nurses
to problem-solve. Despite relatively low test-
retest reliability scores (R = 0.68), internal
consistency was good ( = .79), well above
the 0.70 Cronbach expected for a mea-
sure in the early stages of development. The
items and factor loadings for the FIT organi-
zational factors are reported in Supplemental
Digital Content, Table 1 (available at: http:
//links.lww.com/JNCQ/A252).
Unit
The next section of the instrument focuses
on the nurses experience with Lean improve-
ment thinking on the nursing unit. The 9 items
of factor 1 center on Manager Support; an
example of an item is as follows: My man-
ager is receptive to new ideas. Internal con-
sistency was high ( = .94), and test-retest
reliability was acceptable (R = 0.70). Seven
items encompass the concept of Mentoring
(factor 2), such as: There is an education
program to teach nurses about process im-
provement. Despite relatively low test-retest
reliability scores (R = 0.66), internal consis-
tency ( = .90) was high. The items fit loosely
around mentoring or education on the unit
and differ from the organizational learning
factor.
Factor 3, called Patient-Centered Focus,
contains 11 items. A sample item is The
patient care on my unit is performed in a
systematic way. Since Lean managements
philosophy prioritizes process improvement
for the customer, these 11 items may prove
to be some of the most important items for
nurses. Once again, the Cronbach was high
( = .88) and test-retest reliability adequate
(R = 0.53). The fourth factor embraced Vi-
sual Management, with 2 items, for exam-
ple, Statistics on falls, central line infections,
and other quality indicators are displayed for
staff to see. This factor had a high Cron-
bach ( = .86), and its test-retest reliabil-
ity was the highest of all the FIT unit fac-
tors (R = 0.77), which may mean that units
were aware of the need to involve nurses
in QI and to post their results to gain nurse
buy-in. The items and factor loadings for the
FIT unit factors are reported in Supplemen-
Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
5
tal Digital Content, Table 2 (available at: http:
//links.lww.com/JNCQ/A253).
Individual
The third section of the measure addressed
individual nurses. Factor 1, Takes Action, in-
cludes 9 items, such as I know that it is impor-
tant to confront problems, not ignore them.
The factor items describe a nurse who not
only thinks about care and the problems but
also takes action to improve care. The Cron-
bach was high ( = .88), but test-retest relia-
bility was only fair (R = 0.35). Factor 2, Solves
Problems, includes some of the Lean thinking
elements that are important in a system that
has been changed by Lean management. The
12-item factor includes such items as I iden-
tify ways to eliminate waste in patient care.
The Cronbach was exceptional ( = .91).
Test-retest reliability was fair (R = 0.46).
Factor 3, Reduces Variation, loaded on 2
highly correlated items: I do not think that it
is important to perform care in a similar man-
ner to other nurses in the organization, and
I do not think that it is important to perform
care in a similar manner to other nurses on
the unit. The 2-item factor had a high Cron-
bach ( = .94) but low test-retest reliability
(R = 0.28). Despite, or perhaps because of the
negative wording, the 2 items held together
with each attempted rotation. They seem to
indicate that nurses are aware of variation in
care.
Factor 4, Uses Improvement Tools, in-
cludes 3 items such as: I solve problems using
a methodical, written approach to commu-
nicate the ideas. The Cronbach was .75,
but test-retest reliability was poor (R = 0.38).
Finds Opportunity, factor 5, includes 2 items:
I find opportunities to improve how I pro-
vide care every work-day and I compare the
way I perform care to how my coworkers per-
form care ( = .66 and R = 0.38). These 2
are important concepts because to analyze im-
provement, everyone needs to be performing
care in the same way. Then, opportunities for
improvement can be more easily identified.
Any change requires a mind that is
open to the change and flexible enough to
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40
6 JOURNAL OF NURSING CARE QUALITY/00 2016
embrace improvements. Factor 6, Mind-set
for Change, includes 8 items such as: I think
that process improvement work can cause
positive change to occur. This factor is an-
other important reflection of Lean thinking.
The Cronbach was acceptable ( = .84),
and the items described the concept appro-
priately. Test-retest reliability continued to be
fair (R = 0.38). The items and factor loadings
for the FIT individual factors are reported in
Supplemental Digital Content, Table 3 (avail-
able at: http://links.lww.com/JNCQ/A254).
DISCUSSION
This study developed and tested a compre-
hensive assessment of Lean management in a
health care organization so that a researcher
or administrator can measure how much Lean
presence is in the organization, the unit, and
an individual nurses thinking. A particular
strength of the study was the expert panel
used to test content validity. Their substantial
input reflected their Lean management exper-
tise. The expert panelists depicted a Lean or-
ganization of leaders who communicated the
vision and mission of patient-centered, waste-
free care. They described managers who were
involved and informed enough to stimulate
new ways of thinking about doing the work.
Finally, the experts discussed the availability
of educators and resources to support im-
provement ideas.
However, as the panelists ideas progressed
through the 3 rounds, Lean jargon proved
troublesome. Since Lean is rooted in Japanese
manufacturing, nurses may know the ideas
but not the terminology. To address these con-
cerns, Lean terminology was placed in paren-
theses to explain an item. For example, one
item read: I know to inform the patient . . .
of the expected routine (standard work) for
the day.
Limitations
Using a convenience sample introduces
bias into the study. It is difficult to determine
what constitutes Lean and to identify hospi-
tals that were appropriate for the study. In
addition, although the statistical program re-
Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
ported sampling adequacy, the sample may
not reflect the knowledge and actions of most
nurses. The nurses who responded may be
the ones most knowledgeable or comfortable
with the items described in the instrument.
Furthermore, although s were acceptable for
a new instrument, s and Pearson correlations
were low. Since the FIT instrument is in the
initial stage of development, Pett et al13 sug-
gest the next step is to increase the sample
with 900 or more nurses and subject the sam-
ple to confirmatory analysis.
Because the survey instrument took about
20 minutes to complete and items relating to
the individual nurse were placed in the mid-
dle of the instrument, raters may have expe-
rienced test fatigue with resulting inconsis-
tency of responses. It also is possible that
individual nurses may not have considered
their Lean skill level before taking the FIT sur-
vey and that the items in the measure were
not those typically thought about during pa-
tient care. The 2-week period between survey
administrations gave them time to consider
how they thought about Lean problem solv-
ing and opportunities for improvement. Con-
sequently, their scoring would be expected to
be different on the retest.
CONCLUSION
The importance of assessing the state of QI
thinking in nurses cannot be overemphasized.
Change in practice is not easy, and the need
for change is not always obvious. Comparing
hospitals helps leadership see where their or-
ganization ranks in overall performance. Com-
parative data often serve as a stimulus for
poorly performing hospitals or units to im-
prove outcomes. Evaluating units within the
hospital identifies areas to focus resources and
programs for team alignment with organiza-
tional strategic priorities. Tracking changes
in unit performance after implementation of
a Lean intervention may give insight into
quality of care. Understanding the individual
nurses thinking provides a way to identify
mentors and coaches as well as change agents.
Measures are a means for encouraging im-
provement or may be tools for monitoring
LWW/JNCQ JNCQ-D-15-00206 March 15, 2016 7:40
A Measure of Lean Quality Improvement for Hospital Staff Nurses
performance. The FIT instrument has the po-
tential to provide such a service to hospital
leadership.
Measuring the success of a Lean improve-
ment program requires selecting an environ-
ment at a point in time and taking the tem-
perature of the conditions at that point.
One way of measuring is to find the person
who makes contact with the patient to as-
sess if the organizations Lean management or
other QI initiative has reached the point of
care. Since nurses are at the hub of patient
care activities, asking nurses who have con-
tact with the customer to report on improve-
ments gets at the essence of Lean philosophy,
that is, to provide value as defined by the
customer.
Because the FIT instrument incorporates
widely used QI terminology, it can be used
REFERENCES
1. Reid PP, Compton WD, Grossman JH, Fanjiang G; Na-
tional Academy of Engineering, Institute of Medicine.
Building a Better Delivery System: A New Engi-
neering Health Care Partnership. Washington, DC:
National Academies Press; 2005:262.
2. Wang MC, Hyun JK, Harrison M, Shortell SM, Fraser I.
Redesigning health systems for quality: lessons from
emerging practices. Jt Comm J Qual Patient Saf.
2006;32(11):599-561.
3. Chassin MR. Improving the quality of health care:
whats taking so long? Health Aff (Millwood).
2013;32(10):1761-1765.
4. Chassin MR, Loeb JM. High-reliability health care: get-
ting there from here. Milbank Q. 2013;91(3):459-490.
5. Albliwi S, Antony J, Lim SAH, van der Wiele T.
Critical failure factors of Lean Six Sigma: a system-
atic literature review. Int J Qual Reliabil Manage.
2014;31(9):1012-1030.
6. DelliFraine JL, Langabeer JR II, Nembhard IM. As-
sessing the evidence of Six Sigma and Lean in the
health care industry. Qual Manage Health Care.
2010;19(3):211-225.
7. Brackett T, Comer L, Whichello R. Do Lean practices
lead to more time at the bedside? J Healthc Qual.
2013;35(2):7-14.
Copyright 2016 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
7
to assess not only Lean management but also
the current state of QI. Each item addresses
key elements of improvement thinking; con-
sequently, a score on an item gives ideas
about opportunities for enhancing quality ef-
forts and provides a focus for educational
programs. A baseline assessment followed by
yearly assessments may help organizations or
units enact strategies that ensure that im-
provement efforts are sustained. The FIT in-
strument addresses the organization, unit, and
individual levels and is an early effort to cap-
ture the quality of leanness and, by extension,
Lean implementation on the front line. Using
this measure of leanness, future studies can
explore outcomes such as quality, safety, cost,
and delivery of care and their relationships to
the amount of Lean QI in a hospital or other
health care organization.
8. Groene O, Botje D, Sunol R, Lopez MA, Wagner C.
A systematic review of instruments that assess the
implementation of hospital quality management sys-
tems. Int J Qual Health Care. 2013;25(5):525-541.
9. Hsu CC, Sandford BA. The Delphi technique; mak-
ing sense of consensus. Pract Assess Res Eval.
2007;12(10):1-8.
10. Yousuf MI. Using experts opinions through the Del-
phi technique. Pract Assess Res Eval. 2007;12(4):1-8.
11. Pett MA, Lackey NR, Sullivan JJ. Making Sense of
Factor Analysis: The Use of Factor Analysis for In-
strument Development in Health care Research.
Thousand Oaks, CA: Sage; 2003:348.
12. Lynn MR. Determination and quantification of con-
tent validity. Nurs Res. 1986;35(6):382-385.
13. Readability Formulas. http://www.readability
formulas.com/free-readability-formula-tests.php.
Accessed March 3, 2016.
14. Damrosch SP. Ensuring anonymity by use of subject-
generated identification codes. Res Nurs Health.
1986;9(1):61-63.
15. Costello AB, Osbourne JW. Best practices in ex-
ploratory factor analysis: four recommendations for
getting the most from your analysis. Pract Assess Res
Eval. 2005;10(7):1-9.