AACC Improving Infusion Pump Safety Through Usability Testing Summary Please see the attached reading assignment for your review. Please let me know if you

AACC Improving Infusion Pump Safety Through Usability Testing Summary Please see the attached reading assignment for your review. Please let me know if you have any questions or concerns.Weekly Assignments:Students are assigned one weekly reading assignment (ten total) to include peer-reviewed seminal or breakthrough articles specific to the upcoming lecture. Students are required to summarize the findings of the article in a single page summary (one page +/- one paragraph, single-line spacing, 11pt font) and include a comment and/or question for discussion. The comment may include how materials relate to other course concepts or examples from your own life experience. The question may address any aspect of the paper including something that was confusing or about applying the material to different contexts. The completed assignments will be submitted before the lecture when the reading will be discussed . Full credit will be given for submitting on time with reasonable content (1 point), including both summary questions and/or comment (1 point), and meeting length requirements (1 point). A brief grace period of 30 minutes or less will apply in order to account for occasional technical difficulties in submitting work. Timely submission of work is important to keep pace with the course and to allow time to review grading feedback to guide maximum performance on subsequent work. J Nurs Care Qual
Vol. 32, No. 2, pp. 141–149
c 2017 Wolters Kluwer Health, Inc. All rights reserved.
Copyright
Improving Infusion Pump
Safety Through Usability
Testing
Kristen E. Miller, DrPH, MSPH;
Ryan Arnold, MD, MA; Muge Capan, PhD;
Michele Campbell, BSN, RN, MSM, CPHQ, FABC;
Susan Coffey Zern, MD, CHSE;
Robert Dressler, MD, MBA;
Ozioma O. Duru, PharmD, BCPS;
Gwen Ebbert, MSN, BA, RN, RN-BC;
Eric Jackson, Jr, MD, MBA;
John Learish, CHTM, CBET, CRES; Danielle Strauss, BA;
Pan Wu, PhD; Dean A. Bennett, RPh, CPHQ
With the recognition that the introduction of new technology causes changes in workflow and
may introduce new errors to the system, usability testing was performed to provide data on nursing
practice and interaction with infusion pump technology. Usability testing provides the opportunity
to detect and analyze potentially dangerous problems with the design of infusion pumps that could
cause or allow avoidable errors. This work will reduce preventable harm through the optimization
of health care delivery. Key words: adverse events, infusion pump, patient safety, technology,
usability testing
Author Affiliations: Value Institute, Christiana
Care Health System, Newark, Delaware (Drs Miller,
Arnold, Capan, Jackson, and Wu and Ms Strauss);
Christiana Care Health System, Newark, Delaware
(Drs Miller, Arnold, Capan, Zern, Dressler, Duru,
Jackson, and Wu, Messrs Campbell, Learish, and
Bennett, and Mss Ebbert and Strauss); Virtual
Education and Simulation Training Center,
Christiana Care Health System, Newark, Delaware
(Dr Zern); and University of Delaware, Newark
(Ms Strauss).
The authors declare no conflict of interest.
Supplemental digital content is available for this article.
Direct URL citation appears in the printed text and is
provided in the HTML and PDF versions of this article
on the journal’s Web site (www.jncqjournal.com).
Correspondence: Kristen E. Miller, DrPH, MSPH,
Value Institute, Christiana Care Health System, 4755
Ogletown-Stanton Rd, Newark DE 19718 (krmiller@
christianacare.org).
Accepted for publication: May 22, 2016
Published ahead of print: August 5, 2016
T
HE VAST MAJORITY of hospitalized
patients encounter one of the most
widely used medical technologies in health
care: infusion pumps. Increasingly, computercontrolled “smart pumps” can be programmed to deliver controlled amounts of
painkillers, antibiotics, insulin, chemotherapy
drugs, nutrients, or other fluids. While Christiana Care Health System had been using an
advanced “smart pump” system-wide since
2009, the system introduced a replacement
infusion pump from a different vendor in
January 2015. The implementation of technologies changes the work of end users in
foreseen and unforeseen ways, ultimately affecting an individual’s quality of working
life, such as job satisfaction and stress, and
DOI: 10.1097/NCQ.0000000000000208
141
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
142
JOURNAL OF NURSING CARE QUALITY/APRIL–JUNE 2017
perceived safety and quality of care.1-3 With
the recognition that technology changes
workflow and, therefore, requires changes
to processes already in place, potentially introducing new errors to the system, usability testing was performed to evaluate nursing
practice and interaction with infusion pump
technology.
Infusion pumps are an indispensable tool
in health care, but their complex design is
recognized as a risk factor for administration errors. Because infusion pumps are frequently used to administer critical fluids, including high-risk medications, pump failures
can have significant implications for patient
safety.4 Adverse events may be the result of
user error or related to deficiencies in device design and engineering including software defects, mechanical or electrical failures,
and user-interface issues. User-interface issues
are commonly considered to be the leading
cause of dosing errors, frequently resulting
from pump-programming errors.5-7 The scale
of the problem is troubling: medication errors cause injury or death to 1% to 2% of patients admitted to the hospital in the United
States annually.8 Despite numerous studies
identifying user-interface issues with infusion
pumps, there is an assumption regarding safe
and effective use by the ultimate end user,
a nurse.9
Well-known usability problems involving
intravenous infusion pumps are identified as a
breakdown of one or more primary steps (eg,
nurses administering the wrong dose or the
wrong drug) in the drug-delivery protocol.10
Automation can lead to a variety of unintended effects, such as automation surprise
(when an automated system behaves in ways
that the operators do not expect) and increased complacency.11 This has the paradoxical potential of actually increasing harm to patients by obviating health care providers from
their perceived responsibility to check proper
drug delivery.
Ergonomics and human factors engineering
offer useful frameworks for examining many
of the mediating and moderating factors that
may affect the use of equipment and out-
comes of its use.12,13 Observational methodology provides rich, detailed information on
tasks performed by nurses when administering medication and using various technologies. Usability testing provides direct information on the interaction between people and
their work environment or tools. Simulated
use can capture the complexity and preserve
the context of the work environment within
which health information technology is implemented. To address questions important
to inform system-wide clinical implementation and evaluate issues identified by health
care providers, we evaluated the new infusion
pump through simulated use, an approach
that provides the opportunity to detect and
analyze potentially dangerous problems with
the design of infusion pumps that could cause
or allow avoidable errors.
MATERIALS AND METHODS
Testing environment
To represent the intended use environment
for infusion pumps, the study was conducted
in the Christiana Care Health System Virtual
Education and Simulation Training Center.
Each room was set up like a patient room,
with a manikin in each bed with an intravenous reservoir in place. Participants were
instructed to use no-fluid-required “infinity”
infusion pump tubing segments (equipment
used for infusion training), removing the need
to create simulated medications that had no
impact on programming, the focus of the
study. Each participant operated an infusion
pump equipped with a central processing unit
and at least 1 large volume infusion pump
module. Medication labels were printed directly from the computerized physician order
entry clinical system and represent label formatting in actual use. The order detail was
developed to represent the details a nurse
would have access to via the electronic medical record. Other written materials needed
to program the infusions (ie, screenshots of
the computerized physician order entry order and electronic medication administration
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Improving Infusion Pump Safety Through Usability Testing
record from the electronic medical record order) were also provided.
Participant recruitment for usability
testing sessions
In general, an infusion pump is operated by
a nurse who programs the rate and duration
of medication delivery through a built-in software interface. Representative user groups
include nurses from the operating room, intensive care units, and medical-surgical units.
Inclusion criteria included nurses providing
direct patient care having experience with
this model infusion pump. Usability testing
occurred in 3 rounds based on participation
of nursing staff from nursing-shared decisionmaking councils. Testing took approximately
1 hour per person, requiring a substantial investment of hospital resources. Availability for
participation was made possible through the
nursing councils and occurred during their
standing bimonthly council meetings. Over a
2-month testing period, 22 participants completed the usability testing.
Task selection
The first and most important criterion for
selecting tasks is to use tasks that probe the
potential usability problems with the infusion
pump. Functional and clinical tasks were selected to represent the different types of tasks
expected of health care providers. Functional
tasks represent operational tasks associated
with the use of the machine. Clinical tasks represent tasks necessary to achieve the medical
benefit of the device. The medication safety
officer and medication safety clinical pharmacist selected tasks on the basis of the potential
for certain drugs to contribute to adverse outcomes in patients if the drug dosage is incorrectly calculated. The clinical task scenarios
also represent true clinical scenarios that have
resulted in facility adverse event reports. The
different clinical scenarios represent drugs
from different drug classes. Based on Christiana Care drug library creation, these drug
classes have set dosing guidelines through established concentrations, dose limits, and clinical advisories. The participants’ role and unit
143
determine their drug library access, thereby
impacting the clinical tasks conducted in the
simulated use evaluation.
Task assignment
Tasks were assigned to participants in library profile categories on the basis of patient care unit–level designations or specialty
function. In our hospital, medications are restricted to “levels” on the basis of the type
and degree of monitoring required: levels A,
B, C, and Oncology. Level A medications need
the least monitoring while level C medications
(eg, vasopressors) need the highest monitoring. Oncology contains all drugs from level
C with the addition of cancer-specific medications. Similarly, patient care units are assigned
a level (A, B, or C) on the basis of monitoring capabilities available, nurse-to-patient ratio, and annual nursing competency requirements.
Participants were asked to complete
10 functional tasks and 7 clinical tasks.
Functional tasks were the same for every
participant. Each clinical task was designed
to address a specific drug class and drug combination. The clinical tasks associated with a
subset of the libraries were randomly assigned
to each participant depending on his or her access level, without repetition, such that every
participant was asked to complete 7 clinical
tasks to achieve a balanced outcome sample.
Performance assessment
Data collection included facilitator observation for data capture of objective components and subjective assessment from participants. Primary data were collected at the
scene and reviewed after the simulated use
evaluation for secondary data collection. Performance measures (observed) included task
failures and difficulties. A failure was defined
as an action (or failure to act) that would lead
to an undesirable treatment outcome to the
patient or the user. A difficulty was defined as
any instance of failure that was avoided by vigilance on the part of the user. Interface alerts
and warning screens alone would not classify
as a difficulty.
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
144
JOURNAL OF NURSING CARE QUALITY/APRIL–JUNE 2017
Preference assessment
A subjective analysis after completion of
the simulated use evaluation captured participants’ reaction to the interface. In addition to questions developed by the simulated
use evaluation team, participants were asked
questions specific to the infusion pump using
the Post-Study System Usability Questionnaire
(PSSUQ). The PSSUQ is a standardized instrument developed for use in scenario-based usability evaluation at IBM.14 It consists of 19
items aimed to address the following 5-system
usability characteristics: quick completion of
work, ease of learning, high-quality documentation and online information, functional adequacy, and rapid acquisition.15 The resulting
scores are between 1 and 7, with lower scores
indicating a higher degree of satisfaction.
Statistical analysis
Statistical analyses were performed using STATA/IC 11.0 statistical software (Stata
Corp., College Station, Texas). Standard descriptive measures (ie, frequencies, means,
medians, and SDs) were computed for variables in the data set. Descriptive statistics
were used to describe basic features and
compare different participants’ demographics and performance and preference assessments. Qualitative data regarding participant
feedback were analyzed by the research team
to identify trends in device vulnerabilities.
RESULTS
Participant demographics
There were a total of 22 participants representing diverse units, experience, positions,
and drug-leveling assignments. Before beginning the testing session, moderators documented demographic information for each
participant. Usability testing was performed
once the pumps had been in system-wide
patient care use for more than 3 months.
Participants were all previously trained on
pump usability by infusion pump representatives (hands-on training). This training was required for all staff prior to use in direct patient
care. The majority of participants participated
in facility computer-based training (86%) and
used practice pumps provided on their unit
(64%). Less than half of participants had reviewed the product manual (45%), pump policy (41%), or unit policy (9%).
Functional task performance
During the functional task performance, 15
participants (68%) experienced at least 1 difficulty and 1 failure. Of the 10 tasks, 7 created
difficulties and failures (Figure). The ability to
lock and unlock the tamper resist (a button
located on the back of the machine) created
the most difficulty as participants noted that
this is a function they were not aware of or
ever used. Adjusting the audio volume also
proved difficult as the function is not located
under the “options” soft key, a common assumption. Attaching and detaching the pump
models was difficult for more than 10% of participants, reflecting that this is a task not commonly completed on the unit. A commonly
identified stressor was noise resulting from
delays in programming. Only a few participants used the “silence” button while others
acknowledged the beeping and ensuing anxiety. Many participants powered down or shut
off the channel to change a drug. Many noted
that there is no easy way to clear a drug other
than to shut down the machine.
Clinical task performance
There were difficulties or failures by at
least 1 participant in 13 of the 15 drug
scenarios (see Supplemental Digital Content,
Figure, available at: http://links.lww.com/
JNCQ/A277). Drug scenarios that resulted
in the most difficulties included abciximab
(12/12, 100%), parenteral nutrition (13/13,
100%), intravenous immune globulin (14/14,
100%), methylprednisolone continuous infusion (not in the library) (13/13, 100%), alteplase (11/12, 92%), vasopressin (7/8, 88%),
rituximab (10/12, 83%), vancomycin (8/11,
73%), fentanyl (5/7, 71%), and midazolam
(5/7, 71%). Drug scenarios that resulted in
the most failures included abciximab (12/12,
100%), intravenous immune globulin (13/14,
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
Improving Infusion Pump Safety Through Usability Testing
145
Figure. Functional task performance results.
93%), methylprednisolone (10/13, 77%)
(a drug not in the library), and vancomycin
(8/11, 73%). Each of the 22 participants
(100%) experienced at least 1 clinical difficulty and 1 clinical failure. Common causes
of difficulty and failure included a lack of familiarity with the drug, difficulty locating the
drug in the drug library, titration, and weightbased calculations.
Overall reported ease of use and human
error
The majority of participants (86%, 19/22)
said that they experienced usability problems during simulated use. Participants recognized safeguards including Guardrail infusion
dose or rate limits, clinical advisory reminders
(pop-ups), and reminders and hard stops (limits that cannot be overridden) as 82% felt that
the infusion pump promotes safe and effective use by the end user (18/22). The majority
of participants (64%, 14/22) felt that it would
be easy to make an error with this pump
considering their clinical environment. Few
participants (23%, 5/22) recognized that their
peers, when programming a drug they cannot
find in the drug library, will use another medication or choose the “basic infusion” function. In doing so, there are no guardrails and
no safety net. Christiana Care trained staff to
avoid the use of “basic infusion” and use the
“DrugCalc” feature in the pump when a medication is not in the library. Participants recognized concerns with rate and duration defaults
as compared to the previous pump.
Ease of use was assessed for overall use,
operation, loading, and programming. The
majority of participants believed that the infusion pump was very easy or easy to use
(67%, 14/21), operate in general (86%, 18/21),
load (86%, 18/21), and program (62%, 13/21).
Three participants reported that programing
the pump was difficult and 5 were neutral.
Post-Study System Usability
Questionnaire
Our usability testing indicated that despite
significant difficulty and failure in programming, participants were satisfied with the system (overall PSSUQ 2.75). Individual qualities
Copyright © 2017 Wolters Kluwer Health, Inc. Unauthorized reproduction of this article is prohibited.
146
JOURNAL OF NURSING CARE QUALITY/APRIL–JUNE 2017
also ranked fairly high and included system
usefulness (2.78), information quality (2.76),
and interface quality (2.84).
DISCUSSION
Participant performance
The simulated use evaluation proved especially useful as the majority of participants
admitted experiencing problems during testing. Rather than learning about the trends
in use error through adverse event reports,
the team was able to proactively capture vulnerabilities in a safe environment. Through
self-reflection, participants recognized issues
regarding weight calculations, uncertainty
when drugs are not listed in the library, a lack
of a multistep function, dangers related to unfamiliar medications, and frustrations with a
quick, loud, audio alarm. Overall, a lack of familiarity was the most common precursor to
error or difficulty. Participants expressed concern when asked to perform a clinical task
(program an infusion) they rarely perform on
their unit but for which their unit has the
“level” designation, capability, and expectation to be competent to perform.
The results of the simulated use evaluation showed a number of interesting vulnerabilities and potential opportunities for improvement. First, this study made clear the
importance of research that systematically examines interference effects associated with
switching from one “smart pump” to a new
one with a different design approach to hardware and interface software. One common
error, in particular, was the incidental substitution of rate and dose for certain medications
constructed on what may be explained by
muscle memory on the basis of the previous
pump placement of display and programming
fields for dose and rate. Christiana Care used
a “smart pump” from a different vendor 2009
through January 2015. The dose field was at
the top of the screen with the rate field displayed below. The fields are reversed in the
new pump such that the rate field is at the top.
Developing and integrating knowledge of spe-
cific interference problems may assist in the
unlearning of old interactions and learning of
new tasks.
Second, in nearly every difficulty and failure, participants requested assistance from
colleagues including their peers, a charge
nurse, nurse staff development specialist, registered nurse III (advanced education and expe…
Purchase answer to see full
attachment