Contemporary Research Methods in Pharmacy and Health Services

By Shane P. Desselle and Claire Anderson

Emerging methods, as well as best practices in well-used methods, in pharmacy are of great benefit to researchers, graduate students, graduate programs, residents and fellows also in other health science areas.

Researchers require a text to assist in the design of experiments to address seemingly age-old problems. New interventions are needed to improve medication adherence, patients’ lived experiences in health care, provider-patient relationships, and even various facets of pharmacogenomics. Advances in systems re-engineering can optimize health care practitioners’ roles.

Contemporary Research Methods in Pharmacy and Health Services includes multi-authored chapters by renowned experts in their field. Chapters cover examples in pharmacy, health services and others transcendent of medical care, following a standardized format, including key research points; valid and invalid assumptions; pitfalls to avoid; applications; and further inquiry.

This is a valuable resource for researchers both in academia and corporate R&D, primarily in pharmacy but also in health services, and other health disciplines. Social science researchers and government scientists can also benefit from the reading.

• Provides multi-authored chapters by renowned experts in their field
• Includes examples for pharmacy and health services and others that are transcendent of medical care
• Covers key research points, valid and invalid assumptions, pitfalls to avoid, applications, and further inquiry

• Language: English
• Publisher: Academic Press
• Release date: May 10, 2022
• ISBN: 9780323914260

Book preview

Section I

Overarching Design and Reviews Considerations

Chapter 1: Applying human factors and ergonomics methods to pharmaceutical health services research

Richard J. Holdena,b,c,d; Ephrem Abebed,e; Alissa L. Russ-Jarae; Michelle A. Chuif,g    a Department of Health & Wellness Design, Indiana University School of Public Health, Bloomington, IN, United States

b Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, IN, United States

c Center for Health Innovation and Implementation Science, Indiana Clinical and Translational Sciences Institute, Indianapolis, IN, United States

d Department of Medicine, Indiana University School of Medicine, Indianapolis, IN, United States

e Department of Pharmacy Practice, College of Pharmacy, Purdue University, West Lafayette, IN, United States

f Social and Administrative Sciences Division, University of Wisconsin-Madison School of Pharmacy, Madison, WI, United States

g Sonderegger Research Center, University of Wisconsin-Madison School of Pharmacy, Madison, WI, United States

Abstract

Human factors and ergonomics (HFE) is a scientific and practical human-centered discipline that studies and improves human work performance and wellbeing in sociotechnical systems. HFE in pharmacy involves the human-centered design of systems to support individuals and teams who perform medication-related work. We define the scope of HFE methods in pharmacy as applications to pharmacy settings, such as inpatient or community pharmacies, as well as to medication-related phenomena such as medication safety, adherence, or deprescribing. This chapter presents seven categories of HFE methods suited to widespread use for pharmacy research and clinical practice, with examples of how these methods have been previously applied in pharmacy. These categories of methods are work system analysis; task analysis; workload assessment; medication safety and error analysis; user-centered and participatory design; usability evaluation; and physical ergonomics. HFE methods from these categories are used in three broad phases of human-centered design and evaluation: study, design, and evaluation. Three cases illustrate the comprehensive application of HFE to (1) medication package, label, and information design; (2) human-centered design of a digital decision aid for medication safety; and (3) risk analysis of medication cancellation.

Keywords

Human-centered design; Usability; Medication safety; Sociotechnical systems; Human factors/ergonomics

Objectives

•Define the field of human factors and ergonomics generally and specifically as it relates to pharmacy research and practice.

•Explain the major categories of human factors and ergonomics methods applicable to pharmacy, their uses, strengths, and limitations.

•Illustrate how projects have applied human factors and ergonomics methods in published pharmacy projects.

•Demonstrate how larger scale projects combine multiple human factors and ergonomics methods in practice.

Acknowledgments

Funding: Dr. Abebe received support from the National Institutes of Health (NIH), National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award, Grant Numbers, KL2TR002530 (PI: Carroll), and UL1TR002529 (PI: Shekhar). For some studies described in this article, Dr. Russ-Jara received funding support from the Department of Veterans Affairs (VA), Veterans Health Administration, Health Services Research and Development Service (HSR&D), Center for Health Information and Communication CIN 13-416 (PI: Weiner) at the Richard L. Roudebush Veterans Affairs Medical Center in Indianapolis, IN and from a VA HSR&D Career Development Award 11-214 (PI: Russ-Jara). Work presented in Case Study 2 was supported by Agency for Healthcare Research and Quality (AHRQ) grants P30 HS024384 (PI: Callahan) and R18 HS024490 (PI: Chui) and NIH grant R01AG056926 (PI: Holden). Views expressed in this article are those of the authors and do not necessarily represent the views of the NIH, AHRQ, Department of Veterans Affairs, or the U.S. government.

Human factors and ergonomics (HFE) is a scientific and practical human-centered discipline that studies and improves human work performance and wellbeing in sociotechnical systems.¹,² HFE in pharmacy involves the human-centered design of systems to support individuals and teams who perform medication-related work.³

HFE methods are effective⁴ and therefore constitute standard practice in several industries, especially safety-critical arenas of aviation, surface transportation, military, and energy.⁵,⁶ HFE methods are also routinely applied in office settings, service industries, leisure, consumer products, and medical devices.⁷–¹⁰

HFE first appeared in healthcare in 1960s studies of hospital medication safety.¹¹ Later that century, HFE grew in other inpatient settings such as anesthesiology and surgery.¹²,¹³ Turn-of-the-century reports from the U.S. National Academy of Medicine (then the Institute of Medicine)¹⁴–¹⁶ accelerated the application of HFE to patient safety and quality, particularly the discipline’s incident analyses, team training, and aviation tools such as checklists.¹⁷ Mass adoption of health information technology (IT) increased the demand for HFE methods for user-centered design and usability testing.¹⁸,¹⁹ HFE methods have been used to study and improve the work and outcomes of healthcare professionals, nonprofessionals such as patients or families, and teams.²⁰ Applying HFE to study and improve the health-related work of patients, families, and other nonprofessionals is called patient ergonomics, simply defined as the science and engineering of patient work.²¹ Patient ergonomics is a branch of HFE that has grown in parallel to paradigm shifts toward patient engagement and empowerment, consumer health IT, data democratization, and shared decision making.²² There is also emerging interest in applying HFE to study and improve public health behaviors outside the context of illness and clinical care.²³

Overall, HFE methods have gained penetration and success in healthcare²⁴–²⁸ and are increasingly seen in pharmacy. This chapter reviews select HFE methods well suited to address pharmacy challenges and offers examples of how they have been applied in pharmacy.

Opportunities for HFE in pharmacy

HFE in pharmacy can be defined in two ways. First, it is the application of HFE in specific pharmacy settings, such as inpatient or community pharmacies. Second, it is the application of HFE to study or improve medication-related phenomena such as medication safety,²⁹ adherence,³⁰ or deprescribing.³¹

Many pharmacy settings can benefit from HFE methods, and there is growing realization that one such setting is the community pharmacy. HFE can be useful for community pharmacy research and design as community pharmacy practice evolves, expanding beyond dispensing to include immunization, medication therapy management, and point of care testing.³²,³³ Roles in these settings are changing, as pharmacy technicians perform complex tasks traditionally assumed by pharmacists.³⁴–³⁶ HFE is needed to address safety in community pharmacy settings, including to address an estimated medication error rate between 1.7% and 22%, of which 6.5% are clinically significant events.³⁷–³⁹ Using the most conservative estimate, for a typical U.S. community pharmacy dispensing 250 prescriptions per day, this means approximately 4 errors per day, including 2 clinically significant errors per week—or 51.5 million dispensing errors annually nationwide. HFE offers tools to examine and address the systems factors contributing to errors in these complex environments.⁴⁰,⁴¹ Indeed, HFE methods have already been applied in community pharmacy settings to address e-prescribing workflow and cognitive needs,⁴²,⁴³ medication safety,⁴⁴ stress and workload during dispensing,⁴⁵–⁴⁷ and over-the-counter (OTC) medication decisions.⁴⁸,⁴⁹

HFE methods can address medication use and medication safety phenomena independent of settings and often across settings where professional care and self-care occur. Table 1 illustrates how studies using HFE methods have been performed across the medication use process,⁵⁰ to better understand or improve the medication-related performance of prescribers, pharmacy workers, nurses, and patients. Wherever medication-related work takes place, HFE can be used to redesign the system to be more human-centered and thus to improve the performance of the work and the wellbeing of those who perform it.⁵⁵

Table 1

HFE, human factors/ergonomics; SEIPS, systems engineering initiative for patient safety.

HFE methods for pharmacy research and clinical practice

HFE offers a broad toolkit of methods taught to HFE professionals and learnable by others.⁵⁶ At an introductory level, there are over 100 individual HFE methods, some with over 100 variations each.⁵⁷–⁵⁹ This section describes seven categories of HFE methods that are well suited to widespread use for pharmacy research and clinical practice, with examples of how they have been used in published work on HFE in pharmacy.

Work system analysis methods

Work system analysis identifies, defines, and analyzes factors contributing to performance in sociotechnical systems.⁶⁰ Several models of work systems are available,⁶¹ each depicting broad categories of interacting performance-shaping factors; for example, the Systems Engineering Initiative for Patient Safety (SEIPS) model’s factors are persons, tasks, tools/technologies, organization, and environment.⁶²–⁶⁴ These models are used to structure data collection or analyze data.⁶⁵ Work system analysis is also used to examine interactions between factors, such as the degree of alignment between a tool, its user, and the associated task. Work system analyses in pharmacy have primarily used interview or focus group data collection and applied deductive qualitative analysis to code findings using an existing work system model such as SEIPS,⁶² SEIPS 2.0,⁶³ or others.¹,⁶⁶,⁶⁷ Quantitative analysis can be used, as in a nationwide survey of Australian community pharmacists⁶⁸ or a nursing home study correlating adverse drug events to clinicians’ perceptions of work system factors assessed via surveys.⁶⁹ The method has been used to study factors affecting antipsychotic medication use by patients with serious mental illness,⁷⁰ barriers and facilitators to e-prescribing errors in community pharmacies,⁷¹,⁷² barriers to providing safe OTC medication recommendations to older adults,⁷¹ and performance of persons caring for patients with dementia.⁷³ Other applications in pharmacy include studying implementation (e.g., of collaborative practice agreements⁷⁴ or cognitive pharmaceutical services⁷⁵) or developing interventions (e.g., for interruptions management in inpatient pediatric pharmacies⁷⁶ or medication use in nursing homes⁷⁷). Holden and colleagues⁶⁶ also argue work system analysis can be leveraged to better understand patient work systems and performance in a way that is comprehensive, theory-based, and methodologically rigorous.

To aid practitioners in the use of work system analysis methods, Holden and Carayon introduced SEIPS 101 and the seven simple SEIPS tools.⁷⁸ The SEIPS 101 model is a simplified depiction of the most critical components of the various SEIPS models; for example, it simplifies the work system component into the factors People, Environments, Tools, and Tasks or PETT. The seven tools, described in greater depth elsewhere⁷⁸ and summarized in Table 2, are designed to be used by both novices and experts.

Table 2

SEIPS, systems engineering initiative for patient safety; PETT, people, environments, tools, tasks.

Another type of work analysis, cognitive work analysis (CWA), is a comprehensive framework to identify requirements, constraints, and affordances for individuals’ cognitive work within a system of interest, known as a work domain.⁷⁹,⁸⁰ CWA follows five phases, beginning with a high-level work domain analysis resembling the work system analysis above.⁸¹,⁸² Subsequent analysis progressively narrows to characterize individual tasks (control task analysis), strategies used to perform tasks (strategies analysis), allocation of tasks across individuals (social organization and cooperation analysis), and cognitive requirements of workers (worker competencies analysis). As an example, researchers used CWA to model the medication management system in ambulatory care settings.⁸³,⁸⁴

An advantage of the CWA method is it allows system designers to focus on what is possible in terms of human performance to achieve system goals, given the constrains in the work domain (i.e., what can be done instead of what should be done). As a result, this method is ideally suited for the design of novel systems that do not yet exist. A fundamental principle of CWA is that users can achieve their end goals via different means and that there is no single path to doing so, thus promoting the adaptation and flexibility of workers.

Traditionally, CWA has mostly focused on the analytical stages (e.g., work domain analysis) and few instructive examples exist guiding analysts to translate CWA outputs into designs that benefit end users. In one example, combining CWA with user-centered design methods (reviewed below), researchers developed and tested a novel emergency department information system to serve as a decision-support tool for emergency care staff.⁸⁵ When combined with other methods (e.g., participatory design), the multilevel system analysis focus of CWA makes it ideal for designing novel systems with a transformative effect on how end users engage with and benefit from the new system.

Task analysis methods

Task analysis encompasses methods to deeply understand the tasks performed by individuals or teams. Task analysis can be used to model the work of pharmacy professionals, identify training needs, and formulate requirements for the design of work settings, processes, and job aids.

Cognitive task analysis (CTA) allows analysts to capture cognitive processes (e.g., information processing, decision making) underlying observable behaviors.⁸⁶ CTA has three aspects: knowledge elicitation; data analysis; and data representation. A popular knowledge elicitation approach is the critical incident technique or critical decision method, wherein interviewers ask probing questions to gather details about an incident: what happened, strategies used to detect problems, why a decision was made. A limitation of this method is reliance on individuals recalling past events, often spontaneously. To minimize recall bias, Russ et al. collected information about medication safety incidents prospectively from healthcare professionals.⁸⁷ They conducted follow-up critical decision method interviews with healthcare professionals, where healthcare professionals could access the electronic health record (EHR) to aid their recall as they responded to interview questions. This CTA adaptation was used to identify healthcare professionals’ decision-making process for detecting and responding to medication safety incidents.⁸⁷,⁸⁸ Similarly, Holden et al. developed a patient-centered CTA adapted for older adults, with whiteboard drawings to facilitate incident recall and interactive scenarios to simulate real-time decision making.⁸⁹,⁹⁰ These studies analyzed elicited data to produce models of naturalistic decision making⁸⁸,⁹⁰ and personas depicting different approaches to decision making.⁸⁹ Other common uses of CTA are comparing cognitive work of experts versus novices⁹¹ and identifying which cognitive processes are involved in routine tasks such as medication adherence.⁹²

Another popular task analysis is hierarchical task analysis (HTA). HTA is used to depict the hierarchy of tasks or goals, decomposing these into subtasks or subgoals until they are described at a level of resolution fit for the intended purpose, such as for training workers or designing decision-support tools.⁹³ Different data collection approaches can be used to inform HTA including observations, interviews, focus group discussions, and document reviews. For example, Lane et al. performed HTA to model inpatient medication administration errors.⁹⁴

Link analysis is another specific type of task analysis used to examine how people interact with their physical environment. Lester and Chui used link analysis to describe the impact of the physical layout of a community pharmacy on pharmacists' task performance.⁹⁵ Using observation data, they developed a link diagram depicting movement of pharmacists between locations within a pharmacy.

Workload assessment methods

Workload is a multidimensional, multifaceted concept affecting performance outcomes such as error and healthcare professional outcomes such as stress and burnout.⁹⁶ Workload in healthcare has been defined as the ratio of demands to resources.⁹⁷,⁹⁸ Workload thus depends on interactions of real or perceived work demands, the circumstances under which work is performed, and worker characteristics (e.g., pharmacist’s years of experience, patient’s skill level).⁹⁹

Workload can be assessed at multiple levels, including organizational unit, job, and task levels.⁹⁸–¹⁰⁰ At the organizational level, workload may be conceptualized as the amount or volume of work versus staffing resources, which may be directly related to the skill of pharmacists and technicians, or the perceived adequacy of technology. Job-level workload could be measured by the amount and kind of work required for a role (e.g., hospital pharmacist) relative to the training and tools provided. Job-related tasks may include monitoring and multitasking demands. The workload associated with these tasks might require reacting quickly to prevent problems or keeping track of more than one process at once. Task-level workload can be measured as the complexity, difficulty, and multitasking requirements of specific tasks relative to cognitive capacity or tools for the tasks.⁴⁷,⁹⁸,¹⁰⁰ Tasks may include activities such as reviewing a patient profile. The demands associated with that task may require concentration and mental effort or feeling rushed. Thus, HFE measures of workload can address the high fluctuation and unpredictability of the pharmacy work system and extend beyond the number of prescriptions dispensed at a pharmacy or medications prescribed to a patient. Both demands and resources, across all levels of analysis, can and should be assessed.¹⁰¹

Demands contributing to workload may be physical or cognitive (i.e., mental workload). Physical workload is measured using subjective and objective methods, the former including validated measures of self-reported exertion.¹⁰² More objective measures of physical workload include assessing physiologic variables (e.g., oxygen consumption, heart rate), task outcomes (e.g., time to perform tasks), worker outcomes (e.g., fatigue), and the ratio of physical demands (e.g., task load, frequency, duration, distance) to available resources such as a person’s work capacity, skill or fitness, or access to assistive equipment and tools.¹⁰³

Increasingly, changes in pharmacy work such as new technologies and greater supervision have primarily increased cognitive demands. Cognitive (or mental) workload measures include the popular, validated NASA Task Load Index (TLX), one of several self-report approaches scholars have used to study pharmacy workload.⁴⁷,¹⁰⁰,¹⁰⁴,¹⁰⁵ Other measures use physiological markers such as a person’s pupillary dilation, brain activity, or heart rate;¹⁰⁶ task performance indicators used to infer workload;¹⁰⁷ or analytic modeling of demands inherent to known tasks.¹⁰⁸

Medication safety and error analysis methods

Various HFE techniques inform the analysis of medication errors and safety incidents to generate interventions and sustainable safety solutions. Failure modes and effects analysis (FMEA) is one technique used to prospectively identify safety vulnerabilities and assess strengths of safety interventions, with the goal of preventing errors.¹⁰⁹ Other prospective HFE techniques applicable to healthcare include bowtie analysis,¹¹⁰ probabilistic risk assessment, and proactive hazard analysis,¹¹¹ and some have applied these methods to medication safety.¹¹²–¹¹⁴ Medication error reporting systems at national or local healthcare organizational levels provide essential data to help identify and analyze errors retrospectively. These data are especially useful to capture unexpected and rare medication errors. Data from medication error reports can be used for research on medication errors and combined with HFE techniques to further investigate incidents and generate solutions for implementation.

Root cause action and analysis (RCA²) is used to retrospectively investigate an incident to identify all plausible causes and generate actionable solutions.¹¹⁵,¹¹⁶ RCA², which is endorsed by the American Society of Health-System Pharmacists, was developed by HFE and clinical experts to explicitly overcome flaws in the traditional use of root cause analysis (RCA).¹¹⁷

Importantly, unlike traditional RCA methods, RCA² guidance includes these Five Rules of Causation:

•Rule 1. Clearly show the cause and effect relationship.

•Rule 2. Use specific and accurate descriptors for what occurred, rather than negative and vague.

•Rule 3. Human errors must have a preceding cause.

•Rule 4. Violations of procedure are not root causes, but must have a preceding cause.

•Rule 5. Failure to act is only causal when there is a preexisting duty to act.

The five rules are intended to shift analysts’ focus away from inherent human shortcomings (e.g., human error as the primary cause) and instead direct their efforts to identifying and correcting the real, underlying system issues that facilitate errors. The five rules were first developed for the aviation industry and more recently adapted for healthcare. These rules are meant to guide analysts in the development of written causal statements, with the intent that one written statement is crafted for each underlying systems cause, and with each causal statement including a specific cause, effect, and event.

To investigate or model past incidents or prospective risks, one could use task analysis methods described in a prior section. For example, one type of CTA, the critical decision method, can be used to retrospectively reconstruct a safety incident.¹¹⁸ An interviewer and subject matter expert (e.g., healthcare professional involved in the incident) would create a basic timeline of events, then uncover goals, decision-making cues, cognitive challenges, and other factors that may have influenced the incident. These data can be subsequently used to elucidate decision-making requirements relevant to the incident type and to generate medication safety interventions.⁸⁷,⁸⁸

Once potential solutions are identified via RCA² or other HFE methods, the hierarchy of hazard control is a complementary tool that can be applied to evaluate the relative strengths of proposed safety solutions prior to implementation.¹¹⁹ This hierarchy provides a broad view of whether a solution aligns with a relatively weak (e.g., training/policy changes), moderate (alerts/warnings), or strong (e.g., safeguards, designing out the possibility of an error) systems approach, with various degrees of effectiveness in between these levels. This hierarchy is a broad, generalizable tool that is independent from the context of the medication error, while the FMEA technique can once again be used prospectively at this stage of error mitigation to assess the strengths and weaknesses of a solution within the system context and constraints. Together, these two techniques can help inform which safety solutions warrant further investment of resources and time and are most promising as sustainable safety solutions.

User-centered and participatory design methods

Design-based approaches are increasingly adopted to address pressing healthcare quality and safety challenges. User-centered (or human-centered) and participatory design approaches are used to develop solutions that fit users’ tasks, needs, and contexts.

With increasing demand for more accountability and public participation, several industries are turning to design-based approaches to develop systems that improve the experience of their customers. The design firm, IDEO, and Stanford University’s d-school are among the organizations that have increased popularity of these approaches in recent years. IDEO has also created a human-centered design toolkit to increase accessibility of this methodology to a broader audience (www.ideo.org/tools). Interested readers can find free, openly accessible courses on human-centered design from IDEO and Massive Open Online Course websites such as Coursera and edX.

In typical user-centered design, designers and researchers conduct user needs assessment from observation, interviews, analysis of artifacts and documents, and secondary data analysis. It is best to derive needs from studying actual (or future) users doing actual work in actual settings.¹²⁰ Needs assessment forms the basis for iteratively developing a form of the design, progressing from early ideas and sketches to more interactive prototypes, which will be subsequently evaluated by target end users to ensure the product is usable and useful.¹²¹ Other commonly used design tools are use-case scenarios, journey maps depicting the user journey over time and settings,⁶⁴ and personas—empirically derived archetypes of user types.¹²²

In participatory design, end-user representatives codesign an intervention that meets the user population’s unique needs.¹²³,¹²⁴ That means end users take the role of designer and can be involved in different stages of the design process, often working in a team.¹²⁴ For example, Reddy et al. used participatory design with pharmacy staff and older adults to develop an OTC medication safety intervention in a community pharmacy setting.¹²³ The researchers employed an iterative, multistep process of problem identification, solution generation and convergence, prototyping, and evaluation. Siek et al. used participatory design to develop a digital personal health application to help older adults manage and share their medication regimens during care transitions.¹²⁵

Despite slight variations in their approach, user-centered and participatory design methods both espouse the principle that the end users of the system must be the central focus of all design efforts. Thus, an important consideration during the design process is a deliberative effort to identify and engage the right mix of individuals and groups that will be affected by the planned system. Done the right way, these methods can create opportunities to design tools and services that promote health equity by removing barriers due to differential access to information, services, and decision-making power.

Design is close kin to innovation, or the introduction of novel concepts, tools, or techniques to solve problems. However, HFE methods for user-centered design are about systematically producing and testing design, not about the creative thinking that is often—erroneously—associated with innovation. Holden and colleagues¹²⁶ argue that innovation is not a light bulb event brought into being by creative disruptors; they introduce a systematic process of innovation based on user-centered design methods that can be implemented as an everyday approach to innovation. This Agile Innovation process, detailed elsewhere,¹²⁶,¹²⁷ has the following eight steps:

•Step 1. Confirm demand for solving a particular problem.

•Step 2. Study the problem in need of solving.

•Step 3. Scan for solutions to similar or related problems.

•Step 4. Plan for evaluation and termination of future solutions.

•Step 5. Ideate and select candidate solutions.

•Step 6. Run innovation development sprints to iteratively produce minimum viable products (MVPs).

•Step 7. Validate solutions on user-centered outcomes.

•Step 8. Package for launch, so the solution can be implemented.

Usability evaluation methods

An important component of user-centered design is conducting usability evaluations of devices, technologies, software, and other solutions.¹²⁸ Usability is defined as the measure of the quality of a user’s experience when interacting with a product or system¹²⁹ and refers to the degree to which a product or system [supports] users [in their efforts] to achieve specified goals with effectiveness, efficiency, and satisfaction….¹³⁰ Usability is vital for all healthcare devices, technologies, and tools, to ensure their effectiveness for healthcare delivery, efficiency for users, and safety for patients. To date, usability evaluations have been conducted on a range of pharmacy-related technologies, including computerized provider order entry,¹³¹ hospital¹³² and community pharmacy medication alerts,¹³³ and infusion pumps.¹³⁴ Usability evaluation can be accomplished through techniques such as heuristic evaluation,¹³⁵,¹³⁶ cognitive walkthrough,¹³⁷ usability questionnaires,¹³⁸ and usability testing,¹³⁹,¹⁴⁰ sometimes used in combination. These and additional usability techniques are further examined in other healthcare literature.¹⁸,¹³⁷–¹³⁹,¹⁴¹

Of these usability evaluations methods, formal usability testing is recognized as the most rigorous usability testing approach.¹²⁸ During usability testing, a moderator with usability expertise asks an end-user (e.g., pharmacist, technician) to complete realistic clinical tasks, without assistance, using one or more systems being tested. For example, usability testing might be conducted with a healthcare professional or patient to assess the usability of technology for medication ordering or refilling a medication, respectively. One example research study conducted usability testing with 20 prescribers in a head-to-head comparison of two different designs for EHR medication alerts, which each included a set of drug-allergy, drug–drug interaction, and renal-drug alerts.¹³² Prescribers and their computer screen actions were recorded as they completed several realistic prescribing tasks. Usability outcomes from this study demonstrated that EHR alert designs incorporating known HFE design principles led to significant gains in prescribing efficiency along with a reduction in prescribing errors.

Guidance exists on selecting clinical task scenarios, as they are central to the quality of usability findings.¹⁴¹ Performance is typically captured by video/screen recording.¹³⁹ Multiple usability measures are typically collected, and often assess usability errors (e.g., number and type of usability problems encountered); efficiency (e.g., time on task, number of mouse clicks); effectiveness (e.g., rate of successful task completion); and/or satisfaction (e.g., usability questionnaires, debrief interview with pharmacist). Examples of common usability questionnaires include the NASA Task Load Index,¹⁴² which measures perceived workload; the 19-item Computer System Usability Questionnaire (CSUQ),¹⁴³ and the 10-item System Usability Scale (SUS).¹⁴⁴ For usability assessments where a goal is to examine visualization, eye-tracking technology can be used to estimate usability and cognitive load from gaze patterns.¹⁴⁵,¹⁴⁶ Other usability techniques might include concurrent think aloud protocol (verbalizing reactions during use),¹⁴⁷ patient actors,¹⁴⁸ and safety probes.¹⁴⁹ As an example of safety probes, one study inserted artificial but realistic medication discrepancies into a medication reconciliation task for healthcare professionals and assessed the extent to which the software supported professionals’ and patients’ detection of discrepancies.¹⁴⁹ The probes for discrepancies included a missing medication, inappropriate medication, and incorrect dose. Usability testing can range from brief quality improvement projects in healthcare institutions¹³⁹,¹⁵⁰ to sophisticated research studies.¹³²,¹⁵¹

Usability evaluations and associated research are valuable throughout the product lifecycle, whether that be a health IT, medical device, or other medication-related tool. Usability evaluations are ideally first applied by manufacturers during product development and used in an iterative manner to improve product effectiveness and safety. Once a product becomes commercially available, healthcare administrators can coordinate with HFE professionals to apply usability testing with a small sample of end users to directly compare products and inform purchasing decisions. After implementation, usability evaluation methods can be a valuable way to further investigate product-related safety incidents and to identify opportunities to enhance product designs. Research can be conducted at any of these stages of the product lifecycle to provide knowledge for the scientific community and improve the delivery of healthcare.

Physical ergonomics methods

Physical ergonomics is the study and design of physical work, by attending to the interaction of human anatomical, anthropometric, physiological, and biomechanical characteristics with work system elements such as lighting, noise, vibration, layout, tools, furniture, forces, hazards, and climate. Many specific HFE theories, methods, and design guidelines are available, in service of improving physical safety (e.g., reducing falls or work-related musculoskeletal disorders), performance (e.g., increasing accuracy and speed of lifting or fine-motor tasks), and comfort.¹⁵² In pharmacy work with a physical component—for instance, medication dispensing¹⁵³,¹⁵⁴—HFE methods can be used to assess:

•Lighting needed (vs. provided) for accurate label-reading under variable pace of work;

•Noise levels and other disruptive or interruptive conditions (e.g., overheard conversations) that might increase risk of error;

•Seated and standing work postures of pharmacists and technicians, including frequency and duration of each posture;

•Walking patterns and steps or distances traveled as a function of potentially modifiable conditions such as work processes, policies, layout, and storage design;

•Physical workload, fatigue, stress, strain, and worker physical complaints over a time (e.g., day, week, year) for various tasks or roles;

•Loads and other contributing factors (e.g., posture, distance, object shape) of supply lifting and carrying tasks;

•Available vs. used equipment for repeated physical tasks such as sitting, typing, screen (or paper) reading and navigation, or waste disposal;

•Visual angles and field of view between customers and staff, to identify any obstructions or other barriers to greeting, helping, or communicating with customers;

•Hazards in the environment such as vibration, spills, sharp objects, studied proactively or reactively (based on incidents or reports).

Just as important are HFE methods for the design of the physical workplace and other work system factors that shape the performance of physical work.¹⁵⁵ These methods address the design of physical tools and equipment (hardware, software, automation), workstations and workplace layout, the environment (e.g., changing temperature, vibration, noise, lighting, surface materials), tasks (e.g., to reduce repetition, fatigue, workload), processes (e.g., introducing break schedules or teamwork), and organizational programs (e.g., training, incentives, staffing levels). One example is supporting the physical work of pharmacists and technicians working in a hazardous drug compounding facility of a central pharmacy. Some institutions are redesigning their facilities to comply with the new USP 800 standards for handling hazardous drugs. HFE experts can collaborate with facility planners and designers to ensure that implementation of appropriate engineering controls and the workspace fully consider the physical work demands of these workers.

Comprehensive application of HFE methods in pharmacy research and practice

HFE methods are used in three broad phases of human-centered design and evaluation (Fig. 1).¹⁵⁶ They are used in the study phase to help understand the problem or situation to be addressed. For example, task analysis and cognitive workload assessment can be used to determine the cognitive tasks performed in an inpatient pharmacy and the demands of each task. Methods are used in the intervention design phase to create a solution or adapt existing solutions to the problem at hand. Methods in the evaluation phase are used to test a solution in a laboratory setting, through simulation, or when implemented in a clinical or patient setting, with measures often focusing on human-centered outcomes such as usability, use errors, mental workload, and effect on workflow. HFE measures can be collected alongside traditional measures of clinical effectiveness, safety, and cost to comprehensively assess the intervention’s consequences. The most robust applications of HFE involve the combination of methods across all three phases: study, design, and evaluation. Such comprehensive application of HFE involves both research and solution development,¹²⁶ as illustrated by the three cases below and elsewhere.¹⁵⁶–¹⁵⁸

Fig. 1

Fig. 1 The three phases of human-centered design.

Case 1: Medication package, label, and information design

HFE researchers have studied human processing of medication information to design and evaluate patient-centered instructions and labels.¹⁵⁹ Findings show, for instance, that designs matched with older and younger adults’ mental schema for taking medications enhance their memory of medication information,¹⁶⁰–¹⁶² and consequently improve their medication adherence.¹⁶³ As another example, people prefer larger print and line spacing, additional white space, instructions organized as lists, and extended surface areas (pull-out labels) on medication containers.¹⁶⁴–¹⁶⁷ Labels incorporating such designs led to an improvement in patients’ response time and knowledge acquisition.¹⁶⁸–¹⁷⁰ Pictorials and icons were found to be useful to those patients with low health literacy or inadequate reading skills.¹⁷¹,¹⁷²

Researchers have also used eye tracking to evaluate how the design of medication labels may impact understanding and safety. One study focused on child resistant and product tampering warnings on OTC pain relievers.¹⁷³ Eye tracking was used in the study to quantify three measures related to the relative prominence and conspicuousness of the warnings: time spent examining the warnings compared to other areas of the label; recall of information from the OTCs viewed; and legibility of the warnings (how decipherable was the message) relative to the other label elements. Results showed less than 20% of participants registered any time in the product tampering warning zone and less than 50% of participants viewed the child resistant warning zone. Among all label information types, child resistant and product tampering warnings were least likely to be recalled and least legible. Therefore, despite legal requirements to highlight these warnings, the study demonstrated that the current design of OTC pain reliever packaging failed to effectively convey these important safety messages.

In a follow-up study, the team investigated how different OTC label designs attract attention to critical information, promote decision making, and facilitate rapid cross-product comparisons.¹⁷⁴ They sought to produce a medication label that successfully communicated critical drug information to at-risk older adults, thereby empowering them to make better medication selection decisions, and ultimately reducing adverse drug events. This study demonstrated improved patient attention to interactive and horizontal warning placements versus auxiliary labels placed vertically on prescription vials.¹⁷⁴

Case 2: Human-centered design of a digital decision aid for medication safety

A multidisciplinary team, including HFE and pharmacy experts, sought to improve safety and brain health for older adults by addressing use of potentially unsafe prescription and OTC anticholinergic medications. In the study phase, interviews and observations of OTC shopping behaviors were conducted with older adults who take anticholinergic medications. This was complemented by a simulation study of OTC decision making using standard scenarios¹⁷⁵ and a realistic OTC aisle mockup.⁴⁹ The goal was to understand how older adults made decisions, barriers to making safe choices, and knowledge about anticholinergics. To study cognitive decision making, the contextual inquiry technique was employed as individuals made decisions in actual pharmacies or in scenario-based simulations; this entailed opportunistically asking questions about a person’s thoughts while observing their behavior. In the simulation study, participants were also asked to rank order eight factors (e.g., cost, effectiveness, safety) influencing their OTC choices.

In the project’s intervention design phase, data were analyzed to create personas and workflow maps depicting decision steps and barriers.⁴⁹ Using findings from this and a parallel study,⁴⁸,¹⁷⁶ the team and its professional designers iteratively created multimedia content and mobile app software.¹⁷⁷ A prototype app was formatively tested with a sample of older adults (n = 11), iteratively refined over three design-test cycles, then programmed as working software.¹⁷⁷

In the testing phase, the team performed summative usability testing on the Brain Buddy app with 23 older adult anticholinergic users and feasibility tested the app with a subset (n = 17) at medium or high risk of anticholinergic brain harm.¹⁷⁷ Usability testing combined task-based testing with observation of performance as well as a self-report questionnaire, Holden's Simplified SUS for Older and Cognitively Impaired Adults.¹⁷⁸ The Brain Buddy app performed well on usability, scoring in the Good to Excellent range for the SUS, while uncovering opportunities for further redesign. Feasibility findings showed 100% felt better informed, 94% indicated planning to talk to their physician about anticholinergic medication, and for 82% their physician confirmed having the conversation in an actual visit.¹⁷⁷ The app was further refined, rebranded as the Brain Safe app, and is being evaluated for efficacy and safety in an ongoing clinical trial, which also includes measures of self-reported usability and passively logged usage.¹⁷⁹ Developed multimedia content were also included in a clinical trial of a multicomponent intervention in a safety-net primary care system.¹⁸⁰

Case 3: Clinic and pharmacy work system analysis of a medication cancellation health IT functionality

A multidisciplinary team partnered with a large health system in the Midwest United States to evaluate the implementation of CancelRx, a health IT functionality that sends a medication cancellation message from the clinic’s EHR to the outpatient dispensing software. The team sought to describe the impact of CancelRx implementation on two work systems: the outpatient clinic and the community pharmacy.

A proactive risk assessment was conducted to identify vulnerabilities and consequences associated with each step of the CancelRx process.¹⁸¹ Using the process map generated from the interviews and observations as a guide, 35 vulnerabilities that encompassed the clinic and pharmacy work systems were identified and organized into vulnerability themes (see Fig. 2). For example, an accurate clinic medication list was dependent upon the patient sharing information about which medications they were currently taking, clinic staff correctly documenting that information in the EHR, and proper interpretation of the medication list by the prescriber. Failure in any one of these steps could lead to the consequence of the medication list not being up to date or the patient continuing to take a medication that they should not be taking.

Fig. 2

Fig. 2 Process map with identified vulnerabilities in clinic and pharmacy work systems.

The fact that these consequences were not identified and addressed prior to CancelRx implementation may be due in part to the lack of transparency of the vulnerabilities across the clinic and pharmacy systems as well as a lack of diverse front-line users on the implementation team. While individual actors may understand the vulnerabilities that exist in their own sociotechnical system (i.e., in the clinic), they may not understand the vulnerabilities in other systems (i.e., the pharmacy) and the impact of their actions on one another.¹⁸²

Conclusions and considerations

Pharmacy, including the places where pharmacy professionals work and the multistep process of medication use across people and settings, can benefit from HFE. This is because pharmacy is a human-centered sociotechnical system with an existing tradition of studying or analyzing the present state, designing solutions to problems, and evaluating those solutions in laboratory or practice settings. To put it simply, HFE fits pharmacy.

HFE can be implemented in pharmacy as a mindset or set of principles that drive research or clinical practice operations. HFE principles and underlying theories¹⁸³–¹⁸⁵ can be usefully adopted as an operating system or culture, the way some organizations strive to become lean,¹⁸⁶ agile,¹⁸⁷ patient-centered,¹⁸⁸ or high-reliability¹⁸⁹ organizations.

HFE methods can be used to operationalize the HFE mindset or, importantly, as tools for individual projects. Several, but not all, of varieties of HFE methods suitable for studying and improving pharmacy phenomena were presented here. Interested readers are encouraged to consult other comprehensive methods texts.⁵,¹⁷,⁵⁶,⁹¹,¹⁵⁵,¹⁹⁰–¹⁹³ Training on HFE methods is also available, from webinars and workshops to short courses and degree-granting programs. In the United States, several health professional schools now offer HFE courses or degrees. In the UK, HFE education has been proposed as part of the pharmacy curriculum to address patient safety.¹⁹⁴ However, the best learning opportunities will likely come from hands-on experience, especially when applying the methods in field settings.¹⁹⁵,¹⁹⁶ A few articles and books have been written specifically about practical considerations and uses of HFE and related methods in the field,¹²¹,¹⁹⁷ but most HFE textbooks assume the methods will be used in both laboratory research and practice settings.

Those considering HFE methods or using them for the first time often wonder whether their use requires an HFE professional’s involvement, special training or certification, or a level of practical experience. When formal training or education is available, it is advisable to obtain it, because HFE methods are grounded in theories and mindsets that take time and tutelage to learn and internalize. We also believe collaborating with HFE professionals generally strengthens the quality of work. Moreover, many HFE professionals are available and motivated to partner with researchers, clinicians, administrators, and other pharmacy stakeholders to design, implement, or guide the use of HFE methods in pharmacy. These HFE professionals—and professionals from other human-centered disciplines or branches of HFE such as human-computer interaction—can also serve as consultants, mentors, and collaborators. Furthermore, the increasing community of pharmacy experts who have used HFE methods (some for decades!) can be the bridge for those just beginning to use HFE approaches. It is important to engage HFE professionals not only for the education they have but for the practical experiences they gathered modifying and implementing the methods across application areas. HFE experts can also help newer users select and customize off-the-shelf methods to fit the circumstances.¹⁷⁷

At the same time, many HFE methods can be taught to and learned by novices or obtained in the form of tools.¹⁹⁸ One might be able to learn the rudiments of a method by reading about it and experimenting with its use (preferably with some input from more masterful users). Some HFE experts are also developing do-it-yourself (DIY) HFE method supports (e.g., software, tools, how-to books, video content) to help early-stage users adopt HFE methods without committing malpractice.¹⁹⁹

Whatever the path, HFE methods can be fruitfully adopted and used for pharmacy research and practice, toward improved medication-related outcomes and better health.

Questions for further discussion

1.In what ways are human factors and ergonomics methods distinct from other approaches?

2.Of the many human factors and ergonomics methods available, how does one select which one(s) to use in a particular situation? Are there times when some methods are not suitable?

3.How might one combine human factors and ergonomics methods in a multi-part project?

4.How does someone who does not have an advanced degree in human factors and ergonomics learn to use human factors and ergonomics methods?

Application exercise/scenario

You are working with a technology startup company launched to develop a medication-management application (app) for people with dementia and their family caregivers. Your team is committed to the general idea of the app, has a competent team of software developers, and just secured seed funding. However, your main investor is a former dementia caregiver herself and insists that the app be nothing short of user-centered. The backer will give your team one year and an operating budget to apply the appropriate professional user-centered methodology. What methods might you use to deeply study the problem you are trying to solve, assess the current way medications are managed, and to characterize user needs? What methods will you choose and how will you apply them to design a truly user-centered prototype, before your software developers write their first line of code? How and when will you test the product to evaluate its user-centeredness? How will you organize your activities to ensure on-time, on-budget completion? How will you convince the investor that you used the appropriate professional methods and that your product is truly user-centered?

References

1 Karsh B., Holden R.J., Alper S.J., Or C.K.L. A human factors engineering paradigm for patient safety – designing to support the performance of the health care professional. Qual Saf Health Care. 2006;15:i59–i65.

2 Dul J., Bruder R., Buckle P., et al. A strategy for human factors/ergonomics: developing the discipline and profession. Ergonomics. 2012;55:377–395.

3 Chui M.A., Holden R.J., Russ A.L., et al. Human factors in pharmacy. Proc Hum Factors Ergon Soc Annu Meet. 2017;61:666–670.

4 Stanton N.A., Young M.S. What price ergonomics?. Nature. 1999;399:197–198.

5 Salvendy G. Handbook of Human Factors and Ergonomics. 4th ed. Hoboken, NJ: Wiley & Sons; 2012.

6 Proctor R.W., Van Zandt T. Human Factors in Simple and Complex Systems. Boca Raton, FL: CRC Press; 2018.

7 IEC/ISO 62366:2007. Medical devices – Application of Usability Engineering to Medical Devices 2007. 2007.

8 ISO Standard 9241-210:2010. Ergonomics of Human-System Interaction – Part 210: Human-Centred Design for Interactive Systems.. 2010;vol. ISO 9241-210:2010.

9 ISO/TR Standard 18529:2000. Ergonomics – Ergonomics of Human-System Interaction – Human-Centred Lifecycle Process Descriptions.. 2000;vol. ISO/TR 18529:2000.

10 Weinger M.B., Wiklund M., Gardner-Bonneau D. Handbook of Human Factors in Medical Device Design. Boca Raton, FL: CRC Press; 2011.

11 Chapanis A., Safrin M.A. Of misses and medicines. J Chronic Dis. 1960;12:403–408.

12 Cohen T., Ley E.J., Gewertz B.L. Human Factors in Surgery: Enhancing Safety and Flow in Patient Care. Boca Raton, FL: CRC Press; 2020.

13 Donchin Y., Gopher D. Around the Patient Bed: Human Factors and Safety in Health Care. Boca Raton, FL: CRC Press; 2013.

14 Institute of Medicine. To Err Is Human: Building a Safer Health System. Institute of Medicine Report on Medical Errors. Washington, DC: National Academies Press; 2000.

15 Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.

16 Institute of Medicine. Patient Safety: Achieving a New Standard for Care. Washington, DC: National Academies Press; 2004.

17 Carayon P. Handbook of Human Factors and Ergonomics in Patient Safety. 2nd ed. Mahwah, NJ: Lawrence Erlbaum; 2012.

18 Holden R.J., Voida S., Savoy A., Jones J.F., Kulanthaivel A. Human factors engineering and human–computer interaction: supporting user performance and experience. In: Finnell J., Dixon B.E., eds. Clinical Informatics Study Guide. New York: Springer; 2016:287–307.

19 Carayon P., Hoonakker P. Human factors and usability for health information technology: old and new challenges. Yearb Med Inform. 2019;28:71–77.

20 Hignett S., Carayon P., Buckle P., Catchpole K. State of science: human factors and ergonomics in healthcare. Ergonomics. 2013;56:1491–1503.

21 Holden R.J., Valdez R.S. Patient ergonomics: the science (and engineering) of patient work. In: Holden R.J., Valdez R.S., eds. The Patient Factor: Theories and Methods for Patient Ergonomics. Boca Raton, FL: CRC Press; 2021:3–18.

22 Holden R.J., Cornet V.P., Valdez R.S. Patient ergonomics: 10-year mapping review of patient-centered human factors. Appl Ergon. 2020;82:102972.

23 Holden R.J., Valdez R.S. Beyond disease: technologies for health promotion. Proc Int Symp Hum Factors Ergon Health Care. 2019;8:62–66.

24 Xie A., Carayon P. A systematic review of human factors and ergonomics (HFE)-based healthcare system redesign for quality of care and patient safety. Ergonomics. 2015;58:33–49.

25 Russ A.L., Fairbanks R.J., Karsh B., Militello L.G., Saleem J.J., Wears R.L. The science of human factors: separating fact from fiction. BMJ Qual Saf. 2013;22(10):802–808. doi:10.1136/bmjqs-2012-001450.

26 Carayon P., Wooldridge A., Hose B.-Z., Salwei M., Benneyan J. Challenges and opportunities for improving patient safety through human factors and systems engineering. Health Aff. 2018;37:1862–1869.

27 Waterson P., Catchpole K. Human factors in healthcare: welcome progress, but still scratching the surface. BMJ Qual Saf. 2016;25:480–484.

28 Gurses A.P., Ozok A.A., Pronovost P.J. Time to accelerate integration of human factors and ergonomics in patient safety. BMJ Qual Saf. 2012;21:347–351.

29 Carayon P., Wetterneck T.B., Cartmill R., et al. Characterising the complexity of medication safety using a human factors approach: an observational study in two intensive care units. BMJ Qual Saf. 2014;23:56–65.

30 Mickelson R.S., Holden R.J. Medication adherence: staying within the boundaries of safety. Ergonomics. 2018;61:82–103.

31 Holden R.J., Abebe E. Medication transitions: vulnerable periods of change in need of human factors and ergonomics. Appl Ergon. 2020;90:103279.

32 Doucette W.R., Rippe J.J., Gaither C.A., Kreling D.H., Mott D.A., Schommer J.C. Influences on the frequency and type of community pharmacy services. J Am Pharm Assoc. 2017;57:72–76.e71.

33 Trapskin K., Johnson C., Cory P., Sorum S., Decker C. Forging a novel provider and payer partnership in Wisconsin to compensate pharmacists for quality-driven pharmacy and medication therapy management services. J Am Pharm Assoc. 2009;49:642–651.

34 Gilmore V., Efird L., Fu D., LeBlanc Y., Nesbit T., Swarthout M. Implementation of transitions-of-care services through acute care and outpatient pharmacy collaboration. Am J Health Syst Pharm. 2015;72:737–744.

35 Evans J.L., Gladd E.M., Gonzalez A.C., et al. Establishing a clinical pharmacy technician at a United States Army military treatment facility. J Am Pharm Assoc. 2016;56:573–579.e571.

36 Fleagle Miller R., Cesarz J., Rough S. Evaluation of community pharmacy tech-check-tech as a strategy for practice advancement. J Am Pharm Assoc. 2018;58:652–658.

37 Flynn E.A., Barker K.N., Carnahan B.J. National observational study of prescription dispensing accuracy and safety in 50 pharmacies. J Am Pharm Assoc. 2003;43:191–200.

38 Kessler D.O., Arteaga G., Ching K., et al. Interns' success with clinical procedures in infants after simulation training. Pediatrics. 2013;131:e811–e820.

39 Allan E.L., Barker K.N., Malloy M.J., Heller W.M. Dispensing errors and counseling in community practice. Am Pharm. 1995;NS35:25–33.

40 Flynn E.A., Barker K.N. Effect of an automated dispensing system on errors in two pharmacies. J Am Pharm Assoc. 2006;46:613–615.

41 Angelo L.B., Christensen D.B., Ferreri S.P. Impact of community pharmacy automation on workflow, workload, and patient interaction. J Am Pharm Assoc. 2005;45:138–144.

42 Odukoya O.K., Chui M.A. E-prescribing: a focused review and new approach to addressing safety in pharmacies and primary care. Res Soc Adm Pharm. 2013;9:996–1003.

43 Odukoya O.K., Stone J.A., Chui M.A. E-prescribing errors in community pharmacies: exploring consequences and contributing factors. Int J Med Inform. 2014;83:427–437.

44 Al Juffali L., Al-Aqeel S., Knapp P., Mearns K., Family H., Watson M. Using the human factors framework to understand the origins of medication safety problems in community pharmacy: a qualitative study. Res Soc Adm Pharm. 2019;15:558–567.

45 Reilley S., Grasha A.F., Schafer J. Workload, error detection, and experienced stress in a simulated pharmacy verification task. Percept Mot Skills. 2002;95:27–46.

46 Johnson S., O'Connor E., Jacobs S., Hassell K., Ashcroft D. The relationships among work stress, strain and self-reported errors in UK community pharmacy. Res Soc Adm Pharm. 2014;10:885–895.

47 Chui M.A., Mott D.A. Community pharmacists’ subjective workload and perceived task performance: a human factors approach. J Am Pharm Assoc. 2012;52:e153–e160.

48 Chui M.A., Stone J.A., Holden R.J. Improving over-the-counter medication safety for older adults: a study protocol for a demonstration and dissemination study. Res Soc Adm Pharm. 2017;13:930–937.

49 Holden R.J., Srinivas P., Campbell N.L., et al. Understanding older adults' medication decision making and behavior: a study on over-the-counter (OTC) anticholinergic medications. Res Soc Adm Pharm. 2019;15:53–60.

50 Vest T.A., Gazda N.P., Schenkat D.H., Eckel S.F. Practice-enhancing publications about the medication use process in 2017. Am J Health Syst Pharm. 2019;76:667–676.

51 Sutherland A., Ashcroft D.M., Phipps D.L. Exploring the human factors of prescribing errors in paediatric intensive care units. Arch Dis Child. 2019;104:588–595.

52 Zheng Y., Jiang Y., Dorsch M.P., Ding Y., Vydiswaran V.V., Lester C.A. Work effort, readability and quality of pharmacy transcription of patient directions from electronic prescriptions: a retrospective observational cohort analysis. BMJ Qual Saf. 2021;30:311–319.

53 Gerhart J.M., Spriggs H., Hampton T.W., et al. Applying human factors to develop an improved package design for (Rx) medication drug labels in a pharmacy setting. J Saf Res. 2015;55:177–184.

54 Odberg K.R., Hansen B.S., Aase K., Wangensteen S. A work system analysis of the medication administration process in a Norwegian nursing home ward. Appl Ergon. 2020;86:103100.

55 Lee J.D., Wickens C.D., Liu Y., Boyle L.N. Designing for People: An Introduction to Human Factors Engineering. 3rd ed. Charleston, SC: CreateSpace; 2017.

56 Stanton N.A., Salmon P.M., Rafferty L.A., Walker G.H., Baber C., Jenkins D.P. Human Factors Methods: A Practical Guide for Engineering and Design. Surrey, UK: Ashgate; 2013.

57 Cooke N.J. Varieties of knowledge elicitation techniques. Int J Hum Comput Stud. 1994;41:801–849.

58 Carayon P., Cartmill R., Hoonakker P., et al. Human factors analysis of workflow in health information technology implementation. In: Carayon P., ed. Handbook of Human Factors and Ergonomics in Patient Safety. 2nd ed. Mahwah, NJ: Lawrence Erlbaum; 2012:507–521.

59 Helander M. A Guide to Human Factors and Ergonomics. Boca Raton, FL: CRC Press; 2005.

60 Carayon P., Wetterneck T.B., Rivera-Rodriguez A.J., et al. Human factors systems approach to healthcare quality and patient safety. Appl Ergon. 2014;45:14–25.

61 Carayon P. Human factors of complex sociotechnical systems. Appl Ergon. 2006;37:525–535.

62 Carayon P., Schoofs Hundt A., Karsh B., et al. Work system design for patient safety: the SEIPS model. Qual Saf Health Care. 2006;15:i50–i58.

63 Holden R.J., Carayon P., Gurses A.P., et al. SEIPS 2.0: a human factors framework for studying and improving the work of healthcare professionals and patients. Ergonomics. 2013;56:1669–1686.

64 Carayon P., Wooldridge A., Hoonakker P., Hundt A.S., Kelly M.M. SEIPS 3.0: human-centered design of the patient journey for patient safety. Appl Ergon. 2020;84:103033.

65 Werner N.E., Ponnala S., Doutcheva N., Holden R.J. Human factors/ergonomics work system analysis of patient work: state of the science and future directions. Int J Qual Health Care. 2021;33:60–71.

66 Holden R.J., Schubert C.C., Mickelson R.S. The patient work system: an analysis of self-care performance barriers among elderly heart failure patients and their informal caregivers. Appl Ergon. 2015;47:133–150.

67 Holden R.J., Valdez R.S., Schubert C.C., Thompson M.J., Hundt A.S. Macroergonomic factors in the patient work system: examining the context of patients with chronic illness. Ergonomics. 2017;60:26–43.

68 Saha S.K., Kong D.C., Thursky K., Mazza D. Antimicrobial stewardship by Australian community pharmacists: uptake, collaboration, challenges and needs. J Am Pharm Assoc. 2021;61:158–168.

69 Al-Jumaili A.A., Doucette W.R. A systems approach to identify factors influencing adverse drug events in nursing homes. J Am Geriatr Soc. 2018;66:1420–1427.

70 Abraham O., Myers M.N., Brothers A.L., Montgomery J., Norman B.A., Fabian T. Assessing need for pharmacist involvement to improve care coordination for patients on LAI antipsychotics transitioning from hospital to home: a work system approach. Res Soc Adm Pharm. 2017;13:1004–1013.

71 Odukoya O.K., Stone J.A., Chui M.A. Barriers and facilitators to recovering from e-prescribing errors in community pharmacies. J Am Pharm Assoc. 2015;55:52–58.

72 Odukoya O.K., Chui M.A. E-prescribing: characterisation of patient safety hazards in community pharmacies using a sociotechnical systems approach. BMJ Qual Saf. 2013;22:816–825.

73 Look K.A., Stone J.A. Medication management activities performed by informal caregivers of older adults. Res Soc Adm Pharm. 2018;14:418–426.

74 Bacci J.L., Coley K.C., McGrath K., Abraham O., Adams A.J., McGivney M.S. Strategies to facilitate the implementation of collaborative practice agreements in chain community pharmacies. J Am Pharm Assoc. 2016;56:257–265.

75 Chui M.A., Mott D.A., Maxwell L. A qualitative assessment of a community pharmacy cognitive pharmaceutical services program, using a work system approach. Res Soc Adm Pharm. 2012;8:206–216.

76 Craig C.M., Patzer B., Morris N.L. Minimizing the impact of interruptions in a pediatric retail pharmacy. Proc Hum Factors Ergon Soc Annu Meet. 2018;62:480–484.

77 Strauven G., Vanhaecht K., Anrys P., De Lepeleire J., Spinewine A., Foulon V. Development of a process-oriented quality improvement strategy for the medicines pathway in nursing homes using the SEIPS model. Res Soc Adm Pharm. 2020;16:360–376.

78 Holden R.J., Carayon P. SEIPS 101 and seven simple SEIPS tools. BMJ Qual Saf. 2021;doi:10.1136/bmjqs-2020-012538.

79 Jiancaro T., Jamieson G.A., Mihailidis A. Twenty years of cognitive work analysis in health care: a scoping review. J Cogn Eng Decis Mak. 2014;8:3–22.

80 Vicente K.J. Cognitive Work Analysis: Toward Safe, Productive, and Healthy Computer-Based Work. 1st ed. London: Lawrence Erlbaum Associates; 1999.

81 Naikar N. Cognitive Work Analysis: Foundations, Extensions, and Challenges. Fishermans Bend, Australia: Defence Science and Technology Organisation; 2012.

82 Naikar N. Beyond interface design: further applications of cognitive work analysis. Int J Ind Ergon. 2006;36:423–438.

83 Abebe E., Scanlon M.C., Lee K.J., Chui M.A. What do family caregivers do when managing medications for their children with medical complexity?. Appl Ergon. 2020;87:103108.

84 Baumgartner A., Kunkes T., Clark C.M., et al. Opportunities and recommendations for improving medication safety: understanding the medication management system in primary care through an abstraction hierarchy. JMIR Hum Factors. 2020;7:e18103.

85 Clark L.N., Benda N.C., Hegde S., et al. Usability evaluation of an emergency department information system prototype designed using cognitive systems engineering techniques. Appl Ergon. 2017;60:356–365.

86 Crandall B., Hoffman R.R. Cognitive task analysis. In: Lee J., Kirlik A., eds. The Oxford Handbook of Cognitive Engineering. Oxford: Oxford University Press; 2013:229–239.

87 Russ A.L., Militello L.G., Glassman P.A., Arthur K.J., Zillich A.J., Weiner M. Adapting cognitive task analysis to investigate clinical decision making and medication safety incidents. J Patient Saf. 2019;15:191–197.

88 Elkhadragy N., Ifeachor A.P., Diiulio J.B., et al. Medication decision-making for patients with renal insufficiency in inpatient and outpatient care at a US veterans affairs medical centre: a qualitative, cognitive task analysis. BMJ Open. 2019;9:e027439.

89 Holden R.J., Daley C.N., Mickelson R.S., et al. Patient decision-making personas: an application of a patient-centered cognitive task analysis (P-CTA). Appl Ergon. 2020;87:103107.

90 Daley C.N., Cornet V.P., Toscos T.R., Bolchini D.P., Mirro M.J., Holden R.J. Naturalistic decision making in everyday self-care among older adults with heart failure. J Cardiovasc Nurs. 2020;doi:10.1097/JCN.0000000000000778.

91 Crandall B., Klein G., Hoffman R.R. Working Minds: A Practitioner’s Guide to Cognitive Task Analysis. Cambridge, MA: MIT Press; 2006.

92 Mickelson R.S., Unertl K.M., Holden R.J. Medication management: the macrocognitive workflow of older adults with heart failure. JMIR Hum Factors. 2016. ;3:e27. https://humanfactors.jmir.org/2016/2012/e2027/.

93 Stanton N.A. Hierarchical task analysis: developments, applications, and extensions. Appl Ergon. 2006;37:55–79.

94 Lane R., Stanton N.A., Harrison D. Applying hierarchical task analysis to medication administration errors. Appl Ergon. 2006;37:669–679.

95 Lester C.A., Chui M.A. Using link analysis to explore the impact of the physical environment on pharmacist tasks. Res Soc Adm Pharm. 2016;12:627–632.

96 Vidulich M.A., Tsang P.S. Mental workload and situation awareness. In: Salvendy G., ed. Handbook of Human Factors and Ergonomics. 4th ed. Hoboken, NJ: Wiley & Sons; 2012:243–273.

97 Carayon P., Alvarado C.J. Workload and patient safety among critical care nurses. Crit Care Nurs Clin North Am. 2007;19:121–129.

98 Carayon P., Gurses A.P. A human factors engineering conceptual framework of nursing workload and patient safety in intensive care units. Intensive Crit Care Nurs. 2005;21:284–301.

99 Holden R.J., Scanlon M.C., Patel N.R., et al. A human factors framework and study of the effect of nursing workload on patient safety and employee quality of working life. BMJ Qual Saf. 2011;20:15–24.

100 Holden R.J., Patel N.R., Scanlon M.C., Shalaby T.M., Arnold J.M., Karsh B. Effects of mental demands during dispensing on perceived medication safety and employee well being: a study of workload in pediatric hospital pharmacies. Res Soc Adm Pharm. 2010;6:293–306.

101 Grasha A.F. Misconceptions about pharmacy workload. Can Pharm J. 2001;134:26–35.

102 Borg G.A.V. Psychophysical bases of perceived exertion. Med Sci Sports Exerc. 1982;14:377–381.

103 Lee J.D., Wickens C.D., Liu Y., Boyle L.N. Work physiology. In: Designing for People: An Introduction to Human Factors Engineering. 2017:448–477.

104 Grasha A.F., Schell K. Psychosocial factors, workload, and human error in a simulated pharmacy dispensing task. Percept Mot Skills. 2001;92:53–71.

105 Chui M.A., Look K.A., Mott