Advertisement

Pragmatic Trials in Long-Term Care: Research Challenges and Potential Solutions in Relation to Key Areas of Care

      Abstract

      As a method of research, pragmatic trials are recommended so as to generate results that are applicable to real-world care. This intent is especially important for the millions of older adults who receive long-term care in thousands of nursing homes and assisted living communities across the country—and many millions more around the globe. This article presents key points raised by experts participating in a conference funded by the National Institute of Aging held at the 2021 conference of the Society for Post-Acute and Long-term Care Medicine. The purpose of the conference was to convene leading clinicians, researchers, and industry partners to address special considerations of pragmatic trials in long-term care. Cross-cutting and unique challenges and solutions to conducting pragmatic trials were discussed focusing on 3 areas of clinical relevance to long-term care: (1) functional care and outcomes, (2) psychosocial care and quality of life, and (3) medical care and outcomes, with a special focus on persons with dementia. Challenges and innovative solutions were organized across the 9 domains of the revised Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) Tool, and future research recommendations for pragmatic trials in long-term care were identified.

      Keywords

      There have been calls to improve the quality of long-term care for more than 30 years,
      Institute of Medicine (US) Committee on Nursing Home Regulation
      Improving the Quality of Care in Nursing Homes. National Academies Press; 1986.
      but changing behavior is challenging and implementing new approaches to care is slow. When new care practices are developed, they are typically evaluated in well-controlled clinical trials, meaning that the extent to which they are suited for practical real-world implementation is unclear. On the other hand, pragmatic trials examine real-world efficacy of an intervention in relevant settings and provide the opportunity to adapt it to address implementation challenges. Pragmatic trials are needed to better disseminate and implement effective approaches to care and deimplement ineffective approaches. They also encourage clinicians and others working in care settings to partner with researchers in the development of innovative approaches to care.
      • Voorberg W.
      • Bekkers V.
      • Tummers L.
      A systematic review of co-creation and co-production: Embarking on the social innovation journey.
      The National Institute on Aging provides guidance in designing research to effect change through their Stage Model, which defines the stages of research from basic science (Stage 0) to dissemination and implementation (Stage V); the conduct of pragmatic trials fits in Stage IV, effectiveness.
      • Onken L.
      • Carroll K.
      • Shoham V.
      • Cuthbert B.
      • Riddle M.
      Reenvisioning clinical science: unifying the discipline to improve the public health.
      Nursing homes and assisted living communities are complex adaptive systems, composed of numerous individuals interacting in ways that are not always predictable and always having to adapt.
      • Anderson R.
      • Issel L.
      • McDaniel R.J.
      Nursing homes as complex adaptive systems: relationship between management practice and resident outcomes.
      ,
      • Zimmerman S.
      • Mitchell C.
      • Beeber A.
      • et al.
      Strategies to reduce potentially inappropriate antibiotic prescribing in assisted living and nursing homes and inform other quality improvement efforts.
      Thus, optimal strategies to conduct pragmatic trials in long-term care are not straightforward. In response to the need to address special considerations of pragmatic trials in long-term care, especially for persons living with dementia, the National Institute of Aging funded a conference to bring together leaders to discuss related research challenges and potential solutions. To create focus, the organizers identified 3 areas of clinical relevance to long-term care: (1) functional care and outcomes, (2) psychosocial care and quality of life, and (3) medical care and outcomes. Clinical trials have demonstrated effective approaches to care in these areas, including to increase physical activity,
      • Galik E.
      • Resnick B.
      • Lerner N.
      • Hammersla M.
      • Gruber-Baldini A.
      Function focused care for assisted living residents with dementia.
      ,
      • Resnick B.
      • Boltz M.
      • Galik E.
      • et al.
      Testing the implementation of function focused care in assisted living settings.
      decrease falls,
      • Resnick B.
      • Boltz M.
      • Galik E.
      • et al.
      The impact of function focused care and physical activity on falls in assisted living residents.
      ,
      • Cameron I.
      • Dyer S.
      • Panagoda C.
      • et al.
      Interventions for preventing falls in older people in care facilities and hospitals.
      improve management of behavioral expressions,
      • Cohen-Mansfield J.
      Nonpharmacologic interventions for inappropriate behaviors in dementia: a review, summary, and critique.
      ,
      • Kolanowski A.M.
      • Litaker M.
      • Buettner L.
      • Moeller J.
      • Costa P.
      A randomized clinical trial of theory-based activities for the behavioral symptoms of dementia in nursing home residents.
      prevent transfers to acute care settings and address goals of care,
      • Hanson L.
      • Zimmerman S.
      • Song M.
      • et al.
      Effect of the goals of care intervention for advanced dementia: a randomized clinical trial.
      improve medication management,
      • Garland C.
      • Guénette L.
      • Kröger E.
      • Carmichael P.
      • Rouleau R.
      • Sirois C.
      A new care model reduces polypharmacy and potentially inappropriate medications in long-term care.
      and enhance infection/viral control.
      • Mody L.
      • Krein S.
      • Saint S.
      • et al.
      A targeted infection prevention intervention in nursing home residents with indwelling devices: a randomized clinical trial.
      ,
      • Gravenstein S.
      • Dahal R.
      • Gozalo P.
      • et al.
      A cluster randomized controlled trial comparing relative effectiveness of two licensed influenza vaccines in US nursing homes: design and rationale.
      However, evidence of efficacy based on randomized controlled trials does not generally change care practices in real world settings. Further, some care practices persist despite questionable effectiveness such as obtaining computed tomography of the head and neurologic checks following all falls
      • Pages P.
      • Boncoeur-Martela M.
      • Dalmay F.
      • et al.
      Relevance of emergency head CT scan for fall in the elderly person.
      and use of bed and chair alarms to prevent falls.
      • Shorr R.
      • Chandler A.
      • Mion L.
      • et al.
      Effects of an intervention to increase bed alarm use to prevent falls in hospitalized patients: a cluster randomized trial.

      Conducting Pragmatic Trials in Long-Term Care

      Investigators must understand the unique characteristics of people who live in long-term care settings when conducting pragmatic trials. The majority of residents have cognitive impairment among other comorbidities.
      • Hyde J.
      • Perez R.
      • Forester B.
      Dementia and assisted living.
      • Leon J.
      • Cheng C.
      • Neumann P.
      Alzheimer's disease care: costs and potential savings.
      • Zimmerman S.
      • Sloane P.
      • Reed D.
      Dementia prevalence and care in assisted living.
      • Harris-Kojetin L.
      • Sengupta M.
      • Park-Lee E.
      • et al.
      Long-term care providers and services users in the United States: data from the National Study of Long-Term Care Providers, 2013–2014.
      They frequently exhibit behavioral and psychological symptoms of dementia (now recognized as behavioral expressions)
      • Steinberg M.
      • Shao H.
      • Zandi P.
      • Lyketsos C.G.
      • Welsh-Bohmer K.
      Norton M; Cache County Investigators
      Point and 5-year period prevalence of neuropsychiatric symptoms in dementia: the Cache County study.
      and experience undertreated pain, functional decline, and limited physical activity.
      • Shiells K.
      • Pivodic L.
      • Holmerová I.
      • Van den Block L.
      Self-reported needs and experiences of people with dementia living in nursing homes: a scoping review.
      In addition, 25% of hospitalizations are considered potentially avoidable.
      Healthy People 2020. Dementias, including Alzheimer’s disease.
      Thus, there is a need to provide long-term care for persons with dementia and other geriatric syndromes that is not only efficacious but also scalable and practical for these complex care environments. Appreciation of other key aspects of these environments that influence intervention design and implementation include such things as staffing models, reimbursement, and regulatory constraints.
      In contrast to the development of new knowledge, funders and investigators have traditionally been less focused on dissemination or implementation of research findings into real world settings.
      • Glasgow R.
      • Vinson C.
      • Chambers D.
      • et al.
      National Institutes of Health approaches to dissemination and implementation science: current and future directions.
      Evidence-based research protocols are frequently too complex for implementation
      • Onken L.
      • Carroll K.
      • Shoham V.
      • Cuthbert B.
      • Riddle M.
      Reenvisioning clinical science: unifying the discipline to improve the public health.
      or not developed for actual users,
      • Miller H.
      • Ward K.
      • Zimmerman S.
      Implementation effectiveness of psychosocial and environmental care practices in assisted living.
      and clinical settings lack management support, organizational policies and practices, financial resource availability, organizational readiness for change, and measurement that is acceptable, compatible, practical, and useful.
      • Powell B.
      • Stanick C.
      • Halko H.
      • et al.
      Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping.
      ,
      • Weiner B.
      A theory of organizational readiness for change.
      To gain a better understanding of how to best integrate optimal care practices into long-term care settings, there has been an increased focus on conducting pragmatic trials.
      • Glasgow R.
      • Vinson C.
      • Chambers D.
      • et al.
      National Institutes of Health approaches to dissemination and implementation science: current and future directions.

      Differences Between Explanatory Trials vs Pragmatic Trials

      Explanatory trials are used to demonstrate the initial efficacy of an intervention and are conducted to support or refute a clear clinical hypothesis. These trials are critically important to determine whether an intervention will work under optimal conditions. In explanatory trials, interventions are tested under ideal and controlled situations and generally focus on individuals who are most likely to receive the greatest benefit; also, resources and staff are often provided beyond what is available in usual care environments. Explanatory trials strive to maximize the internal validity of results so that investigators have confidence that findings are due primarily to the effects of an intervention and not confounding factors. These trials create an evidence base for efficacy. Conversely, results from pragmatic trials are used to inform clinical or policy decisions by providing evidence that the intervention demonstrates effectiveness in any relevant setting.
      • Schwartz D.
      • Lellouch J.
      Explanatory and pragmatic attitudes in therapeutical trials.
      There are often, however, varying degrees to which a trial is considered “pragmatic” vs “explanatory” based on the type of intervention and the participants involved. To help determine the extent to which a trial is pragmatic, the Pragmatic-Explanatory Continuum Indicator Summary (PRECIS-2) Tool
      • Loudon K.
      • Treweek S.
      • Sullivan F.
      • Donnan P.
      • Thorpe K.
      • Zwarenstein M.
      The PRECIS-2 tool: Designing trials that are a fit for purpose.
      was developed. The purpose of PRECIS-2 is to help researchers design studies that are truly pragmatic.

      Description of PRECIS-2

      PRECIS-2 includes the following 9 domains (Table 1): Eligibility, Recruitment, Setting, Organization, Flexibility in Delivery, Flexibility in Adherence, Follow-up, Primary Outcome, and Primary Analysis. Each of the domains is evaluated by the individuals designing the study to determine to what degree the trial is pragmatic. Scoring on each domain ranges from very explanatory (1), rather explanatory (2), equally pragmatic and explanatory (3), rather pragmatic (4), or very pragmatic (5). Use of PRECIS-2 can help assure that truly practical trials are being conducted in long-term care and that they will be relevant for those living and working in these communities.
      Table 1Description of the Domains Within PRECIS-2
      • Loudon K.
      • Treweek S.
      • Sullivan F.
      • Donnan P.
      • Thorpe K.
      • Zwarenstein M.
      The PRECIS-2 tool: Designing trials that are a fit for purpose.
      DomainDescription
      EligibilityWho is eligible to be in the trial?
      RecruitmentHow are participants recruited into the trial?
      SettingWhere is the trial being conducted?
      OrganizationWhat expertise and resources are needed from the organization to deliver the intervention?
      Flexibility in deliveryHow should the intervention be delivered?
      Flexibility in adherenceWhat is being done to ensure adherence to the intervention?
      Follow-upHow closely are the participants monitored or followed?
      Primary outcomeHow relevant is the outcome to participants?
      Primary analysisAre all data included regardless of each individual’s level of participation?

      Pragmatic Trials in Long-Term Care: Challenges and Solutions

      The NIA-funded Pragmatic Trials in Long-Term Care conference included presentations by 10 experts (see acknowledgments) who discussed research challenges and solutions in 10 areas related to the 3 clinical foci delineated above: falls prevention, activities of daily living, function-focused care (related to functional care and outcomes); nonpharmacologic practices, aging in place, advance care planning (related to psychosocial care and quality of life); and medication use, prevention, infections, and health service use (related to medical care and outcomes). Using the PRECIS-2, they commented on up to 4 domains constituting particular challenges in pragmatic trials in long-term care in their area. Table 2 summarizes key challenges for the 9 PRECIS-2 domains, all of which are described in detail below. Although Table 2 is not a comprehensive list of all challenges, it demonstrates that challenges can occur across multiple domains of the PRECIS-2 Tool.
      Table 2Challenges Identified in Pragmatic Trials in Long-Term Care Based on Categories in the PRECIS-2
      TopicChallenges
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      ParticipantsRecruitmentSettingsOrganizationDeliveryAdherenceMeasurement Follow-upPrimary OutcomePrimary Analyses
      Falls prevention
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Heterogeneity among residents; resident goals may conflict with adherenceHeterogeneity in staff training and background; appropriate level of randomization may vary across facilitiesHeterogeneity in staffing models; turnoverMultifactorial interventions require multiple fidelity or process measuresUnreliable facility data; Hawthorne effect; surrogate outcomes not important to residents; process outcomes difficult to document
      Activities of daily living
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Variability of length of stay; timing of assessment and interventionVariability of settings; contamination risk if not randomized by facilityLack of sensitivity of assessments; attrition in target groupBias in proxy reports of function
      Function-focused care
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Challenges to assenting and consenting residents with dementiaIdentifying invested champion(s); securing oversight at the facility level; anticipating and managing staff turnover; allowing flexibility in deliveryCompetition with new initiatives; lack of accountability; need to educate new staffLack of sensitivity in proxy reports; timing of routine data collection may not capture results
      Nonpharmacologic practices
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Intervention must be relevant for the setting and have low initiation and maintenance costsIntervention must not require initiation by resident and should not exceed routine care activitiesAdherence must be monitored over timeOutcome measure must take into account that the week is 168 h
      Aging in place
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Need to consider cognition and function and presence of informal caregiver; should include persons in remote areas and diverse populations (eg, those with difficult family, mental illness)Remote or underserved settings are difficult to access; communities may change over time (eg, in food availability, walkability, gentrification, transportation)Ensuring a variety of preferences and individualized needs may result in low intervention specificity; requiring an infusion of resources may be impracticalOutcomes are broad (eg, relate to location, health status, mortality, social roles, activities)
      Advance care planning (ACP)
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Vulnerable population with complex consent issues; trials require assessment of capacity for participationPoor data integration between settings; longitudinal data require robust plan for input from diverse settingsIntervention fidelity data should be collected to understand outcomesACP research lacks consensus on key outcome measures; there are few validated measures of the quality of communication and goal-concordant care; hospitalization and utilization occur for a subset, limiting power
      Medication use
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Obtaining consent from residents who are decisionally impaired vs waiving consentDifferences in care models across countries; financing of long-term care and implications; provider expertise and resource availabilityStudy fidelity; stakeholder engagement; organizational support and readinessCompleteness of regulatory data and completeness and accuracy of existing data
      Prevention
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Recruit facilities, not residentsInfection control regulations vary by statePractice change is slow absent formal recommendationChallenges to maintaining access and completing entries for primary data, and data matching between data sets (provider and resident IDs)Evolving coding practicesAdjusting for facility-level factors
      Infections
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Targeted vs inclusive approach requires considerationShort-stay residents may be discharged before recruitment, introducing selection biasFacilities are often diverse in their focus, services, case mix, infection control infrastructure, availability and engagement of providers, and strength of connections with local hospitalsFocus on QI and research engagement can vary across facilitiesTime constraints, staff turnover, distance from academic centers, competing prioritiesStaff turnover, prior staff training, leadership engagement, priorities of the facility and their parent corporationsOutcome measures that require frequent follow-upOutcome definition cannot be complex (ie, definitions such as for UTI are often very specific); expectations from academic journals require rigorous assessmentsNeed to measure at the specimen, pathogen, visit, or patient level depending on the research question
      Health services use
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.
      Scarce resources, staff resistance, competing demands, instability of leadershipInterventions are complex; may need evidence to justify more intensive interventionsFidelity and fidelity monitoring need attention
      Total number635676572
      EHR, electronic health record; QI, quality improvement; UTI, urinary tract infection.
      Each presenter was asked to identify the 3 or 4 most pressing challenges to highlight; therefore, this list is not complete.

      Eligibility of Participants

      Highly pragmatic trials include broad eligibility criteria for participants so as to include diverse settings of care, residents, and/or staff. Even broad criteria require parameters, though, in that some interventions may be more useful for select residents and settings; such parameters need to be articulated and justified. One consideration for determining eligibility is ensuring that residents’ personal goals and preferences do not conflict with the intervention, such as a resident who prefers autonomy over falls-risk reduction when the setting is focused on falls prevention. Temporal considerations also influence eligibility considerations as for some studies it may be more appropriate to target new admissions regardless of the potential merit for all residents. Again, such decisions must be carefully weighed and justified if the trial is to be fully pragmatic.
      Further, different interventions may be necessary in pragmatic trials to recognize the heterogeneity of residents, which may affect eligibility or the intervention itself. Implementation of interventions for advance care planning, for example, requires inclusion of surrogate decision makers; the absence of such decision makers may affect eligibility. Likewise, health literacy can influence the appropriateness of the intervention for certain groups of participants, suggesting a need to attend to modify the intervention lest it not be optimally pragmatic.

      Potential solutions

      To overcome some of the challenges associated with the heterogeneity of potential participants, it may be necessary to utilize administrative data and identify key subgroups that will most likely benefit from the intervention or that reduce confounding. Conducting secondary or post hoc analyses on original efficacy trial data to obtain a better understanding of which subgroups responded positively or with the least amount of variance to the intervention can help target specific participants in the pragmatic trial phase. In this manner, the intervention could deliberately address different subgroups within a setting and tailor interventions for each of those groups. Consideration of advanced trial designs, such as sequential, multiple assignment, randomized trials (SMARTs) where participants are randomized at multiple stages during an intervention to allow for more adaptive approaches, may also be warranted to ensure effective tailoring of an intervention in long-term care.
      The Penn State Methodology Center
      Why Use a SMART Design to Build an Adaptative Intervention? The Pennsylvania State University College of Education and Human Development.
      ,
      • Almirall D.
      • Compton S.
      • Gunlicks-Stoessel M.
      • Duan N.
      • Murphy S.
      Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy.
      Shared decision-making tools are another way to match individual goals with the intervention being implemented.
      • Hanson L.
      • Zimmerman S.
      • Song M.
      • et al.
      Effect of the goals of care intervention for advanced dementia: a randomized clinical trial.

      Recruitment of Participants

      When conducting pragmatic trials, the goal is to include all eligible participants to best reflect standard practice. Therefore, recruitment must avoid strategies for recruitment that may bias participation. Challenges to recruitment in pragmatic trials may include the need to consent individuals if gathering identifiable data, making them de facto less pragmatic. Cognitive impairment impacts both consent and data collection. The inability to access caregivers or guardians and the lack of assent among the very residents who might benefit the most from the intervention (eg, those with behavioral expressions) are major challenges to recruitment to all trials in long-term care.
      • Resnick B.
      • Kolanowski A.
      • Van Haitsma K.
      • et al.
      Current psychotropic medication use and contributing factors among nursing home residents with cognitive impairment.

      Potential solutions

      Various approaches to avoid the need for or to facilitate consent include such things as consent waivers, broadcast notification, integrated consent, and targeted consent.
      • Carpenter J.
      • Connie Ulrich C.
      • Nancy Hodgson N.
      • Hanson L.
      • Ersek M.
      Alternative consent models in pragmatic palliative care.
      Consent waivers allow researchers to gather deidentified data already collected. Broadcast notification involves placing notices in prominent locations that inform potential patients of the ability to participate in research related to their care. Integrated consent integrates clinical and research consent into a single clinical encounter (often at the time of admission), whereas targeted consent involves a brief consent process followed by an information sheet for participants informing them that they are helping researchers explore a specific topic.
      One overall approach that ensures a trial is more pragmatic is to reduce reliance on primary data collection and instead use deidentified data extracted from health records; this approach reduces the need for informed consent but also limits the nature of the data available for study. It is also helpful to use a modified consent procedure and obtain verbal informed consent to reduce the burden of signed consent for a low-risk intervention. Limiting the complexity of an intervention will increase willingness to participate as will highlighting the relevance of the effort for potential participants, particularly when reaching out to those from underrepresented racial and ethnic groups.

      Inclusion of Settings

      Pragmatic trials should strive to include all types of long-term care settings as appropriate for their target population, but inherent differences in settings (such as nursing homes vs assisted living) may require separate trials for each setting type. In addition, pragmatic trials often rely on cluster randomization to avoid contamination between intervention and control within a given setting, thereby requiring a large sample of randomization units such as nursing homes to ensure adequate statistical power (eg, because individuals are nested within settings) and address variation. It may be challenging to ensure that settings randomized to the intervention vs usual care are in fact similar, particularly when a relatively small number of settings are included. It is also challenging for research teams to include settings that may be more rural and therefore more difficult to access. In addition, whether or not settings are part of a larger entity (eg, health system or corporation) may affect the resources available for an intervention; a pragmatic trial is one that is suitable for all relevant settings. Other considerations for pragmatic trials in long-term care include the services that are available (eg, separate dementia care units, electronic health records that connect with other sites such as hospitals), and also variations in relevant regulations due to state or regional differences.

      Potential solutions

      To best address or overcome these challenges, consideration should be given to the match between components of the intervention and care settings. For example, an intervention focused on optimizing function and physical activity would not match with a setting that focused on managing pain by (erroneously) decreasing mobility in an effort to lessen pain. Instead, a better match would be with a setting focused on falls prevention that supported the philosophy that physical activity can prevent falls. If it is feasible to include a large number of settings, it may be useful to account for variation in settings by creating subgroups (eg, within and without memory care units) and proceed with stratified random sampling. In addition, because imbalances between treatment and control group settings may occur when using cluster randomization, it may be advisable to use multiple-stage constrained randomization techniques to limit imbalance.
      • Roth D.
      • Huang J.
      • Gitlin L.
      • Gaugler J.
      Application of randomization techniques for balancing site covariates in the adult day service plus pragmatic cluster-randomized trial.
      Further to the point of matching the intervention with the setting, some have suggested assessing organizational readiness to determine the setting’s readiness and capability to carry out the various elements of the pragmatic trial.
      • Miake-Lye I.
      • Delevan D.
      • Ganz D.
      • Mittman D.
      • Finley E.
      Unpacking organizational readiness for change: an updated systematic review and content analysis of assessments.
      However, there is no uniform endorsement to limit intervention opportunities to those that are the most ready to enact them because doing so may disadvantage the settings that are most needy.
      • Levy C.
      • Zimmerman S.
      • Mor V.
      • et al.
      Pragmatic trials in long-term care: Implementation and dissemination challenges and opportunities.
      Finally, because some administrators may not prioritize a given intervention, it may be more useful to focus on settings with a particular interest in the intervention being offered; once effectiveness and pragmatism are established, hesitant administrators may be persuaded.

      Organization of Care Delivery

      Because pragmatic trials conduct interventions in “real-world” practice rather than highly controlled research settings, the organization of care delivery in long-term care is highly consequential. In this regard, challenges include heterogeneity in staff training and staffing ratios; limited and variable organizational resources; few or no policy requirements, recommendations, or accreditation standards related to the intervention content; and hands-off involvement of administrative staff, to name but a few. It is usually the case that time is limited to educate staff regarding a new intervention (especially if it is impractical and complex), that there are competing demands on staff time (even if time is available for training), that high industry-wide turnover rates affect sustainability, and that the lack of stable leadership or “champions” impedes adoption.

      Potential solutions

      To overcome infrastructure challenges, it may be helpful to assess the culture of the organization using culture assessment tools
      American Nurses Credentialing Center
      Self-assessment of organizational culture (for long term care).
      prior to starting the trial as well as organizational readiness as described earlier, perhaps with an eye toward promoting the culture and readiness of those less able. A principal objective when doing so is to match the intervention and study outcomes to the culture and priorities of care of the partner organization. Use of electronic health records and other tools embedded within the healthcare system can facilitate intervention delivery processes and should be fully incorporated into the data collection of a proposed pragmatic trial as appropriate. Other implementation approaches that may be helpful include using outside resources to provide education or training of staff (or relying on remote or asynchronous methods to do so)
      • Gitlin L.
      • Marx K.
      • Scerpellac D.
      • et al.
      Embedding caregiver support in community-based services for older adults: a multi-site randomized trial to test the Adult Day Service Plus Program (ADS Plus).
      and making sure that initiation and maintenance costs are low. Overall, it is best that the intervention fit the existing infrastructure, such as capitalizing on electronic learning management systems where modules can be uploaded, or regularly scheduled staff in-service sessions that can be leveraged to deliver intervention-related trainings.

      Flexible Delivery of the Intervention

      Pragmatic trials typically ensure that the delivery of the intervention is flexible so that it can be individualized for participant needs and fit better into real-world settings. A key concern in this regard is that it is not often clear which components of the intervention are modifiable if the outcomes shown to be efficacious in an explanatory trial are to be achieved. Further, the components that are modified by the staff may in fact be those that are central to achieving the intended outcome. For example, one study of a multicomponent intervention conducted in long-term care found that the combination of components (in this case, families attending a workshop, a care plan being developed, and being followed) was more related to outcomes than was any single component.
      • Washington T.
      • Zimmerman S.
      • Cagle J.
      • et al.
      Fidelity decision making in social and behavioral research: alternative measures of dose and other considerations.
      In this case, it was critical that all components be included to achieve the desired outcomes. Had the workshop not been offered, the intervention would have been less effective; it may have been possible to modify the workshop, but not to omit it completely.

      Potential solutions

      To optimize delivery, it is critical to ensure that the intervention is clearly defined and fits within the current workflow of the setting; doing so may be facilitated by offering a menu of potential modification strategies from which staff can choose to minimize the practice change required. When doing a pragmatic trial, the use of hybrid effectiveness designs are recommended because outcomes are considered alongside key measures of implementation (eg, appropriateness, feasibility, acceptability). This design can help ensure that the intervention aligns with the capacity and resources of the organization and can be implemented as intended.
      • Curran G.
      • Bauer M.
      • Mittman B.
      • Pyne J.
      • Stetler C.
      Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact.
      For a pragmatic trial to be successful, researchers must work with the care organization to optimize delivery of all components as intended and to identify the core elements of an intervention that must remain as well as the flexible elements that can be adapted to better fit the preferences and needs of residents, families, and staff. Doing so requires that the pragmatic trial include pragmatic measures of fidelity.

      Flexibility of Adherence

      Beyond delivering the intervention with suitable flexibility, it must be adhered to and sustained. Challenges to adherence include competing initiatives, resource-intensive interventions, and those perceived as useless to the staff or residents. There is widespread recognition among pragmatic trialists that fidelity must be assessed in an ongoing manner, must be done as unobtrusively as possible, and in a flexible manner that fits with daily workflow. Asking staff to monitor adherence can be burdensome, as found in a recent trial in which charting new care practices was more often missing than not.
      • Zimmerman S.
      • Wretman C.
      • Ward K.
      • Tandan M.
      • Sloane P.
      • Preisser J.
      Fidelity and sustainability of Mouth Care Without a Battle and lessons for other innovations in care.
      Moreover, it is not clear when to measure adherence and whether the focus is relevant for short- (less than 6 months) vs long-term adherence (greater than 6 months). Also challenging is when supervisors do not require accountability for changed care, which relates to the earlier pragmatic consideration regarding the setting itself.

      Potential solutions

      Much has been written about the benefits of identifying and involving a committed champion and stakeholder team to promote adoption and fidelity. Toward this end, embedding pragmatic trials in a model of quality improvement is likely to achieve the most buy-in and will by design include follow-up monitoring. Alternate modes of monitoring might also be considered, especially those that do not impose on staff. Technology is one such option, such as when the intervention itself provides counts or an observable means regarding use; in fact, it has been noted there is need to better use technology in care for persons with dementia.
      • Kolanowski A.
      • Fortinsky R.
      • Calkins M.
      • et al.
      Advancing research on care needs and supportive approaches for persons with dementia: recommendations and rationale.
      For example, the use of actigraphy with devices such as the Motionwatch8 is a useful way to capture data on physical activity in a pragmatic trial. Another option for monitoring may be including resident and family reports—if such can be done in a pragmatic way.

      Measurement of Follow-Up

      There are numerous challenges related to follow-up measurement of residents in long-term care. When care is intended to benefit a resident with dementia, the person may not be able to self-report, or self-report may be limited to responses with limited sensitivity to change, such as yes/no or simple Likert-type scales. Family may be called on as respondents, but proxy reports are known to be biased, often in a negative direction.
      • Magaziner J.
      • Zimmerman S.
      • Gruber-Baldini A.
      • Hebel J.R.
      • Fox K.M.
      Proxy reporting in five areas of functional status. Comparison with self-reports and observations of performance.
      Also, both of these options may create undue burden, contrary to the intent of pragmatic trials. Observational measures may be resource intensive and similarly not pragmatic unless they are technology-based. Further, use of secondary data may not coincide with optimal timing to detect change, and items such as from Medicare claims data or the Minimum Data Set may not be optimally relevant or sensitive to change.

      Potential solutions

      To overcome measurement challenges, it is helpful to capitalize on existing data. In so doing, it may be advisable to triangulate those data with other data sources to determine reliability (eg, physical activity data from Minimum Data Set data and actigraphy). If leadership is supportive of the trial, it may be possible to collaborate to incorporate new structured data elements within paper or electronic templates. Indeed, focusing on outcomes that are included in the clinical record, such as basing them on documented notes or goal attainment
      • Gordon J.E.
      • Powell C.
      • Rockwood K.
      Goal attainment scaling as a measure of clinically important change in nursing home patients.
      is a promising option to facilitate the availability of data if the records are sufficiently detailed. Regardless the strategy, it is critical that data elements be clearly defined (eg, definition of pneumonia or a urinary tract infection) and that if others are asked to provide the data, that they find it to be important.

      Relevance of Primary Outcomes

      Outcomes should be relevant to residents and other stakeholders in long-term care if the trial is to be pragmatic. In this regard, a key challenge is reconciling outcomes that are important to residents and stakeholders with available data (eg, recorded hospitalizations) and validated outcome measures. For example, researchers tend to consider hospitalizations as something to be avoided, whereas in some cases residents and their family may be reassured if there is a hospitalization. Relatedly, it may be that the wrong component of an outcome is being measured—such as measuring the completion of an advance directive when what truly mattered was having the related conversation. Process outcomes, such as whether or not the activity associated with the intervention was done, may be particularly relevant for pragmatic trial results, but difficult to document and capture.

      Potential solutions

      Pilot studies and review of existing literature may be the best strategy to ensure that the outcomes being collected are relevant to stakeholders. Toward this end, collaborating with long-term care settings during the development of the trial is indicated, and has been noted as a critical gap.
      • Levy C.
      • Zimmerman S.
      • Mor V.
      • et al.
      Pragmatic trials in long-term care: Implementation and dissemination challenges and opportunities.
      If residents, families, and staff endorse the importance of key outcomes, they may be more likely to support the collection of other data they find less central, including fidelity data noted earlier.

      Nature of Primary Analyses

      Pragmatic trials use an intention to treat analysis so as to ascertain effects of an intervention in real-world settings—as opposed to per-protocol analyses that examine effects when the intervention is provided as intended. As such, data from all participants are relevant, but in long-term care, attrition and dropout are common given resident death and staff turnover. Not only might their outcome data be unavailable but the resulting data may be biased given the selective nature of attrition.

      Potential solutions

      Analytically, adjusting for covariates that relate to outcomes may allow a more valid indication of the treatment effect. To do so, it is necessary that those covariates be known in advance and collected in advance, presumably from residents’ medical records (even if they are not fully sufficient for adjustment). Further, prior research may suggest setting specific factors known to influence outcomes and attrition that can be included in analytic models. Use of theory or conceptual models that consider the multilevel effects of interventions in long-term care may help inform setting-level characteristics to address in pragmatic trials.

      Successful Pragmatic Trials and Approaches Used for Sustainability of Interventions

      Across multiple areas, experts reported that effective and sustainable interventions were those that were feasible and allowed for flexibility in implementation. Feasible interventions are those that are integrated into the care plan or part of ongoing care processes,
      • Resnick B.
      • Boltz M.
      • Galik E.
      • et al.
      Testing the implementation of function focused care in assisted living settings.
      ,
      • Palmer J.
      • Parker V.
      • Barre L.
      • et al.
      Understanding implementation fidelity in a pragmatic randomized clinical trial in the nursing home setting: a mixed-methods examination.
      aligned with policy, or important to the setting based on relevant organizational, patient-centered, and/or regional/national goals.
      • Iida K.
      • Ryan A.
      • Hasson F.
      • Payne S.
      • McIlfatrick S.
      Palliative and end-of-life educational interventions for staff working in long-term care facilities: An integrative review of the literature.
      Active participation of a champion and stakeholder team was consistently mentioned as essential to successful implementation and sustainability.
      • Palmer J.
      • Parker V.
      • Barre L.
      • et al.
      Understanding implementation fidelity in a pragmatic randomized clinical trial in the nursing home setting: a mixed-methods examination.
      ,
      • Papaioannou A.
      • Loannidis G.
      • McArthur C.
      • et al.
      Preventing fractures in long-term care: translating recommendations to clinical practice.
      ,
      • Seshadri S.
      • Felsen C.
      • Sellers C.
      • Dumyati G.
      "There is no one to pick up the pieces": sustainability of antibiotic stewardship programs in nursing homes.
      Care initiatives that are perceived as beneficial to staff and residents are also more likely to be sustained by staff providing the care.
      • Resnick B.
      • Aboff-Pretzer I.
      • Galik E.
      • et al.
      Barriers and benefits to implementing a restorative care intervention in nursing homes.
      • Sloane P.D.
      • Hoeffer B.
      • Mitchell C.M.
      • et al.
      Effect of person-centered showering and the towel bath on bathing-associated aggression, agitation, and discomfort in nursing home residents with dementia: a randomized, controlled trial.
      • Calleson D.
      • Sloane P.
      • Cohen L.
      Effectiveness of mailing ‘Bathing Without a Battle' to all US nursing homes.
      The use of technology was repeatedly encouraged to facilitate scalability of interventions and approaches.
      • Germain A.
      • Markwald R.
      • King E.
      • et al.
      Enhancing behavioral sleep care with digital technology: study protocol for a hybrid type 3 implementation-effectiveness randomized trial.
      ,
      • Ben-Zeev D.
      • Buck B.
      • Meller S.
      • Hudenko W.
      • Hallgren K.
      Augmenting evidence-based care with a texting mobile interventionist: a pilot randomized controlled trial.
      Technology included wearable devices, use of electronic health records, and the development and use of apps to facilitate intervention delivery and staff training. There was strong consensus that successful implementation required that the intervention be adaptable or adjustable based on feedback from organizational partners and that consideration be given to the business of health care and cost/benefit of the new approach when compared to usual care. Lastly, deimplementation of ineffective interventions was recognized as an important component to successful implementation of new care practices.
      • Shelton R.
      • Brotzman L.
      • Johnson D.
      • Erwin D.
      Trust and mistrust in shaping adaptation and de-implementation in the context of changing screening guidelines.
      Deimplementation involves the removal or the replacement of care interventions that are known to be ineffective, harmful, or not beneficial.
      • Norton W.
      • Chambers D.
      Unpacking the complexities of de-implementing inappropriate health interventions.

      Conclusions and Implications

      Challenges to designing and implementing pragmatic trials in long-term care were noted across all 9 domains of PRECIS-2. Although not a comprehensive list, the most commonly noted challenges included identifying and obtaining primary outcomes that are relevant to residents and families as well as practical to collect, adherence to the intervention, allowing flexibility in delivery, including heterogeneous participants while simultaneously ensuring sufficient control to determine intervention effectiveness, and the variable nature of the organization.
      In long-term care, flexibility in delivery is critical and it may be beneficial to help facilities become ready to initiate the intervention.
      • Ojo T.
      • Kabasele L.
      • Boyd B.
      • et al.
      The role of implementation science in advancing resource generation for health interventions in low- and middle-income countries.
      In so doing, it is possible to ensure that more disadvantaged facilities are able to participate successfully. If a trial is to be optimally pragmatic, exclusions should be limited to settings in which the intervention lacks relevance. Recognizing setting-specific differences, there may be need to revise training or intervention materials so that they are culturally appropriate and relevant for staff and participants.
      • Green H.A.
      community-based evaluation of Screening, Brief Intervention, and Referral to Treatment (SBIRT) for the Black community.
      Technology and use of existing data are critical going forward, yet both need further development and evaluation.
      In a pragmatic trial, recruitment is done by providers and is offered to all who may benefit. Future work is required to establish strategies to ethically include residents who may not have the capability to assent or consent and may not have an identified proxy.
      • Shepherd V.
      How nurses can support the inclusion in research of older people who lack capacity to consent.
      • O'Reilly M.
      • Fetherstonhaugh D.
      • McMaster M.
      • Moyle W.
      • Fielding E.
      • Beattie E.
      Supporting autonomy of nursing home residents with dementia in the informed consent process.
      • Gustavson A.
      • Drake C.
      • Lakin A.
      • et al.
      Conducting clinical research in post-acute and long-term nursing home care settings: regulatory challenges.
      Waiving consent is allowable in situations in which the intervention is low risk, does not impact the welfare or rights of the participants, and when comparing 2 different but equally effective care approaches.
      United States Department of Human Health and Services
      Assuring compliance with this policy research conducted or supported by any Federal department or agency. 45 CFR.
      The waiver of consent, however, impacts the type of outcomes that can be obtained.
      Table 3 provides an overview of the areas of pragmatic trial research suggested by the expert participants in the 2021 conference. Some topics relate to methodology such as using new measurement approaches; others address learning more about strategies to ensure successful implementation by establishing factors within organizations that inform success, or translating interventions for new settings. The majority of the suggested areas of research involve using pragmatic trials to learn more about factors that influence functional care and outcomes (eg, falls), psychosocial care and quality of life (eg, advance care planning), and medical care and outcomes (eg, medication prescribing, infection prevention). The ideas promoted by the group at this meeting are not likely to be comprehensive of all possible challenges and solutions. For example, there may be a benefit to engaging with resident or family councils to vet potential outcome measures of importance to them or to seek the help of an ombudsman to facilitate recruitment of a participant.
      Table 3Future Areas of Suggested Research
      Functional care and outcomes
       • Evaluation of the environment as an active intervention ingredient (eg, Green Care Farms)
       • Use of Ecological Momentary Assessment and assessment of resident participation in activities of daily living
       • Strategies to motivate staff to engage residents in physical activity
       • Interventions to increased administrative support of Function Focused Care
       • Evaluation of phenotypes of residents who fall
       • Use of wearable devices to measure falls
      Psychosocial care and quality of life
       • Evaluation of what organizational assessment tools inform successful implementation
       • Impact of the environment on stimulation of persons with dementia
       • How activities of daily living can result in positive experiences between persons with dementia and their caregivers
       • Whether evidence of outcomes related to psychosocial care and quality of life is transferable to rural settings
       • Cognitive capacity and frailty as related to completion of advance care plans
       • Advance care planning outside the nursing home setting and in diverse populations
      Medical care and outcomes
       • Interventions to improve the quality of prescribing, including overprescribing
       • Impact of immunization on functional loss, cardiovascular events, and outbreak prevention, and comparisons of enhanced vaccines
       • Engaging residents, family, and visitors in infection prevention
       • The role of in-room surfaces in the transfer of bacteria and viruses
       • Development of systematic solutions to decrease transmission of pathogens, and implementation of interventions with proven evidence-based infection prevention
       • Evaluation of the potential of telehealth to provide services to residents
       • Connecting electronic health records across systems to improve patient transfers
       • Regaining a focus on person-centered care and the match of patient/family goals with treatment
      Pragmatic trials are critically important to our ability to disseminate and implement useful and sustainable interventions in long-term care. These types of trials ensure that the intervention is not only effective but can be implemented in real-world settings and organizations among a range of residents and staff. Although there are some challenges to designing and implementing truly pragmatic trials based on PRECIS-2 domains, challenges can be overcome using innovative approaches. In doing so, it is possible to build from the explanatory to the pragmatic and help ensure that dissemination and implementation of new interventions will be successful and sustainable in real-world settings.

      Acknowledgment

      Conference presenters and moderators included the authors and also Laura Hanson, MD, MPH, University of North Carolina at Chapel Hill.

      References

        • Institute of Medicine (US) Committee on Nursing Home Regulation
        Improving the Quality of Care in Nursing Homes. National Academies Press; 1986.
        https://pubmed.ncbi.nlm.nih.gov/25032432/
        Date accessed: November , 2021
        • Voorberg W.
        • Bekkers V.
        • Tummers L.
        A systematic review of co-creation and co-production: Embarking on the social innovation journey.
        Public Manage Rev. 2015; 17: 1333-1357
        • Onken L.
        • Carroll K.
        • Shoham V.
        • Cuthbert B.
        • Riddle M.
        Reenvisioning clinical science: unifying the discipline to improve the public health.
        Clin Psychol Sci. 2014; 2: 22-34
        • Anderson R.
        • Issel L.
        • McDaniel R.J.
        Nursing homes as complex adaptive systems: relationship between management practice and resident outcomes.
        Nurs Res. 2003; 52: 12-21
        • Zimmerman S.
        • Mitchell C.
        • Beeber A.
        • et al.
        Strategies to reduce potentially inappropriate antibiotic prescribing in assisted living and nursing homes and inform other quality improvement efforts.
        Am J Med Res. 2015; 2: 41-52
        • Galik E.
        • Resnick B.
        • Lerner N.
        • Hammersla M.
        • Gruber-Baldini A.
        Function focused care for assisted living residents with dementia.
        Gerontologist. 2015; 55: S13-S26
        • Resnick B.
        • Boltz M.
        • Galik E.
        • et al.
        Testing the implementation of function focused care in assisted living settings.
        J Am Med Dir Assoc. 2021; 22: 1706-1713
        • Resnick B.
        • Boltz M.
        • Galik E.
        • et al.
        The impact of function focused care and physical activity on falls in assisted living residents.
        Can J Nurs Res. 2020; 52: 45-53
        • Cameron I.
        • Dyer S.
        • Panagoda C.
        • et al.
        Interventions for preventing falls in older people in care facilities and hospitals.
        Cochrane Database Syst Rev. 2018; 9: CD005465
        • Cohen-Mansfield J.
        Nonpharmacologic interventions for inappropriate behaviors in dementia: a review, summary, and critique.
        Am J Geriatr Psychiatry. 2001; 9: 361-381
        • Kolanowski A.M.
        • Litaker M.
        • Buettner L.
        • Moeller J.
        • Costa P.
        A randomized clinical trial of theory-based activities for the behavioral symptoms of dementia in nursing home residents.
        J Am Geriatr Soc. 2011; 59: 1032-1041
        • Hanson L.
        • Zimmerman S.
        • Song M.
        • et al.
        Effect of the goals of care intervention for advanced dementia: a randomized clinical trial.
        JAMA Intern Med. 2017; 177: 24-31
        • Garland C.
        • Guénette L.
        • Kröger E.
        • Carmichael P.
        • Rouleau R.
        • Sirois C.
        A new care model reduces polypharmacy and potentially inappropriate medications in long-term care.
        J Am Med Dir Assoc. 2021; 22: 141-147
        • Mody L.
        • Krein S.
        • Saint S.
        • et al.
        A targeted infection prevention intervention in nursing home residents with indwelling devices: a randomized clinical trial.
        JAMA Intern Med. 2015; 175: 714-723
        • Gravenstein S.
        • Dahal R.
        • Gozalo P.
        • et al.
        A cluster randomized controlled trial comparing relative effectiveness of two licensed influenza vaccines in US nursing homes: design and rationale.
        Clin Trials. 2016; 13: 364-374
        • Pages P.
        • Boncoeur-Martela M.
        • Dalmay F.
        • et al.
        Relevance of emergency head CT scan for fall in the elderly person.
        J Neurol. 2020; 47: 54-58
        • Shorr R.
        • Chandler A.
        • Mion L.
        • et al.
        Effects of an intervention to increase bed alarm use to prevent falls in hospitalized patients: a cluster randomized trial.
        Ann Intern Med. 2012; 157: 692-699
        • Hyde J.
        • Perez R.
        • Forester B.
        Dementia and assisted living.
        Gerontologist. 2007; 47: 51-67
        • Leon J.
        • Cheng C.
        • Neumann P.
        Alzheimer's disease care: costs and potential savings.
        Health Aff. 1998; 17: 206-216
        • Zimmerman S.
        • Sloane P.
        • Reed D.
        Dementia prevalence and care in assisted living.
        Health Aff. 2014; 33: 658-666
        • Harris-Kojetin L.
        • Sengupta M.
        • Park-Lee E.
        • et al.
        Long-term care providers and services users in the United States: data from the National Study of Long-Term Care Providers, 2013–2014.
        Vital Health Stat. 2016; 38 (x–xii; 1–105)
        • Steinberg M.
        • Shao H.
        • Zandi P.
        • Lyketsos C.G.
        • Welsh-Bohmer K.
        • Norton M; Cache County Investigators
        Point and 5-year period prevalence of neuropsychiatric symptoms in dementia: the Cache County study.
        Int J Geriatr Psychiatry. 2018; 23: 170
        • Shiells K.
        • Pivodic L.
        • Holmerová I.
        • Van den Block L.
        Self-reported needs and experiences of people with dementia living in nursing homes: a scoping review.
        Aging Ment Health. 2019; 1: 1-16
      1. Healthy People 2020. Dementias, including Alzheimer’s disease.
        • Glasgow R.
        • Vinson C.
        • Chambers D.
        • et al.
        National Institutes of Health approaches to dissemination and implementation science: current and future directions.
        Am J Public Health. 2012; 102: 1274-1281
        • Miller H.
        • Ward K.
        • Zimmerman S.
        Implementation effectiveness of psychosocial and environmental care practices in assisted living.
        Geriatr Nurs. 2021; 42: 295-302
        • Powell B.
        • Stanick C.
        • Halko H.
        • et al.
        Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping.
        Implement Sci. 2017; 12: 118
        • Weiner B.
        A theory of organizational readiness for change.
        Implement Sci. 2009; 4: 67
        • Schwartz D.
        • Lellouch J.
        Explanatory and pragmatic attitudes in therapeutical trials.
        J Chron Dis. 1967; 20: 637-648
        • Loudon K.
        • Treweek S.
        • Sullivan F.
        • Donnan P.
        • Thorpe K.
        • Zwarenstein M.
        The PRECIS-2 tool: Designing trials that are a fit for purpose.
        BMJ. 2015; 350: 1-11
        • The Penn State Methodology Center
        Why Use a SMART Design to Build an Adaptative Intervention? The Pennsylvania State University College of Education and Human Development.
        https://www.methodology.psu.edu/ra/adap-inter/research/
        Date: 2020
        Date accessed: November , 2021
        • Almirall D.
        • Compton S.
        • Gunlicks-Stoessel M.
        • Duan N.
        • Murphy S.
        Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy.
        Stat Med. 2012; 31: 1887-1902
        • Resnick B.
        • Kolanowski A.
        • Van Haitsma K.
        • et al.
        Current psychotropic medication use and contributing factors among nursing home residents with cognitive impairment.
        Clin Nurs Res. 2021; 30: 59-69
        • Carpenter J.
        • Connie Ulrich C.
        • Nancy Hodgson N.
        • Hanson L.
        • Ersek M.
        Alternative consent models in pragmatic palliative care.
        J Pain Symptom Manag. 2021; 62: 183-190
        • Roth D.
        • Huang J.
        • Gitlin L.
        • Gaugler J.
        Application of randomization techniques for balancing site covariates in the adult day service plus pragmatic cluster-randomized trial.
        Contemp Clin Trials Commun. 2020; 19: 100628
        • Miake-Lye I.
        • Delevan D.
        • Ganz D.
        • Mittman D.
        • Finley E.
        Unpacking organizational readiness for change: an updated systematic review and content analysis of assessments.
        BMC Health Serv Res. 2020; 20: 106
        • Levy C.
        • Zimmerman S.
        • Mor V.
        • et al.
        Pragmatic trials in long-term care: Implementation and dissemination challenges and opportunities.
        J Am Med Dir Assoc. 2022; 23: 339-344
        • American Nurses Credentialing Center
        Self-assessment of organizational culture (for long term care).
        • Gitlin L.
        • Marx K.
        • Scerpellac D.
        • et al.
        Embedding caregiver support in community-based services for older adults: a multi-site randomized trial to test the Adult Day Service Plus Program (ADS Plus).
        Contemp Clin Trials. 2019; 83: 97-108
        • Washington T.
        • Zimmerman S.
        • Cagle J.
        • et al.
        Fidelity decision making in social and behavioral research: alternative measures of dose and other considerations.
        Soc Work Res. 2014; 38: 154-162
        • Curran G.
        • Bauer M.
        • Mittman B.
        • Pyne J.
        • Stetler C.
        Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact.
        Med Care. 2012; 50: 217-226
        • Zimmerman S.
        • Wretman C.
        • Ward K.
        • Tandan M.
        • Sloane P.
        • Preisser J.
        Fidelity and sustainability of Mouth Care Without a Battle and lessons for other innovations in care.
        Geriatr Nurs. 2020; 41: 878-884
        • Kolanowski A.
        • Fortinsky R.
        • Calkins M.
        • et al.
        Advancing research on care needs and supportive approaches for persons with dementia: recommendations and rationale.
        J Am Med Dir Assoc. 2018; 19: 1047-1053
        • Magaziner J.
        • Zimmerman S.
        • Gruber-Baldini A.
        • Hebel J.R.
        • Fox K.M.
        Proxy reporting in five areas of functional status. Comparison with self-reports and observations of performance.
        Am J Epidemiol. 1997; 146: 418-428
        • Gordon J.E.
        • Powell C.
        • Rockwood K.
        Goal attainment scaling as a measure of clinically important change in nursing home patients.
        Age Ageing. 1999; 28: 275-281
        • Palmer J.
        • Parker V.
        • Barre L.
        • et al.
        Understanding implementation fidelity in a pragmatic randomized clinical trial in the nursing home setting: a mixed-methods examination.
        Trials. 2019; 20: 2-10
        • Iida K.
        • Ryan A.
        • Hasson F.
        • Payne S.
        • McIlfatrick S.
        Palliative and end-of-life educational interventions for staff working in long-term care facilities: An integrative review of the literature.
        Int J Older People Nurs. 2021; 16: 1-14
        • Papaioannou A.
        • Loannidis G.
        • McArthur C.
        • et al.
        Preventing fractures in long-term care: translating recommendations to clinical practice.
        J Am Med Dir Assoc. 2021; 22: 36-42
        • Seshadri S.
        • Felsen C.
        • Sellers C.
        • Dumyati G.
        "There is no one to pick up the pieces": sustainability of antibiotic stewardship programs in nursing homes.
        Infect Control Hosp Epidemiol. 2021; 42: 440-447
        • Resnick B.
        • Aboff-Pretzer I.
        • Galik E.
        • et al.
        Barriers and benefits to implementing a restorative care intervention in nursing homes.
        J Am Med Dir Assoc. 2008; 2: 103-108
        • Sloane P.D.
        • Hoeffer B.
        • Mitchell C.M.
        • et al.
        Effect of person-centered showering and the towel bath on bathing-associated aggression, agitation, and discomfort in nursing home residents with dementia: a randomized, controlled trial.
        J Am Geriatr Soc. 2004; 52: 1795-1804
        • Calleson D.
        • Sloane P.
        • Cohen L.
        Effectiveness of mailing ‘Bathing Without a Battle' to all US nursing homes.
        Gerontol Geriatr Educ. 2006; 27: 67-79
        • Germain A.
        • Markwald R.
        • King E.
        • et al.
        Enhancing behavioral sleep care with digital technology: study protocol for a hybrid type 3 implementation-effectiveness randomized trial.
        Trials. 2021; 22: 1-14
        • Ben-Zeev D.
        • Buck B.
        • Meller S.
        • Hudenko W.
        • Hallgren K.
        Augmenting evidence-based care with a texting mobile interventionist: a pilot randomized controlled trial.
        Psychiatr Serv. 2020; 71: 1218-1224
        • Shelton R.
        • Brotzman L.
        • Johnson D.
        • Erwin D.
        Trust and mistrust in shaping adaptation and de-implementation in the context of changing screening guidelines.
        Ethn Dis. 2021; 31: 119-132
        • Norton W.
        • Chambers D.
        Unpacking the complexities of de-implementing inappropriate health interventions.
        Implement Sci. 2020; 15: 2
        • Ojo T.
        • Kabasele L.
        • Boyd B.
        • et al.
        The role of implementation science in advancing resource generation for health interventions in low- and middle-income countries.
        Health Services Insights. 2021; 3: 1-12
        • Green H.A.
        community-based evaluation of Screening, Brief Intervention, and Referral to Treatment (SBIRT) for the Black community.
        Qual Health Res. 2018; 28: 418-432
        • Shepherd V.
        How nurses can support the inclusion in research of older people who lack capacity to consent.
        Nurs Older People. 2021; 33: 26-31
        • O'Reilly M.
        • Fetherstonhaugh D.
        • McMaster M.
        • Moyle W.
        • Fielding E.
        • Beattie E.
        Supporting autonomy of nursing home residents with dementia in the informed consent process.
        Dementia. 2019; 18: 2821-2835
        • Gustavson A.
        • Drake C.
        • Lakin A.
        • et al.
        Conducting clinical research in post-acute and long-term nursing home care settings: regulatory challenges.
        J Am Med Dir Assoc. 2019; 20: 798-803
        • United States Department of Human Health and Services
        Assuring compliance with this policy research conducted or supported by any Federal department or agency. 45 CFR.

      Linked Article

      • Erratum
        Journal of the American Medical Directors AssociationVol. 23Issue 8
        • Preview
          Regarding the March 2022 article, Resnick B, et al. A Pragmatic Trials in Long-term Care: Research Challenges and Potential Solutions in Relation to Key Areas of Care. J Am Med Dir Assoc 2022;23(3):330-338.
        • Full-Text
        • PDF