Original Study| Volume 21, ISSUE 4, P538-544.e1, April 2020

The Post-Acute Delayed Discharge Risk Scale: Derivation and Validation With Ontario Alternate Level of Care Patients in Ontario Complex Continuing Care Hospitals

Published:February 20, 2020DOI:



      To describe and validate the Post-acute Delayed Discharge Risk Scale (PADDRS), which classifies patients by risk of delayed discharge on admission to post-acute care settings using information collected with the interRAI Minimum Data Set (MDS) 2.0 assessment.


      Retrospective cohort study of individuals admitted to Ontario Complex Continuing Care (CCC) hospitals. Person-level linkage between interRAI MDS 2.0 assessments and Cancer Care Ontario Wait Time Information System records was performed.

      Setting and Participants

      Sample of 30,657 patients who received care in an Ontario CCC hospital and were assessed with the interRAI MDS 2.0 assessment between January 1, 2010, and March 31, 2013.


      Alternate Level of Care (ALC) designation of 30 or more days was used as the marker of delayed discharge. Scale validation was performed through computation of class-level effect sizes and receiver operating characteristic curves for each of Ontario's geographic health regions. Additionally, Clinical Assessment Protocol (CAP) decision-support tool trigger rates by PADDRS risk level were computed for problem areas that are clinically relevant with the delayed discharge outcome.


      Overall, 9.4% of the sample experienced the delayed discharge outcome. The PADDRS algorithm achieved an overall area under the curve (AUC) statistic of 0.74, which indicates good discriminatory ability for predicting delayed discharge. PADDRS is generalizable across geographic regions, with AUC statistics ranging between 0.61 and 0.81 across each of Ontario's 14 Local Health Integration Networks. PADDRS demonstrated strong concurrent validity, as the percentage of patients triggering CAPs increased with the risk of delayed discharge.

      Conclusions and Implications

      PADDRS combines numerous important clinical factors associated with delayed discharge from a post-acute hospital into a cohesive decision-support tool for use by discharge planners. In addition to early identification of patients who are most likely to experience delayed discharge, PADDRS has applications in risk-adjusted quality measurement of discharge planning efficiency.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of the American Medical Directors Association
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Côté M.J.
        Understanding patient flow.
        Decision Line. 2000; : 8-10
        • Marshall A.
        • Vasilakis C.
        • El-Darzi E.
        Length of stay-based patient flow models: Recent developments and future directions.
        Health Care Manag Sci. 2005; 8: 213-220
        • Victor C.R.
        • Healy J.
        • Thomas A.
        • Seargeant J.
        Older patients and delayed discharge from hospital.
        Health Soc Care Community. 2000; 8: 443-452
        • Lenzi J.
        • Mongardi M.
        • Rucci P.
        • et al.
        Sociodemographic, clinical and organisational factors associated with delayed hospital discharges: A cross-sectional study.
        BMC Health Serv Res. 2014; 14: 1-8
        • McCloskey R.
        • Jarrett P.
        • Stewart C.
        The untold story of being designated an alternate level of care patient.
        Healthc Policy. 2015; 11: 76-89
        • New P.W.
        • Cameron P.A.
        • Olver J.H.
        • Stoelwinder J.U.
        Defining barriers to discharge from inpatient rehabilitation, classifying their causes, and proposed performance indicators for rehabilitation patient flow.
        Arch Phys Med Rehabil. 2013; 94: 201-208
        • Gassoumis Z.D.
        • Fike K.T.
        • Rahman A.N.
        • et al.
        Who transitions to the community from nursing homes? Comparing patterns and predictors for short-stay and long-stay residents.
        Home Health Care Serv Q. 2013; 32: 75-91
        • Hirshon J.M.
        • Risko N.
        • Calvello E.J.
        • et al.
        Health systems and services: The role of acute care.
        Bull World Health Organ. 2013; 91: 386-388
        • New P.W.
        • Jolley D.J.
        • Cameron P.A.
        • et al.
        A prospective multicentre study of barriers to discharge from inpatient rehabilitation.
        Med J Aust. 2013; 198: 104-108
        • New P.
        Prospective study of barriers to discharge from a spinal cord injury rehabilitation unit.
        Spinal Cord. 2014; 53: 358-362
        • Bradley L.J.
        • Kirker S.G.B.
        • Corteen E.
        • et al.
        Inappropriate acute neurosurgical bed occupancy and short falls in rehabilitation: Implications for the national service framework.
        Br J Neurosurg. 2006; 20: 36-39
        • Worthington A.D.
        • Oldham J.B.
        Delayed discharge from rehabilitation after brain injury.
        Clin Rehabil. 2006; 20: 79-82
        • Challis D.
        • Hughes J.
        • Xie C.
        • Jolley D.
        An examination of factors influencing delayed discharge of older people from hospital.
        Int J Geriatr Psychiatry. 2014; 29: 160-168
        • Tan W.S.
        • Chong W.F.
        • Chua K.S.G.
        • et al.
        Factors associated with delayed discharges after inpatient stroke rehabilitation in Singapore.
        Ann Acad Med Singapore. 2010; 39: 435-441
        • Costa A.P.
        • Poss J.W.
        • Peirce T.
        • Hirdes J.P.
        Acute care inpatients with long-term delayed-discharge: Evidence from a Canadian health region.
        BMC Health Serv Res. 2012; 12: 172
        • Gaughan J.
        • Gravelle H.
        • Siciliani L.
        Testing the bed-blocking hypothesis: Does nursing and care home supply reduce delayed hospital discharges?.
        Health Econ. 2015; 24: 32-44
        • Landeiro F.
        • Leal J.
        • Gray A.M.
        The impact of social isolation on delayed hospital discharges of older hip fracture patients and associated costs.
        Osteoporos Int. 2015; 27: 737-745
        • Complex Continuing Care and Rehabilitation Provincial Leadership Council of the Ontario Hospital Association
        Optimizing the Role of Complex Continuing Care and Rehabilitation in the Transformation of the Health Care Delivery System.
        Ontario Hospital Association, Toronto, Ontario, Canada2006
      1. Access to Care—Alternate Level of Care Informatics. Provincial Monthly Alternate Level of Care Performance Summary. Ontario, Canada: Cancer Care Ontario, 2019
        • Nord P.
        Alternate level of care: Ontario addresses the long waits.
        Can Fam Physician. 2009; 55: 786
        • Turcotte L.
        • Hirdes J.
        Long-stay alternate level of care in Ontario Complex Continuing Care beds. Ontario Hospital Association Health System Reconfiguration.
        Ontario Hospital Association, Toronto, Ontario, Canada2015
        • Bernabei R.R.
        • Landi F.F.
        • Onder G.G.
        • et al.
        Second and third generation assessment instruments: The birth of standardization in geriatric care.
        J Gerontol. 2008; 63: 308-313
        • Hirdes J.P.
        • Ljunggren G.
        • Morris J.N.
        • et al.
        Reliability of the interRAI suite of assessment instruments: A 12-country study of an integrated health information system.
        BMC Health Serv Res. 2008; 8: 277
        • Carpenter I.
        • Hirdes J.
        Using interRAI assessment systems to measure and maintain quality of long-term care.
        in: A Good Life in Old Age? OECD, Paris2013: 93-139
        • Morris J.N.
        • Nonemaker S.
        • Murphy K.
        • et al.
        A commitment to change: Revision of HCFA's RAI.
        J Am Geriatr Soc. 1997; 45: 1011-1016
        • Hirdes J.P.
        • Mitchell L.
        • Maxwell C.J.
        • White N.
        Beyond the “iron lungs of gerontology”: Using evidence to shape the future of nursing homes in Canada.
        Can J Aging. 2011; 30: 371-390
        • Heckman G.A.
        • Gray Leonard C.
        • Hirdes J.P.
        Addressing health care needs for frail seniors in Canada: The role of interRAI instruments.
        CGS J CME. 2013; 3: 8-16
        • Heckman G.A.
        • Hirdes J.P.
        • Hébert P.C.
        • et al.
        Predicting future health transitions among newly admitted nursing home residents with heart failure.
        J Am Med Dir Assoc. 2019; 20: 438-443
        • Hirdes J.P.
        • Poss J.W.
        • Caldarelli H.
        • et al.
        An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): Secondary analyses of Ontario data submitted between 1996 and 2011.
        BMC Med Informa Decis Mak. 2013; 13: 27
        • Foebel A.D.
        • Hirdes J.P.
        • Heckman G.A.
        • et al.
        Diagnostic data for neurological conditions in interRAI assessments in home care, nursing home and mental health care settings: A validity study.
        BMC Health Serv Res. 2013; 13: 457
        • Poss J.W.
        • Jutan N.M.
        • Hirdes J.P.
        • et al.
        A review of evidence on the reliability and validity of Minimum Data Set data.
        Healthc Manage Forum. 2008; 21: 33-39
        • Morris J.N.
        • Jones R.N.
        • Fries B.E.
        • Hirdes J.P.
        Convergent validity of minimum data set-based performance quality indicators in postacute care settings.
        Am J Med Qual. 2004; 19: 242-247
        • Hirdes J.P.
        • Fries B.E.
        • Morris J.N.
        • et al.
        Integrated health information systems based on the RAI/MDS series of instruments.
        Healthc Manage Forum. 1999; 12: 30-40
        • Pfaff S.
        • Guerriero L.
        • Martalog J.
        • et al.
        Transitioning initial success into sustainable results: the future of the WTIS.
        Healthc Q. 2009; 12: 84-88
        • Williams B.C.
        • Li Y.
        • Fries B.E.
        • Warren R.L.
        Predicting patient scores between the functional independence measure and the minimum data set: Development and performance of a FIM-MDS “crosswalk”.
        Arch Phys Med Rehabil. 1997; 78: 48-54
        • Morris J.N.
        • Fries B.E.
        • Mehr D.R.
        • et al.
        MDS Cognitive Performance Scale.
        J Gerontol. 1994; 49: M174-M182
        • Hartmaier S.L.
        • Sloane P.D.
        • Guess H.A.
        • et al.
        Validation of the Minimum Data Set Cognitive Performance Scale: Agreement with the Mini-mental State Examination.
        J Gerontol A Biol Sci Med Sci. 1995; 50: M128-M133
        • Jones K.
        • Perlman C.M.
        • Hirdes J.P.
        • Scott T.
        Screening cognitive performance with the Resident Assessment Instrument for Mental Health Cognitive Performance Scale.
        Can J Psychiatry. 2010; 55: 736-740
        • Morris J.N.
        • Howard E.P.
        • Steel K.
        • et al.
        Updating the cognitive performance scale.
        J Geriatr Psychiatry Neurol. 2016; 29: 47-55
        • Hirdes J.P.
        • Frijters D.H.
        • Teare G.F.
        The MDS-CHESS scale: A new measure to predict mortality in institutionalized older people.
        J Am Geriatr Soc. 2003; 51: 96-100
        • Hirdes J.P.
        • Poss J.W.
        • Mitchell L.
        • et al.
        Use of the interRAI chess scale to predict mortality among persons with neurological conditions in three care settings.
        PLoS One. 2014; 9: e99066
        • Perlman C.M.
        • Hirdes J.P.
        The aggressive behavior scale: A new scale to measure aggression based on the minimum data set.
        J Am Geriatr Soc. 2008; 56: 2298-2303
        • Poss J.
        • Murphy K.M.
        • Woodbury M.G.
        • et al.
        Development of the interRAI Pressure Ulcer Risk Scale (PURS) for use in long-term care and home care settings.
        BMC Geriatr. 2010; 10: 67
        • Mor V.
        • Branco K.
        • Fleishman J.
        • et al.
        The structure of social engagement among nursing home residents.
        J Gerontol B Psychol Sci Soc Sci. 1995; 50: P1-P8
        • Gerritsen D.L.
        • Steverink N.
        • Frijters D.H.M.
        • et al.
        A revised Index for Social Engagement for long-term care.
        J Gerontol Nurs. 2008; 34: 40-48
        • Morris J.N.
        • Berg K.
        • Björkgren M.
        • et al.
        InterRAI Clinical Assessment Protocols (Caps): For Use With Community and Long-Term Care Assessment Instruments.
        interRAI, Washington, DC2010
        • Baumann M.
        • Evans S.
        • Perkins M.
        • et al.
        Organisation and features of hospital, intermediate care and social services in English sites with low rates of delayed discharge.
        Health Soc Care Community. 2007; 15: 295-305
        • Rojas-García A.
        • Turner S.
        • Pizzo E.
        • et al.
        Impact and experiences of delayed discharge: A mixed-studies systematic review.
        Health Expect. 2017; 21: 41-56
        • McCoy D.
        • Godden S.
        • Pollock A.M.
        • Bianchessi C.
        Carrot and sticks? The community care act (2003) and the effect of financial incentives on delays in discharge from hospitals in England.
        J Public Health (Oxf). 2007; 29: 281-287