Patient Returns to the Emergency Department - Wiley Online Library

ORIGINAL CONTRIBUTION

Patient Returns to the Emergency Department: The Time-to-return Curve Kristin L. Rising, MD, MS, Timothy W. Victor, PhD, Judd E. Hollander, MD, and Brendan G. Carr, MD, MS

Abstract Objectives: Although 72-hour emergency department (ED) revisits are increasingly used as a hospital metric, there is no known empirical basis for this 72-hour threshold. The objective of this study was to determine the timing of ED revisits for adult patients within 30 days of ED discharge. Methods: This was a retrospective cohort study of all nonfederal ED discharges in Florida and Nebraska from April 1, 2010, to March 31, 2011, using data from the Agency for Healthcare Research and Quality (AHRQ) Healthcare Cost and Utilization Project (HCUP). ED discharges were followed forward to identify ED revisits occurring at any hospital within the same state within 30 days. The cumulative hazard of an ED revisit was plotted. Parametric and nonparametric modeling was performed to characterize the rate of ED revisits. Results: There were 4,782,045 ED discharges, with 7.5% (95% confidence interval [CI] = 7.4% to 7.5%) associated with 3-day revisits, and 22.4% (95% CI = 22.3% to 22.4%) associated with 30-day revisits, inclusive of the 3-day revisits. A double-exponential model fit the data best (p < 0.0001), and a single hinge point at 9 days (multivariate adaptive regression splines [MARS] model) yielded the best linear fit to the data, suggesting 9 days as the most reasonable cutoff for identification of acute ED revisits. Multiple stratified and subgroup analyses produced similar results. Future work should focus on identifying primary reasons for potentially avoidable return ED visits instead of on the revisit occurrence itself, thus more directly measuring potential lapses in delivery of high-quality care. Conclusions: Almost one-quarter of ED discharges are linked to 30-day ED revisits, and the current 72hour ED metric misses close to 70% of these patients. Our findings support 9 days as a more inclusive cutoff for studies of ED revisits. ACADEMIC EMERGENCY MEDICINE 2014;21:864–871 © 2014 by the Society for Academic Emergency Medicine

T

he Affordable Care Act has rekindled efforts to increase the value of health care. Hospital readmissions within 30 days of inpatient discharge are frequent,1–5 costly,6 and actionable for improvement,7 and reducing 30-day hospital readmissions has become a prime target for decreasing cost and improving quality. The Centers for Medicare and Medicaid Services (CMS) implemented the Hospital Readmission Reduction Program on October 1, 2012, under which hospitals with excessive readmissions for patients with

the diagnoses of congestive heart failure, pneumonia, and acute myocardial infarction now receive reduced reimbursement. As a result of this program, multiple evidence-based interventions to reduce inpatient readmissions have been developed.7,8 Compared with the focus on inpatient admissions, little attention has been focused on recurrent emergency department (ED) visits. The fact that a patient returns to the ED is not inherently an adverse event,9 yet return visits within an acute time period after ED discharge

From the Department of Emergency Medicine (KLR, JEH), Thomas Jefferson University, Philadelphia, PA; the Department of Emergency Medicine and the Department of Biostatistics and Epidemiology (BGC), Graduate School of Education (TWV), University of Pennsylvania, Philadelphia, PA; and Kantar Health (TWV), Philadelphia, PA. Received January 15, 2014; revision received March 3, 2014; accepted March 13, 2014. Presented at the American College of Emergency Physicians Research Forum, Seattle, WA, October 2013. Funded by the Emergency Medicine Foundation/Emergency Nursing Association Foundation Directed Team Grant awarded to Drs. Rising and Carr, July 2013. The authors have no relevant financial information or potential conflicts of interest to disclose. Supervising Editor: Richard T. Griffey, MD, MPH. Address for correspondence and reprints: Kristin L. Rising, MD, MS; e-mail: [email protected].

864 864

ISSN 1069-6563 864 PII ISSN 1069-6563583

© 2014 by the Society for Academic Emergency Medicine doi: 10.1111/acem.12442

ACADEMIC EMERGENCY MEDICINE • August 2014, Vol. 21, No. 8 • www.aemj.org

warrant a closer look to determine potentially preventable reasons for return. A first step in this work is to determine what most reasonably qualifies as the “acute time period” during which a patient is most likely to have a return ED visit (ED revisit) following a recent ED discharge. Prior studies assessing adverse events after ED discharge have generally used three different times periods for ED revisits: 2 to 3,10–23 7 to 8,24–30 or 30 days.31–34 The rationale provided in these studies for the use of any of these time periods is limited. Regarding the use of 2 to 3 days, Lerman and Kobernick23 performed one of the first structured assessments of 72-hour returns, referencing American College of Emergency Physicians (ACEP) guidelines that identify these patients as “redflag patients” without further justification for selection of this time period. Keith et al.10 followed this work with another study of 72-hours returns, referencing Lerman and Kobernick23 and the same ACEP guidelines as their rationale for the use of this time period.10 Despite limited rationale for this time period by either group, subsequent studies such as those by Pierce et al.11 and Gordon et al.12 use little justification for their decisions to use this time period other than referencing back to the earlier studies. More recent studies have generally stopped referencing these early works, although the decision for time periods used is still without a clear evidence base, as evidenced in studies such as that by Pham et al.,35 which notes that the use of 72-hour returns as the measure of interest derives from “the commonly held belief that these patients represent premature discharges from the first ED visit.” Emergency department revisits that occur within 72 hours of prior ED discharges are increasingly used as a hospital quality metric, although there is no clear empiric basis for 72 hours as the most appropriate time period of focus. Determination of the timing of ED revisits is fundamental to future work aimed at identifying patients at high risk of unexpected ED revisits and at developing targeted strategies to predict and prevent this phenomenon. The objective of this study was to characterize the distribution of time to an ED revisit after a prior ED discharge. METHODS Study Design This was a retrospective analysis of ED discharges for patients 18 years and older. The study was approved by the University of Pennsylvania Institutional Review Board. Study Setting and Population The visits studied occurred in nonfederal hospitals in Florida and Nebraska from April 1, 2010, through March 31, 2011. We obtained data from the Agency for Healthcare Research and Quality (AHRQ) Healthcare Cost and Utilization Project (HCUP).36 HCUP maintains a national information resource of patient-level health care data that includes the State Emergency Department Database (SEDD) and the State Inpatient Database (SID). The SEDD contains discharge-level data on all ED visits to nonfederal hospitals that do not lead to hospital

865

admission. The SID contains discharge-level data on all hospital discharges from short-term, acute care, nonfederal hospitals, including those that originated in the ED. Combining these two data sources allows for inclusion of all ED visits regardless of patient disposition or the hospital at which the visits have occurred. We chose Florida and Nebraska as the states for this analysis because they have especially robust HCUP data for the variables included in this analysis, with 95.2% to 98.9% of person identifiers verified in the Florida data sets and 98.6% to 100% verified in the Nebraska data sets.34 Study Protocol We included all ED visits for adult (18 years and older) patients that ended with patient discharge from April 1, 2010 to March 31, 2011 as index ED encounters. The unit of analysis was each individual ED discharge, and thus the same patient may have had multiple index ED encounters included in analysis. We followed patients from the index encounters forward to determine details of the next visit by the same patient. This was done with use of two revisits variables within the HCUP data sets (visitlink and daystoevent) that allow for tracking of patients across time and hospital within both data sets. Cases in which the subsequent visit was another ED visit occurring within 30 days were flagged as ED revisits. ED revisits that again resulted in patient discharge were counted both as revisits and as new index ED discharges. We used an additional 30-day lookback period prior to April 1, 2010 to determine utilization prior to the study start date and a 30-day follow-up period after the end of the study period to identify ED revisits linked to ED discharges occurring in the last month of the study. For potential index cases, we excluded ED encounters that resulted in patient admission or patient death or had unknown disposition (see Data Supplement S1, available as supporting information in the online version of this paper). In addition, as our analysis was focused on returns after the decision to discharge a patient, we excluded cases in which patients decided to leave (against medical advice, left without treatment complete, left without being seen) without formal discharges. ED visits were included as revisit cases regardless of the final patient disposition. We created one master analytic file in which we excluded records missing data for the variables required for linking patients across visits and merged records involving transfers to avoid inappropriately counting a transfer to another hospital as a revisit. ED visits that were coded as ending with transfers out with no subsequent visit for linkage within 1 day were considered to have final dispositions of discharge. For cases in which patients were coded as having died prior to their final visits, all visits linked with these patients were omitted, as we considered there to be significant coding or linkage errors. Outcome Measures The primary outcome was to assess the time to an ED revisit after an index ED discharge, up to a maximum of 30 days after the index encounter. This assessment includes the derivation of models to mathematically

866

characterize the timing of ED revisits with a specific focus on identifying salient time points of change along the time-to-return curve. We calculated summary statistics to characterize the overall study population of index ED discharges as well as the subgroup of index ED discharges for which there was no evidence of prior 30-day ED or inpatient utilization, defined as “new episode of care” ED visits. In addition, we report patient-level statistics by reporting data as recorded at the first encounter for that patient in the data set. We derived a mathematical model of the time to an ED revisit after an index ED discharge. This data-driven exploratory analysis37 was performed in the following three steps: Step 1. We plotted the cumulative risk of an ED revisit after an index ED discharge by calculating the daily cumulative hazard function. Data were censored at 30 days for all index discharges that were without ED revisits within that time period. Step 2. We determined the mathematical function that best described the cumulative hazard function from the initial analysis. Visual inspection of the cumulative hazard suggested a logarithmic curve, and thus we performed parametric exponential modeling. We fit single and double exponential models for comparison to determine which model fit the data better, using the following equation: Survival = b0 + b1e–k1 9 Time + b2e–k2 9 Time. Step 3. We created nonparametric models using multivariate adaptive regression splines (MARS). MARS modeling is a form of regression analysis that is designed to fit the simplest linear piecewise regression possible to a data set.38,39 This method was used to 1) explore the robustness of the inferences from the parametric models regarding how well the exponential models fit the data and 2) assist in identification of any inflection points along the curve that represent salient points of change in the rate of return to the ED. We performed sensitivity analyses by repeating this three-step process for three subgroups to address the concern of high-utilizer patients overly influencing the results. The first subgroup was limited to index ED discharges determined to be “new episodes of care,” which we defined as ED discharges for which there was no evidence of prior 30-day ED or inpatient utilization. This analysis was performed to eliminate inclusion of multiple index ED discharges for the same patient occurring within less than 30 days. The second subgroup included only one index ED discharge per patient, with each discharge selected at random. This selection was done by first setting a seed number, so the process would be repeatable, and then assigning a random number to every observation with use of the uniform function from STATA 13-MP. Data were then sorted first by patient identifier and then by random number, and the first observation for each patient was selected. The third subgroup excluded “high-utilizers” altogether, which we defined based on prior literature as patients with more than four index ED discharges over 1 year.40,41 In addition, we repeated analyses stratified by: age; sex; primary payer; same discharge diagnosis on both ED visits compared to different diagnoses for the two visits; discharge diagnosis at return visit of congestive

Rising et al. • ED TIME-TO-RETURN CURVE

heart failure, pneumonia, and acute myocardial infarction versus all other diagnoses; disposition from return ED visit (admitted vs. not admitted); and hospital state (Florida versus Nebraska). Discharge diagnoses were examined in two ways, first aggregated with the singlelevel Clinical Classifications Software (CCS) for the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) developed by AHRQ42 and then grouped into 17 major disease categories per methodology developed in prior studies.43 This second grouping was done to allow for better matching of visits potentially related to each other but with different specific diagnoses (i.e., dyspnea and pneumonia). Finally, we explored differences in discharges with revisits occurring before versus after the time inflection point(s) identified in the analysis above (9 days). We used half-lives from the exponential model derived to estimate revisit rates of the more acutely returning population that dominates the initial part of the return curve. In addition, we fit a logistic multivariable regression model to characterize the contribution of each variable to the likelihood of a revisit within 9 days. We used a direct logistic regression model where all predictors were included in the model simultaneously. Standard errors were computed using the Huber-White sandwich estimator to account for the lack of independence between observations in the data. Model assumptions were verified through visual inspection. Further, the results from Lumley et al.44 suggest that with large data sets, linear models appear to be quite robust even to the most significant assumption violations. Data Analysis All data management and survival analyses were performed using STATA 13-MP. Parametric and MARS models were performed using R 3.0.1.45 RESULTS We identified 4,782,045 index ED discharges meeting inclusion criteria. Overall 357,092 ED discharges (7.5%, 95% confidence interval [CI] = 7.4% to 7.5%) were associated with 3-day revisits, and 1,069,936 ED discharges (22.4%, 95% CI = 22.3% to 22.4%) were associated with 30-day revisits, inclusive of the 3-day revisits. At the patient level, the mean age of patients was 45.6 years (95% CI = 45.6 to 45.6 years) and over half (58%) were female. The majority of index ED discharges (77%) were new episodes of care (not linked to any ED or inpatient discharges in the prior 30 days). The most notable difference between the overall group of index ED discharges and those reported at the patient level or those that were new episodes of care was in the distribution of primary payer and patient age, with increased representation of Medicaid patients and patients age 25 to 44 years noted in the overall sample (Table 1). Figure 1 is a patient-level analysis depicting the number of index ED discharges contributed by each patient to the entire sample of index ED discharges (dark bars) as well as to the subgroup of index ED discharges associated with subsequent 30-day ED revisits (light bars). Over the 1-year study period, most patients (69%) had


867

Table 1 Demographics of Study Population: All ED Discharges (Primary Study Group), One ED Discharge per Patient (Patient Level), and ED Discharges Associated With New Episodes of Care

Characteristic Age (yr), mean (range) 18–24 25–44 45–64 ≥65 Sex, female Payer Medicare Medicaid Private Other* Median household income (patient zip)† Quartile 1 Quartile 2 Quartile 3 Quartile 4 Missing Location, urban Weekend visit

All ED Discharges, N = 4,782,045 44 852,179 1,862,499 1,274,472 792,894 2,879,484

(18–111) (17.8) (39.0) (26.7) (16.6) (60.2)

1,003,325 973,820 1,248,134 1,556,766 1,003,325

(21.0) (20.4) (26.1) (32.6) (21.0)

1,622,698 1,512,356 1,117,254 419,587 110,150 4,180,639 1,338,889

(33.9) (31.6) (23.4) (8.8) (2.3) (87.4) (28.0)

One ED Discharge per Patient Patient Level), n = 2,943,291 46 486,081 1,059,428 818.577 579,205 1,698,522

ED Discharges Associated with New Episodes of Care, n = 3,391,162

(18–111) (16.5) (36.0) (27.8) (19.7) (57.7)

44 659,856 1,416,897 984,473 629,936 2,215,919

(18–111) (17.9) (38.4) (26.7) (17.1) (60.0)

656,953 433,495 928,812 924,031 656,953

(22.3) (14.7) (31.6) (31.4) (22.3)

753,003 664,532 1,071,440 1,202,187 753,003

(20.4) (18.0) (29.0) (32.6) (20.4)

930,029 915,377 728,928 306,672 62,285 2,578,948 840,314

(31.6) (31.1) (24.8) (10.4) (2.1) (87.6) (28.6)

1,211,428 1,161,968 882,776 346,126 78,864 3,232,755 1,041,162

(33.1) (31.5) (23.9) (9.4) (2.1) (87.6) (28.2)

Data are reported as n (%) unless otherwise specified. *Other payer = self, no pay, other. †2010—Quartile 1 = $0–$40,999, Quartile 2 = $41,000–$50,999, Quartile 3 = $51,000–$66,999, Quartile 4 = $67,000+; 2011—Quartile 1 = $0–$38,999, Quartile 2 = $39,000–$47,999, Quartile 3 = $48,000–$63,999, Quartile 4 = $64,000+.

Figure 1. Number of index ED discharges contributed, by patient: provided for the entire study population and a subgroup of index ED discharges with associated 30-day revisits. Number of index ED discharges, by patient (x-axis label).

Figure 2. Cumulative hazard of an ED revisit within 30 days of index ED discharges, by day: multiple models (overall study population of all index ED discharges). MARS = multivariate adaptive regression splines.

one single index ED discharge, and 96% of patients had four or fewer index ED discharges (range = 1 to 55 visits). Among the subgroup of index ED discharges associated with 30-day ED revisits, 67% of patients had one single index ED discharge with a 30-day revisit, and 93% had four or fewer index discharges linked to 30day revisits. Eighty percent of patients (2,347,775 of 2,943,291) did not have any index discharges associated with 30-day revisits.

of an ED revisit is depicted by the 31 discreet data points. The smooth line connecting the data points is the double-exponential curve, which was determined to fit the data significantly better than the single-exponential model (p < 0.0001). This suggests there are at least two processes (or populations) responsible for the timing of ED revisits. The fit of the double-exponential model, as assessed by comparison of predicted versus observed hazards, was excellent (R2 = 0.9997). The two straight lines depict results from the MARS model, which placed a single hinge point at 9 days, further supporting the inference from the exponential models that there are at least two populations contributing to ED

Distribution of Time to Revisit Figure 2 demonstrates results from all three steps of the primary outcome analysis. The daily cumulative hazard

868


Table 2 Comparison of Index ED Discharges With and Without Subsequent 9-Day Revisits Including Results of Multivariable Logistic Regression Predicting 9-Day Revisits

Characteristic

ED Discharges With 9-Day ED Revisits, n = 624,466

Age (yr), mean (range) 43 (18–109) 18–24 107,827 (17.3) 25–44 257,028 (41.2) 45–64 167,720 (26.9) ≥65 91,891 (14.7) Sex, female 365,572 (58.5) Payer Medicare 138,097 (22.1) Medicaid 163,081 (26.1) Private 107,637 (17.2) Other* 215,651 (34.5) Median house income (patient zip)† Quartile 1 224,934 (36.0) Quartile 2 198,670 (31.8) Quartile 3 137,281 (22.0) Quartile 4 45,057 (7.2) Missing 18,524 (3.0) Location, urban 543,774 (87.1) Weekend visit 175,497 (28.1) New episode of care 362,178 (58.0)

ED Discharges Without 9-Day Revisits, n = 4,157,579

Multivariable Logistic Regression for 9-Day Revisit, AOR (95% CI)

44 744,352 1,605,471 1,106,753 701,003 2,513,912

(18–111) (17.9) (38.6) (26.6) (16.9) (60.4)

856,228 810,739 1,140,497 1,341,115

(20.8) (19.5) (27.4) (32.3)

1.22 (1.20–1.24) 1.18 (1.17–1.19) 0.67 (0.66–0.68)

1,397,764 1,313,686 979,973 374,530 1,397,764 3,636,865 1,163,392 3,328,984

(33.6) (31.6) (23.6) (9.0 (33.6) (87.5) (28.0) (80.1)

91,626 (2.2) 1.10 (1.08–1.11) 1.07 (1.06–1.09) 1.05 (1.04–1.07) 91,626 (2.2) 1.04 (1.03–1.06) 1.03 (1.02–1.04) 0.37 (0.37–0.38)

1.30 (1.28–1.33) 1.43 (1.41–1.46) 1.38 (1.36–1.41) 0.90 (0.90–0.91)

Data are reported as n (%) unless otherwise specified. Overall model fit: McFadden’s R2 = 0.042, LR = 152,002, p < 0.001. AOR = adjusted odds ratio. *Other payer = self, no pay, other †2010—Quartile 1 = $0–$40,999, Quartile 2 = $41,000–$50,999, Quartile 3 = $51,000–$66,999, Quartile 4 = $67,000+; 2011—Quartile 1 = $0–$38,999, Quartile 2 = $39,000–$47,999, Quartile 3 = $48,000–$63,999, Quartile 4 = $64,000+.

Table 3 Proportion of Early Return ED Revisits That Have Occurred at Each Half-Life, as Derived From the Double Decay Model % of Discharges With ED Revisit 50* 75 87.5 93.8 96.9 98.4 99.2 99.6

Half-life

Days

1 2 3 4 5 6 7 8

1.4 2.8 4.2 5.6 7.0 8.4 9.8 11.2

*This row represents the first half-life.

revisits. Results were similar for analyses of all three subgroups as well as all stratified samples, with a hinge point at 9 days in all cases: for life tables see Data Supplement S2, for model estimates see Data Supplement S3, and for the figure of model fit see Data Supplement S4 (all available as supporting information in the online version of this paper). Early and Late Returns The hinge point identified in the MARS model at 9 days represents the time at which a transition happens from the first process (the “early return” population) dominating the curve to the second process (the late return population) taking over the curve. Table 2 provides a comparison between index ED discharges with and without subsequent 9-day ED revisits, including results

of a logistic regression with the dependent outcome of an ED revisit within 9 days. All comparisons were statistically significant due to the extremely large sample size, although younger patient age, having a primary payer of Medicare or Medicaid, and presence of at least one ED or inpatient visit within the past 30 days were most highly associated with having a 9-day revisit. The half-lives from our double exponential model give the rate of decay (return) of the early return population (Table 3). Half of this population returned within 1.4 days, 75% returned within 2.8 days, and close to 99% returned within 9 days. DISCUSSION To our knowledge, this is the first study to rigorously quantify the rate of ED revisits after index ED discharges. On average, over 7% of ED discharges had ED revisits within 3 days, and over 22% of ED discharges had revisits within 30 days. Our findings suggest that patients return to the ED at a rate that is logarithmically decaying and that 72 hours is an inadequate time period to capture acutely returning ED patients. Indeed, use of 72-hour ED revisits misses close to 70% of patients who will re-present within 30 days following index ED visits and misses nearly one-quarter of patients returning within our empirically derived time frame of 9 days. We describe two distinct populations within our exponential model, which we identify as “early” and “late” return patients. We interpret the early return population as including a mix of patients who are “expected” to return after failing a reasonable trial of


outpatient treatment, as well as “unexpected” returns for whom there has been a problem in some portion of the discharge process or outpatient treatment plan. In contrast, we consider the late return population to represent primarily baseline ED revisits comprised of patients who return to the ED at a steady rate day after day for a number of reasons that may or may not be related to prior visits. The hinge point at 9 days represents the transition from the early return population to the late return population and thus is potentially a more appropriate cutoff to capture the majority of patients who are returning due to unexpected postdischarge complications. We recognize that this classification of patients into early and late returns is an oversimplification of ED revisits. In reality, there are two other key dimensions of revisits to consider in addition to the timing of the occurrence: relatedness and expectedness. Relatedness addresses whether the reason for return is due to a similar or associated problem compared to the prior ED visit. Expectedness strives to further categorize revisits with related returns and identifies returns that were anticipated or even requested by providers (e.g., wound check, suture removal) compared to those that failed for other unexpected reasons (e.g., unable to fill medications). Emergency care providers rely on the fact that a certain proportion of patients will return to the ED, especially those with “related expected returns,” as well as those who have failed reasonable trials of outpatient therapy. The goal, therefore, is not to entirely eliminate ED revisits, but rather to develop a system by which we can more reliably identify those revisits occurring due to potential lapses in quality so that we can better address the underlying quality issues. This preliminary work is an important first step in moving forward with a dialogue regarding how and whether to use ED returns as a quality measure, and there are multiple implications of these findings both for future research and for improving the quality of health care delivered. We highlight the importance of developing an evidence-based standard for the use of ED revisits in future studies. We are unable to find an empirically derived evidence base for 72 hours as the critical time period criterion in the peer-reviewed literature, CMS reporting, or the empirical study in hand. These findings suggest that researchers may be inadequately capturing primary events of interest. Further work is needed to explore the validity of 9 days as a more appropriate time period for use, but for now, we caution against routine use of 72 hours without careful consideration of the outcomes of interest to be captured, as there is no empirical basis for the use of this time period compared to others. In addition, we point out that ED revisits are not unusual occurrences, while also highlighting the finding that less than 5% of the overall patient population included in this study qualified as superutilizers (more than four ED visits per year), and analysis did not change with their exclusion. We suggest that there are a number of distinct issues contributing to patient returns and that the underlying drivers of both expected and unexpected revisits may be related to the patient, the provider, or systems factors. Thus, further study of whether there are distinct

869

types of returns that are more associated with potential lapses in quality are important before more widespread adoption of ED returns as a quality measure. This is the first study in which rigorous modeling is used to identify salient time points for returns. Our finding of increased rate of revisit for Medicaid-insured patients is consistent with prior literature suggesting especially limited access to outpatient service for the Medicaid population compared to those who are otherwise insured.46,47 The 7.5% 3-day revisit rate is high compared to prior literature, in which rates vary from 1.3% to 5.5%,10,19,35,48 although this is likely explained by the fact that prior studies have been limited to primarily single-institution analyses. Additional preliminary analysis of our data reveals that the majority of these returns occurring with the first 3 days are to different hospitals compared to the index ED discharges, thus explaining a significantly higher revisit capture rate. Future studies are required to assess the relevance of this timing and to replicate this work in other geographic settings and with distinct populations of patients and potentially health care networks. LIMITATIONS This work begins an important dialogue regarding the use of ED revisits as a quality measure, yet there are a number of limitations to address. A primary limitation of our study is the use of administrative claims data, which lack information regarding clinical situations. We were unable to determine whether the return visits were anticipated because of this lack of detailed clinical information. We did compare relation of discharge diagnoses between the two visits, obtaining the same results regardless of whether or not the diagnoses between the two visits were the same; however, more detailed review to examine the exact reasons for return and whether the returns were anticipated is not possible due to the nature of claims data. This limits our ability to draw conclusions regarding clinical differences between groups returning at different time points after ED discharges. Studies with more clinically rich data sets are needed to further explore details of individual clinical encounters. We also recognize that there are inherent limitations in any attempt to mathematically model complex clinical environments, and we spent substantial effort balancing model fit with ready interpretability. We censor our analysis at 30 days, and we recognize that findings could change with the selection of a different time point. Our decision was made based on an exhaustive literature review in which 30 days was identified as the longest time period used to assess adverse events potentially related to prior ED discharges. Finally, there are a number of limitations of the study related to data quality. The analysis was limited to the two states of Florida and Nebraska, which may have unique patients and care patterns that limit generalizability, yet it is reassuring that there were no substantive differences when comparing the two states. Records missing revisit variables (11%) were unable to be included as there was no means of imputing this linkage. Patient disposition was also difficult to verify

870

for certain cases. Linkage of transfers involved multiple steps and data were incomplete; thus, some cases may not have been matched correctly. In addition, observation stays were also not routinely coded, and thus disposition may have been incorrectly documented for patients if identified as ED discharges instead of as observation stays in the data set. Despite these limitations, with such a large data set, we believe that our findings are robust and add valuable insight into the occurrence of ED revisits. CONCLUSIONS As we move forward in establishing a performancebased health care system, we must continue to strive to develop empirically supported quality measures. Further research is needed to better characterize at-risk populations and to determine the main reasons that patients return in the acute time period after ED discharges. It may be that different reasons for return present within different time intervals and that there is not a “one-sizefits-all” answer regarding the time period most appropriate for monitoring quality. With better understanding of reasons for return, we can establish evidence-based quality measures focused on actual causes for patient return, instead of on the revisit occurrence itself, thus more directly measuring potential lapses in delivery of high-quality care. We find that 9 days is the most appropriate time period to use in the examination of ED revisits when analyzed from a strict mathematical viewpoint, as this is the time at which a distinction may be made between patients with early versus late revisits. Further work is under way to determine whether there are distinct and important differences in the populations that return within different time periods after ED discharges and how and whether distinct patient outcomes vary with the timing of returns. We encourage policy-makers, researchers, health professionals, and patients to work together to identify primary reasons for potentially avoidable return ED visits to inform future quality measurement initiatives. References 1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. NEJM 2009;360:1418–28. 2. Goodman DC, Fisher ES, Chang CH. After Hospitalization: A Dartmouth Atlas Report on Post-acute Care for Medicare Beneficiaries. Available at: http:// www.dartmouthatlas.org/downloads/reports/Post_discharge_events_092811.pdf. Accessed May 27, 2014. 3. Ross JS, Chen J, Lin ZQ, et al. Recent national trends in readmission rates after heart failure hospitalization. Circ Heart Fail 2010;3:97–103. 4. Bernheim SM, Grady JN, Lin Z, et al. National patterns of risk-standardized mortality and readmission for acute myocardial infarction and heart failure. Update on publicly reported outcomes measures based on the 2010 release. Circ Cardiovasc Qual Outcomes 2010;3:459–67.


5. Dharmarajan K, Hsieh AF, Lin Z, et al. Diagnoses and timing of 30-day readmissions after hospitalization for heart failure, acute myocardial infarction, or pneumonia. JAMA 2013;309:355–63. 6. Berwick DM, Hackbarth AD. Eliminating waste in US health care. JAMA 2012;307:1513–6. 7. Institute for Healthcare Improvement. STate Action on Avoidable Rehospitalizations (STAAR) Initiative: Applying Early Evidence and Experience in Frontline Process Improvements to Develop a State-based Strategy. Available at: http://www.ihi.org/offerings/ Initiatives/STAAR/Documents/STAAR State Based Strategy.pdf. Accessed Jan 9, 2014. 8. Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med 2009;150:178–87. 9. Abualenain J, Frohna WJ, Smith M, et al. The prevalence of quality issues and adverse outcomes among 72-hour return admissions in the emergency department. J Emerg Med 2013;45:281–8. 10. Keith KD, Bocka JJ, Kobernick MS, Krome RL, Ross MA. Emergency department revisits. Ann Emerg Med 1989;18:964–8. 11. Pierce JM, Kellerman AL, Oster C. “Bounces”: an analysis of short-term return visits to a public hospital emergency department. Ann Emerg Med 1990;19:752–7. 12. Gordon JA, An LC, Hayward RA, Williams BC. Initial emergency department diagnosis and return visits: risk versus perception. Ann Emerg Med 1998;32:569–73. 13. Martin-Gill C, Reiser RC. Risk factors for 72-hour admission to the ED. Am J Emerg Med 2004;22: 448–53. 14. Hu SC. Analysis of patient revisits to the emergency department. Am J Emerg Med 1992;10:366–70. 15. Khan NU, Razzak JA, Saleem AF, Khan UR, Mir MU, Aashiq B. Unplanned return visit to emergency department: a descriptive study from a tertiary care hospital in a low-income country. Eur J Emerg Med 2011;18:276–8. ~ ez S, Hexdall A, Aguirre-Jaime A. Unscheduled 16. Nun returns to the emergency department: an outcome of medical errors? Qual Saf Heal Care 2006;15:102–8. 17. Salluzzo RF, Bartfield JM. Evaluating the effectiveness of continuous quality improvement. in: Siegel DM, Crocker PJ, eds. Continuous Quality Improvement for Emergency Departments. Dallas, TX: American College of Emergency Physicians, 1994:83–88. 18. Lindsay P, Schull M, Bronskill S, Anderson G. The development of indicators to measure the quality of clinical care in emergency departments following a modified-Delphi approach. Acad Emerg Med 2002;9:1131–9. 19. Wu CL, Wang FT, Chiang YC, et al. Unplanned emergency department revisits within 72 hours to a secondary teaching referral hospital in Taiwan. J Emerg Med 2010;38:512–7. 20. Imsuwan I. Characteristics of unscheduled emergency department return visit patients within


21.

22.

23. 24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

48 hours in Thammasat University Hospital. J Med Assoc Thai 2011;94(Suppl 7):S73–80. Fan JS, Kao WF, Yen DH, Wang LM, Huang CI, Lee CH. Risk factors and prognostic predictors of unexpected intensive care unit admission within 3 days after ED discharge. Am J Emerg Med 2007;25:1009–14. Hu KW, Lu YH, Lin HJ, Guo HR, Foo NP. Unscheduled return visits with and without admission post emergency department discharge. J Emerg Med 2012;43:1110–8. Lerman B, Kobernick MS. Return visits to the emergency department. J Emerg Med 1987;5:359–62. Kefer MP, Hargarten SW, Jentzen J. Death after discharge from the emergency department. Ann Emerg Med. 1994;24:1102–7. Sklar DP, Crandall CS, Loeliger E, Edmunds K, Paul I, Helitzer DL. Unanticipated death after discharge home from the emergency department. Ann Emerg Med 2007;49:735–45. Gunnarsdottir OS, Rafnsson V. Death within 8 days after discharge to home from the emergency department. Eur J Public Health 2008;18:522–6. Lee DS, Schull MJ, Alter DA, et al. Early deaths in patients with heart failure discharged from the emergency department: a population-based analysis. Circ Heart Fail 2010;3:228–35. Guttmann A, Schull MJ, Vermeulen MJ, Stukel T. Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada [abstract]. BMJ 2011;342:d2983. Forster AJ, Rose NG, van Walraven C, Stiell I. Adverse events following an emergency department visit. Qual Saf Heal Care 2007;16:17–22. Gabayan GZ, Derose SF, Asch SM, et al. Patterns and predictors of short-term death after emergency department discharge. Ann Emerg Med 2011;58:551–8. Friedmann PD, Jin L, Karrison TG, et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med 2001;19:125–9. LaMantia MA, Platts-Mills TF, Biese K, et al. Predicting hospital admission and returns to the emergency department for elderly patients. Acad Emerg Med 2010;17:252–9. Reed MJ, Newby DE, Coull AJ, Prescott RJ, Jacques KG, Gray AJ. The ROSE (risk stratification of syncope in the emergency department) study. J Am Coll Cardiol 2010;55:713–21. Hess EP, Brison RJ, Perry JJ, et al. Development of a clinical prediction rule for 30-day cardiac events in emergency department patients with chest pain and possible acute coronary syndrome. Ann Emerg Med 2012;59:115–2. Pham JC, Kirsch TD, Hill PM, DeRuggerio K, Hoffmann B. Seventy-two-hour returns may not be a good indicator of safety in the emergency department: a national study. Acad Emerg Med 2011;18:390–7.

871

36. Agency for Healthcare Research and Quality. Healthcare Cost and Utilization Project (HCUP). Available at: http://www.hcup-us.ahrq.gov. Accessed May 26, 2014. 37. Tukey JW. Exploratory Data Analysis. Upper Saddle River, NJ: Pearson, 1977. 38. Friedman JH. Multivariate adaptive regression splines. Ann Stat 1991;19:1–67. 39. LeBlanc M, Crowley J. Adaptive regression splines in the Cox model. Biometrics 1999;55:204–13. 40. Locker TE, Baston S, Mason SM, Nicholl J. Defining frequent use of an urban emergency department. Emerg Med J 2007;24:398–401. 41. Althaus F, Paroz S, Hugli O, et al. Effectiveness of interventions targeting frequent users of emergency departments: a systematic review. Ann Emerg Med 2011;58:41–52. 42. Agency for Healthcare Research and Quality (AHRQ). Clinical Classifications Software (CCS) for ICD-9-CM. 2013. Available at: http://www.hcup-us. ahrq.gov/toolssoftware/ccs/ccs.jsp. Accessed Feb 26, 2014. 43. Karaca Z, Wong HS, Mutter RL. Duration of patients’ visits to the hospital emergency department. BMC Emerg Med 2012;12:15. 44. Lumley T, Diehr P, Emerson S, Chen L. The importance of the normality assumption in large public health data sets. Annu Rev Public Health 2002;23:151–69. 45. Milborrow S. Derived from mda:mars by Trevor Hastie and Rob Tibshirani. Uses Alan Miller’s Fortran utilities with Thomas Lumley’s leaps wrapper. Earth: Multivariate Adaptive Regression Spline Models. R package version 3.2-6. Available at: http://cran.r-project.org/web/packages/earth/earth. pdf. Accessed May 26, 2014. 46. Asplin B, Rhodes KV, Levy H, et al. Ambulatory care follow-up appointments. JAMA 2005;294:1248– 54. 47. Rhodes KV, Bisgaier J, Lawson CC, Soglin D, Krug S, Van Haitsma M. “Patients who can’t get an appointment go to the ER”: access to specialty care for publicly insured children. Ann Emerg Med 2013;61(4):394–403. 48. Liaw SJ, Bullard MJ, Hu PM, Chen JC, Liao HC. Rates and causes of emergency department revisits within 72 hours. J Formos Med Assoc 1999;98 (6):422–5. Supporting Information The following supporting information is available in the online version of this paper: Data Supplement S1. Study inclusion/exclusion criteria. Data Supplement S2. Life tables for subgroup analyses. Data Supplement S3. Model estimates. Data Supplement S4. Assessment of model fit.