Customer Satisfaction in a Large Urban Fire ... - Wiley Online Library

5 downloads 108105 Views 57KB Size Report
threatening emergencies received at the 911 call center. Forty-eight emergency medical technician. (EMT)-staffed BLS ambulances support paramedic services.
106

Persse et al.

d

CUSTOMER SATISFACTION IN AN URBAN EMS SYSTEM

Customer Satisfaction in a Large Urban Fire Department Emergency Medical Services System David E. Persse, MD, Jeffrey L. Jarvis, MS, EMT-P, Jerry Corpening, MPA, Bobbie Harris, MSN, RN Abstract Objectives: The purpose of this study was to determine if emergency medical services (EMS) customer satisfaction could be assessed using telephone-survey methods. The process by which customer satisfaction with the EMS service in a large, fire department–based EMS system is reported, and five month results are presented. Methods: Ten percent of all patients transported during the period of October 15, 2001, through March 15, 2002, were selected for study. In addition, during the same period, all EMS incidents in which a patient was not transported were identified for contact. Customer-service representatives contacted patients via telephone and surveyed them from prepared scripts. Results: A total of 88,528 EMS incidents occurred during the study period. Of these, 53,649 resulted in patient transports and 34,879 did not. Ten percent of patients transported (5,098) were selected for study participation, of which 2,498 were successfully contacted; of these,

2,368 (94.8%) reported overall satisfaction with the service provided. Of the 34,879 incidents without transport, only 5,859 involved patients who were seen but not transported. All of these patients were selected for study. Of these, 2,975 were successfully contacted, with 2,865 (96.3%) reporting overall satisfaction. The most common reason given for nonsatisfaction in both groups was the perception of a long response time. Conclusions: It is possible to conduct a survey of EMS customer satisfaction using telephonesurvey methods. Although difficulties exist in contacting patients, useful information is made available with this method. Such surveys should be an integral part of any EMS system’s quality-improvement efforts. In this survey, the overwhelming majority of patients, both transported and not transported, were satisfied with their encounter with EMS. Key words: satisfaction; emergency medical services. ACADEMIC EMERGENCY MEDICINE 2004; 11:106–110.

There has been much written about the evaluation of an emergency medical services (EMS) system’s performance.1–3 Recently, work has been performed on assessing the attitudes of patients in several different aspects of EMS practice, including complaints received by a fire department system,4 actual versus perceived response times,5 reader response to EMS continuing education journal articles,6 and the effects of feedback loops on patient satisfaction with nontransports in a quality improvement (QI) process.7 The importance of patient satisfaction also has been recognized in the EMS Agenda for the Future.8–10 Additionally, patient satisfaction was identified by an EMS system’s field paramedics as a critical indicator of the quality of an EMS system.11 Although there is common agreement on the need for a systematic process of quality improvement, little attention has

been paid to the role of patient satisfaction. Only Doering’s study addressed patient satisfaction, reporting a satisfied average of 4.75 on a 5.0 scale.8 The purpose of our study was to assess the satisfaction of the patients with encounters with a large fire department–based EMS system during a five-month study period utilizing a telephone survey.

From the Houston Fire Department, Emergency Medical Services, Houston, TX (DEP, JC, BH); the University of Texas Medical Branch at Galveston School of Medicine, Galveston, TX (JLJ); the University of Texas Medical School–Houston, Houston, TX (DEP); and Baylor College of Medicine, Houston, TX (DEP). Received December 27, 2002; revisions received May 1 and July 8, 2003; accepted July 8, 2003. Address for correspondence and reprints: David E. Persse, MD, 601 Sawyer Street, Suite 500, Houston, TX 77007. Fax: 713-865-4175; e-mail: [email protected]. doi:10.1197/S1069-6563(03)00594-3

METHODS Study Design. This was a descriptive study without therapeutic intervention or randomization. This study was approved by the institutional review board of the Baylor College of Medicine. Study Setting and Population. Houston Emergency Medical Services is operated through the Houston Fire Department, which provides basic life-support (BLS) first-responder coverage via 118 automated external defibrillator (AED)-equipped engine companies and ladder trucks to a 620–square mile metropolitan area. A priority dispatch system is used to dispatch one of 37 paramedic ambulances or nontransport response vehicles to all potentially lifethreatening emergencies received at the 911 call center. Forty-eight emergency medical technician (EMT)-staffed BLS ambulances support paramedic services. The annual call volume for fiscal year 2002

ACAD EMERG MED

d

January 2004, Vol. 11, No. 1

d

107

www.aemj.org

(July 2001 through June 2002) was 226,502 incidents with 142,455 patient transports. Patient satisfaction surveys are routinely performed as part of an ongoing quality-improvement program within our system. The survey results from October 15, 2001, through March 15, 2002, were analyzed for this study. All surveys of nontransported patients with apparent complete contact information were included, along with a 10% sample of those transported. The sample of transported patients selected for study was obtained by automated selection using a computer program that selected every tenth EMS transport call with complete patient contact information. A sample of 10% of transported patients was arbitrarily chosen for study after statistical consultation, because there were no pilot data to perform a power analysis. Survey Content and Administration. Survey questions were developed in consultation with faculty from the University of Texas School of Public Health and written into a script. Identical questions were utilized for both transported and nontransported patients. All scripted questionnaires began by asking the respondent’s consent to be surveyed. Consenting participants were asked to respond ‘‘yes’’ or ‘‘no’’ to seven questions, and two additional questions were asked that solicited open-ended responses. The final question asked patients to describe their degree of satisfaction with the care provided. Patients were contacted by telephone by one of our two full-time customer-service representatives. These representatives received training consisting of reviewing script questions, making practice calls, observing an experienced representative, and making 100 to 150 calls under the supervision of an experienced representative. The training sessions were competencybased so that the amount of time each trainee spent varied based on his or her abilities. If initial contact attempts were unsuccessful, up to a total of three attempts were made over a three-week period. The reason for all unsuccessful contacts was recorded. Representatives always asked the questions directly from the script in an effort to improve interrater reliability. All representatives were proficient in English and Spanish. Pediatric patients were included in the analysis if a parent or guardian who was present during the EMS encounter was available to complete the survey. Surveys were administered only if the representative was able to speak directly with the patient or parent of a minor patient. The initial contact attempt was made within three weeks of the date of EMS contact so that the encounter would be fresh in the patient’s mind. Calls were attempted from 9:00 AM to 11:00 PM Monday through Friday. At least one of the three attempts was made between 6:00 PM and 11:00 PM. All patients who provided a long-distance contact number were excluded from the study.

Patient demographics such as age, gender, race, chief complaint, or emergency categorization level were not tracked in the study’s database and could not be correlated with the survey results. Data Analysis. Responses were compared between transported and nontransported patients using chisquare analysis. A p-value of less than 0.05 was considered to be statistically significant. All computations were performed using SAS version 8.02 running under Microsoft Windows 2000 Professional (Microsoft Corp., Redmond, WA).

RESULTS The fire department’s EMS system responded to 88,528 calls for service between October 15, 2001, and March 15, 2002. Of those, 53,649 (60.6%) resulted in transport to a hospital, and 34,879 (39.4%) resulted in notransport. A 10% sample of transported patients was randomly selected for contact, resulting in 5,098 patients. Of those not transported, 29,020 (83.2%) had absent or incomplete information and thus were excluded from the study, leaving 5,859 (16.8%) nontransported patients for the study. Reasons for incomplete information included false alarms, the patient was gone from the scene when EMS arrived, incomplete information provided by the patient, incomplete information obtained by the EMS crew, or field termination of a cardiac arrest on the scene. Telephone contact was not possible in 1,773 (34.8%) of the sample of transported patients selected for study, and 1,960 (33.5%) of the nontransported cases. Reasons for failure are noted in Table 1. Telephone contact was made with 827 transported patients and 924 nontransported patients who declined or were unable to participate in the survey. This resulted in 2,498 transported and 2,975 nontransported patients completing the survey (Table 1). Overall, 62.8% of those contacted were contacted on the first attempt, 28.4% on the second attempt, and 16.5% on the third attempt. Table 2 reports the responses to the structured ‘‘yes/no’’ portion of the survey. Responses obtained between the transported and nontransported groups were statistically significant, with the exception of the question of professional demeanor. Responses to open-ended questions are reported in Table 3, and were not different for each group. The most important part of service for both groups was a quick response time. Responses to the question of overall satisfaction are reported in Table 4.

DISCUSSION Customer satisfaction has been almost universally recognized as an important component in assessing

108

Persse et al.

TABLE 1. Final Participant Number and Reasons for Noncontact Transports n ¼ 5,098 Phone answered, wrong number Phone number not In service Phone rings, not answered No phone number on chart Not called because of long- distance call Cases eligible for survey Phone answered, patient not available to talk Phone answered, patient unwilling to talk Phone answered, patient completed survey

%

Nontransports n ¼ 5,859

%

637

(12.5)

593

(10.1)

443

(8.7)

588

(10.0)

548 98

(10.7) (1.9)

577 119

(9.8) (2.0)

47 3,325

(0.9) (63.7)

83 3,899

(1.4) (66.6)

801

(24.)

866

(2.2)

26

(0.8)

58

(1.5)

2,498

(48.9)

2,975

(50.7)

overall quality in most industries. In recent years, increasing attention has been paid to this issue by EMS agencies. This study describes the initial experience of a large fire-based metropolitan EMS service in assessing patient (customer) satisfaction with the use of a telephone follow-up survey. The study results were overwhelmingly positive (95.6% composite satisfaction rate). Although the process utilized in this study is relatively simple, it requires the investment of substantial resources. However, such an investment likely will result in positive returns, such as insight into what the public believes is important with the agency, where they believe the agency is lacking, and potential ways to improve. This also may be beneficial to public image and aid in obtaining political support for the EMS agency. The staff required to conduct

d

CUSTOMER SATISFACTION IN AN URBAN EMS SYSTEM

these surveys could be used for other qualityimprovement efforts such as assessing the hospital and other public safety department staff satisfaction with the EMS service. Topic areas with the highest negative responses were related to failure to provide the patient with educational information about his or her illness/ injury (74.3% among transported patients and 32.3% among nontransported patients), failure to explain what EMS personnel were doing (17% transported and 7.8% nontransported), and the perception of extended response times (16.2% transported and 14.2% nontransported). These response differences were statistically significant in each of these areas. Failure of personnel to provide educational information about the patient’s illness was much higher among transported versus nontransported patients. This may be because EMTs and paramedics are not generally trained to provide education to the patients they transport, and are more likely to focus on providing treatment. Many EMS agencies, including ours, inform nontransported patients of potential risks of their illness/injury, as well as how to care for their injury or illness until they receive medical attention. The relatively low rate of educational efforts in transported patients also may have resulted from an expectation that patient education would occur at the hospital. In our system, the failure to provide education for 32.3% of nontransported patients is an opportunity for improvement. Further investigation into this issue is warranted. Patient perception of failure of EMS personnel to explain what they were doing is another potential target area for future study and improvement. The finding of patient displeasure with response time is consistent with previous reports.6 Previous studies have demonstrated poor correlation between actual and perceived EMS re-

TABLE 2. Responses to ‘‘Yes/No’’ Questions Transported Patients (n ¼ 2,498) Question 1. When you called 9-1-1, were you satisfied with how long it took to receive the service? 2. Did the fire fighters take care of your problems or concerns? 3. Did they explain what they were doing? 4. Did they possess the necessary skills to provide you with the care you needed? 5. Did they give you educational guidelines or information concerning your illness/injury? 6. Were they courteous and professional? 7. Were they caring and supportive?

Nontransported Patients (n ¼ 2,975)

Yes

No

Yes

No

p-value

2,092 (83.7%)

406 (16.2%)

2,552 (85.8%)

423 (14.2%)

0.036

2,210 (88.5%)

288 (11.5%)

2,698 (90.7%)

277 (9.3%)

0.007

2,073 (83.0%)

425 (17.0%)

2,744 (92.2%)

231 (7.8%)

\0.001

2,137 (85.5%)

361 (14.5%)

2,865 (96.3%)

110 (3.7%)

\0.001

641 (25.7%)

1,857 (74.3%)

2,015 (67.7%)

960 (32.3%)

\0.001

2,198 (88.0%)

300 (12.0%)

2,653 (89.2%)

322 (10.8%)

0.169

2,157 (86.3%)

341 (13.7%)

2,648 (89.0%)

327 (11.0%)

0.003

ACAD EMERG MED

d

January 2004, Vol. 11, No. 1

d

109

www.aemj.org

TABLE 3. Responses to Open-ended Questions Transported Patients (n ¼ 2,498)

Nontransported Patients (n ¼ 2,975)

Quick response: 49% Professional behavior: 29% Everything: 15% No suggestion: 49% No response: 31% A positive-reinforcement comment: 13%

Quick response: 50% Professional behavior: 30% Everything: 14% No suggestion: 48% No response: 31% A positive-reinforcement comment: 15%

Question What do you feel is the most important part of our service? Do you have any suggestions or comments about how we can improve our service?

sponse times, with patients overestimating the time of EMS arrival.7 Half of all patients we surveyed reported that a quick response was the most important part of the EMS service. A negative response regarding whether their problem was addressed was obtained in 7.8% and 17% of nontransported and transported patients, respectively (p # 0.001). This is a particularly curious finding, because 96.3% of nontransported patients indicated that EMS possessed necessary skills to provide the needed care. Conversely, 85.5% of transported patients indicated that EMS took care of their problem and possessed the necessary skills to do so. The percentage of patients who indicated that EMS personnel could be more courteous and professional was 12% for the transported patients and 10.8% for the nontransported patients. This was the only response among the ‘‘yes/no’’ questions that did not have a statistically significant difference between transported and nontransported patients. Percentages of respondents indicating that the firefighters could be more caring and supportive were 13.7% of transported patients and 11% of nontransported patients, (p ¼ 0.003). Although these negative responses were fewer in number compared with perceptions of response time, they indicate areas for system improvement. Educational efforts aimed at improving interpersonal communication skills and offering methods of explaining why things are being done in a particular way may provide a means of increasing the public’s positive experiences with EMS. Transported respondents had a 94.8% overall satisfaction rate, and nontransported patients had a satisfaction rate of 96.3%. Similar findings were noted by Doering, who reported a satisfaction rating of 4.75 on a 5.0 scale.8 Our study differed because it was conducted in a larger, urban, fire- based system, whereas the Doering study was conducted in a suburban, hospital-based system.

p-value 0.478

0.131

Because of the large number of respondents, the data obtained between transported and nontransported patients reached statistical significance, although the actual percentage differences might be considered relatively small. Our results suggest that EMS personnel are perceived differently between transported and nontransported patients, with nontransported patients uniformly indicating a higher level of satisfaction. These data suggest that nontransport decisions should be looked at carefully, not simply from a medically appropriate standpoint, but also potentially as a measure of patient satisfaction. Assessing patient satisfaction provided valuable information to both the system administrators and medical directors, allowing them to identify areas of strengths and weaknesses.

LIMITATIONS A potential bias exists in which patients contacted directly by telephone may be hesitant to report negative experiences, thus artificially elevating satisfaction. This type of bias would exist in any such survey. Other survey methods, such as mailings, carry additional financial expense and may not provide feedback or data that are more accurate than our method. Other limitations include an inability of this survey to provide insight into why transported and nontransported patients may have answered differently to the same questions. The design of this study was not intended to explore response differences between transported and nontransported patients, but to establish a benchmark from which future investigations may be compared. Further studies could address these issues. Although the absolute number of contacts was large, a bias may exist because of the different contact rates between transported and nontransported patients. The overall number of successful telephone contacts was a relatively small

TABLE 4. Response to Satisfied, Neutral, or Dissatisfied Question Question

Transported Patients (n ¼ 2,498)

Overall, would you say you were satisfied, neutral, or dissatisfied with the quality of care the Houston Fire Department provided?

Satisfied: 2,368 (94.8%) Neutral: 75 (3.0%) Dissatisfied: 55 (2.2%)

Nontransported Patients (n ¼ 2,975) Satisfied: 2,866 (96.3%) Neutral: 60 (2.0%) Dissatisfied: 49 (1.6%)

p-value 0.019

110

Persse et al.

percentage of the total number of patients transported by this agency. Thus, the findings from the study population may not be truly representative of the larger patient population. It is possible that the contact rates could have been improved by increasing the number of attempts, as well as more frequent calling during nonbusiness hours. Customer-service representatives were not given the opportunity to make long-distance calls, and although the number of these cases was relatively small, study results may have been impacted. We were not able to correlate patient demographics, such as age, gender, race, chief complaint, or emergency categorization level with survey results. It is possible that certain groups may have responded differently to the survey, and could be the focus of additional studies on quality improvement. Future investigations of this type could target receiving hospitals, police agencies, and other intradepartmental agencies.

CONCLUSIONS Patient (customer) satisfaction surveys conducted by telephone are relatively easy to perform and provide valuable information to both system administrators and medical directors. In this system over the period studied, overall patient satisfaction was very positive. However, several opportunities for improvement were identified. These include provision of education regarding the patient’s illness/injury at the scene, conveying to the patient what EMS personnel are doing when providing treatment, and educating the public as to actual EMS response times.

d

CUSTOMER SATISFACTION IN AN URBAN EMS SYSTEM

The authors thank the men and women of the Houston Fire Department for their dedicated service to this community. They also thank Ms. Austrebertha Aranda and the customer-service representatives for their many hours of work contributing to this project, and Ms. Dong Sun for statistical analysis.

References 1. Ryan JL. Quality management. In: Kuehl A (ed). Prehospital Systems and Medical Oversight, 3rd ed. Dubuque, IA: Kendall/Hunt Publishing, 2002, pp 355–87. 2. Kallsen GW. Quality Assurance in EMS. In: Swor RA, Rottman SJ, Pirrallo RG, Davis E (eds). Quality Management in Prehospital Care. St. Louis, MO: Mosby Publishing, 1993, pp 3–13. 3. Moore L. Measuring quality and effectiveness of prehospital EMS. Prehosp Emerg Care. 1999; 3:325–31. 4. Curka PA, Pepe PE, Zachariah BS, Gray GD, Matsumoto C. Incidence, source, and nature of complaints received in a large, urban emergency medical services system. Acad Emerg Med. 1995; 2:508–12. 5. Harvey AL, Gerard WC, Rice GF, Finch H. Actual vs. perceived EMS response times. Prehosp Emerg Care. 1999; 2:11–14. 6. Moshinskie JF. Reader attitudes toward journal-based continuing education for EMS personnel. Prehosp Disaster Med. 1997; 12:258–63. 7. Persse DE, Key CB, Baldwin JB. The effect of a quality improvement feedback loop on paramedic-initiated nontransport of elderly patients. Prehosp Emerg Care. 2002; 6:31–5. 8. Doering GT. Customer care: patient satisfaction in the prehospital setting. Emerg Med Serv. 1998; 9:69–74. 9. Hanzely D, Higgins B, Popil V. The use of customer-satisfaction surveys by an air medical program. Air Med J. 1993; 7:230–3. 10. Delbridge TR, Manz D, Suter RE, et al (eds). Emergency Medical Services: Agenda for the Future. Washington, DC: National Highway Traffic Safety Administration, August 1996, DOT HS 808-441, NTS-42, pp 57–59. 11. Greenberg MD, Garrison HG, Delbridge TR, et al. Quality indicators for out-of-hospital emergency medical services: the paramedic’s perspective. Prehosp Emerg Care. 1997; 1:23–7.