UNIVERSITY OF CALIFORNIA Los Angeles
Pilot Test of an Innovative Interprofessional Education Assessment Strategy
A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Education by Michelle Christine Emmert
2011
© Copyright by Michelle Christine Emmert 2011
ii
TABLE OF CONTENTS TABLE OF CONTENTS ........................................................................................... III LIST OF TABLES ........................................................................................................ V LIST OF FIGURES ...................................................................................................... V ACKNOWLEDGEMENTS ........................................................................................ VI VITA .......................................................................................................................... VII ABSTRACT OF THE DISSERTATION................................................................ VIII CHAPTER 1 STATEMENT OF THE PROBLEM .....................................................1 THE FACE OF MEDICAL ERROR......................................................................................1 CONSEQUENCES OF MEDICAL ERROR ............................................................................2 LEADING CAUSES OF MEDICAL ERROR ..........................................................................3 DRIVING FORCES OF CHANGE........................................................................................4 WHY INTERPROFESSIONAL EDUCATION .........................................................................5 EVALUATING INTERPROFESSIONAL EDUCATION .............................................................6 WESTERN UNIVERSITY OF HEALTH SCIENCES IPE CURRICULUM ....................................7 MY STUDY ...................................................................................................................8 STUDY SIGNIFICANCE ...................................................................................................9 SUMMARY .................................................................................................................. 10 CHAPTER 2 LITERATURE REVIEW ..................................................................... 12 DECIPHERING THE TERMINOLOGY ............................................................................... 12 A SNAPSHOT OF INTERPROFESSIONAL EDUCATION....................................................... 14 INTERPROFESSIONAL EDUCATION IN NON-CLINICAL SETTINGS .................................... 20 SELECTING AN ASSESSMENT APPROACH ...................................................................... 28 CREATING A PERFORMANCE-BASED ASSESSMENT....................................................... 30 SUMMARY .................................................................................................................. 39 CHAPTER 3 METHODOLOGY ............................................................................... 40 RESEARCH QUESTIONS ................................................................................................ 41 RESEARCH DESIGN ..................................................................................................... 41 METHODOLOGY .......................................................................................................... 42 SITE SELECTION .......................................................................................................... 47 SAMPLE SELECTION .................................................................................................... 48 INSTRUMENTATION ..................................................................................................... 49 DATA COLLECTION ..................................................................................................... 51
iii
DATA ANALYSIS ......................................................................................................... 52 LIMITATIONS .............................................................................................................. 53 SUMMARY .................................................................................................................. 53 CHAPTER 4 RESULTS .............................................................................................. 54 QUESTION 1A: TEAMWORK SCORE DIFFERENCES BETWEEN GROUPS ........................... 54 QUESTION 1B: TEAMWORK SCORE DIFFERENCES BETWEEN PROGRAMS....................... 59 QUESTION 1C: TEAMWORK SCORE DIFFERENCES BETWEEN GENDERS ......................... 60 QUESTION 1D: TEAMWORK SCORE DIFFERENCES BETWEEN RATER TYPES................... 61 QUESTION 2: METHODOLOGICAL SOUNDNESS OF RATING SCALES ............................... 62 QUESTION 3: KEY SUGGESTIONS FOR IMPROVEMENT .................................................. 65 SUMMARY .................................................................................................................. 75 CHAPTER 5 CONCLUSIONS ................................................................................... 77 RELATIONSHIP TO PRIOR RESEARCH ............................................................................ 77 NOTEWORTHY OUTCOMES .......................................................................................... 78 STUDY SIGNIFICANCE ................................................................................................. 83 LIMITATIONS .............................................................................................................. 84 RECOMMENDATIONS ................................................................................................... 87 FUTURE RESEARCH ..................................................................................................... 91 CONCLUSION .............................................................................................................. 92 APPENDICES ............................................................................................................. 94 APPENDIX A TEAMSTEPPS PERFORMANCE CRITERIA ................................................. 94 APPENDIX B PILOT STUDY DESIGN PLAN..................................................................... 96 APPENDIX C PILOT STUDY EXPERTS ............................................................................ 97 APPENDIX D PARTICIPANT RECRUITMENT COMMUNICATIONS ...................................... 98 APPENDIX E STANDARDIZED PATIENT TRAINING DOCUMENT .................................... 100 APPENDIX F DOORWAY INFORMATION ...................................................................... 105 APPENDIX G PATIENT’S MEDICAL CHART ................................................................. 106 APPENDIX H SCRIPTED REACTIONS FOR HEALTHCARE PROVIDERS ............................. 111 APPENDIX I TOSCE ORIENTATION/WRAP-UP TOPICS................................................ 112 APPENDIX J ID NUMBER ASSIGNMENT SHEET ............................................................ 115 APPENDIX K RATING SCALES .................................................................................... 117 APPENDIX L FEEDBACK SURVEYS ............................................................................. 118 APPENDIX M INFORMED CONSENTS........................................................................... 123 APPENDIX N PARTICIPANT FEEDBACK SUMMARY ...................................................... 132 REFERENCES .......................................................................................................... 141
iv
LIST OF FIGURES FIGURE 1 FIGURE 2 FIGURE 3
TYPICAL TOSCE ASYNCHRONOUS TOSCE TEAMWORK SCORES BY RATER TYPE AND GROUP
35 36 57
LIST OF TABLES TABLE 1 TABLE 2 TABLE 3 TABLE 4 TABLE 5 TABLE 6 TABLE 7 TABLE 8 TABLE 9 TABLE 10 TABLE 11 TABLE 12 TABLE 13
MEDICAL ERROR STATISTICS T-VALUES FOR SPCG RATERS T-VALUES FOR LIVE RATERS T-VALUES FOR VIDEO RATERS ANOVA FOR SPCG RATERS ANOVA FOR LIVE RATERS ANOVA FOR VIDEO RATERS GENDER DIFFERENCE VALUES FOR SPCG RATERS GENDER DIFFERENCE VALUES FOR LIVE RATERS GENDER DIFFERENCE VALUES FOR VIDEO RATERS PEARSON CORRELATION COEFFICIENTS EXPERT FEEDBACK ON RATING SCALE QUESTIONS NUMBER OF AVAILABLE PARTICIPANTS BY PROGRAM
v
1 55 56 56 59 59 59 60 60 60 63 76 86
ACKNOWLEDGEMENTS Four years ago when my first project fell through I didn’t know if I would have the energy to start a new dissertation and earn my degree, but thanks to a number of people, I did and I will! I would first like to thank my Chair, Dr. Li Cai, who has been so very supportive, patient and informative during this process. I won’t forget the night during data analysis when he responded to an email inquiry within a few minutes at 1:30am! To my committee members; Dr. Noreen Webb, Dr. Mary Cadogan and Dr. Bill Sandoval, thank you for helping me survive the nerve wracking process of oral examinations! And of course a huge note of thanks to Marisela Diaz and Kaitlin Gibson for always being ready and willing to answer questions and send me documents when I needed them! To the IPE and Standardized Patient Department staff at WesternU, thank you for allowing me to base my project at WesternU and for the amazing expertise you brought to the table – I couldn’t have done it without you! Special thanks to my boss and friend, Dr. Beverly Guidry for her flexibility and encouragement. To Mom and Dad; thank you for always believing in me and finding ways to support and encourage me even when we were literally a world apart. To my best friend Stephanie Zia and her husband Andrew Doolittle; thank you for keeping me sane every week with date night and for helping me double check my data entry! And thank you to all of my dear friends who asked time and time again how things were going with my dissertation and celebrated even the minor victories with me! Thank you all, and PRAISE GOD IT’S DONE!
vi
VITA 1977
Born, Walnut Creek, California
1999
B.A., Psychology and Biology and Graduated Cum Laude Claremont McKenna College Claremont, California Awarded Deans List and Distinguished Scholar
1999-2002
University Recruiter Western University of Health Sciences Pomona, California
2002-2003
Director of Admissions, College of Veterinary Medicine Western University of Health Sciences Pomona, California
2003-2004
Director of Admissions, College of Veterinary Medicine and College of Graduate Nursing Western University of Health Sciences Pomona, California
2006-2011
Director for University Student Affairs Western University of Health Sciences Pomona, California
2006-2011
Assistant Dean of Student Affairs College of Osteopathic Medicine of the Pacific Western University of Health Sciences Pomona, California
vii
ABSTRACT OF THE DISSERTATION
Pilot Test of an Innovative Interprofessional Education Assessment Strategy
by
Michelle Christine Emmert Doctor of Education University of California, Los Angeles, 2011 Professor Li Cai, Chair
The primary goal of this study was to test an innovative way of assessing students’ teamwork skills in a controlled environment. Twenty-four second year students from Western University of Health Sciences (WesternU) participated in the experimental group and 22 third year students from WesternU participated in the control group. Students in the experimental group had taken part in a two-year interprofessional education (IPE) curriculum while third year students had not had any formal IPE training at WesternU. Students took part in a Team Objective Structured Clinical Exam (TOSCE) performance task in which they interacted with a standardized patient and her caregiver son after which three to five raters scored each student on their teamwork skills using newly designed teamwork rating scales. Findings indicate that students in the
viii
experimental group performed significantly better (p=.0003) than students in the control group suggesting that the IPE curriculum is successfully equipping students with teamwork skills. We also determined that there were no significant differences in how students were scored based upon their gender or program. Pearson Correlation Coefficient calculations determined that the teamwork scores of faculty who rated students “live” during the TOSCE (Live Raters) and scores of faculty who rated students after the fact via video (Video Raters) were significantly correlated (p95%. 5. CAD with known HTN and Hyperlipidemia - Continue with home regimen of aspirin, lipitor. 6.
GI prophylaxis - Initiate proton pump inhibitor regimen
Hospital Discharge Summary ®
PATIENT NAME:
KOWALSKI,MIRIAM
DATE OF ADMISSION:
Would fill in 7 days prior to date of discharge
DATE OF DISCHARGE:
Would fill in day before students are given this case
DISCHARGE DIAGNOSES: 1. CVA, residual left sided-weakness/dysarthria/oropharyngeal dysphagia 2. Atrial fibrillation, rate controlled 3. Diabetes, Type 2 4. Hypertensive heart disease with renal insufficiency and congestive heart failure 5. COPD 6. Functional dual incontinence secondary to #1 7. Stage 2 Coccyx pressure ulcer 8. Bilateral carotid stenosis 9. Depression CONSULTS: 1. Neurology 2. Cardiology
108
3. Physical Medicine & Rehabilitation (PT/OT/ST) 4. Pulmonology 5. Wound care specialist PROCEDURES: echocardiogram
CT of the brain, Bilateral carotid ultrasound, Cardiac
HISTORY OF PRESENT ILLNESS: Patient is a 78-year-old female with past medical history significant for coronary artery disease with unstable angina, hypertension, hyperlipidemia, diabetes type 2 and COPD who presented with acute left sided weakness and confusion reflective of CVA. Please see H&P for details. HOSPITAL COURSE BY PROBLEM: 1. CVA. Multiple CT scans of the brain were done and demonstrated a nonhemorrhagic right parietal infarct in the area of the right anterior cerebral artery supply. Bilateral carotid ultrasound demonstrated a stenosis of 20% on the right internal carotid artery and a stenosis of 40% on left common carotid artery. Initial left facial droop noticed on admission cleared during course of hospital progress. Patient also regained sensation in left upper extremity but still with weakness of the leftupper extremity and paralysis of the left lower extremity. Patient upgraded from NPO status to tolerating a soft mechanical diet with thin liquids by day of discharge although regular diet not trialed as dentures are not available and patient is edentulous. 2. Atrial fibrillation. No prior history per medical records. Transesophageal echocardiogram identified multiple thrombi in left atrium. Patient initially heparanized then transitioned to oral warfarin with target INR between 2.0-3.0. Cardiology recommends anticoagulation for three weeks then follow-up in his clinic for possible electrocardioversion. Rate controlled since initiation of beta-blocker regimen. 3. Diabetes, Type 2. Patient initially on tight glycemic control on intravenous insulin with hourly plasma glucose checks. Patient successfully transitioned to oral sulfonyurea (metformin not initiated due to renal insufficiency) during hospitalization and adhered to a no concentrated sweets diet. 4. Hypertensive heart disease with renal insufficiency and congestive heart failure. Once heart rate controlled with beta-blockers and ACE-I resumed, patient's blood pressure normalized during course of hospital stay. Estimated GFR initially identified at 32 upon
109
admission which later improved and stabilized at an eGFR of 48. Echo did reveal mild left ventricular hypertrophy with an ejection fraction of 50% without any fluid collection noted. 5. COPD. Patient did well throughout hospitalization with QID nebulizer treatments and tolerated being on room air on day of discharge. 6. Stage 2 Coccyx pressure ulcer. Due to immobility, pt developed a 1.5cm x 1.0cm Stage 2 coccyx decub which is granulating well by day of discharge without signs or symptoms of infection. Duoderm dressing applied and changed QOD; patient repositioned every 2 hours. 7. Depression. Patient verbalized feelings of hopelessness and depression with decreased level of functioning. This is not an uncommon phenomenon is post-stroke patients. Antidepressant regimen initiated. DISCHARGE MEDICATIONS: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.
Glyburide 5mg one tab PO QD Warfarin 2mg one tab PO QD Lisinopril 20mg one tab PO QD Metoprolol 50mg one tab PO BID Albuterol HFA 2 puffs QID Ipatropium bromide HFA 2 puffs QID Aspirin enteric-coated 81 mg one tab PO QD Pantoprazole 40mg one tab PO QD Escitalopram 20mg one tab PO QD Milk of Magnesia 8% 15mL PO QD as needed for constipation Oxybutynin 5 mg one tab PO QD Vicodin 5/500mg one tab PO Q4H as needed for pain
DIET: Soft mechanical with thin liquids ACTIVITIES: Ad lib with assistance FOLLOWUP: In 5-7 days with primary provider In 3 weeks with cardiology Home health for nursing, physical therapy, occupational therapy, and speech therapy
110
Appendix H Scripted Reactions for Healthcare Providers
If the student calls for results of INR: o Inform the student the INR is 4.0 If the student calls to request a consult: o If they do not specify what service, ask them who they are calling for. o Once identified, ask the student what can I do for you? o If they ask for a consult, ask the student for a brief synopsis of their concern and what they want accomplished with the consult. o Advise the student that the consult will be initiated and thank them for the consult If the student calls to “close the loop”: o If they don’t specify who they are calling for, ask them what profession they wish to speak with. o They should then state they are following up with the ___________regarding the patient. They should provide a short summary of their findings. If they do not do this, you can ask them to summarize their visit with the patient. Thank them for the follow up, and advise that you will continue to follow the patient with them. If they call with a concern as a safety issue (conflict): o If they don’t specify who they are calling for, ask them what profession they wish to speak with. o They should divulge something along the lines of “doctor, I am concerned about starting physical therapy when the patient’s blood pressure is 170/90. You will then PUSH BACK with a comment like “I don’t care what the blood pressure is. I ordered physical therapy, and I want it started TODAY” This will vary with the safety item conflicts, but you should be firm in your insistence that they follow through with your orders. The student should then use something like the CUS or DESC technique. C=I am concerned about__________ U= I am uncomfortable with this because_____________ S= This is a safety issue _____________ D=Describe the specific situation of behavior regarding the conflict E=Express how the situation makes you feel or what your concerns are S=Suggest alternatives and seek agreement C=Consequences should be stated in terms of impact on the team goal If the student calls to obtain more information about the patient: o This will be an ad-lib activity, but should be based off of information on the script, or the discharge summary
111
Appendix I TOSCE Orientation/Wrap-Up Topics Folders: Participants received the following items in their folders: Faculty Raters
Actors
Consent Form Pilot Study Schedule Pre-Labeled Rating Scales Pre-Labeled Feedback Survey Pencil Healthcare Providers
Consent Form Pilot Study Schedule Pre-Labeled Rating Scales Pre-Labeled Feedback Survey Pencil Students
Consent Form Pilot Study Schedule Scripted Reactions Pre-Labeled Feedback Survey Pencil
Consent Form Pilot Study Schedule Pre-Labeled Rating Scales Pre-Labeled Feedback Survey Plan Summary Forms Pencil
Pre-Preparatory Course Orientation Topics (Students) Orientation to Experience o Why we’re doing this study o How course will prepare them for TOSCE (experimental group only) o Confidentiality reminder Orientation to pre-TOSCE Paperwork o Sign Informed Consent Read forms out loud. Questions? Obtain signatures. Collect forms and verify signatures. o Complete Assessment Battery Quick overview of instruments in the packet. Questions? Reminder to check both sides of instruments. Collect completed instruments. Verify completion. o Remember to read the article we sent you o Remember to bring study/reading materials for downtime Pre-TOSCE Orientation Topics (Students) Take attendance and check IDs.
112
Orientation to Experience o What is a TOSCE? o Overview of what will happen today, when, and for how long o Study in SGA room when your group is not participating in the TOSCE o What to do in the SP lab, doorway instructions, phone usage o Don’t limit your thinking to just the professions participating in TOSCE o “Do a visual assessment as appropriate but do not perform a hands on physical exam” o Need to verbalize everything they notice and are doing o It’s OK to write notes during downtime before encounter o What to do if they end the encounter before time’s up o Stay in room after interaction & conduct any necessary follow-up in their “office” o There will be some artificiality – flow with it o Explain what will be assessed in a general sense and who’s assessing o What to do during downtime – reading material provided o Let them know they are being recorded o Do not discuss the case before, during or after the TOSCE – study room should remain quiet o “Extra” students will stay in the SGA room and study – we’ll let you know if you’re needed o Reminder of confidentiality of study – not to discuss with classmates o Questions? Restroom/food break and proceed to stations Put bottled water in exam rooms
Pre-TOSCE Orientation Topics (Faculty and Actors) Orientation to Experience o Why we’re doing this study o What is a TOSCE? o Overview of what will happen today, when, and for how long o What will happen in the SP lab, doorway instructions, phone usage o What will faculty do during the TOSCE (control room vs. VIP room) o We will be available in control room to answer questions about assessment tools o Discussion in control room should not be distracting or contaminating o Students can call ANY professional not just those participating in TOSCE
113
o Students stay in room after interaction to conduct follow-up in their “office” o Turn phones on silent o Questions? Sign Informed Consent o Read forms out loud. Questions? o Obtain signatures. o Collect forms and verify signatures. Evaluation Form Training o Give participants an overview of how to use the evaluation form o Walk participants through each question on the evaluation form o Tell raters that scale items worded with “noticed”, “recognized” etc. should observed through verbal cues. o Remind participants to confirm all questions are answered after each student finishes o Pass out pencils Restroom/food break and proceed to stations
Final Wrap-Up Topics (All Participants) Restroom/food break o Orientation to Remaining Paperwork o Double-check both sides of 2-sided documents completed o Double-check all evaluation forms included o Place all paperwork back into folder Questions/Topics to address during student debriefing o Disclosure Information o Did you feel like you were adequately prepared to take part in the TOSCE? o Do you feel that the TOSCE tested what was taught in the preparatory course? o Was any information noticeably missing that inhibited your performance? o How did you feel about the amount of time allotted for each aspect of the study? Questions? Reminder of confidentiality Collect paperwork folders Give $100 gift cards to students as they hand in their folders
114
Appendix J ID Number Assignment Sheet KEY
Student = Degree + Group# Standardized Patient = SP Standardized Caregiver = SC Healthcare Provider = HP
SPs, SC’s and HPs will be pre-assigned to either groups 1, 3 & 5 or groups 2, 4 & 6. We will randomly assign students to groups as they arrive. If extra students are available, they will be added into the blank spots at the end of each group as additional raters. Group 1 Pilot Study ID# DPT1 DO1 PharmD1 DMD1 SP1 SC1 HP1
Participant Name
Group 2 Pilot Study ID# DPT2 DO2 PharmD2 DMD2 SP2 SC2 HP2
Participant Name
115
Group 3 Pilot Study ID# DPT3 DO3 PharmD3 DMD3 SP1 SC1 HP1
Participant Name
Group 4 Pilot Study ID# DPT4 DO4 PharmD4 DMD4 SP2 SC2 HP2
Participant Name
Group 5 Pilot Study ID# DPT5 DO5 PharmD5 DMD5 SP1 SC1 HP1
Participant Name
Group 6 Pilot Study ID# DPT6 DO6 PharmD6 DMD6 SP2 SC2 HP2
Participant Name
116
Appendix K Rating Scales Please contact author regarding availability of rating scales.
117
Appendix L Feedback Surveys
Student Feedback Survey Study ID # ______________________ Thank you very much for participating in this study. We are interested in your thoughts and opinions regarding your experience. Your answers will be kept strictly confidential. Strongly Disagree 1
Disagree
Neutral
Agree
2
3
4
Strongly Agree 5
The material was effectively organized and presented.
1
2
3
4
5
The preparatory course helped develop my understanding of interprofessional issues.
1
2
3
4
5
The training I received during the preparatory course helped my performance during the simulation.
1
2
3
4
5
The disclosures I received provided the right amount of information.
1
2
3
4
5
The standardized patient encounter flowed well.
1
2
3
4
5
Instructions for what was expected of me during the standardized patient encounter were clear.
1
2
3
4
5
The assessment rubric was effective for evaluating my performance.
1
2
3
4
5
The Behavioral Intentions Survey was easy to understand.
1
2
3
4
5
This TOSCE model could be implemented on a larger scale.
1
2
3
4
5
As a result of this experience, I am more likely to consider other perspectives when reviewing cases.
1
2
3
4
5
Strongly Unfavorable
Unfavorable
Neutral
Favorable
Strongly Favorable
1
2
3
4
5
The objectives for the preparatory course were clear.
How would you characterize your attitude toward interprofessional education?
118
Given that the purpose of this experience is to provide an opportunity to learn about other professionals’ roles, build teamwork, and develop communication skills, please provide feedback to the following questions: 1. What aspects of this study do you think are most valuable?
2. What, if anything, could be done to improve the preparatory course?
3. What, if anything, could be done to improve the standardized patient scenarios?
4. What, if anything, could be done to improve the assessment rubrics?
5. Please provide any additional thoughts, suggestions, or comments.
6. In which program are you enrolled? (circle one) a. b. c. d.
DMD DO DPT PharmD
7. Do you have previous health care employment experience? (circle one) Yes No If yes, please explain:
119
Faculty Feedback Survey Study ID # _________________________________ Thank you very much for participating in this study. We are interested in your thoughts and opinions regarding your experience. Your answers will be kept strictly confidential. Strongly Disagree 1
Disagree
Neutral
Agree
2
3
4
Strongly Agree 5
The assessment rubric was effective for evaluating student performance.
1
2
3
4
5
This TOSCE model could be implemented on a larger scale.
1
2
3
4
5
Strongly Unfavorable
Unfavorable
Neutral
Favorable
Strongly Favorable
1
2
3
4
5
The standardized patient encounter flowed well.
How would you characterize your attitude toward interprofessional education?
Given that the purpose of this experience is to provide an opportunity for students to learn about other professionals’ roles, build teamwork, and develop communication skills, please provide feedback to the following questions: 1. What aspects of this study do you think are most valuable?
2. What, if anything, could be done to improve the standardized patient scenarios?
3. What, if anything, could be done to improve the assessment rubrics?
4. Please provide any additional thoughts, suggestions, or comments.
5. What is your sex? (circle one)
Male
6. What is your program? (circle one) a. DMD b. DO c. DPT d. PharmD
120
Female
Actor Feedback Survey Study ID # _____________________ Thank you very much for participating in this study. We are interested in your thoughts and opinions regarding your experience. Your answers will be kept strictly confidential. For each question, please circle ONE number on the rating scale.
1. I felt well prepared to perform today’s exercise.
Strongly Disagree Strongly Agree 1 5
Neutral 2
3
4
Comments:
2. The standardized patient encounter flowed well.
Strongly Disagree Strongly Agree 1 5
Neutral 2
3
Comments:
121
4
3. The grading rubric was effective for evaluating student performance.
Strongly Disagree Strongly Agree 1 5
Neutral 2
3
Comments:
4. What aspects of this study do you think are most valuable?
5. Please provide any additional thoughts, suggestions, or comments.
122
4
Appendix M Informed Consents Western University of Health Sciences & University of California, Los Angeles CONSENT TO PARTICIPATE IN RESEARCH STUDENTS Pilot Testing Interprofessional Education Assessment Tools You are asked to participate in a research study conducted by Jordan Orzoff, PhD, Michelle Emmert, BA, and associates from Western University of Health Sciences (WesternU) and the Educational Leadership Program (ELP), at the University of California, Los Angeles. You were selected as a possible participant in this study because you are a 2nd year student currently enrolled in the Doctor of Dental Medicine, Osteopathic Medicine, Pharmacy or Physical TherapyEntry program at WesternU. Your participation in this research study is voluntary. Why is this study being done? We are testing a new evaluation process and assessment tools for a teamwork training and simulation performance task in the Interprofessional Education (IPE) program at WesternU. What will happen if I take part in this research study? If you volunteer to participate in this study, the researcher will ask you to do the following: You will take part in a preparatory course on March 31st or May 12th and a Team Objective Structured Clinical Exam (TOSCE) on April 1st or May 13th. At the preparatory course you will sign an informed consent, complete a Readiness for Interprofessional Learning (RIPLS) and a Behavioral Intentions Survey (BIS) asking about your intentions to utilize teamwork skills in professional practice. You will also learn about issues that specifically relate to geriatric care. At the orientation meeting before the TOSCE, you will take part in an orientation where you will learn how to utilize the assessment tools for this study and then move to the standardized patient labs of the Health Professions Center at WesternU to take part in a TOSCE. You will play the role of a professional in your field while interacting with standardized patients. Faculty observers will watch your interactions on a video screen in another room. After each patient interaction, faculty observers, the standardized patient, caregiver and you will evaluate your performance at that station.
123
After standardized patients have visited all stations, you will return to the orientation room, turn in your evaluations and retake the RIPLS and BIS in addition to completing a Feedback Survey asking for feedback about how the process can be improved. How long will I be in the research study? Participation in the study will take a total of approximately 8 hours including the preparatory course and the Team Objective Structured Clinical Exam (TOSCE). Are there any potential risks or discomforts that I can expect from this study? There are no anticipated risks or discomforts. Please also note that your IPE grade will not be impacted by your participation or non-participation in this study. Are there any potential benefits if I participate? You may benefit from the study by improving the existing interprofessional education curriculum to better prepare you to work in health care teams upon graduation. If you are prepared to work in effective teams when you graduate this could have a significant positive impact on hundreds of thousands of patients who you and your classmates will care for in your lifetimes. What we learn in this study will help us better assess and thus improve teamwork training for current and future students at WesternU and possibly at other health science institutions as well. Will I receive any payment if I participate in this study? You will receive a $100 gas card as a token of our appreciation. Will information about me and my participation be kept confidential? Any information that is obtained in connection with this study and that can identify you will remain confidential. It will be disclosed only with your permission or as required by law. Confidentiality will be maintained by using pilot study ID numbers rather than your names on all assessment documents. The resulting data will be kept by the principal investigator in a locked file at her home and all results will be discussed in aggregate. Withdrawal of participation by the investigator The investigator may withdraw you from participating in this research if circumstances arise which warrant doing so. If you arrive too late to be fully oriented before the start of the pilot study you may have to drop out, even if you would like to continue. The investigator will make the decision and let you know if it is not possible for you to continue. What are my rights if I take part in this study? You may withdraw your consent at any time and discontinue participation without penalty or loss of benefits to which you were otherwise entitled.
124
You can choose whether or not you want to be in this study. If you volunteer to be in this study, you may leave the study at any time without consequences of any kind. You are not waiving any of your legal rights if you choose to be in this research study. You may refuse to answer any questions that you do not want to answer and still remain in the study. Who can answer questions I might have about this study? If you have any questions, comments or concerns about the research, you can talk to the Principal Investigators: Dr. Jordan Orzoff at 909- 706-3811,
[email protected], or Michelle Emmert at 909-469-5616,
[email protected]. If you wish to ask questions about your rights as a research participant or if you wish to voice any problems or concerns you may have about the study to someone other than the researchers, please call the Office of the Human Research Protection Program at (310) 825-7122 or write to Office of the Human Research Protection Program, UCLA, 11000 Kinross Avenue, Suite 102, Box 951694, Los Angeles, CA 90095-1694. SIGNATURE OF STUDY PARTICIPANT I understand the procedures described above. My questions have been answered to my satisfaction, and I agree to participate in this study. I have been given a copy of this form.
Name of Participant
Signature of Participant
Date
SIGNATURE OF PERSON OBTAINING CONSENT In my judgment the participant is voluntarily and knowingly giving informed consent and possesses the legal capacity to give informed consent to participate in this research study. Michelle Emmert Name of Person Obtaining Consent
Contact Number
Signature of Person Obtaining Consent
Date
125
Western University of Health Sciences & University of California, Los Angeles CONSENT TO PARTICIPATE IN RESEARCH FACULTY Pilot Testing Interprofessional Education Assessment Tools You are asked to participate in a research study conducted by Jordan Orzoff, PhD, Michelle Emmert, BA, and associates from Western University of Health Sciences and the Educational Leadership Program (ELP), at the University of California, Los Angeles. You were selected as a possible participant in this study because you are on faculty at Western University of Health Sciences (WesternU). Your participation in this research study is voluntary. Why is this study being done? We are testing a new evaluation process and assessment tools for a teamwork training and simulation performance task in the Interprofessional Education (IPE) program at WesternU. What will happen if I take part in this research study? If you volunteer to participate in this study, the researcher will ask you to do the following: You will take part in an orientation on April 1st or May 13th where you will learn how to utilize the assessment tools used in this study, and sign informed consent paperwork. You will be asked to observe (via video) interactions between students, standardized patient and caregiver in a Team Objective Structured Clinical Exam (TOSCE) format. After each patient interaction you will evaluate the student’s performance at that station. After the final station of the last group, you will return to the orientation room, turn in your evaluations and complete a Feedback Survey asking for feedback about how the process can be improved. How long will I be in the research study? Participation in the study will take a total of approximately 7 hours. Are there any potential risks or discomforts that I can expect from this study? There are no anticipated risks or discomforts.
126
Are there any potential benefits if I participate? Your assistance will help your colleagues to improve the IPE curriculum for your students. More importantly, you will better equip your students to work as productive members of healthcare teams upon graduation. If students are prepared to work in effective teams when they graduate this could have a significant positive impact on hundreds of thousands of patients who these soon-to-be graduates will care for in their lifetimes. What we learn in this study will help us better assess and thus improve teamwork training for current and future students at WesternU and possibly at other health science institutions as well. Will I receive any payment if I participate in this study? While we greatly appreciate your participation you will not receive compensation. Will information about me and my participation be kept confidential? Any information that is obtained in connection with this study and that can identify you will remain confidential. It will be disclosed only with your permission or as required by law. Confidentiality will be maintained by using pilot study ID numbers rather than your names on all assessment documents. The resulting data will be kept by the principal investigator in a locked file at her home and all results will be discussed in aggregate. Withdrawal of participation by the investigator The investigator may withdraw you from participating in this research if circumstances arise which warrant doing so. If you arrive too late to be fully oriented before the start of the pilot study you may have to drop out, even if you would like to continue. The investigator will make the decision and let you know if it is not possible for you to continue. What are my rights if I take part in this study? You may withdraw your consent at any time and discontinue participation without penalty or loss of benefits to which you were otherwise entitled. You can choose whether or not you want to be in this study. If you volunteer to be in this study, you may leave the study at any time without consequences of any kind. You are not waiving any of your legal rights if you choose to be in this research study. You may refuse to answer any questions that you do not want to answer and still remain in the study. Who can answer questions I might have about this study? If you have any questions, comments or concerns about the research, you can talk to the Principal Investigators: Dr. Jordan Orzoff at 909- 706-3811,
[email protected], or Michelle Emmert at 909-469-5616,
[email protected].
127
If you wish to ask questions about your rights as a research participant or if you wish to voice any problems or concerns you may have about the study to someone other than the researchers, please call the Office of the Human Research Protection Program at (310) 825-7122 or write to Office of the Human Research Protection Program, UCLA, 11000 Kinross Avenue, Suite 102, Box 951694, Los Angeles, CA 90095-1694. SIGNATURE OF STUDY PARTICIPANT I understand the procedures described above. My questions have been answered to my satisfaction, and I agree to participate in this study. I have been given a copy of this form.
Name of Participant
Signature of Participant
Date
SIGNATURE OF PERSON OBTAINING CONSENT In my judgment the participant is voluntarily and knowingly giving informed consent and possesses the legal capacity to give informed consent to participate in this research study. Michelle Emmert Name of Person Obtaining Consent
Contact Number
Signature of Person Obtaining Consent
Date
128
Western University of Health Sciences & University of California, Los Angeles CONSENT TO PARTICIPATE IN RESEARCH ACTORS Pilot Testing Interprofessional Education Assessment Tools You are asked to participate in a research study conducted by Michelle Emmert, BA, and associates from the Educational Leadership Program (ELP), at the University of California, Los Angeles. You were selected as a possible participant in this study because you have received training as a standardized patient and agreed to be paid for your participation in this study. Your participation in this research study is voluntary. Why is this study being done? We are testing a new evaluation process and assessment tools to determine whether students are able to demonstrate the teamwork skills they’re being taught in the Interprofessional Education (IPE) program at WesternU. What will happen if I take part in this research study? If you volunteer to participate in this study, the researcher will ask you to do the following: The pilot study will take place the afternoons of April 1st and May 13th in the standardized patient labs of the Health Professions Center at WesternU. Before the first time slot you will take part in an orientation where you will be trained on what symptoms and information you should exhibit/share with students in each of your interactions. You will also learn how to utilize the assessment tools and sign an informed consent. Each day you will rotate through a total of 24 stations (4 stations repeated in each of 6 groups) exhibiting a single set of patient symptoms. You will get a 15 minute break after every other group. You will be asked to evaluate the student’s performance at each station. After you have visited all 24 stations, you will return to the debriefing room, turn in your evaluations and complete a post-pilot study survey asking for feedback about how the process can be improved. How long will I be in the research study? Participation in the study will take a total of approximately 12 hours including orientation, TOSCE’s and debriefing. Are there any potential risks or discomforts that I can expect from this study?
129
There are not anticipated risks or discomforts. Are there any potential benefits if I participate? Your assistance will help WesternU to improve their Interprofessional Education curriculum for students and better equip students to work as productive members of healthcare teams upon graduation. If students are prepared to work in effective teams when they graduate this could have a significant positive impact on hundreds of thousands of patients who these soon-to-be graduates will care for in their lifetimes. Will I receive any payment if I participate in this study? Yes, you will be paid $25/hour for your participation Will information about me and my participation be kept confidential? Any information that is obtained in connection with this study and that can identify you will remain confidential. It will be disclosed only with your permission or as required by law. Confidentiality will be maintained by using pilot study ID numbers rather than your names on all assessment documents. The resulting data will be kept by the principal investigator in a locked file at her home and all results will be discussed in aggregate. Withdrawal of participation by the investigator The investigator may withdraw you from participating in this research if circumstances arise which warrant doing so. If you arrive too late to be fully oriented before the start of the pilot study you may have to drop out, even if you would like to continue. The investigator will make the decision and let you know if it is not possible for you to continue. What are my rights if I take part in this study? You may withdraw your consent at any time and discontinue participation without penalty or loss of benefits to which you were otherwise entitled. You can choose whether or not you want to be in this study. If you volunteer to be in this study, you may leave the study at any time without consequences of any kind. You are not waiving any of your legal rights if you choose to be in this research study. You may refuse to answer any questions that you do not want to answer and still remain in the study. Who can answer questions I might have about this study? If you have any questions, comments or concerns about the research, you can talk to the Principal Investigator: Michelle Emmert at 909-469-5616 or
[email protected]. If you wish to ask questions about your rights as a research participant or if you wish to voice any problems or concerns you may have about the study to someone other than the researchers, please
130
call the Office of the Human Research Protection Program at (310) 825-7122 or write to Office of the Human Research Protection Program, UCLA, 11000 Kinross Avenue, Suite 102, Box 951694, Los Angeles, CA 90095-1694. SIGNATURE OF STUDY PARTICIPANT I understand the procedures described above. My questions have been answered to my satisfaction, and I agree to participate in this study. I have been given a copy of this form.
Name of Participant
Signature of Participant
Date
SIGNATURE OF PERSON OBTAINING CONSENT In my judgment the participant is voluntarily and knowingly giving informed consent and possesses the legal capacity to give informed consent to participate in this research study.
Michelle Emmert Name of Person Obtaining Consent
Contact Number
Signature of Person Obtaining Consent
Date
131
Appendix N Participant Feedback Summary Preparatory Course General Concern - Although students understood the general concepts learned through the TeamSTEPPS modules and in the preparatory course, they did not know how to implement these skills in practice and wanted significantly more guidance. Solution – Instructor should offer many clear examples or case studies that make it clear how students should/can enact the concepts learned through TeamSTEPPS. After having these strategies demonstrated to them, students should be given the opportunity to practice. Specific requests included: o how to best communicate and collaborate in an asynchronous environment (students felt that we talked a lot about a traditional team yet TOSCE was asynchronous). o what typical S-BAR’s might sound like o how to best “hand off” a patient o how to address caregiver o how to deal with a difficult patient/caregiver o when/if it’s appropriate to ask a caregiver to step out of the room. General Concern – Knew that the bruising was an elder abuse issue but didn’t feel like course prepared them to address the issue. Solution – Students wanted more concrete examples or case examples during the course that methodically tell them how to handle anticipated team/geriatric issues. General Concern – Never seen the abuse scenario before, so very enlightening and thought provoking. Not sure what reporting is within their scope of practice. Should they address it at all during this first visit? Should they just forward information along to appropriate authorities for them to handle? What’s within their scope of practice or appropriate/best practice? Solution – Address more clearly when and who should be contacted in elder abuse situations as well as how far the healthcare provider should go in addressing a possible elder abuse situation when they are first made aware of the possibility. DO Student Concern – article on stroke was not very clear/informative. Solution – ask them to do a different article that’s easier to read
132
Observer Concern - Geriatrics workers are taught to be more formal with patient names. Should this be added to the preparatory course (and the rubric)? TOSCE Student Orientation General Concern – really wanted the goals/objectives for the TOSCE to be clarified in advance. o “Not sure what to do as far as physical exam was concerned”. o “Not sure if I was supposed to do a more thorough history”. o “Not sure what to focus on”. o “Realized half way through what I was supposed to be doing but by then it was too late”. o “Tell me it’s OK to gloss over the medical part”. o “Only done one OSCE and was told to spend the whole time building rapport with patient, so that’s what I did in the TOSCE”. Solution – provide clear objectives during the orientation and let them know it’s OK to gloss over the medical aspects of the case and focus on the team/geriatric aspects. DO Concern – Not sure if OK to reference the discharge papers during the TOSCE or if they were supposed to be read and discarded before the TOSCE began. Solution - tell students it’s OK to review and discuss the provided paperwork during the TOSCE PharmD Concern – Not sure when OK to use the phone – during vs. after the patient interaction Solution – make it more clear during the orientation that they can call any time during the TOSCE.
TOSCE Observer Orientation
Provide them with pencils and not only walk them through the rubric itself, but give them a big picture view of how they are expected to use the rubric
TOSCE Performance Task
General Concern – so much information and possible medical issues in the doorway information that they didn’t know what to focus on. Observer questioned whether the case was to complex for a novice (especially DMD students who had little/no OSCE experience. Solution – provide more clear/directed medical information in the doorway so that the medical aspects of the case are clear and practically “resolved” before
133
they even see the patient, this will free them up to focus on the communication/collaboration/teamwork aspects of the case on which we want them to focus. This would also make the goals/objectives more clear (another concern expressed by the majority of student participants). General Concern – students often didn’t pick up on the fact that the INR or BP levels were too high or mention the elder abuse. Solution – make BP and INR information more prominent in the paperwork and possibly increase levels if appropriate. One observer suggests simplifying the TOSCE as perhaps they didn’t pick up on these things because the issues are too advanced for their current level of training. General Concern – Students kept picking up the phone receiver preventing raters from hearing/recording both sides of the conversation. Solution – Remove the phone receiver so that it’s impossible to pick up the phone; this will force them to use the speaker phone. General Concern – felt their hands were tied because not enough information and too many medical issues. Didn’t know if they could do blood pressure or INR levels at their offices. Solution – provide vitals as part of the doorway information. General Concern – students wanted more time for follow-up and to write the plan. Solution – considering increasing the time designated for follow-up/plan writing. DPT Concern – five minutes wasn’t enough time to review the doorway information/discharge papers. Solution – all other students get much more time – give DPT students more time for review. DPT Concern – distracted while reading disclosures because they could hear the patient and caregiver arguing inside the room. Solution – have DPT students read disclosure in another room. DPT Concern – students didn’t address the fact that blood pressure was too high. They said they didn’t address it because they thought that the vitals were from when the patient was in the hospital NOT from the day of the visit. Solution – make it clear on the paperwork that the blood pressure was taken the same day as the home visit to the patient. Same with INR and other results. Also provide doctors name/phone #. PharmD Concern – not clear how much time lapsed between discharge from the hospital and the Coumadin clinic visit – this impacted their reaction to the INR levels.
134
Solution – clarify in the doorway information what today’s date is, what date the patient was discharged and when INR levels were tested. PharmD Concern – not clear about whether they should be looking at all of the medications or not. Do we want them to primarily focus on the Coumadin issue? Solution – If we want them to focus on the Coumadin, then eliminate presentation of other medication issues in the doorway information. DMD Concern – didn’t think radiographs really existed; thought patient was confused when they said they were “in the drawer”. Felt that having radiographs would have moved the consultation along and given them something more to talk about with the patient. Solution - provide radiographs as part of the doorway information.
Assessment Tools
Indicate if OK to give credit for missing information or safety issues other than what’s indicated on the rubric. Is Coumadin the most important safety issue or are others equally important in terms of rating? If they note other missing information but not INR levels, are they equally important in terms of rating? Clarify if Coumadin is the only safety issue that they can be credited for on the rubric. Generally look at all questions to ensure that examples are provided for each and that it’s clear that these are examples and not the only way to earn credit for the question. Provide more concrete examples of what constitutes each scoring level on the rubric. Use video clips from this pilot study to demonstrate each performance level. Eliminate double-barreled questions/Reword questions with “and/or” statements (#7 cited as example). Live Raters, Video Raters and SPCG Raters all noted this. SPCG Raters suggested splitting questions in two allowing each to have a single focus. Q2 o change word “family” to “caregiver” in all three performance descriptions and behavior Q4 o change word “challenged” as some Faculty Raters felt it was inappropriate to “challenge” but seemed OK with the concept we were getting at…. o Consider clarifying on the rubric when the physician is expected to push back on a safety issue
135
Q6 o Competent and Not Performing are essentially the same as currently worded - reword to make it clear that High Performing is A+B+C, Competent is A+B or A+C, and that Not Performing is A or B or C. Suggest adding word “AND” under high performing description after “body posture)”, and capitalizing “AND” under Competent.
Q7 o Add nutrition as an example of a “missing information” issue in addition to INR – let there be more options for scoring well on this question than just addressing missing INR value.
Q8 o change word “patient” to “caregiver under High Performing and Competent
Q9 o Faculty Raters were not asked to rate the “handed off” question, but standardized patients were (Q9). Consider adding this question to the observer evaluation tool if appropriate. o Competent description is effectively the same as High Performing due to “verbal or non-verbal” reference and use of “and/or” statement. Remove “and” to correct. Perhaps change verbal/non verbal as well since it does not offer any additional clarity. o Students usually stated the plan to patient/son, but none elicited confirmation of understanding. Don’t understand the difference between performing and high performing. Perhaps clarify how “confirmation of understanding” could be demonstrated in this TOSCE case or remove/reword this part of the question. Q10 o One of the behaviors indicated “student articulated…” while the grading addressed whether or not “student referred”. Felt like there was a disconnect between the two. Consider rewording to provide better alignment between the two. o “Competency 10 is hard to answer with respect to 5/6. My interpretation is: 5-referral for medication management 6-referral for possible abuse 10-referral to other provider for other reported issues, such as tooth pain or trouble swallowing”
136
Q11 o
this setting does not allow students to “double-check that information conveyed was received”. Suggest changing wording to “AND it was clear that the referring provider heard and understood what had been shared.
Q6&10 o Students get double credit for referring only once. Is this OK? All o If they say you’re going to do something but don’t actually do it during the TOSCE can they get credit for doing it? o Add examples to rubric to clarify what is meant by active vs. passive Active = when the healthcare provider indicates “I will do X”, “I’ll call your doctor”, “I’ll give you a referral” etc. Passive = when the healthcare provider indicates “you (patient/caregiver) should do X”, “You should make an appointment”, “you should talk to your pharmacist” etc. o Should the rubric indicate that the handoff to next office has to be verbal? TOSCE format doesn’t really provide strong opportunity to communicate verbally with other healthcare providers. Like to see communication skills such as empathy open ended questions, reflective listening and attention to body language addressed Challenging because the scenarios didn’t always play out as anticipated (e.g. DPT with patient with HTN Marti says geriatrics workers are taught to be more formal with patient names. Should the request for name preference go back into the rubric as a separate rubric item? Have all SPCG Raters receive the final rubric in advance and take part in a dry run.
General Student Feedback
General – Wished they could have had an opportunity to receive feedback on their performance General – Wished they could have debriefed with the interprofessional team after the TOSCE General – Wished they could have actually interacted with each other during TOSCE General – Would have liked 5-10 more minutes with the patient to delve more deeply
137
General – Felt that the scenario was realistic/something they could expect to see in practice General – Felt that the standardized patient scenarios did not need any major changes General – Felt that the assessment rubric did not need any major changes General – Greatest benefits were: o making us think about how to work with healthcare providers outside of our profession o making us think about how to deal with elder scenarios (e.g. caregiver dynamic) o making us try out our conflict resolution skills o helping us learn about each other’s roles in patient care General – Not sure if it would be OK to send caregiver out of the room General – Think that doing the TOSCE during pre-clinical years is best because they think it’s essential to be prepared should they encounter this kind of thing on rotations/clinical. DPT students just saw this on clinicals last week (2 nd year feedback). General – Thought that this TOSCE was too complex for 2nd year students, but great for 3rd years. Suggest offering a modified version to 2nd year students that’s simpler and more directed (3rd year feedback). DPT – Decrease wait time DPT – Wished they’d had more time for the post session wrap up DO – “Doing IPE in asynchronous environment makes more sense – it helped me learn more” DO – Learn how to handle difficult patients in curriculum so felt prepared for the TOSCE DO/DMD – TOSCE was challenging because haven’t yet covered geriatric care DMD – TOSCE was challenging because haven’t yet had patient encounters in the curriculum DMD – Suggested having students develop a plan of action before seeing the patient DMD – Not sure with whom to close the loop DMD – Having the x-ray in the drawer was weird because patient knew more than they did DMD – The fact that it was new territory was nice because liked challenge PharmD – Enjoyed the TOSCE because it resembled his work setting at Kaiser
138
Dr. Sam – College of Pharmacy teaches conflict resolution which probably helped the pharmacy students in this experience. Not sure if all colleges do, and if so, at what point in the curriculum
General Standardized Patient Feedback
“Some of the wording and terms made it difficult to feel confident we were assessing student performance as we should be”. Exact wording on rubric made it difficult to use the rubric (e.g. “summarize”) Wished they had received a copy of the last minute changes to the assessment form in advance because they didn’t feel as well prepared as they could have been.
General Observer Feedback
It is unclear as to whether bruises were the result of abuse or too strong a grip when lifting Pharmacy students appeared to note abuse though we didn’t expect them to The abscess was presented too dramatically by one Standardized Patient to the dentist. We should pull clips from TOSCE pilot study to use in observer training. Demonstrate what a rating of 5, 3, and 1 look like. This will increase reliability of the evaluation instrument. Standardized Patient 2 (SP2) acted more passive/timid than Standardized Patient 1(SP1) (both in experimental and control groups). One observer suggests that elder abuse and other issues be made more apparent – another observer said that especially with 3rd years they should be recognizing subtle signs of elder abuse at that stage in their education so no need to be more obvious with their cues. Some students used the phone a lot while others used it only a little. Will those who used the phone only a little have more information in their summary plans? Edit out time when student is writing (for video reviews) Provide the plan summaries to Video Raters Students seemed nervous – perhaps have them do it once, receive feedback, then do it again. Mixed review about whether rubric is too generic and needs to be more programspecific. Associated suggestion was that faculty should rate their own students on a more program-specific rubric. Other faculty said that interprofessional skills are generic, so it’s appropriate that the rubric also be generic.
139
Had a lot of technical problems with video review group. First – the video for a few stations got cut in half, and one TOSCE interaction was uploaded twice under two different participant ID numbers. A number of faculty did not have access to the W drive folders, but didn’t realize it until the deadline. Another reviewer was not able to stream video fast enough off campus (she’s distance CGN faculty) and also had volume issues with her laptop. At least two faculty were ultimately unable to review the videos by the deadline, and the backup video reviewer ended up not being able to assist either.
140
REFERENCES Accreditation Council for Graduate Medical Education. (2007, Effective July 1, 2007). Common program requirements: General competencies. Retrieved November 8, 2010, from http://www.acgme.org/outcome/comp/GeneralCompetenciesStandards21307.pdf Accreditation Council for Pharmacy Education. (2006). Accreditation standards and guidelines for the professional program in pharmacy leading to the Doctor of Pharmacy degree. Retrieved November 26, 2010, from http://www.acpeaccredit.org/pdf/ACPE_Revised_PharmD_Standards_Adopted_Jan152006.pdf Airasian, P. (1994). Classroom assessment. In L. Akers & E. Rosenberg (Eds.) (Second ed.). Columbus: McGraw Hill Incorporated. Ajzen, I. (2005). Attitudes, personality and behavior (Second ed.). Polland: Open University Press. Allen, M., Jackson, D., & Youngner, S. (1980). Closing the communication gap between physicians and nurses in the intensive care unit setting. Heart and Lung, 9(5), 836-840. Amin, Z., Seng, C., & Eng, K. (2006). Practical guide to medical student assessment. Singapore: World Scientific Publishing Company. Andrich, D. (1988). Rasch models for measurement. Thousand Oaks: Sage Publications, Inc. Association of American Medical Colleges. (1998). Report I - Learning objectives for medical student education: Guidelines for medical schools. Washington, DC: Association of American Medical Colleges. Association of American Medical Colleges. (2006). Educating doctors to provide highquality medical care: A vision for medical education in the United States. Washington, DC: Association of American Medical Colleges. Baggs, J., & Ryan, S. (1990). Intensive care unit nurse-physician collaboration and nursing satisfaction. Nursing Economics, 8(6), 386-392. Barber, G., Borders, K., Holland, B., & Roberts, K. (1997). Life span forum: An interdisciplinary training experience. Gerontology and Geriatrics Education, 18(1), 4759.
141
Barrett, J., Gifford, C., Morey, J., Risser, D., & Salisbury, M. (2001). Enhancing patient safety through teamwork training. Journal of Health Care Risk Management, 21(4), 5765. Beatty, P. (1987). Attitudes and perceptions of nursing students toward preparation for interdisciplinary health care teams. Journal of Advanced Nursing, 12, 21-27. Biggs, J. (2003). Teaching for quality learning at university. Berkshire: The Society for Research into Higher Education and Open University Press. Blue, A., Zoller, J., Stratton, T., Elam, C., & Gilbert, J. (2010). Interprofessional education in United States medical schools. Journal of Interprofessional Care, 24(2), 204-206. Bogden, P., Abbott, R., Williamson, P., Onopa, J., & Koontz, L. (1998). Comparing standard care with a physician and pharmacist team approach for uncontrolled hypertension. Journal of General Internal Medicine, 13, 740-745. Brickell, J., Huff, F., & Fraley, T. (1997). Educating for collaborative practice. Clinical Laboratory Science, 10(6), 311-314. Burns, C., Smith, A., Hyer, K., Jacobson, H., Lowry, L., Reed, C., et al. (2000). Training the interdisciplinary team in primary care. National Academies of Practice Forum, 2(2), 95-100. Burns, R., Nichols, L., Martindale-Adams, J., & Graney, M. (2000). Interdisciplinary geriatric primary care evaluation and management: Two-year outcomes. Journal of the American Geriatrics Society, 48(1), 8-13. Cameron, A., Rennie, S., DiProspero, L., Langlois, S., Wagner, S., Potvin, M., et al. (2009). Findings from an evaluation of an interprofessional education experience for one thousand first-year health science students. Journal of Allied Health, 38(4), 220-226. Centre for the Advancement of Interprofessional Education. (2010). Defining IPE. Retrieved November 26, 2010, from http://www.caipe.org.uk/about-us/defining-ipe/ Cohen, J. (1999). Letter to medical school deans. Washington, DC: Association of American Medical Colleges. Collaborative Health and Human Services, C. M. B. (2010). CHHS Major Learning Outcomes: Collaboration. Retrieved 04/25, 2010, from http://hhspp.csumb.edu/academic/CHHS/C.htm
142
Committee on Quality of Health Care in America, I. o. M. (2001). Crossing the quality chasm: A new health system for the twenty-first century. Washington: National Academy Press. Cooper, H., Carlisle, C., Gibbs, T., & Watkins, C. (2001). Developing an evidence base for interdisciplinary learning: A systematic review. Journal of Advanced Nursing, 35(2), 228-237. Cooper, J., Newbower, R., & Kitz, R. (1984). An analysis of major errors and equipment failures in anesthesia management: Considerations for prevention and detection. Anesthesiology, 60(1), 34-42. Craig, R. (Ed.). (1996). The ASTD training and development handbook: A guide to human resource development. New York: McGraw-Hill. Curran, V., Sharpe, D., Flynn, K., & Button, P. (2010). A longitudinal study of the effect of an interprofessional education curriculum on student satisfaction and attitudes towards interprofessional teamwork and education. Journal of Interprofessional Care, 24(1), 4152. Davidson, R., & Waddell, R. (2005). A historical overview of interdisciplinary family health: A community-based interprofessional health professions course. Academic Medicine, 80(4), 334-338. DeKruif, K. (2010). Western University of Health Sciences Enrollment Report 20102011. Pomona: Western University of Health Sciences. Dent, J. H., R. (2005). A practical guide for medical teachers. Edinburgh: Elsevier Churchill Livingstone. DeVellis, R. (2003). Scale development: Theory and applications (Vol. 26). Thousand Oaks: Sage Publications. Diller, L. (1990). Fostering the interdisciplinary team, fostering research in a society in transition. Archives of Physical Medicine and Rehabilitation, 71(5), 275-278. Doran, D., Baker, G., Murray, M., Bohnen, J., Zahn, C., Sidani, S., et al. (2002). Achieving clinical improvement: An interdisciplinary intervention. Health Care Management Review, 27(4), 42-56. Emmert, M. (2010). About Western University of Health Sciences. Retrieved January 31, 2011, from http://prospective.westernu.edu/university
143
Finset, A., Krogstad, J., Hansen, H., Verstad, J., Haarberg, D., & Kristansen, G. (1995). Team development and memory training in traumatic brain injury rehabilitation: Two birds with one stone. Brain Injury, 9(5), 495-507. Greenfield, S., Roblin, D., Kaplan, S., Carlton, D., & Roberts, M. (2002). Collaborative clinical culture and primary care outcomes. Journal of General Internal Medicine, 17(Unknown), 195. Hewstone, M., Carpenter, J., Franklyn-Stokes, A., & Routh, D. (1994). Intergroup contact between professional groups: Two evaluation studies. Journal of Community and Applied Social Psychology, 4, 347-363. Hibbard, M., Van Wagenen, L., Lewbel, S., Waterbury-Wyatt, S., Shaw, S., Pelletier, K., et al. (1996). A teacher's guide to performance-based learning and assessment. Alexandria: Association for Supervision and Curriculum Development. Howarth, M., Holland, K., & Grant, M. (2006). Education needs for integrated care: A literature review. Journal of Advanced Nursing, 56(2), 144-156. Interprofessional Education Consortium. (2001). Evaluating interprofessional education programs. San Francisco: Interprofessional Education Consortium. Interprofessional Education Consortium. (2002). Creating, implementing and sustaining interprofessional education. San Francisco: Interprofessional Education Consortium. Joint Commission on Accreditation of Healthcare Organizations. (2004). Sentinel Event Statistics. Retrieved May 27, 2010, from www.jcaho.org/accredited+organizations/ambulatory+case/sentinel+events/sentinel+even t+statistics.htm Katz, S., Halstead, L., & Wierenga, M. (1975). A medical perspective of team care. New York: Spectrum. Katzenbach, J., & Smith, D. (1993). The discipline of teams: Creating the highperformance organization. New York: HarperCollins Publishers, Inc. Keith, R. (1991). The comprehensive treatment team in rehabilitation. Archives of Physical Medicine and Rehabilitation, 72(5), 269-274. King, H., Battles, J., Baker, D., Alonso, A., Salas, E., Webster, J., et al. (2008). TeamSTEPPS: Team strategies and tools to enhance performance and patient safety. Agency for Healthcare Research and Quality.
144
Knaus, W., Draper, E., Wagner, D., & Zimmerman, J. (1986). An evaluation of outcome from intensive-care in major medical centers. Annals of Internal Medicine, 104(3), 410418. Kohn, L., Corrigan, J., & Donaldson, M. (2000). To err is human: Building a safer health system. Washington, DC: National Academy Press. Koppel, I., Barr, H., Scott, R., Freeth, D., & Hammick, M. (2001). Establishing a systematic approach to evaluating the effectiveness of interprofessional education. Issues in Interdisciplinary Care, 3(1), 41-49. Leathard, A. (Ed.). (2003). Interprofessional collaboration: From policy to practice in health and social care. New York: Brunner-Routledge. Lindqvist, S., Duncan, A., Shepstone, L., Watts, F., & Pearce, S. (2005). Case-based learning in cross-professional groups: The development of a pre-registration interprofessional learning programme. Journal of Interprofessional Care, 19(5), 509-520. Litaker, D., Mion, L., Planavsky, L., Kippes, C., Mehta, N., & Frolkis, J. (2003). Physician-nurse practitioner teams in chronic disease management: The impact on costs, clinical effectiveness, and patients' perception of care. Journal of Interprofessional Care, 17(3), 223-237. Mackintosh, S. (2010a). Interprofessional education 5000 course syllabus. Pomona: Western University of Health Sciences. Mackintosh, S. (2010b). Interprofessional education 6000 course syllabus. Pomona: Western University of Health Sciences. McDowell, I. (2006). Measuring Health (Third ed.). New York: Oxford University Press. Mitchell, P., Armstrong, S., Simpson, T., & Lentz, M. (1989). American Association of Critical-Care Nurses demonstration project: Profile of excellence in critical care nursing. Heart and Lung, 18(3), 219-237. Moorthy, K., Munz, Y., Adams, S., Pandey, V., & Darzi, A. (2005). A human factors analysis of technical and team skills among surgical trainees during procedural simulations in a simulated operating theatre. Annals of Surgery, 242(5), 631-639. Morey, J., Simon, R., Jay, G., Wears, R., Salisbury, M., Dukes, K., et al. (2002). Error reduction and performance improvement in the emergency department through formal
145
teamwork training: Evaluation results of the MedTeams Project. Health Services Research, 37(6), 1553-1581. OSCE Home. (2010). What is Objective Structured Clinical Examination (OSCE)? Retrieved December 4, 2010, from http://www.oscehome.com/What_is_ObjectiveStructured-Clinical-Examination_OSCE.html Pollard, K., Miers, M., & Gilchrist, M. (2004). Collaborative learning for collaborative working? Initial findings from a longitudinal study of health and social care students. Health and Social Care in the Community, 12(4), 346-358. Popham, W. (1995). Classroom assessment: What teachers need to know. Needham Heights: Allyn and Bacon. Reeves, S., Zwarenstein, M., Goldman, J., Barr, H., Freeth, D., & Hammick, M. (2008). Interprofessional education: Effects on professional practice and health care outcomes: Cochrane Database of Systematic Reviews. Remington, T., Foulk, M., & Williams, B. (2006). Evaluation of evidence for interprofessional education. American Journal Pharmacy Education, 70(3), 66. Rich, M., Beckham, V., Wittenberg, C., Leven, C., Freedland, K., & Carney, R. (1995). A multidisciplinary intervention to prevent the readmission of elderly patients with congestive-heart-failure. New England Journal of Medicine, 333(18), 1190-1195. Risser, D., Rice, M., Salisbury, M., Simon, R., Jay, G., Berns, S., et al. (1999). The potential for improved teamwork to reduce medical errors in the emergency department. Annals of Emergency Medicine, 34(3), 373-383. Rust, J., & Golombok, S. (1989). Modern psychometrics: The science of psychological assessment. London: Routledge. Schmitt, M. (1994). United States of America: Focus on interprofessional practice, education and research. Journal of Interprofessional Care, 8(1), 9-18. Schmitt, M. (2001). Collaboration improves the quality of care: Methodological challenges and evidence from United States health care research. Journal of Interprofessional Care, 15(1), 47-66. Schmitt, M., Farrell, M., & Heinemann, G. (1988). Conceptual and methodological problems in studying the effects of interdisciplinary geriatric teams. The Gerontological Society of America, 28(6), 753-764.
146
Shortell, S., Zimmerman, J., Rousseau, D., Gillies, R., Wagner, D., Draper, E., et al. (1994). The performance of intensive care units: Does good management make a difference? Medical Care, 32(5), 508-525. Simon, R. (1993). Full scale development of emergency team coordination course. Arlington: United States Army Research Institute for the Behavioral and Social Sciences. Simon, R., Morey, J., Locke, A., & Blair, E. (1995). Emergency team coordination course phase one report. Arlington: United States Army Research Institute for the Behavioral and Social Sciences. Spath, L. (1999). Error reduction in health care: A systems approach to improving patient safety. San Francisco: Jossey-Bass Publishers. Spear, S. (2005). Fixing healthcare from the inside, today. Harvard Business Review, 83(9), 78-91. Stone, N. (2006). Evaluating interprofessional education: The tautological need for interdisciplinary approaches. Journal of Interprofessional Care, 20(3), 260-275. Streiner, D., & Norman, G. (1995). Health measurement scales: A practical guide to their development and use (Second ed.). New York: Oxford University Press. Taft, S., Jones, P., & Minch, E. (1992). Strengthening hospital nursing 3: Differences among professional groups in the hospital planning process. Journal of Nursing Administration, 22(7), 41-50. Taggart, W. (1994). Crew resource management: Achieving enhanced flight operations. Hampshire: Ashgate Publishing Ltd. Tegzes, J. (2010a). Interprofessional education 5100 course syllabus. Pomona: Western University of Health Sciences. Tegzes, J. (2010b). Interprofessional education 6100 course syllabus. Pomona: Western University of Health Sciences. Trojan, L., Suter, E., Arthur, N., & Taylor, E. (2009). Evaluation framework for a multisite practice-based interprofessional education intervention. Journal of Interprofessional Care, 23(4), 380-389. University of Florida. (2007). Interdisciplinary family health 2007-2008 syllabus. Gainesville: University of Florida.
147
Van der Horst, M., Turpie, I., & Nelson, W. (1995). St. Joseph's Community Centre model of community-based interdisciplinary health care team education. Health and Social Care in the Community, 3(Unknown), 33-42. Verma, S., Paterson, M., & Medves, J. (2006). Core competencies for health care professionals: What medicine, nursing, occupational therapy, and physiotherapy share. Journal of Allied Health, 35(2), 109-115. Waddell, R., Burg, M., McCarthy, J., & Davidson, R. (2002). Evaluation of an interdisciplinary course for medical, nursing and pharmacy students. Annals of Behavioral Science and Medical Education, 8(2), 85-91. Wagner, E. (2000). The role of patient care teams in chronic disease management. British Medical Journal, 320(7234), 569-572. Williams, E., Konrad, T., Linzer, M., McMurray, J., Pathman, D., & M, G. (1999). Refining the measurement of physician job satisfaction results from the physician worklife survey. Medical Care, 37(11), 1140-1154. Zimmerman, J., Shortell, S., Rousseau, D., Duffy, J., Gillies, R., & Knaus, W. (1993). Improving intensive-care: Observations based on organizational case studies in nine intensive-care units - a prospective multicenter study. Critical Care Medicine, 31(10), 1443-1451. Zwarenstein, M., Atkins, J., Barr, H., Hammick, M., Koppel, I., & Reeves, S. (1999). A systematic review of interprofessional education. Journal of Interprofessional Care, 13(4), 417-424.
148
