Delineating simulation-based learning, problem-based ... - MedEdWorld

Dong C MedEdPublish 2014, 3: 38 http://dx.doi.org/10.15694/mep.2014.003.0038

Article

Open Access

Delineating simulation-based learning, problem-based learning, and standardized patient instructional approaches Chaoyan Dong Corresponding author: Chaoyan Dong [email protected] Department: Medical Education Unit, Yong Loo Lin School of Medicine, National University

of Singapore, 1E Kent Ridge Road, NUHS Tower Block, Level 11, Singapore 119228. The research was conducted at New York University School of Medicine and Yong Loo Lin School of Medicine, National University of Singapore, Singapore Received: 24/09/2014 Accepted: 24/09/2014 Published: 06/11/2014 Abstract

Many studies compare simulation to other instructional approaches in health profession education journals. However, each approach emphasizes different aspects of learning and can be applied to different learning objectives. Herein, we outline each approach's distinct characteristics and use an example from the literature in which simulation-based learning was compared to problem-based learning and standardized patient to dramatize the limitations in applying and comparing simulation-based learning with other approaches. When employing each instructional approach, medical teachers must determine what kind of learning is sought in what contexts, deliberate about what strategies may be most appropriate for those goals, and consider available personnel and equipment. Because those answers will differ, medical teachers should avoid making facile comparisons among the different approaches and make sure to use rigorous methods in applying each. Simulation-based learning has proven effects and differs from other instructional methods. Research about simulation-based learning should move beyond comparative studies. Practice Points 1. Many studies compare simulation to other instructional approaches in health profession education journals. However, each approach emphasizes different aspects of learning and can be applied to different learning objectives. 2. When employing each instructional approach, medical teachers must determine what kind of learning is sought in what contexts, deliberate about what strategies may be most appropriate for those goals, and consider available personnel and equipment. 3. Simulation-based learning has proven effects and differs from other instructional methods. Page | 1


Research about simulation-based learning should move beyond comparative studies. Keywords: Simulation, PBL, standardisation. Article

Introduction Students vary widely in how they learn and process information (Driscoll, 2005). This has led to the development of a variety of teaching methods, from lecture-based, didactic teaching to more hands-on, practice-based methods, and, in more recent times, towards selfdirected learning. In the past decades, technology innovations such as simulation have demonstrated educational effectiveness (Issenberg, McGaghie, Petrusa, Lee Gordon, & Scalese, 2005; McGaghie, Draycott, Dunn, Lopez, & Stefanidis, 2011; McGaghie, Issenberg, Petrusa, & Scalese, 2010). Many studies in health profession education compare simulation with other instructional approaches, such as problem-based learning and standardized patients. However, each learning approach emphasizes different aspects of learning and can be applied to different learning objectives. More than half of the research abstracts presented at the 2014 International Meeting on Simulation in Healthcare (IMSH) reported on studies to determine whether simulation contributed to better learning outcomes and/or to support use of simulation-based learning. However, meta-analysis studies have already outlined the beneficial effects attributable to simulation-based learning (Issenberg et al., 2005; McGaghie et al., 2010). The purpose of this article is to urge the rigorous methodological application of each approach so as to avoid the limitations in applying and comparing simulation-based learning with other approaches. Simulation-based medical education, problem-based learning, and standardized patients drawn from the literature are used to dramatize such pitfalls. The distinct characteristics of the three approaches are reviewed. Review of Simulation-Based Learning (SBL), Problem-Based Learning (PBL), and Standardized Patient (SP) Simulation-based learning (SBL), problem-based learning (PBL), and standardized patient (SP) are not equivalent because of the unique emphasis of each. Even in the same context, these approaches can be applied to achieve different learning objectives. Misconceptions and misuse of these approaches are common in medical education (G Maudsley, 1999; Savin-Baden & Major, 2004). Shanley (2007) pointed out that “Some will deny that what is being discussed is in fact PBL because it is not the ‘pure’ version … Others will adopt such an all-inclusive view as to suggest that any alternative method of case study is merely another variation on the PBL theme” (p. 480). To make an informed choice about which of the three approaches to use in a particular context, one must consider the major differences among them. Table 1 summarizes their definitions, core steps, hardware required, roles of instructor, trainees’ responsibility, scenario, and assessment.

Page | 2


Table 1 Descriptions of Simulation-Based Learning (SBL), Problem-Based Learning (PBL), and Standardized Patient (SP) Definition PBL is “a continuum of approach rather than one immutable process,” and “a teaching method the can be included in the teacher’s tool-kit along with other teaching methods rather than used as the sole educational strategy” (Davis & Harden, 1999, p. 130).

Process 1. The problem comes first. 2. Students activate prior knowledge for preliminary diagnosis. 3. Students engage in systematic reasoning through history-taking for evidence of diagnosis. 4. Group report (Barrows, 1996; Davis & Harden, 1999).

Hardware Small and appropriately appointed rooms are essential (Taylor & Miflin, 2010).

SBL

1. Use computerized mannequins to mimic human body functions. 2. Present symptoms of different diseases.

1. Case presentation. 2. Care for simulated patients. 3. Debriefing.

SP

1. Portray a clinical case through standard roleplaying (Ainsworth, 1991; Lane et al., 2001). 2. Present existing disease processes and a history consistent with the disease. 3. Provide feedback.

1. Case presentation. 2. Care for simulated patients. 3. Debriefing.

Mannequin, facilitators, simulation technician, and space (Ainsworth, 1991; Lane et al., 2001). Standardized patient, facilitators.

PBL

Page | 3

Trainee 1. Problem solving through group interaction. 2. Self-motivated learners (Ferrier, Marrin, & Seidman, 1988; PBLI, 2003). 3. No obvious group leader. 4. 6-8 students (H.S. Barrows, 1988). [For group work] 1. A clear group leader. 2. Team leader makes diagnosis. 3. Team members facilitate team leader. Same as SBL.

Instructor Content expert and good facilitators (Miflin, 2004).

Scenario Authentic clinical scenarios are prevalent and important in practice (PBLC, 2000).

Assessment By instructor.

1. Present the case. 2. Provide feedback. 3. Debrief.

Authentic clinical scenario.

By instructor and/or peers through checklists.

Same as SBL.

Authentic clinical scenario.

By instructor and/or peers through checklists.


Aspects of Application that Vary among Instructional Approaches Aspects of application vary among SBL, PBL, and SP, affecting the choice of which to use. These are discussed below. Learning process. The learning process in PBL is illustrated step-by-step, and the beginning of each step is likely to be announced by the facilitators (Howard S. Barrows, 1996; Hmelo-Silver & Barrows, 2006), whereas in SBL and SP, the session proceeds without distinct phases (Kachur et al., 2013; McGaghie, 1999). In PBL, students try to arrive at a diagnosis; they are not required to care for the patient based on that diagnosis. However, in SBL and SP, students go on to assess the patients based on their diagnosis. Platform. A small and designated room is required for a PBL session; SBL requires a mannequin and facilitators; and SP requires a live actor and facilitators (Ainsworth, 1991; Lane, Slavin, & Ziv, 2001). Roles of students. In PBL, everyone works collaboratively; there is not an obvious team leader (Savin-Baden & Major, 2004). In a SBL session that requires a group of students to work together, there must be a clear group leader responsible for the diagnosis and assigning each team member a specific role; this has been studied in interprofessional education studies (Baker, Day, & Salas, 2006). The team members make sure that there is no mistake in the diagnosis while performing their own duties. In a SP session that requires a group of clinicians to work together, the role of team leader and each team member are the same as in SBL. Roles of instructors. In all three approaches, instructors facilitate the session through monitoring the learning process. In a PBL session, the instructor presents the case or problems, answers students’ questions, and provides necessary support when students encounter difficulties (H G Schmidt & Moust, 1995). In a SBL session, the instructor presents the case to students, does not participate in the session, but provides feedback through debriefing (Dieckmann, Molin Friis, Lippert, & Ostergaard, 2009; Galloway, 2009). Similarly, during a SP session, the instructor presents the case and provides feedback, but does not participate (Howard S. Barrows & Abrahamson, 1964; Kachur et al., 2013). Scenarios/problems. For all three approaches, the scenarios are authentic clinical scenarios (Howard S. Barrows, 1996; Kachur et al., 2013; McGaghie, 1999). Assessment. For both PBL and SBL, the instructors do the assessment; in a SP session, the SP assesses students’ performance and provides feedback (Kachur et al., 2013). In addition, peer assessment has been widely used in both SBL and SP (Epstein, 2007; Ten Cate & Durning, 2007). Settings and Best Practices that Vary among Instructional Approaches Given the attributes outlined above, each learning approach has been identified as having different areas of applicability, best practices, and set of pros and cons. These are detailed, along with supporting citations, in Table 2 and discussed below.

Page | 4


Table 2 Best Practices for Simulation-Based Learning (SBL), Problem-Based Learning (PBL), and Standardized Patient (SP) PBL

SBL

Best Practice 1. The instructor presents the case or problems, answers questions, and provides scaffolding when students encounter difficulties (Hendry et al., 2003; Gillian Maudsley, 2002; H G Schmidt & Moust, 1995). 2. A good scenario meets students’ level of prior knowledge, engages students in discussion, and encourage students to consider all aspects of the problem (Dolmans et al., 1997; Hmelo-Silver, 2004; Schmidt, Rotgans, & Yew, 2011). 3. Students work collaboratively in the group to solve problems (de Grave, Dolmans, & van der Vleuten, 2001; Dolmans et al., 2001; SavinBaden & Major, 2004). 4. Students should be informed how the PBL session is aligned with learning objectives so they can take the best from it (Howard S. Barrows & Tamblyn, 2000). 1. Feedback should be provided (Issenberg et al., 2005). Effective feedback should identify performance gap, and explore approaches to address the gaps (Rudolph, Simon, Raemer, & Eppich, 2008; Salas, DiazGranados, Weaver, & King, 2008). 2. Trainees are engaged in deliberate practice (McGaghie et al., 2010). 3. Simulation is integrated into curriculum (McGaghie et al., 2010). 4. Simulation fidelity is aligned with educational goals and outcomes (Issenberg et al., 2005). 5. Mastery learning should be used to make sure that “all learners accomplish all educational objectives with little or no outcome variation” (McGaghie et al., 2010, p. 996). 6. Transfer to practice should be one of the ultimate goals of SBL (McGaghie et al., 2011). 7. Team-related skills should be one of the goals of SBL (Salas et al., 2008). 8. If SBL is used in high-stakes testing, standardization, fidelity, and reproducibility should be assured (Gallagher & Cates, 2004). 9. Instructors need training to effectively facilitate, guide, and motivate learners (McGaghie et al., 2010).

Page | 5

Applicable areas 1. Pre-clinical phase of curriculum (Hartling et al., 2010; Irby et al., 2010). 2. Knowledge acquisition, problem solving, collaboration skills (Barrett, 2010; Hmelo-Silver, 2004; Wells, Warelow, & Jackson, 2009). 3. Communication (Koh, Khoo, Wong, & Koh, 2008). 4. Decision making (Gers & Niemer, 2009).

1. ACGME core competencies (Kyle & Murray, 2007). 2. Clinical training in graduate medical education and continuing medical education (Irby et al., 2010).

Pros & cons 1. Easy to access. 2. Instructors have exceptional skills in facilitating the process and designing the scenarios (Savin-Baden & Major, 2004). 3. No patient encounter.

1.

2.

3.

4.

Encounter with simulated patient (Aggarwal et al., 2010). Requirement of mannequins, space, trained instructors, and simulation technicians. Requirement of staff facilitation and coordination for logistic issues. Ethical benefits (Shackford et al., 1999).


SP

Best Practice 1. When assembling a team, the team members should share common goals, carry a variety of skills, meet regularly, and share documents (Kachur et al., 2013). 2. The team agrees on goals and a timeline so that they can identify SPs and secure training times and availability of personnel (Kachur et al., 2013). 3. The blueprint delineates core competencies, establishes performance criteria, and aligns assessment with curriculum (Kachur et al., 2013). 4. The case development: The scenarios should present challenges to trainees, represent real clinical practice, and include a feedback session. The allotted time should be reasonable. A trial run should be done if possible (Kachur et al., 2013). 5. The rating items should be aligned with the blueprint, and a sufficient number of items should be included to assess trainees. Both behavior-specific items and global-rating items should be used (Kachur et al., 2013). 6. SPs and evaluators should be trained for rating and feedback to assure the data quality (Kachur et al., 2013).

Page | 6

Applicable areas 1. Clinical skills (May, Park, & Lee, 2009). 2. Communication (Teutsch, 2003; Ziv, Wolpe, Small, & Glick, 2003). 3. History-taking (Epstein et al., 2005). 4. Licensure and certification testing (Petrusa, 2002).

Pros & cons 1. Encounter with standardized patient (Aggarwal et al., 2010). 2. Requirement of trained actors and trained instructors (H S Barrows, 1993; Kachur et al., 2013; Spencer & Dales, 2006). 3. Requirement of staff facilitation and coordination for logistic issues.


Applicable areas. PBL is widely used in pre-clinical curricula (Hartling, Spooner, Tjosvold, & Oswald, 2010; Irby, Cooke, & O’Brien, 2010), while SBL and SP are more frequently used for clinical training (Irby et al., 2010). However, all three approaches can be customized for similar contexts (Seybert, Smithburger, Kobulinsky, & Kane-Gill, 2012; Smithburger, Kane-Gill, Ruby, & Seybert, 2012). They can also be integrated and combined with each other; for example, SBL and PBL have been integrated in nursing education and showed effectiveness (Gers & Niemer, 2009; Murphy, Hartigan, Walshe, Flynn, & Brien, 2011; Walshe, O’Brien, Murphy, & Hartigan, 2013). High-fidelity simulation and SP have been used together successfully (Gibson & Blumberg, 2012). However, this makes it difficult for an instructor to select which approach to use in a particular context. Some students learn best through a particular instructional method, while others may learn a specific task regardless of the instructional approach (Driscoll, 2005). In addition, determining what effect each learning method may have had in the outcomes is difficult. Best Practices. Best practices for each approach have been widely researched and discussed (see Table 2). For PBL, the instructor should closely monitor the progress (Hendry, Ryan, & Harris, 2003). The scenario should meet students’ level of prior knowledge and encourage students to consider all aspects of the problem (Dolmans et al., 1997; Hmelo-Silver, 2004; Schmidt, Rotgans, & Yew, 2011). Students should work closely in the group (Dolmans, Wolfhagen, Van Der Vleuten, & Wijnen, 2001). For SBL, McGaghie and colleagues validated a list of best practices, such as constructive feedback, deliberate practice, mastery learning, standardization of the process (McGaghie et al., 2011; Mcgaghie, Issenberg, Barsuk, & Wayne, 2014; McGaghie et al., 2010). For SP, the team members should share common goals and agree upon timelines; the blueprint should include performance criteria and rating items; the scenarios should be authentic clinical cases; and SPs and assessors should be trained for rating and feedback (Kachur et al., 2013). Pros and cons. Compared to SBL and SP, PBL is easy to employ because it does not require any hardware except a dedicated room. But PBL does require that instructors have exceptional skills in facilitating the process and designing the learning scenarios (Savin-Baden & Major, 2004). However, the face-to-face encounter with a patient is absent from PBL. Both SBL and SP offer students the opportunity to interact with a patient (Aggarwal et al., 2010). SBL requires mannequins, space, and trained instructors in facilitating and debriefing; SP requires trained actors to perform as the patient, assess the encounter, and provide feedback, as well as SP-trained instructors (H S Barrows, 1993; Kachur et al., 2013; Spencer & Dales, 2006). SBL also requires simulation technicians to run the simulation, and a SP session requires staff facilitation and coordination for logistic issues, which can lead to significant cost. Examples of Studies Comparing Simulation to Other Approaches For discussion purposes, two studies published in a simulation journal are offered as examples. These studies were identified by literature review as the best matches for dramatizing the difficulties of distinguishing among SBL, PBL, and SP; no criticism is intended of the individual researchers or studies. Both studies compared SBL, PBL, and SP. The first study was in an acute care pharmacotherapy course (Smithburger et al., 2012). The design was a prospective, randomized, crossover design. Students rotated among three seizure disorder cases, presented in a PBL, SP, or SBL approach. Outcomes included knowledge change (assessed by quizzes and students’ perceptions) and satisfaction (by survey). Those who received instruction via the SBL approach had higher knowledge gain and higher satisfaction compared with those using the SP and PBL approaches.

Page | 7


The second study was in a seizure disorder laboratory session (Seybert et al., 2012). The method was a randomized, crossover design. Students were randomly assigned to Group 1 (SBL in Week 1 and PBL in Week 2) or Group 2 (PBL in Week 1 and SBL in Week 2). Outcomes included knowledge change assessed by quizzes and clinical assessment evaluated through use of a highfidelity mannequin. For both groups, the topic for Week 1 was management of dysrhythmias and for Week 2 was management of heart failure. Students using the SBL approach had higher knowledge change than those using the PBL approach, and outperformed the other two groups in clinical assessment. Both articles had the following methodological limitations. First, students solved the problems through working in groups. How much students learn in a group setting is not entirely attributable to which instructional approach is used, but also by group dynamics; this has been well documented in the field of organizational psychology (Diehl & Stroebe, 1991; Ingham, Levinger, Graves, & Peckham, 1974; Karau & Williams, 1993). Difference in team members is a critical confounding factor that affects the teamwork outcomes (H.S. Barrows, 1988; Diehl & Stroebe, 1991; Miflin, 2004), but this issue was not discussed in either study. Second, it is unclear whether the three approaches in both studies followed the best practices defined in prior research and practice. SBL can be an effective instructional approach under the right condition (Issenberg et al., 2005), which also holds true for PBL (Hmelo-Silver, Eberbach, 2012) and SP (Lane et al., 2001). However, without evidence of best practices, it is difficult to justify SBL as more effective than the other two approaches. Third, it is unclear how much of the observed effect resulted from the interventions. Confounding factors were not controlled, so the results cannot be reliably generalized. In the First Research Consensus Summit of the Society for Simulation in Healthcare, it was stated that, “It is important for most studies to control as much unwanted variance as possible during data collection and interpretation” (Dieckmann et al., 2011, p. s5). In summary, in these studies it is problematic to conclude that SBL was more effective than PBL and SP. Discussion SBL, PBL, and SP all have distinct characteristics. It is important that teachers acquire an overview of what, why, and how for each approach. Teachers should be armed with a teaching toolkit, as Harden pointed out in the course Essential Skills in Medical Education (ESME) and his writings (Harden & Crosby, 2000). Though many health professional educators have their favorite or most familiar approaches of teaching, they might not be aware of best practices (Gilkison, 2003; Walton & Matthews, 1989) and have different interpretations of the same approach; for example, there is confusion about PBL (G Maudsley, 1999; Shields et al., 2007). It is important to gain a deep understanding of the approach they are comfortable with and master best practices in that particular approach. With the help of the toolkit, they can refine the strategies they have been using to align with best practices. The choice of which instructional approach to use must be based on macro and micro level situations. Until recently, the selection of instructional approaches has been a macro-level decision based on various considerations in balance (Bransford, Brown, & Cocking, 2000; Weston & Cranton, 1986). The decision of selecting one particular approach has to be weighed against the increased cost for the entire training, which has been extensively discussed in using technology in education (Woodhall, 2004). For example, at the macro-level, the cost of SBL is high relative to

Page | 8


other approaches. However, the costs are likely to continue to go down as more faculty members receive training in SBL. The costs of purchasing equipment are usually one-time costs; maintenance costs will likely go down as the market matures; and equipment can be used for multiple students across departments and specialties. The macro-level questions for educators to ask are: 1. What is the overall impact of one approach versus another across students? 2. Is this impact going to be sufficient to justify the additional training and cost that might be involved? The selection of instructional approaches to use should also be a micro-level decision. A shift from macro- to micro-level decisions requires educators to have a thorough understanding of the essences of different learning approaches and the requirements for students (Driscoll, 2005; Harden & Crosby, 2000). Health profession educators should develop a deep understanding of how people learn, what influences motivation, and what in the social context affects learning (Harden & Crosby, 2000). Educators have to consider the various sources of knowledge and theory that exist, take into account the specific context, have a thorough understanding of their students, and determine when and how theory can inform practice. Every student brings unique idiosyncrasies, challenges, personality, and ability to the classroom, and educators need to acknowledge these differences (Driscoll, 2005). Teaching should be built on students’ prior knowledge, languages, and cultures. To do this successfully, the educator should have an understanding of different learning theories (Harden & Crosby, 2000; Oakes & Lipton, 1999). When employing each instructional approach, medical teachers must determine what kind of learning is sought in what contexts, deliberate about what strategies may be most appropriate for those goals, and consider available personnel and equipment. At the micro-level, educators should ask: 1. What is the prior knowledge of an individual student on a specific topic? 2. How is this knowledge represented and structured? 3. How does the student apply prior knowledge to acquire new knowledge and to solve problems? 4. Which teaching approaches can best facilitate learning of specific content? 5. What experience do students have in different learning approaches? 6. How does the technology employed in the process support learning? As the examples from the literature show, there are difficulties in applying and comparing simulation-based learning with other learning approaches. Simulation-based learning differs from other instructional methods and has proven effects, as shown in a series of meta-analysis studies (Issenberg et al., 2005; McGaghie et al., 2010). Research about simulation-based learning should move beyond comparing simulation to other instructional approaches such as lecture, online learning, problem-based learning, or team-based learning. Future simulation research should focus on simulation designs, usage patterns, assessment, and integration into curriculum. As an analogy, when the computer was first introduced to health profession education, the field was filled with studies comparing computer with other formats of instruction. However, soon Friedman (1994) pointed out that “studies undertaken from the media-comparative perspective are not the most important studies to be doing at this time” (p. 456), and recently Cook (2005) reiterated that “media-comparative research is futile” for various reasons (p. 541).

Page | 9


In the First Research Consensus Summit of the Society for Simulation in Healthcare in 2011, ten research priorities were identified. Impact of simulation was only one of the 10 areas (Dieckmann et al., 2011), yet received much attention. Simulation should be studied across different levels, as suggested by McGaghie and colleagues: in a medical simulation laboratory (T1); patient care practice (T2); patient outcomes (T3); and collateral educational effects (T4), such as cost savings, skill retention, and systematic educational and patient care improvements (Barsuk, Cohen, Feinglass, McGaghie, & Wayne, 2011; McGaghie et al., 2011, 2010; Santen, Deiorio, & Gruppen, 2012). This would facilitate understanding more in-depth issues about SBL, such as what factors contribute to translation of behavior from the simulation center to patient care; which confounders and validity issues should be taken into account when considering the results; and what characteristics make debriefing effective. Recently, Kardong-Edgren (2014) shared three research priorities for simulation in 2014: 1. How to apply deliberate practice in simulation to help learners to gain proficiency and competency? 2. What are the best practices for teaching with simulation (e.g., using the flipped classroom model to expose students to patients' stories before encounter with a simulated patient)? 3. How can facilitators use discourse analysis techniques to accurately assess learners' knowledge of the content area?

Conclusion Different learning approaches reflect different underlying views of the philosophy of education. Debates about best teaching practices will continue, but there is growing appreciation of how different strategies are useful for different kind of learning and learners. It is perhaps more productive to think of these issues in terms of what kind of learning is sought in what contexts, to deliberate about what strategies may be most appropriate for those goals, and to consider potentially available personnel and equipment. Simulation holds great potential in health profession training. Realization of this potential requires theory-based research questions, rigorous study design, and multiple data collection methods to address the impact of simulation at the training level and patient care level. This will help to refine and advance best practices.

Acknowledgments The authors acknowledge Dr. Elizabeth Kachur, Medical Education Development, for reading earlier drafts of this article and providing invaluable feedback.

Declaration of Interest The author reports no conflicts of interest. The author alone is responsible for the content and writing of the article.

Page | 10


References Aggarwal, R., Mytton, O. T., Derbrew, M., Hananel, D., Heydenburg, M., Issenberg, B., … Reznick, R. (2010). Training and simulation for patient safety. Quality & Safety in Health Care, 19 Suppl 2, i34–i43. doi:10.1136/qshc.2009.038562 http://dx.doi.org/10.1136/qshc.2009.038562 Ainsworth, M. A. (1991). Standardized Patient EncountersA Method for Teaching and Evaluation. JAMA: The Journal of the American Medical Association, 266(10), 1390. http://dx.doi.org/10.1001/jama.1991.03470100082037 http://dx.doi.org/10.1001/jama.1991.03470100082037 Baker, D. P., Day, R., & Salas, E. (2006). Teamwork as an essential component of high-reliability organizations. Health Services Research. http://dx.doi.org/10.1111/j.1475-6773.2006.00566.x http://dx.doi.org/10.1111/j.1475-6773.2006.00566.x Barrett, T. (2010). The problem‐based learning process as finding and being in flow. Innovations in Education and Teaching International, 47(2), 165–174. doi:10.1080/14703291003718901 http://dx.doi.org/10.1080/14703291003718901 Barrows, H. S. (1988). The tutorial process. Springfield, IL: Southern Illinois University School of Medicine. Barrows, H. S. (1993). An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. In Academic medicine : journal of the Association of American Medical Colleges (Vol. 68, pp. 443–451; discussion 451–453). doi:10.1097/00001888-199306000-00002 http://dx.doi.org/10.1097/00001888-199306000-00002 Barrows, H. S. (1996). Problem-based learning in medicine and beyond: A brief overview. New Directions for Teaching and Learning, 1996(68), 3–12. doi:10.1002/tl.37219966804 http://dx.doi.org/10.1002/tl.37219966804 Barrows, H. S., & Abrahamson, S. (1964). The programmed patient: a technique for appraising student performance in clinical neurology. Journal of Medical Education, 39, 802–805. Barrows, H. S., & Tamblyn, R. (2000). Stimulated recall: Peronalized assessment of clinical reasoniong. Springfield, IL. Barsuk, J. H., Cohen, E. R., Feinglass, J., McGaghie, W. C., & Wayne, D. B. (2011). Unexpected Collateral Effects of Simulation-Based Medical Education. Academic Medicine. doi:10.1097/ACM.0b013e318234c493 http://dx.doi.org/10.1097/ACM.0b013e318234c493 Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How People Learn: Brain, Mind, Experience, and School. Committee on learning research and educational practice (Vol. Expanded E, p. x, 374 p.). doi:10.1016/0885-2014(91)90049-J http://dx.doi.org/10.1016/0885-2014(91)90049-J

Page | 11


Carol Okupniak. (2010). combining a high fidelity simulator with a standardized patient in one scenario - the best of both. Clinical Simulation in Nursing, 6(3), e119. http://dx.doi.org/10.1016/j.ecns.2010.03.049 Cook, D. A. (2005). The research we still are not doing: an agenda for the study of computer-based learning. Academic Medicine : Journal of the Association of American Medical Colleges, 80(6), 541–548. doi:10.1097/00001888-200506000-00005 http://dx.doi.org/10.1097/00001888-200506000-00005 Davis, M. H., & Harden, R. M. (1999). AMEE Medical education guide no. 15: Problem-based learning: a practical guide. Medical Teacher, 21(2), 130–140. http://dx.doi.org/10.1080/01421599979743 De Grave, W. S., Dolmans, D., & van der Vleuten, C. P. M. (2001). Student perceptions about the occurrence of critical incidents in tutorial groups. MEDICAL TEACHER, 23(1), 49–54. doi:10.1080/0142159002005596 http://dx.doi.org/10.1080/0142159002005596 Dieckmann, P., Molin Friis, S., Lippert, A., & Ostergaard, D. (2009). The art and science of debriefing in simulation: Ideal and practice. Medical Teacher, 31(7), e287–e294. doi:10.1080/01421590902866218 http://dx.doi.org/10.1080/01421590902866218 Dieckmann, P., Phero, J. C., Issenberg, S. B., Kardong-Edgren, S., Ostergaard, D., & Ringsted, C. (2011). The first research consensus summit of the society for simulation in healthcare: conduction and a synthesis of the results. Simulation in Healthcare : Journal of the Society for Simulation in Healthcare, 6 Suppl, S1–S9. doi:10.1097/SIH.0b013e31822238fc http://dx.doi.org/10.1097/SIH.0b013e31822238fc Diehl, M., & Stroebe, W. (1991). Productivity loss in idea-generating groups: Tracking down the blocking effect. Journal of Personality and Social Psychology. doi:10.1037/0022-3514.61.3.392 http://dx.doi.org/10.1037/0022-3514.61.3.392 Dolmans, D. H. J. M., Wolfhagen, I. H. A. P., Van Der Vleuten, C. P. M., & Wijnen, W. H. F. W. (2001). Solving problems with group work in problem-based learning: Hold on to the philosophy. Medical Education. http://dx.doi.org/10.1046/j.1365-2923.2001.00915.x Driscoll, M. P. (2005). Psychology of learning for instruction. (Anonymous, Ed.)Learning and Instruction (Vol. 3rd, pp. xvi, 448). Allyn & Bacon. Retrieved from http://eric.ed.gov/ERICWebPortal/recordDetail?accno=ED369772 Epstein, R. M. (2007). Assessment in medical education. The New England Journal of Medicine, 356(4), 387–396. doi:10.1056/NEJMra054784 http://dx.doi.org/10.1056/NEJMra054784 Epstein, R. M., Franks, P., Fiscella, K., Shields, C. G., Meldrum, S. C., Kravitz, R. L., & Duberstein,

Page | 12


P. R. (2005). Measuring patient-centered communication in patient-physician consultations: theoretical and practical issues. Social Science & Medicine (1982), 61(7), 1516–28. doi:10.1016/j.socscimed.2005.02.001 http://dx.doi.org/10.1016/j.socscimed.2005.02.001 Ferrier, B., Marrin, M., & Seidman, J. (1988). student autonomy in learning medicine: some participants' experiences. In D. Boud (Ed.), Developing student autonomy in learning. London: Kogan Page. Friedman, C. (1994). The research we should be doing. Academic Medicine, 69, 455–457. http://dx.doi.org/10.1097/00001888-199406000-00005 Gallagher, A. G., & Cates, C. U. (2004). Virtual reality training for the operating room and cardiac catheterisation laboratory. Lancet. http://dx.doi.org/10.1016/S0140-6736(04)17278-4 Galloway, S. (2009). Simulation Techniques to Bridge the Gap Between Novice and Competent Healthcare Professionals. Online Journal of Issues in Nursing, 14(2), 7. doi:10.3912/OJIN.Vol14No02Man03 Gers, M. A., & Niemer, L. (2009). Utilization of problem based learning and simulation to promote integrated decision making in a baccalaureate nursing program. Clinical Simulation in Nursing, 5(3), e139. http://dx.doi.org/10.1016/j.ecns.2009.04.035 Gibson, N., & Blumberg, K. (2012). what? simulation lab with a real live person! using standardized patients in psychiatric clinical simulations. Clinical Simulation in Nursing, 8(8), e388. http://dx.doi.org/10.1016/j.ecns.2012.07.010 Gilkison, A. (2003). Techniques used by "expert" and "non-expert" tutors to facilitate problembased learning tutorials in an undergraduate medical curriculum. Medical Education, 37(1), 6–14. http://dx.doi.org/10.1046/j.1365-2923.2003.01406.x Harden, R. M., & Crosby, J. O. Y. (2000). AMEE Guide No 20 : The good teacher is more than a lecturer - the twelve roles of the teacher. Medical Teacher, 22(4), 334–347. doi:10.1080/014215900409429 http://dx.doi.org/10.1080/014215900409429 Hartling, L., Spooner, C., Tjosvold, L., & Oswald, A. (2010). Problem-based learning in pre-clinical medical education: 22 years of outcome research. Medical Teacher, 32(1), 28–35. doi:10.3109/01421590903200789 http://dx.doi.org/10.3109/01421590903200789 Hendry, G. D., Ryan, G., & Harris, J. (2003). Group problems in problem-based learning. Medical Teacher, 25(6), 609–616. doi:10.1080/0142159031000137427 http://dx.doi.org/10.1080/0142159031000137427

Page | 13


Hmelo-Silver, C. E. (2004). Problem-Based Learning: What and How Do Students Learn? Educational Psychology Review, 16(3), 235–266. Retrieved from http://www.springerlink.com/content/j360715xw085866r/ http://dx.doi.org/10.1023/B:EDPR.0000034022.16470.f3 Hmelo-Silver, C. E., & Barrows, H. S. (2006). Goals and Strategies of a Problem-based Learning Facilitator. Interdisciplinary Journal of Problem-Based Learning, 1(1). doi:10.7771/1541-5015.1004 http://dx.doi.org/10.7771/1541-5015.1004 Hmelo-Silver, Cindy E., Eberbach, C. (2012). Learning theories and problem-based learning. In Problem-based learning in clinical education: Innovation and change in professional education Volume 8 (pp. 3–17). Ingham, A. G., Levinger, G., Graves, J., & Peckham, V. (1974). The Ringelmann effect: Studies of group size and group performance. Journal of Experimental Social Psychology. doi:10.1016/00221031(74)90033-X http://dx.doi.org/10.1016/0022-1031(74)90033-X Irby, D. M., Cooke, M., & O'Brien, B. C. (2010). Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Academic Medicine : Journal of the Association of American Medical Colleges, 85(2), 220–227. doi:10.1097/ACM.0b013e3181c88449 http://dx.doi.org/10.1097/ACM.0b013e3181c88449 Issenberg, S. B., McGaghie, W. C., Petrusa, E. R., Lee Gordon, D., & Scalese, R. J. (2005). Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher, 27(1), 10–28. doi:10.1080/01421590500046924 http://dx.doi.org/10.1080/01421590500046924

Kachur, E. K., Zabar, S., Hanley, K., Kalet, A. L., Bruno, J. H., & Gillespie, C. C. (2013). Organizing OSCEs (and Other SP Exercises) in Ten Steps. In Objective structured clinical examinations: 10 Steps to Planning and Implementing OSCEs and Other Standardized Patient Exercises (pp. 7–34). New York: Springer. Karau, S. J., & Williams, K. D. (1993). Social loafing: A meta-analytic review and theoretical integration. Journal of Personality and Social Psychology. doi:10.1037/0022-3514.65.4.681 http://dx.doi.org/10.1037/0022-3514.65.4.681 Kardong-Edgren, S. (2014). Three research worthy ideas in 2014. Clinical Simulation in Nursing, 10, e55–e56. http://dx.doi.org/10.1016/j.ecns.2013.12.002 Koh, G. C.-H., Khoo, H. E., Wong, M. L., & Koh, D. (2008). The effects of problem-based learning

Page | 14


during medical school on physician competency: a systematic review. CMAJ : Canadian Medical Association Journal = Journal de l'Association Medicale Canadienne, 178(1), 34–41. Retrieved from http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2151117&tool=pmcentrez&rendertype= abstract http://dx.doi.org/10.1503/cmaj.070565 Kyle, R., & Murray, W. B. (2007). Clinical Simulation (p. 848). Academic Press. Retrieved from http://www.amazon.com/Clinical-Simulation-Richard-Kyle/dp/0123725313 Lane, J. L., Slavin, S., & Ziv, A. (2001). Simulation in Medical Education: A Review. Simulation & Gaming, 32(3), 297–314. doi:10.1177/104687810103200302 http://dx.doi.org/10.1177/104687810103200302 Maudsley, G. (1999). Do we all mean the same thing by "problem-based learning"? A review of the concepts and a formulation of the ground rules. Academic Medicine : Journal of the Association of American Medical Colleges, 74(2), 178–185. doi:10.1097/00001888-199902000-00016 http://dx.doi.org/10.1097/00001888-199902000-00016 Maudsley, G. (2002). Making sense of trying not to teach: an interview study of tutors' ideas of problem-based learning. Academic Medicine : Journal of the Association of American Medical Colleges, 77(2), 162–172. http://dx.doi.org/10.1097/00001888-200202000-00017 May, W., Park, J. H., & Lee, J. P. (2009). A ten-year review of the literature on the use of standardized patients in teaching and learning: 1996-2005. Medical Teacher, 31(6), 487–92. doi:10.1080/01421590802530898 http://dx.doi.org/10.1080/01421590802530898 McGaghie, W. C. (1999). Simulation in professional competence assessment: Basic considerations. In A. Tekian, C. H. McGuire, & W. C. McGaghie (Eds.), Innovative simulations for assessing professional competence. Chicago: University of Illinois at Chicago. McGaghie, W. C., Draycott, T. J., Dunn, W. F., Lopez, C. M., & Stefanidis, D. (2011). Evaluating the impact of simulation on translational patient outcomes. Simulation in Healthcare : Journal of the Society for Simulation in Healthcare, 6 Suppl, S42–7. doi:10.1097/SIH.0b013e318222fde9 http://dx.doi.org/10.1097/SIH.0b013e318222fde9 Mcgaghie, W. C., Issenberg, S. B., Barsuk, J. H., & Wayne, D. B. (2014). A critical review of simulation-based mastery learning with translational outcomes. Medical Education. http://dx.doi.org/10.1111/medu.12391 McGaghie, W. C., Issenberg, S. B., Petrusa, E. R., & Scalese, R. J. (2010). A critical review of simulation-based medical education research: 2003-2009. Medical Education, 44(1), 50–63. doi:10.1111/j.1365-2923.2009.03547.x http://dx.doi.org/10.1111/j.1365-2923.2009.03547.x Miflin, B. (2004). Small groups and problem-based learning: are we singing from the same hymn

Page | 15


sheet? Medical Teacher, 26(5), 444–450. doi:10.1080/01421590410001696425 http://dx.doi.org/10.1080/01421590410001696425 Murphy, S., Hartigan, I., Walshe, N., Flynn, A. V., & O'Brien, S. (2011). Merging ProblemBased Learning and Simulation as an Innovative Pedagogy in Nurse Education. Clinical Simulation in Nursing. http://dx.doi.org/10.1016/j.ecns.2010.01.003 Oakes, J., & Lipton, M. (1999). Teaching to change the world. New York: McGraw Hill. PBLC. (2000). Problem-based learning curriculum. PBLI. (2003). Problem-based learning initiative. Petrusa, E. R. (2002). Clinical performance assessments. In G. R. Norman, PH.D., C. P. M. Van Der Vleuten, & D. I. Newble (Eds.), International Handbook of Research in Medical Education, Volume 1 (pp. 673–709). Springer. Retrieved from http://books.google.com/books?hl=en&lr=&id=ioqS0EOJvIkC&pgis=1 Rudolph, J. W., Simon, R., Raemer, D. B., & Eppich, W. J. (2008). Debriefing as formative assessment: closing performance gaps in medical education. Academic Emergency Medicine Official Journal of the Society for Academic Emergency Medicine, 15(11), 1010–6. doi:10.1111/j.1553-2712.2008.00248.x http://dx.doi.org/10.1111/j.1553-2712.2008.00248.x Salas, E., DiazGranados, D., Weaver, S. J., & King, H. (2008). Does team training work? Principles for health care. Academic Emergency Medicine Official Journal of the Society for Academic Emergency Medicine, 15(11), 1002–1009. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/18828828 http://dx.doi.org/10.1111/j.1553-2712.2008.00254.x Santen, S. A., Deiorio, N. M., & Gruppen, L. D. (2012). Medical education research in the context of translational science. Academic Emergency Medicine : Official Journal of the Society for Academic Emergency Medicine, 19(12), 1323–7. doi:10.1111/acem.12040 http://dx.doi.org/10.1111/acem.12040 Savin-Baden, M., & Major, C. H. (2004). Foundations of problem-based learning. SRHE and Open University Press imprint (p. 216). Retrieved from http://www.worldcat.org/oclc/56965887 Schmidt, H. G., & Moust, J. H. (1995). What makes a tutor effective? A structural-equations modeling approach to learning in problem-based curricula. Academic Medicine : Journal of the Association of American Medical Colleges, 70(8), 708–714. doi:10.1097/00001888-19950800000015 http://dx.doi.org/10.1097/00001888-199508000-00015 Schmidt, H. G., Rotgans, J. I., & Yew, E. H. J. (2011). The process of problem-based learning: what works and why. Medical Education, 45(8), 792–806. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/21752076

Page | 16


http://dx.doi.org/10.1111/j.1365-2923.2011.04035.x Seybert, A. L., Smithburger, P. L., Kobulinsky, L. R., & Kane-Gill, S. L. (2012). Simulation-based learning versus problem-based learning in an acute care pharmacotherapy course. Simulation in Healthcare : Journal of the Society for Simulation in Healthcare, 7(3), 162–5. doi:10.1097/SIH.0b013e31825159e3 http://dx.doi.org/10.1097/SIH.0b013e31825159e3 Shackford, S. R., Rogers, F. B., Osler, T. M., Trabulsy, M. E., Clauss, D. W., & Vane, D. W. (1999). Focused abdominal sonogram for trauma: the learning curve of nonradiologist clinicians in detecting hemoperitoneum. The Journal of Trauma, 46(4), 553–62; discussion 562–4. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10217217 http://dx.doi.org/10.1097/00005373-199904000-00003 Shanley, P. F. (2007). Viewpoint: leaving the "empty glass" of problem-based learning behind: new assumptions and a revised model for case study in preclinical medical education. Academic Medicine : Journal of the Association of American Medical Colleges, 82(5), 479–485. doi:10.1097/ACM.0b013e31803eac4c http://dx.doi.org/10.1097/ACM.0b013e31803eac4c Shields, H. M., Guss, D., Somers, S. C., Kerfoot, B. P., Mandell, B. S., Travassos, W. J., … Hafler, J. P. (2007). A faculty development program to train tutors to be discussion leaders rather than facilitators. Academic Medicine : Journal of the Association of American Medical Colleges, 82(5), 486–492. doi:10.1097/ACM.0b013e31803eac9f http://dx.doi.org/10.1097/ACM.0b013e31803eac9f Smithburger, P. L., Kane-Gill, S. L., Ruby, C. M., & Seybert, A. L. (2012). Comparing effectiveness of 3 learning strategies: simulation-based learning, problem-based learning, and standardized patients. Simulation in Healthcare : Journal of the Society for Simulation in Healthcare, 7(3), 141–6. doi:10.1097/SIH.0b013e31823ee24d http://dx.doi.org/10.1097/SIH.0b013e31823ee24d Spencer, J., & Dales, J. (2006). meeting the needs of simulated patients and caring for the person behind them. Medical Education, 40(1), 3–5. http://dx.doi.org/10.1111/j.1365-2929.2005.02375.x Taylor, D., & Miflin, B. (2010). AMEE Guide No. 36: Problem-based learning: Where are we now? Woodhall, M. (2004). Cost-benefit analysis in educational planning (Fourth.). Paris: UNESCO: International Institute for Educational Planning. Ten Cate, O., & Durning, S. (2007). Dimensions and psychology of peer teaching in medical education. Medical Teacher, 29(6), 546–552. doi:10.1080/01421590701583816 http://dx.doi.org/10.1080/01421590701583816 Teutsch, C. (2003). Patient–doctor communication. Medical Clinics of North America, 87(5), 1115– 1145. doi:10.1016/S0025-7125(03)00066-X

Page | 17


http://dx.doi.org/10.1016/S0025-7125(03)00066-X Walshe, N., O'Brien, S., Murphy, S., & Hartigan, I. (2013). Integrative Learning Through Simulation and Problem-Based Learning. Clinical Simulation in Nursing, 9(2). http://dx.doi.org/10.1016/j.ecns.2011.08.006 Walton, H. J., & Matthews, M. B. (1989). Essentials of problem-based learning. Medical Education, 23, 542–558. http://dx.doi.org/10.1111/j.1365-2923.1989.tb01581.x Wells, S. H., Warelow, P. J., & Jackson, K. L. (2009). Problem Based Learning (PBL): a conundrum. Contemporary Nurse, 33(2), 191–201. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/19929163 http://dx.doi.org/10.5172/conu.2009.33.2.191 Weston, C., & Cranton, P. A. (1986). Selecting instructional strategies. The Journal of Higher Education, 57(3), 259–288. http://dx.doi.org/10.2307/1981553 Ziv, A., Wolpe, P. R., Small, S. D., & Glick, S. (2003). Simulation-Based Medical Education: An Ethical Imperative. Academic Medicine, 78(8), 783–788. http://dx.doi.org/10.1097/00001888-200308000-00006

Page | 18