The Potential Role of Intermediates in Preclinical

Lessons from a Pilot Project in Cognitive Task Analysis: The Potential Role of Intermediates in Preclinical Teaching in Dental Education Judith Walker, PhD; HsingChi von Bergmann, PhD Abstract: The purpose of this study was to explore the use of cognitive task analysis to inform the teaching of psychomotor skills and cognitive strategies in clinical tasks in dental education. Methods used were observing and videotaping an expert at one dental school thinking aloud while performing a specific preclinical task (in a simulated environment), interviewing the expert to probe deeper into his thinking processes, and applying the same procedures to analyze the performance of three second-year dental students who had recently learned the analyzed task and who represented a spectrum of their cohort’s ability to undertake the procedure. The investigators sought to understand how experts (clinical educators) and intermediates (trained students) overlapped and differed at points in the procedure that represented the highest cognitive load, known as “critical incidents.” Findings from this study and previous research identified possible limitations of current clinical teaching as a result of expert blind spots. These findings coupled with the growing evidence of the effectiveness of peer teaching suggest the potential role of intermediates in helping novices learn preclinical dentistry tasks. Dr. Walker is Assistant Professor, Educational Studies, Faculty of Education, The University of British Columbia; and Dr. von Bergmann is Associate Professor, Education Research, Faculty of Dentistry, The University of British Columbia. Direct correspondence to Dr. HsingChi von Bergmann, Faculty of Dentistry, The University of British Columbia, JBM 288, Vancouver, BC, Canada V6T 1Z3; 604-827-5699; [email protected]. Keywords: dental education, preclinical education, cognitive task analysis, psychomotor skills, educational methodologies, peer teaching Submitted for publication 1/24/14; accepted 7/28/14

C

ognitive task analysis is a method applied in a wide variety of disciplines to investigate the thought processes and discrete actions behind performance of a task. Although task analytic methods have been around since before World War I,1 this methodology has rarely been used to study clinical tasks in dentistry. The aim of our study was to understand whether cognitive task analysis can help dental educators systematically identify the steps and critical incidents in performing a preclinical dental task. Understanding the discrete steps involved in a task and identifying the areas that present high cognitive challenge can potentially provide important information to educators in designing curriculum, instruction, and assessment that further support students to better master the overall procedure. In this pilot study, we applied cognitive task analysis to an operative dentistry procedure being taught at The University of British Columbia Faculty of Dentistry. Specifically, we explored the cognitive processes and steps involved in undertaking a Class II wax carving by observing and interviewing an expert (a clinical educator) and

286

three second-year dental students trained in the task (thus called “intermediates” because their expertise was between that of the educators and untrained students). Our goal was to understand where these participants’ struggles with this task overlapped (if at all). Analysis of the procedures performed by our participants and of the interview transcripts led us to the literature on peer teaching and expert blind spots and helped us identify the potential contribution of intermediates in facilitating student learning of preclinical dentistry tasks.

Defining Cognitive Task Analysis Cognitive task analysis (CTA) is a family of methods for studying and describing reasoning and knowledge.2 It has been used to isolate the various incidents embedded in a particular procedure or task (e.g., performing an appendectomy or undertaking a root canal treatment on a molar) and to identify the elements of the task with the highest cognitive load.3

Journal of Dental Education ■ Volume 79, Number 3

In a typical CTA study, an expert talks through what she is doing in a particular task and the reasons for her actions. In subsequent analysis of the transcript from the session, researchers isolate the incidents (or steps) and assess cognitive load.3,4 An example of applying CTA to uncover the reasoning process in clinical diagnosis is shown in Figure 1. Drawing on insights from neuroscience and cognitive science, cognitive task analysis suggests tasks that take the largest cognitive load can be identified by someone slowing down, hesitating, or pausing in her speech. These places are called “critical incidents.”2 CTA has proven useful to capture the incidents that make up a task. For example, one study found that surgeons’ descriptions of how to insert a femoral artery shunt were 70% more accurate when captured using CTA than with free recall.5 A number of studies have utilized this approach to inform teaching and curriculum development in the health sciences.5-11 The thinking is that, by distilling the separate steps and the places where an expert struggles, we can develop more appropriate teaching tools for novices.

There are very few published studies of CTA in dentistry. A basic search in Medline conducted in November 2013 returned only two articles and one poster presented at the American Dental Education Association (ADEA) Annual Session & Exhibition.4,12,13 In an article published in 2000, Cameron et al. reported on their study that used a CTA to identify all the knowledge and tasks that are important to dental hygiene.12 In isolating these discrete tasks, the authors designed nine cases, expecting that the solutions would differ depending on whether the participant was an expert, competent, or novice dental hygienist. Interviews to capture solutions to the cases confirmed that respondents differed in nine areas (gathering and using information, formulating problems and investigating hypotheses, communication and language, scripting behavior, ethics, patient assessment, treatment planning, treatment, and evaluation) on the basis of their professional perspective. This research enabled the creation of a simulation exam to test dental hygienists’ competence in those nine areas.

Figure 1. Sequence of reasoning underlying an oral clinical diagnosis Source: Used with permission from Crespo K, Torres J, Recio M. Reasoning process characteristics in the diagnostic skills of beginner, competent, and expert dentists. J Dent Educ 2004;68(12):1235-44.

March 2015 ■ Journal of Dental Education

287

In another article, Crespo et al. described their evaluation of the differences in dental practitioners’ diagnostic reasoning processes depending on their levels of expertise.4 Practitioners were asked to determine the diagnosis of an oral condition based on a clinical case. The researchers found that cognitive processes and reasoning differed depending on experience; for example, “experts [but not novices] evidenced cognitive diagnostic schemas that integrate pathophysiology of disease” (p. 1235). Finally, the poster abstract reported on a study utilizing CTA to identify cognitive activity of dentists reviewing patient information for diagnosis and treatment planning purposes.13 From these authors’ research, they were able to identify the steps taken by dental experts in this process (information retrieval followed by information analysis and then hypothesis generation during treatment planning). In other health sciences, CTA has been used to inform teaching. One article in medicine described the use of CTA to create performance assessments for intraoperative decision making,14 and another reported on how CTA was used to create a simulationbased training for medical tasks.11 Also in the field of medicine, a 2008 article used CTA to capture the steps of three surgical practitioners and educators performing a colonoscopy to identify relevant steps and decision points.7 The researchers found that the participants often omitted some of the steps when they were engaged in traditional teaching, so the results of the CTA informed how this procedure could be taught in the future. Similarly, in veterinary medicine, researchers have reported on how CTA was useful in defining the technical and cognitive steps required to both perform and teach bovine dystocia.10 In other studies, researchers explored how teaching methods derived from CTA might affect student learning in medicine. For example, the curriculum for a laboratory course on central venous catheterization (CVC) was amended based on the result of a CTA.6 The new curriculum resulted in improvement in the technical skills of new surgical interns. In a similar study, Campbell et al. examined the effects of a CTA-informed curriculum on teaching open cricothyrotomy.8 While this article provides little detail on the revised curriculum, it reports that the CTA-informed curriculum resulted in an increase in students’ self-confidence and ability to perform a cricothyrotomy. Along with suggestions for teaching, learning, and curriculum, another study offered implications for patient care. In emergency medicine, a study examined paramedics’ handling

288

of cognitive challenges related to sense-making and task management in two challenging scenarios.15 The authors found that the expert paramedics used their basic level emergency medical technician partners more than expected, which could lead to provision of more advanced level care for patients in both scenarios. In all these studies in dentistry and other health sciences, researchers have focused especially on experts’ decision making, clinical skills, and cognitive processes, mostly to the benefit of improved student learning. While we do not disagree that experts generally use sounder judgment in differential diagnoses, show more adept communication skills, and have tacit, embodied knowledge of how to perform surgical tasks, we also argue for the role of intermediates in exploring the steps involved in a task and its critical incidents. While intermediates are likely to make more mistakes than experts, these mistakes can shed light on which areas of a procedure are more challenging to learn. In our study, the experts were clinical educators who had practiced more than five years; the intermediates were students who had completed formal learning of a specific clinical task; and the novices were students who had not begun formal clinical training in that specific task.

Methods This research received approval from The University of British Columbia ethics board (approval number H11-01253). The study took place in several stages. First was the selection of the task and the experts. We spoke to all educators in the Faculty of Dentistry at our university who taught in the areas of operative dentistry and endodontics to determine the best clinical tasks to analyze. We sought to identify tasks that contained core cognitive and psychomotor elements necessary to the particular procedure that would also be transferable to a number of other clinical dentistry procedures. After many conversations, we decided to focus on a Class II wax carving restoration of tooth 25 (an upper left second premolar in the international numbering system). This procedure encompasses a number of steps within a task requiring various psychomotor and cognitive capabilities. The instructor responsible for teaching this task was asked to be Dental Expert 1 (DE1), and the director of clinical education was asked to serve as Dental Expert 2 (DE2).


Next, DE1 performed the task three times in the clinic, from preparation of the wax to the final output, using a mannequin head set up on a dental chair as a simulated patient. The entire procedure took him an average of five minutes to perform. Throughout this procedure, the expert talked through what he was doing and sometimes why (e.g., instrument choice, particular part of the tooth, description of the actual carving/motion of instrument, and rationale behind decisions). This is a classic CTA approach.2 The task performance was filmed both intraorally (using a Futudent camera mounted on the bridge of the clinician’s loupes; Novocam Medical Innovations Oy Ltd., Helsinki, Finland) and from a distance with a regular video camera focusing on the clinician. The procedure was filmed three times to ensure consistency. The footage of one performance from both videos was merged and edited (see Figure 2 for a screenshot of the merged video). The audio from the video was transcribed by a professional transcriber. The merged video and transcribed procedure were sent to DE1 for his review. The third part of the study involved an interview between the two experts. The merged/edited video was sent to the second dental expert (DE2) along with the transcript. DE2 was asked to watch

the video along with the transcription, to make a task timeline of the key parts of the procedure, and to note where he witnessed pauses, slowing down, and hesitations (potential critical incidents). DE2 sent this timeline to DE1 for review. Following this, an interview was set up between DE1 and DE2. In this video recorded interview, DE2 asked DE1 his overall thoughts on how someone would best learn the procedure, the places he felt students struggled, the skills associated with competent performance of the task, etc. In the second part of the interview, the merged video of the Class II wax carving was shown. DE2 stopped at all the places in the video that corresponded with the critical incidents he had earlier identified. In each place, DE2 asked DE1 to reflect on why he thought he paused, hesitated, or slowed down. Not all previously labelled critical incidents were deemed as such by DE1. Some pauses seemed to correspond with getting used to speaking, thinking about how to communicate on camera, or reflecting on how to break down the task so it would be most meaningful for student learning. In fact, only around 30% of the pauses, hesitations, and moments of slowing down appeared to relate to increases in cognitive load in a particular part of the task. The timeline was thus amended to highlight the true critical incidents.

Figure 2. Screenshot of cognitive task analysis video Note: The larger intraoral view was filmed by a camera mounted on the bridge of the clinician’s eyeglasses. The smaller frontal clinician operating view was filmed using a video camera. These two films were merged to create a cognitive task analysis video.


289

Students included in the study were secondyear dental students who had recently completed learning of this wax-carving task, since we intended to identify the location of students’ and experts’ critical incidents and noting where there was any overlap. The students’ instructor placed them into three groups—below average, average, and above average—in terms of their psychomotor/cognitive skills associated with learning the task. The criteria for placing students into their respective grouping were as follows. Students who passed all three psychomotor quizzes in the wax carving module without retake were in the above average group. Students who had to retake the quizzes once were in the average category, and those who required intervention from the instructors and retook the quizzes more than once fell in the below average category. All students had learned the task a few months prior to our study, and all had practiced the task approximately eight or nine times. Every student in each of the three groups was contacted and invited to participate. The group to which they belonged was not made known to the researchers to avoid bias. We asked for ability groups only to ensure that we would have full representation of the whole class; we had no intention to compare participants across ability levels. None of the students in the studied cohort were aware of our selecting method and thus were unaware of their ability group. Multiple students from each ability group expressed interest in participating. Three students were chosen at random to represent the three groups and to perform the procedure at separate occasions in the same manner of DE1. Each student was asked to perform the Class II wax carving procedure of tooth 25, filmed by a regular video camera and an intraoral camera. As with DE1, they were instructed to talk through what they were doing and why. Each student received up to 15 minutes to perform the procedure, a limit considered reasonable for a novice. The small and large video footage of each student was merged, edited, and sent to each of the students to watch in preparation for the interview. Critical incidents were identified by one of the researchers (JW) during editing of the videos and were noted in preparation for the interviews. Separate video-recorded interviews were arranged with each of the three students. Each interview took between 60 and 90 minutes. At each of the interviews, the students were asked to reflect on a) how it was to learn the procedure; b) which parts they struggled with and why in learning to perform a restoration with wax on a simulated patient; c)

290

their experience in talking through their actions and thoughts in the CTA study; and d) the teaching strategies used during their learning of basic wax carving restorations that worked or did not work for them. We then proceeded to view the 15-minute video of the student’s performance together, pausing during potential critical incidents (places they slowed down, hesitated, or paused in their speech). At each perceived critical incident, the student was asked to reflect on what was going through his or her mind, the perceived cognitive load (on a scale from 0=I can happily multitask to 5=absolute cognitive load). We asked each student for an example of what absolute cognitive load looked like to him or her and then provided the example of being so focused on a task that a person cannot think about or do anything else but the task (e.g., rock climbing on very dangerous terrain). This was to uncover the relative differences in perceptions of cognitive load in the participants, including why they think they slowed down, paused, or hesitated. We recognize that a 5 for one might be a 4 for someone else, but what was important were the changes the students noted between each step of the task in terms of their perceived cognitive load. At the end of the interview, the students were asked to further reflect on the teaching of the procedure and on potential teaching materials that would be valuable in helping them to learn. The interviews were recorded and analyzed to identify the true critical incidents (i.e., to exclude pauses that were not content-related), to document the students’ described experiences in learning the procedure, and to gather the students’ suggestions to improve teaching. Interviews were comparatively analyzed across all three participants to identify shared critical incidents and shared experiences. Individual thoughts on learning and teaching were retained.

Results The results of this study showed little overlap between the critical incidents of the expert (a clinical educator) and the intermediates (the three dental students), mainly because there were so few critical incidents uncovered in the performance of the dental expert (DE1). However, all four agreed that one part of the task had the highest cognitive load: carving of the marginal ridge of the tooth. This part of the task resulted in slowing down and pauses for the expert and the three students. There was a great deal of overlap in the three intermediates’ critical incidents,


which were not shared by the expert. These parts of the process involved carving the pits, moving from the lingual/palatal to the buccal side of the mouth, and angulating the reflective vision/mirror on the buccal side of the mouth. Issues with angulation and mirror positioning were identified by the dental expert in our interview as a possible place of challenge for students, but did not comprise any of his critical incidents. What initially looked like critical incidents (places of slowing down, pausing, or hesitation) for the intermediates and expert sometimes turned out to be related to other factors; this was especially true for the expert. For example, it appears that 70% of the initially identified critical incidents of the expert were due to what he defined as “trying to make a teaching video,” including using language students would find most useful, making sure the camera captured the appropriate field of operation, getting used to wearing the intraoral camera, or thinking about what to say. For the students, in about 25% of the initially identified critical incidents, they slowed down in their speaking as they were grasping for the technical name of an instrument or part of the tooth, which was unrelated to what they were actually doing in their carving at the time. Searching for words places a larger cognitive load on the non-expert, as students are still learning the nomenclature; however, this load cannot be identified with one point in the procedure being more challenging than another. The intermediates also concurred in their suggestions of what worked in learning the procedure. Completely independently and unprompted, all three dental students mentioned the crucially important role a peer-mentor had played in their learning of this procedure. This peer-mentor was an upper-level (third- or fourth-year) student who was assigned to the clinic floor. What these students seemed to appreciate most was the mentor’s modelling of the procedure and provision of immediate feedback. The participants mentioned in their interviews that they were able to film one of the peer-mentors performing the procedure, which included his explaining what he was doing and why (almost like a CTA). In the students’ interviews, they informed us that the Mp4 video file of the procedure was in circulation and was of great help to them and their peers. The students remarked that their peers who were just above them could tell stories of what they had struggled with in learning to carve wax and perform a Class II restoration and how they overcame those challenges. In contrast, the expert revealed that, while he identified


the marginal ridge as difficult to learn, he did not identify the other areas noted by the intermediates and revealed by the video to constitute critical incidents and could not really remember what worked for him in learning the procedure.

Discussion Given the predominance of the master-apprentice tradition in teaching clinical dentistry, the role of a non-competent dental student as clinical teacher may seem like a weak alternative, at best, compared to a master dentist apprenticing a novice. However, our research suggests that dental students who have recently learned a clinical task could greatly help novices learn that task. Peer teaching is not new and should not be confused with Lasry et al.’s peer instruction model, which features groups of novices learning together through making their thinking visible and collectively working through possible solutions.16 Peer teaching in our study refers to a method in which a student who has mastered a concept/skill teaches another who is a novice in that concept/ skill. Our CTA study suggests a potential role for intermediate-novice peer teaching, in which a slightly more competent peer (an intermediate) can scaffold the learning of a novice. This concept relates to previous research that found dental students preferred an apprenticeship model to assist them in mastering clinical dentistry.17 In our interview data, the student participants reported that the second-year students felt enriched in their clinical dentistry learning from the peer teaching of third- and fourth-year students, their previous lectures relevant to the learning tasks, the modeling and feedback from their clinical instructor, and their own practice, trials, and errors. In reviewing our findings, consulting the literature, and talking to other dental faculty members, we have become more and more aware of the “expert blind spots”18 in teaching novices. That is, experts often do not make the same mistakes as novices, will not experience the same cognitive load, and may indeed be blind to the areas novices struggle with in learning a task, having learned it often decades earlier. This may potentially impact both teaching—how and what is taught—and student learning. In an empirical study of math education, for example, Nathan and Koedinger found that high school teachers can overestimate the accessibility of symbol-based representations and procedures for students learning introductory algebra.19 Nathan

291

and Petrosino claimed that “think aloud reports from experts and novices show that experts are less likely to have access to memory traces of their cognitive processes when engaged in tasks within their domain of expertise… it can make them blind to the processes of novices who are struggling to understand new ideas during their constructive learning process.”18 While it appears to be the case that experts can tap into their processes more in a CTA compared to free recall,4 there may still be gaps. Nathan et al. argued that, “among novices, [cognitive] processes are deliberate and stepwise, and so they leave a memory trace which is more likely to be inspectable and verbalizable.”20 Cognitive task analysis studies of experts only would be limited in their ability to identify critical incidents or those places in a task that might constitute the largest cognitive load for novices or near-novices. As shown in our research, a dental expert no longer experiences the same critical incidents as intermediates. For example, mirror angulation is a challenge for most new dental students and is indicated by slowing down and pausing in speech in executing a clinical dentistry task. In our expert’s performance, the cognitive load no longer appears high, and a student viewing a CTA of an expert may consequently fail to understand that this is a critical incident in learning. Of course, it is possible that a different expert may reveal a different set of critical incidents in the task we studied. Therefore, a much larger number of experts and intermediates is needed to test the generalizability of this observation. The expert blind-spot, it appears, is in large part an issue of memory. As Redelmeier and Kahneman21 and Miron-Shatz et al.22 have noted, we base our memory of pain on two things: its intensity (not its duration) and the outcome. We may forget how painful something was as long as the worst moments were not absolutely terrible. Furthermore, we forget about this pain because of a favorable outcome. Experienced dentists all learned to perform expected clinical tasks, and the “hard parts” that took more effort are pretty much forgotten with time. It is important to note that experienced teachers do not solely rely on their memory in teaching and are attuned to the difficulties students may have and draw attention to them. However, in performing a procedure, even good clinical educators may not always be aware of all the areas in which a novice may struggle. Furthermore, videos of experts performing dental procedures can be misleading as they do not alert novices to the main parts of a task that cause the highest cognitive load in a learner.

292

Another challenge we find in master practitioners who are teaching novices is that the experts may expect the students to be as they were. In exploring the expert blind spot in English teachers, Nathan et al. note, “as successful students themselves, [these teachers] expected their students to be as knowledgeable and as interested in literature as they remembered themselves being.”20 Not every dental student is the same: some are knowledgeable, some are faster, some slower, some more motivated by external rewards, some perfectionists, others not. While great dental educators have—and do—work to overcome these blind spots, others may need assistance and targeted faculty development to support them in this endeavor. The peer teaching research also asserts the invaluable experiences intermediates will gain from teaching novices. Similar to dental education, medical education has a long tradition of expert-apprentice arrangements to train future doctors. In a large-scale literature review of peer teaching in the medical education field, Cate and Durning23 argued that peer teaching is widely discussed but rarely developed in medical curricula regardless of its associated strong evidence in supporting learning. This process resembles the medieval German concept of the journeyman (Geselle). After the Geselle completed an apprenticeship, he was expected to travel around to practice in order to develop expertise in what he was trained, and he was also permitted to teach based on the idea that through teaching he can begin to perfect his learned skills and knowledge. An intentionally structured “cognitive journeyman” in clinical dentistry (i.e., pairing upper-year students with loweryear students) may thus help both the intermediates and the novices. Our purpose here is not to deny the importance of an expert-novice paradigm. With our findings from CTA, however, we want to expand the discussion to include an intermediate-novice pair mentorship model. There were certain limitations to this study. First, we chose to analyze a preclinical task instead of a clinical one mainly because it is not as easy to repeatedly perform a clinical task three times with equivalent conditions (e.g., the right kind of clinical condition of patient). It is likely that applying CTA to a clinical environment may help uncover even more interesting factors that can affect performance of the task. Second, we studied only one dental educator and three intermediates, which can make the outcomes of this study limited in their generalizability. Repeated study of more dental experts and intermediates may


help ascertain the power of CTA as a tool to assist preclinical and clinical dental education.

casions; and the three DMD students who inspired us to explore and examine the issue of expert blind spots in clinical dentistry.

Conclusion

REFERENCES

Based on our reading of previous research and on the results obtained from this study, we believe that CTA is a method that allows us to make the hidden visible in breaking down a task. The method has proven useful in many health professions for the development of more accurate, comprehensive, and tailored teaching materials for students. However, most studies have focused either solely on experts or have failed to further explore the contribution of knowing how intermediates perform a procedure and where their critical incidents lie. In our pilot study, we found CTA useful in systematically identifying cognitively challenging incidents in performing a Class II wax carving in a simulated environment. Our inclusion of intermediates, to compare their critical incidents with those of the expert, led us to the research on peer teaching and expert blind spots. As a result, we think the inclusion of intermediate-novice pairs or alternative forms of utilizing intermediates in teaching preclinical dentistry may be helpful in dental education. We also hope that, through our use of CTA in a task of restorative clinical dentistry, researchers will explore CTA in other dental areas and examine both experts and intermediates to further understand where experts’ blind spots occur, where intermediates and experts overlap in their critical incidents, and, most importantly, how teaching approaches, curriculum, and assessment design can be reformed to better help dental students identify their struggles in learning particular psychomotor tasks and work through them as they move from novice to mastery.

Acknowledgments

This project was funded by the UBC Teaching and Learning Enhancement Fund, the Canada Summer Jobs Program, and an award from the UBC Faculty of Dentistry Research Committee. The idea of using CTA to re-examine clinical dentistry came from a conversation with Dr. Charles F. Shuler in September 2010, and we would like to acknowledge the various conversations we had with him. We would also like to specially thank two clinical educators, Dr. Mark Fogelman and Dr. James Richardson, for their tremendous assistance in participating in this CTA project; Dr. Ingrid Emanuels for her helpful explanation of the studied task to us on various oc-


1. Hoffman RR, Militello LG. Perspectives on cognitive task analysis: historical origins and modern communities of practice. New York: Psychology Pree-Taylor & Francis Group, 2009. 2. Schraagen JM, Chipman SF, Shalin VL, eds. Cognitive task analysis. Mahwah, NJ: Lawrence Erlbaum Associates, 2000. 3. Crandall B, Klein G, Hoffman R. Working minds: a practitioner’s guide to cognitive task analysis. Cambridge: MIT Press, 2006. 4. Crespo K, Torres J, Recio M. Reasoning process characteristics in the diagnostic skills of beginner, competent, and expert dentists. J Dent Educ 2004;68(12):1235-44. 5. Clark RE, Pugh CM, Yates KA, et al. The use of cognitive task analysis to improve instructional descriptions of procedures. J Surg Res 2012;173(1):e37-42. 6. Velmahos GC, Toutouzas KG, Sillin LF, et al. Cognitive task analysis for teaching technical skills in an inanimate surgical skills laboratory. Am J Surg 2004;187(1):114-9. 7. Sullivan ME, Ortega A, Wasserberg N, et al. Assessing the teaching of procedural skills: can cognitive task analysis add to our traditional teaching methods? Am J Surg 2008;195(1):20-3. 8. Campbell J, Tirapelle L, Yates K, et al. The effectiveness of a cognitive task analysis informed curriculum to increase self-efficacy and improve performance for an open cricothyrotomy. J Surg Educ 2011;68(5):403-7. 9. Clark RE, Pugh CM, Yates KA, et al. The use of cognitive task analysis to improve instructional descriptions of procedures. J Surg Res 2012;173(1):e37-42. 10. Read EK, Baillie SJ. Using cognitive task analysis to create a teaching protocol for bovine dystocia. Vet Med Educ 2013;40(4):397-402. 11. Cannon-Bowers J, Bowers C, Stout R, et al. Using cognitive task analysis to develop simulation-based training for medical tasks. Mil Med 2013;178(10):15-21. 12. Cameron CA, Beemsterboer PL, Johnson LA, et al. A cognitive task analysis for dental hygiene. J Dent Educ 2000;64(5):333-51. 13. Thyvalikakath TP, Torres-Urquidy MH, Dziabiak MP, Schleyer TKL. Cognitive task analysis of clinicians’ decision making activities during diagnosis and treatment planning. J Dent Educ 2008;72(2):Abstract 62. 14. Pugh CM, DaRosa DA. Use of cognitive task analysis to guide the development of performance-based assessments for intraoperative decision making. Mil Med 2013;178(10):22-7. 15. Smith MW, Bentley MA, Fernandez AR, et al. Performance of experienced versus less experienced paramedics in managing challenging scenarios: a cognitive task analysis study. Ann Emerg Med 2013;62(4):367-79. 16. Lasry N, Mazur E, Watkins J. Peer instruction: from Harvard to the two-year college. Am J Phys 2008;76:1066-9. 17. Horst JA, Clark MD, Lee AH. Observation, assisting, apprenticeship: cycles of visual and kinaesthetic learning in dental education. J Dent Educ 2009;73(8):919-33.

293

18. Nathan MJ, Petrosino A. Expert blind spot among preservice teachers. Am Educ Res J 2003;40(4):905-28. 19. Nathan MJ, Koedinger KR. An investigation of teachers’ beliefs of students’ algebra development. Cognit Instr 2000;18(2):209-37. 20. Nathan MJ, Koediger KR, Alibali MW. Expert blind spot: when content knowledge eclipses pedagogical content knowledge. Proc Third Int Conf Cognit Sci 2001:644-8.

294

21. Redelmeier N, Kahneman D. Patients’ memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures. Pain 1996;66(1):3-8. 22. Miron-Shatz T, Stone A, Kahneman D. Memories of yesterday’s emotions: does the valence of experience affect the memory-experience gap? Emotion 2009;9(6):885-91. 23. Cate OT, Durning S. Peer teaching in medical education: twelve reasons to move from theory to practice. Med Teach 2007;29:591-9.
