Continuing Education Meetings on Adverse ... - Wiley Online Library

Original Research

A 3-Armed Randomized Controlled Trial of Nurses’ Continuing Education Meetings on Adverse Drug Reactions

AMIR SARAYANI, PHARMD, MPH; FAHIMEH NADERI-BEHDANI, PHARMD; NASER HADAVAND, PHARMD; MOHAMMADREZA JAVADI, PHARMD; FARIBORZ FARSAD, PHARMD; MOLOUK HADJIBABAIE, PHARMD; KHEIROLLAH GHOLAMI, PHARMD Introduction: Nurses’ insufficient knowledge of adverse drug reactions is reported as a barrier to spontaneous reporting. Therefore, CE meetings could be utilized to enhance nurses’ competencies. Methods: In a 3-armed randomized controlled trial, 496 nurses, working in a tertiary medical center, were randomly allocated to a didactic lecture, brainstorming workshop, or the control group (delayed education). Similar instructors (2 clinical pharmacists) prepared and delivered the educational content to all 3 groups. Outcomes were declarative/procedural knowledge (primary outcome), participation rate, and satisfaction. Knowledge was evaluated using a validated researcher-made questionnaire in 3 time points: immediately before, immediately after, and 3 months after each session. Participants’ satisfaction was assessed immediately after each meeting via a standard tool. Data were analyzed using appropriate parametric and nonparametric tests. Results: Rate of participation was 37.7% for the lecture group and 47.5% for the workshop group. The workshop participants were significantly more satisfied in comparison with the lecture group (p < .05). Mean knowledge scores were similar at baseline in the 3 study groups (43–47). Immediately after the meeting, knowledge was significantly higher in the lecture group (79.1 ± 11.9 vs 73.7 ± 11.3; p = .01). At the follow-up, knowledge scores of the lecture and workshop groups were similar, while significantly higher than the control group. However, the reduction of knowledge score was significantly higher in the lecture group (–13.0 ± 15.9% vs –5.7 ± 15.1%, p = .02). Discussion: Educational interventions can improve nurses’ knowledge of adverse drug reactions. Short-term learning could be achieved with lecture, but the retention of knowledge will be enhanced by simple interactive techniques. Key Words: patient safety, profession—nurse, evaluation–educational intervention, experimental/quasiexperimental design, adverse drug reaction, pharmacovigilance, continuing education, cardiovascular medication

Disclosures: The authors report none. Dr. Sarayani: Research Associate, Research Center for Rational Use of Drugs, Tehran University of Medical Sciences; Dr. Naderi-Behdani, Pharmacist, Pharmacy student, Azad University of Pharmaceutical Sciences; Dr. Hadavand: Assistant Professor, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences; Dr. Javadi: Associate Professor, Faculty of Pharmacy and Research Center for Rational Use of Drugs, Tehran University of Medical Sciences; Dr. Farsad: Assistant Professor, Rajaie Cardiovascular Medical and Research Center, Iran University of Medical Sciences; Dr. Hadjibabaie: Professor, Faculty of Pharmacy and Research Center for Rational Use of Drugs, Tehran University of Medical Sciences; Dr. Gholami: Professor, Faculty of Pharmacy and Research Center for Rational Use of Drugs, Tehran University of Medical Sciences. Correspondence: Naser Hadavand, Pharmaceutical Care Department, Rajaie Cardiovascular Medical and Research Center, Vali-Asr Ave, Next to Mellat Park, Tehran, Iran, 1996911151; e-mail: [email protected] © 2015 The Alliance for Continuing Education in the Health Professions, the Society for Academic Continuing Medical Education, and the Council

Introduction Medical care is highly dependent on medications; however, adverse drug reactions (ADRs) are inevitable. The prevalence of ADRs among hospitalized patients is estimated to be 16.8%.1,2 Drug safety surveillance is essential to prevent catastrophic events in health systems, and “spontaneous reporting” by health care professionals is the backbone of surveillance systems.3 Nurses, as the medication administrators and the caregivers of hospitalized patients, could play an important role in ADR reporting.4 Several factors might result in underreporting of ADRs, including lack of time, fear of legal consequences, and insufficient knowledge.3,5 In a Swedish study, approximately on Continuing Medical Education, Association for Hospital Medical Education • Published online in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/chp.21276

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS, 35(2):123–130, 2015

Sarayani et al.

50% of the nurses were unaware of their role in ADR reporting. Moreover, only 15% of the nurses had received ADR training, which was associated with a 2-fold higher reporting rate in comparison with their untrained counterparts.6 There are few studies on the efficacy of different educational approaches to improve nurses’ knowledge of ADRs. A recent before-and-after study in Iran showed that a 3-hour training session on the importance of the ADR and pharmacovigilance process could significantly improve knowledge and attitude of practicing nurses in a tertiary hospital.7 Therefore, educational interventions could be utilized to improve nurses’ competencies to report ADRs. Nevertheless, the comparative efficacy of different educational interventions should be investigated. Continuing education (CE) has been shown to be effective in improving knowledge, performance, and patient health outcomes8 ; however, the positive effects of CE decline as the level of outcome assessment moves from knowledge toward patient health.9 Regarding knowledge, a systematic review showed that CE activities could have short-term and long-term positive effects. Use of multiple educational techniques (lecture, role play, case-based learning, discussion group, simulation, etc) is associated with more knowledge improvement.10 Didactic lecture remains the most common instructional method, particularly in resource-limited settings.11 Incorporation of simple interactive techniques (eg, brainstorming) seems to be feasible in most settings. Moreover, the education environment could be modified to provide face-to-face contact among learners. These modifications might lead to improvement in CE outcomes at diverse levels; however, the margin of effect needs to be quantified. In the present study, we compared the participation rate, satisfaction, and the learning outcome of workshop-based (brainstorming) versus lecture-based meetings for nurses working in a referral cardiovascular care hospital in Iran. Methods

vices are provided at the Center, including adult and pediatric cardiology, cardiac surgery, catheterization, ablation, cardiac electrophysiology, rehabilitation, angiography, and cardiac imaging. According to the database of the human resource department, 635 nurses were working at the hospital before study commencement. Study Groups and Educational Design Lecture Intervention. Didactic lecture sessions were held in a conference hall with a capacity of 100 participants. The instructors used PowerPoint presentations to deliver the educational content. They asked participants a few questions during the instruction period. Workshop Intervention. A workshop was designed to incorporate the “brainstorming technique” and was held in a conference room equipped with a round table to enhance face-to-face contact among participants and motivate group discussions.12 Each instructor introduced a topic and provided a group of related questions. Participants were encouraged to answer the questions and the instructor noted their answers. He tried to gather nurses’ views and experiences about the topic. After brainstorming and group discussion, the instructor presented brief PowerPoint slides to provide a summary of the educational content. The training time was approximately 4 hours divided into 2 equal parts with a 20-minute tea break in between. At the end of each part, the instructor answered participants’ questions for 15 minutes. Two faculty members, who were clinical pharmacists, instructed the participants, and the meetings were held using available facilities within the hospital. The first instructor taught the ADRs of cardiovascular drugs, and the second instructor provided training on the ADRs of analgesics and antibiotics. The sequence and the time devoted to each educational topic were the same for all meetings. One of the authors observed each meeting to confirm the accurate delivery of the content. Educational design and objectives are summarized in TABLE 1.

Study Design A 3-armed randomized controlled trial was designed to compare the didactic lecture meeting, brainstorming workshop, and a control group. The study protocol was approved by the Institutional Review Board of the Azad University, College of Pharmaceutical Sciences. The trial was nested within a hospital-wide educational program. Study Setting and Participants The study was conducted at Shaheed Rajaei Cardiovascular Medical and Research Center, which is a 600-bed referral hospital in Tehran, Iran. Several cardiovascular care ser124

Control Group. This group of nurses did not receive training during the trial period. Educational Content In a previous educational program at the Center, nurses were trained on the pharmacovigilance concept and methods to report ADRs. In the present study, the objective of the educational program was to improve nurses’ competencies for detecting ADRs. The references for content development were Meyler’s Side Effects of Drugs (Cardiovascular Drugs, Antimicrobial Drugs, Analgesics and Antiinflammatory Drugs)13–15 ; Adverse Drug Reactions, 2nd

JOURNAL OF CONTINUING EDUCATION IN THE HEALTH PROFESSIONS—35(2), 2015 DOI: 10.1002/chp

Nurses’ Continuing Education Meetings on ADR

The third and fourth meetings were assigned to the lecture intervention, and the fifth and sixth meetings were devoted to the workshop intervention. The control subgroups (N = 3) were merged together, and 2 sessions (seventh and eighth) were held for this group due to administrative issues. The strata were hospital wards (eg, intensive care unit), and this approach was used to achieve a homogenous population in our study groups regarding workload and nurses’ capabilities.

TABLE 1. Educational Design and Objectives

Educational themes: ●

A review of drug categories

●

Adverse drug reactions

●

Drug-drug interactions

●

Contraindications and precautions

Learning objectives: ●

Achieve the competency for detecting adverse drug reactions

●

Recognize the medication responsible for the adverse drug reaction

Study Outcomes

●

Identify the interaction of drugs consumed concomitantly

We assessed study outcomes based on Moore and colleagues’ (2009) framework, which has been proposed for evaluation of continuing medical education (CME).20

Instructional designs: Didactic lecture

Predisposing

Workshop

Full lecture

Brief lecture

(PowerPoint

(PowerPoint

approach

presentation) Application exercise

presentation)

—

Brain storming group

Participation (Level 1) The Nursing Management Office informed the chief nurse of each hospital ward about the educational program. Nurses were asked to attend the program on the designated date and time according to the timetable. The participation rate for each educational meeting and the average of rates for the lecture and workshop were calculated.

discussion Exposure time

∼4 hours

∼4 hours

Satisfaction (Level 2)

Seating arrangement

Traditional classroom

Circular seating

We used a prevalidated satisfaction assessment questionnaire developed by the Office of Continuing Education for Health Personnel, Ministry of Health. This is an anonymous tool that contains 11 Likert items (5 points, ranging from very satisfied to very unsatisfied) to evaluate participants’ satisfaction with educational content, instructional design, and organizational aspects. A separate item assesses the participants’ goal of attending the educational program, and 2 open-ended questions are provided to ask for comments and suggestions.

rows

format

edition16 ; and Lexicomp’s Drug Information Handbook.17 In addition, a list of medications that were highly consumed in the hospital was extracted, and the ADR reports of pharmacovigilance unit were reviewed. The instructors developed the educational content, prepared their PowerPoint slides, and planned for the workshop sessions.

Declarative and Procedural Knowledge (Levels 3A and 3B) Sample Size and Randomization All of the nurses working at the hospital were eligible for participation in the study (N = 635). The sample size was estimated based on a moderate effect size for the learning outcome,18,19 alpha error of .05, and power of 80%, which suggested a minimum of 53 participants in each group. Two authors, not involved in the training, allocated nurses to 9 subgroups using a stratified random algorithm generated by computer software (68–74 nurses in each subgroup). Two subgroups attended the pilot sessions (first and second meetings) to harmonize the educational content before the trial. Each of the remaining subgroups was invited to attend 1 of the meetings (lecture, workshop, or control arm).

Multiple-choice questions were designed to evaluate declarative knowledge and brief case vignettes were used to assess procedural knowledge. The initial tool consisted of 24 multiple-choice questions and 1 case vignette. Through the pilot sessions, we refined the questionnaire and the final tool included 16 multiple-choice questions (16 points) and 2 case vignettes (4 points). Afterwards, we asked senior clinical pharmacy residents and junior pharmacy students to fill out the questionnaire to confirm the discriminative capability of the tool. Participants’ knowledge scores were calculated as the percentage of correct responses. Three of the authors developed a questionnaire based on the educational content, and the instructors were blind to the outcome assessment


125

Sarayani et al.

FIGURE 1. Flow Diagram of Study Participants

questionnaire. The multiple-choice questions and case vignettes were similar at all measurement time points. Data Collection The final tool was a paper questionnaire including demographics section, multiple-choice questions, and case vignettes. One of the authors presented the objectives of the project at the beginning of each session. It was emphasized that the study findings will be reported in form of aggregate data. The researcher distributed the questionnaire and supervised the test period. For the intervention groups, the study tool was administered at 3 time points: immediately before, immediately after, and 3 months after the interventions. The researcher visited nurses in the wards and collected the follow-up data. Participants’ satisfaction was assessed immediately after each meeting. The educational meetings for the control group were delivered in 2 consecutive weeks immediately after conducting the lecture and workshop sessions due to administrative issues. Baseline data for the control group was collected after the study tool was finalized (1 month before the lecture sessions). We had to collect the follow-up data immediately before the educational meeting as the final assessment. This assessment could quantify possible improvements in the knowledge that might be observed based on the repetitive nature of the study tool and possible contamination of study groups. Statistical Analysis The normality of data was investigated by the One-Sample Kolmogorov-Smirnov test. We compared age and years since 126

graduation between study groups by the analysis of variance (ANOVA) test. Monthly work hours were analyzed by the Kruskal-Wallis test, as the data were not normally distributed. Chi-square was used to compare categorical data between study groups (gender, degree, ADR reporting, and main practice ward). We used the Mann-Whitney test to analyze satisfaction data (ordinal data) and the ANOVA test for the knowledge outcome in pre- and follow-up time points among the 3 study groups. The postmeeting knowledge scores were analyzed by the Mann-Whitney test between lecture and workshop groups (normality of data was not confirmed). Comparison of knowledge outcome within study groups was done by paired t-test. Statistical significance level was p < .05. Participants who did not complete the questionnaire at each of the time points were excluded from data analysis.

Results The trial was conducted from January to May 2013; FIGURE 1 illustrates the participants’ flow diagram. Mean age of participants was 34.6 ± 7.0, and the average practice experience was 10.2 ± 6.6 years. Only 3 nurses had previously reported ADRs to pharmacovigilance centers. Baseline characteristics were not significantly different among study groups. (TABLE 2).

Participation Mean rate of participation (2 sessions for each intervention) was 37.7% for the lecture group and 47.5% for the workshop group. Overall, 228 nurses out of 635 eligible participants (35.9%) attended the educational program.


Nurses’ Continuing Education Meetings on ADR TABLE 2. Summary of Study Participants’ Characteristics

Factor

Age (mean ± SD)

Workshop (n = 67)

Lecture (n = 54)

Control (n = 37)

p valuea

33.7 ± 6.9

35.1 ± 7.4

35.4 ± 6.6

.40

Gender (Female)

55 (82.1%)

50 (92.6%)

29 (78.4%)

.13

Degree (BS)

62 (92.5%)

50 (92.6%)

35 (94.6%)

.94

Years since graduation (mean ± SD)

9.6 ± 6.6

10.8 ± 6.1

11.5 ± 6.2

.32

211.1 ± 41.7

205.4 ± 39.9

201.2 ± 66.1

.81

1 (1.6%)

1 (1.9%)

1 (2.7%)

.93

ICU

10 (15.2%)b

13 (24.1%)

7(18.9%)

.60

CCU

18 (27.3%)

18 (33.3%)

10 (27%)

2 (3%)

4 (7.4%)

2 (5.4%)

6 (9.1%)

2 (3.7%)

1 (2.7%)

Monthly work hour (mean ± SD) Previously reported ADR Main practice ward:

Pediatrics Surgery Internal Emergency

2 (3%)

3 (5.6%)

0 (0%)

6 (9.1%)

3 (5.6%)

4 (10.8%)

a

Age and years since graduation (ANOVA test); monthly work hours (Kruskal-Wallis test); chi-square test (gender, degree, ADR reporting, and main practice ward). b Frequency (percentage within group).

Satisfaction

TABLE 3. Summary of Participants’ Satisfaction With the Instructional Pro-

The workshop participants were significantly more satisfied with the educational meeting (p < .05) (TABLE 3). About half of the participants (50.9%) mentioned updating their knowledge as the most important goal of attending the program. Other participants listed sharing their experiences with colleagues and finding solutions to their practice challenges as the most important goal (25.0% and 24.1%, respectively). Content analysis of the open-ended questions revealed a number of positive and negative themes. “Knowledgeable lecturers” and “presenting new knowledge and practical information” were the positive points. Moreover, the nurses stated that the program helped them with correct drug administration methods and increased their clinical information. Negative points about the program were “extensive educational content” and “short time duration devoted to each topic.” They suggested a series of classes instead of a 1-day meeting on the ADR topic in order to be more effective.

gram

Factor

Workshop

Lecture

Complying with previous knowledge

4 (59.3)a

4 (46.5)

.02

Presenting up to date knowledge

4 (63.4)

4 (42.9)

< .01

Satisfying professional requirements

4 (59.4)

4 (46.3)

.01

Motivating for further study

4 (58.6)

4 (46.4)

.02

Appropriateness of instructional

4 (61.8)

3 (42.5)

< .01

4 (58.6)

3 (44.5)

.01

design Appropriateness of training time Participation in group discussion

4 (63.8)

3 (41.0)

< .01

Satisfaction with program

4 (66.5)

4 (39.5)

< .01

coordination Satisfaction with instructional aids

4 (58.2)

3 (43.6)

< .01

Satisfaction with the time of program

4 (58.3)

3 (46.9)

.05

4 (59.3)

3 (44.9)

.01

Declarative and Procedural Knowledge At baseline, the scores were similar for all 3 groups. Between-group comparisons showed that the lecture group scored significantly higher immediately after the meeting in comparison with the workshop group (p = .01). At followup, lecture and workshop groups had similar scores that were significantly higher than the control group (FIGURE 2).

p value

delivery Satisfaction with place of program delivery a

Mann-Whitney test was used to compare satisfaction between groups. Median (mean rank) is reported. There was a 5-point Likert scale for each item: very unsatisfied (1) to very satisfied (5).


127

Sarayani et al.

FIGURE 2. Knowledge Scores During the Study Period (Reported as Percentage of Correct Responses) TABLE 4. Change of Knowledge Scores Within Study Groups

Study group

Lecture Workshop Control

Post/Pre

Follow-up/Pre

Follow-up/Post

33.00 ± 17.4**

20.1 ± 15.1**

–13 ± 15.9**

30.7 ± 16.9**

24.4 ± 17.8**

–5.7 ± 15.15*

NA

**

8.6 ± 13.3

NA

Note The values are the difference between 2 time points of knowledge assessment. Paired t-test was used to compare means between 2 time points. ** p < .001; * p < .01. NA = not applicable.

Within-group comparisons showed that the scores of the lecture and workshop groups improved significantly during the meetings (p < .001). The follow-up assessment revealed significant score reductions in both lecture and workshop groups (p < .001). The score reduction was significantly higher in the lecture group in comparison with the workshop group (–13.0 ± 15.9% vs –5.7 ± 15.1%, p = .02). Overall, knowledge scores improved in all groups during the 3month follow-up period (p < .001); however, the difference in the control group was smaller than the lecture and workshop groups (8.6 ± 13.3 vs 20.1 ± 15.1 and 24.4 ± 17.8, respectively). TABLE 4 summarizes the within-group comparisons. Discussion Few studies have evaluated the effects of educational interventions on improving nurses’ pharmacovigilance 128

competencies.7,21 In the present study, the workshop intervention was designed to ensure participants’ face-to-face contact, and a simple technique (brainstorming) was employed to facilitate interactive training.22,23 Both lecture and workshop interventions were effective in improving declarative/procedural knowledge. Although the lecture showed higher efficacy in improving knowledge immediately after the meeting, the retention of learning was significantly greater in the workshop group. The participants were more satisfied with the workshop meeting. Participation is the first level of outcomes in continuing medical education.20 Incentives or sanctions are used to promote participation of health professionals in continuing education programs.24 In the present study, the participation rate of the workshop was relatively higher than the lecture (47.5% vs 37.7%). The overall participation rate was relatively low in spite of possible monetary incentives administered by the nursing management office. The method of training might not have influenced the participation rate, as the nurses were not informed about the method of education before attending the session. However, 2 explanations could be proposed: Nurses might have considered the ADR topic to be irrelevant to their routine practice. In addition, the meetings were conducted weekly on Wednesday (8:30 am–12:30 pm) and heavy workload or working the previous night shift might have been a barrier to attend the meetings. E-learning options including self-learning digital media and Web-based modules may also provide modalities to improve participation rate.25–27 Future comparative studies should evaluate the outcomes of ADR e-learning modules for nurses. In our previous trials on continuing pharmacy education, we observed higher satisfaction with the interactive or mixed-method instructional designs using the same


Nurses’ Continuing Education Meetings on ADR

satisfaction assessment tool.18,19 This finding was confirmed in the present study; nevertheless, we expected to see similar satisfaction with 3 items including “complying with previous knowledge,” “presenting up-to-date knowledge,” and “satisfying professional requirements” because the content of education was similar for both lecture and workshop groups. The difference in the educational environment (a modernized conference room) and the instructional method might have enhanced learners’ understanding of the content and its relevance to their practice. There are inconsistencies among continuing education studies regarding learning and practice outcomes.28–30 The characteristics of educational interventions should be evaluated carefully when the results of the studies are being compared to each other. Davis et al have defined didactic sessions as “predominantly lectures or presentations with minimal audience interaction or discussion”;31(p868) interactive sessions as “those sessions using techniques to enhance physician participation, such as role-play, discussion groups, hands-on training, problem solving or case solving”;31(p868) and mixed interventions if they “specified the use of both interactive and didactic methods.”31(p868) For instance, lecture-based education is reported to be equally effective in improving knowledge, while performance was more enhanced in the problembased learning group. Follow-up knowledge scores of both groups were similar.30 However, the lecture-based program could be categorized as a mixed-method intervention according to Davis et al categorization.31 Regarding the learning outcome in the present study (declarative/procedural knowledge), the findings are comparable to our previous trials. Didactic lecture can improve short-term learning (immediately after the meeting), but the retention of knowledge could not be assured.18 Interactive education could provide a more sustainable learning; however, frequent or long-term exposure with interactive education techniques might be required to achieve the optimal level of knowledge.10 Another explanation could be related to the nature of the interactive technique that we used in our workshop. The brainstorming method might not be powerful enough to improve knowledge. Nevertheless, in our previous trial with short-term exposure, we used a “simulated patient” technique, and the findings were similar to the present results.18 Mixed-method training can be the most effective approach for 1-day continuing education meetings.19 However, further studies are required to investigate the effect of these educational interventions on nurse’s ADR reporting behavior. The improvement of learning in the control group requires elaboration. Our study participants were working in 1 hospital, and knowledge could be transferred to the control group from the lecture and the workshop participants during the study period. Although no concurrent ADR training program

was conducted at the hospital, nurses could be exposed to other sources of training during the study period. Strength and Limitations We tried to maximize the internal validity of the trial by taking into account the following considerations: 1. Random allocation was done by the authors, who were not involved in conducting the interventions. 2. The instructors were blind to the content of the questionnaire. 3. The time duration of lecture and workshop interventions was similar. 4. The educational content was tested and harmonized through pilot sessions.

As mentioned earlier, the risk of contamination could not be neglected in our study, particularly for the control groups. Another caveat could be the time and location of the data collection. The baseline data of the control group was collected when the nurses were available in the hospital wards and the follow-up test was taken in the conference hall. On the contrary, follow-up data of the lecture and the workshop groups were collected in the wards. Thus, the knowledge score might be affected by different test-taking environments. However, we did not observe significant differences among baseline scores of the 3 groups; therefore, the effect could be negligible. Conclusion Educational interventions can improve nurses’ knowledge of ADRs. Short-term learning of a lecture-based meeting could be higher than interactive training, but the retention of knowledge will be enhanced by implementing simple interactive education techniques. Future trials should evaluate the outcomes of mixed-method interventions and frequent training sessions. Moreover, studies are required to investigate whether improved learning outcomes would be translated into desired ADR reporting behavior. Supporting Information Additional Supporting Information may be found in the online version of this article at the publisher’s web site: Appendix S1: Sample of Educational Content Appendix S2: Sample of Knowledge Assessment Questions As a service to our authors and readers, this journal provides supporting information supplied by the authors. Such materials are peer reviewed and may be reorganized for online delivery, but are not copy edited or typeset. Technical support issues arising from supporting information (other than missing files) should be addressed to the authors.


129

Sarayani et al.

Lessons for Practice ●

●

●

Nurses’ continuing education meetings could be utilized to improve their knowledge of adverse drug reactions. Modification of education environment and incorporation of the brainstorming technique can enhance learning retention and participants’ satisfaction. Monetary incentives may not be sufficient to assure nurses participation in continuing education meetings.

Acknowledgment The authors would like to thank Dr. Hamed Hosseini for his valuable contribution to study design and Ms. Asieh Ashouri for her assistance in data analysis.

References 1. Miguel A, Azevedo LF, Araujo M, Pereira AC. Frequency of adverse drug reactions in hospitalized patients: a systematic review and meta-analysis. Pharmacoepidemiol. Drug Saf. 2012;21(11): 1139–1154. 2. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998;279(15):1200–1205. 3. Herdeiro MT, Polonia J, Gestal-Otero JJ, Figueiras A. Factors that influence spontaneous reporting of adverse drug reactions: a model centralized in the medical professional. J. Eval. Clin. Pract. 2004;10(4):483– 489. 4. Bäckström M, Mjörndal T, Dahlqvist R. Spontaneous reporting of adverse drug reactions by nurses. Pharmacoepidemiol. Drug Saf. 2002;11(8):647–650. 5. Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Determinants of underreporting of adverse drug reactions. Drug Saf. 2009;32(1):19–31. 6. Ekman E, Petersson G, Tagerud S, Backstrom M. Awareness among nurses about reporting of adverse drug reactions in Sweden. Drug Healthc Patient Saf. 2012;4:61–66. 7. Hanafi SH, Torkamandi H, Hayatshahi AH, Gholami KG, Ashrafi Shahmirzadi N, Javadi MRJ. An educational intervention to improve nurses’ knowledge, attitude, and practice toward reporting of adverse drug reactions. Iran J Nurs Midwifery Res. 2014;19(1):101–106. 8. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Contin Educ Health Prof. 2007;27(1):6–15. 9. Cervero RM, Gaines JK. Effectiveness of Continuing Medical Education: Updated Synthesis of Systematic Reviews. Chicago, IL: Accreditation Council for Continuing Medical Education; 2014. 10. Bordage G, Carlin B, Mazmanian PE. Continuing medical education effect on physician knowledge: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009;135(3 Suppl):29s–36s.

130

11. Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009(2):Cd003030. 12. Hutchinson L. Educational environment. BMJ. 2003;326(7393):810– 812. 13. Aronson JK. Meyler’s Side Effects of Antimicrobial Drugs. Amsterdam, The Netherlands: Elsevier Science; 2009. 14. Aronson JK. Meyler’s Side Effects of Analgesics and Anti-inflammatory Drugs. Amsterdam, The Netherlands: Elsevier Science; 2009. 15. Aronson JK. Meyler’s Side Effects of Cardiovascular Drugs. Amsterdam, The Netherlands: Elsevier Science; 2009. 16. Lee A. Adverse Drug Reactions. London, UK: Pharmaceutical Press; 2006. 17. Lexicomp. Drug Information Handbook: A Comprehensive Resource for All Clinicians and Healthcare Professionals. Hudson, OJ: LexiComp, Inc.; 2013. 18. Javadi M, Kargar A, Gholami K, et al. Didactic lecture versus interactive workshop for continuing pharmacy education on reproductive health: a randomized controlled trial. Eval. Health Prof. DOI: 10.1177/0163278713513949. 19. Sarayani A, Rashidian A, Gholami K, Torkamandi H, Javadi M. Efficacy of continuing education in improving pharmacists’ competencies for providing weight management service: three-arm randomized controlled trial. J Contin Educ Health Prof. 2012;32(3):163–173. 20. Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29(1):1–15. 21. Bäckström M, Ekman E, Mjörndal T. Adverse drug reaction reporting by nurses in Sweden. Eur J Clin Pharmacol. 2007;63(6): 613–618. 22. Newble DI, Cannon RA. A Handbook for Medical Teachers. New York, NY: Springer; 2001. 23. Jeffries WB, Huggett KN. An Introduction to Medical Teaching. New York, NY: Springer; 2010. 24. Eustace LW. Mandatory continuing education: past, present, and future trends and issues. J Contin Educ Nurs. 2001;32(3):133–137. 25. Khatony A, Nayery ND, Ahmadi F, Haghani H, Vehvilainen-Julkunen K. The effectiveness of Web-based and face-to-face continuing education methods on nurses’ knowledge about AIDS: a comparative study. BMC Med Educ. 2009;9:41. 26. Wutoh R, Boren SA, Balas EA. eLearning: A review of Internetbased continuing medical education. J Contin Educ Health Prof. 2004;24(1):20–30. 27. Lam-Antoniades M, Ratnapalan S, Tait G. Electronic continuing education in the health professions: An update on evidence from RCTs. J Contin Educ Health Prof. 2009;29(1):44–51. 28. Leemans L, Laekman G. Teaching and counselling behaviour: an applied study with community pharmacists. Pharm World Sci. 1998;20(6):238–247. 29. Watson MC, Bond CM, Grimshaw JM, Mollison J, Ludbrook A, Walker AE. Educational strategies to promote evidence-based community pharmacy practice: a cluster randomized controlled trial (RCT). Fam. Pract. 2002;19(5):529–536. 30. Smits PB, de Buisonje CD, Verbeek JH, van Dijk FJ, Metz JC, ten Cate OJ. Problem-based learning versus lecture-based learning in postgraduate medical education. Scand J Work Environ Health. 2003;29(4):280– 287. 31. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, TaylorVaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282(9):867–874.
