I
N
N
O
V
A
T
I
O
N
S
A
N
D
I
D
E
A
S
TELEMEDICAL EDUCATION: TEACHING SPIROMETRY ON THE INTERNET Esther H. Lum and Thomas J. Gross Division of Pulmonary, Critical Care, and Occupational Medicine, Department of Internal Medicine, University of Iowa Hospitals and Clinics, Iowa City, Iowa 52242
dvances in portable equipment have led to routine spirometry testing outside of formal pulmonary function laboratories. Practitioners ordering these tests are not formally trained in spirometry interpretation. Providing effective off-site training can be challenging. Our objective was to develop a remotely accessible computer-based tutorial for teaching spirometry interpretation to nonpulmonologists. We designed an educational module that was accessible via the Internet and tested by 65 medical trainees at a major university medical center. In addition, the module was posted within the Virtual Hospital on the World Wide Web. Increases in spirometry interpretative skills were assessed using pre- and post-tests submitted electronically. The spirometry module significantly improved spirometry interpretation by nonspecialist trainees. This improvement included a broad increase in knowledge base and was observed independent of training level and prior spirometry reading experience. We conclude that computerbased tutorials can effectively train off-site practitioners in spirometry interpretation. This technology allows for the dissemination of educational material from a central site of expertise and provides a valuable adjunct to limited teaching resources.
A
AM. J. PHYSIOL. 276 (ADV. PHYSIOL. EDUC. 21): S55–S61, 1999.
Key words: pulmonary physiology; pulmonary function tests; Virtual Hospital; medical education
The practitioners ordering these tests, however, may not have had formal training in the performance or interpretation of spirometry. Whereas automated interpretation programs are available, these rigid algorithms cannot, at present, fully replace an experienced reader (3). Educating primary caregivers in spirometry interpretation could lead to more appropriate test acquisition, data utilization, and specialty referral (4, 15, 16). Providing convenient, effective educational opportunities for these off-site practitioners can be a daunting task. Increased personal and institutional access to the information superhighway, the Internet, offers new opportunities for the remote delivery of computer-aided instruction (7).
Pulmonary function tests (PFT) are among the first diagnostic studies employed in the evaluation of suspected lung disease. Indeed, the measurement of expired airflow by simple spirometry is invaluable in many clinical settings including the diagnosis and management of bronchial asthma, evaluation of chronic cough, and quantification of disability impairment. Recent innovations in PFT equipment have made portable spirometry devices reliable and accurate (19). This has allowed for the increasing use of simple spirometry in the primary care setting. Thus spirograms can now be performed routinely outside of traditional PFT laboratories in outpatient office practices, industrial clinics, and community hospital emergency rooms (13, 19).
1043 - 4046 / 99 – $5.00 – COPYRIGHT r 1999 THE AMERICAN PHYSIOLOGICAL SOCIETY VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S55
I
N
N
O
V
A
T
I
O
N
S
The University of Iowa has developed and instituted an electronic medical education forum, The Virtual Hospital (11). This multimedia, integrated teaching package is accessible through the Internet (http:// indy.radiology.uiowa.edu/) and is also available to University of Iowa-affiliated clinical outreach centers via a statewide telemedicine network (17). We have developed a computer-based educational module to test whether such a teaching network could be used to effectively improve spirometry interpretation skills for off-site nonpulmonologists.
A
N
D
I
D
E
A
S
grams. The spirograms and clinical identifiers were used as stems for R-type multiple matching questions using the same list of interpretations as possible answers for each question (5). A lead-in instructed the students to choose the single best interpretation and stated that each answer could be used once, more than once, or not at all. The spirograms and the designated correct answers were reviewed by clinical experts in PFT interpretation (see above). These questions were grouped as a ‘‘pre-test’’ that was taken before reading the educational module. The same series of spirograms was relabeled with different clinical vignettes, randomly reordered, and presented with the same set of potential answers as a ‘‘post-test’’ immediately on completion of the educational module. Completion of each test triggered submission of the results to a dedicated electronic mailbox. Feedback on performance was provided via electronic mail to those who requested it.
METHODS Educational module. Using published American Thoracic Society (ATS) guidelines, we designed a computer-based tutorial on the interpretation of simple spirometry (1). In addition to stressing recognition of specific disease patterns, standards for assessing spirogram quality were also addressed (2). To ensure that our recommendations were consistent with standard practice, the guidelines were reviewed and approved by the University of Iowa Hospitals and Clinics PFT laboratory director as well as an outside expert reviewer (Dr. William Eschenbacher, Baylor College of Medicine, Houston). Textual content was composed on the authors’ personal computer and then transferred by electronic mail to the Virtual Hospital librarian for conversion into hypertext markup language (HTML) files. Representative spirometry tracings were obtained from clinical spirograms performed at the University of Iowa PFT laboratory. These figures were scanned into Adobe Photoshop v3.0 (Adobe Systems, San Jose, CA) using a HewlettPackard ScanJet IIc and inserted within the text document. To aid reader visualization, a normal volunteer (E. H. Lum) performing spirometry and the flow-volume loop accompanying this effort were video recorded (Sony Handycam, 8 mm). These video images were digitized and inserted with hot text buttons within the body of the spirometry module.
Subjects were recruited from among the medical students, interns, and more senior house staff rotating through the Department of Internal Medicine according to a protocol approved by the University of Iowa Institutional Review Board. Subjects were diverted from scheduled noon conferences to an educational resource center complete with multiple personal computer stations that provided Internet access with bookmarks directing them to the Virtual Hospital. After a brief introduction, participants were free to complete the pre-test, module, and post-test at their own pace with the option of submitting their test scores under pseudonyms. Those who completed the exercise were compensated with a free meal. Remote distribution of spirometry module. A site was created on the Virtual Hospital that contained the pre-test, module, and post-test. This site was posted within the Pulmonary Core Curriculum section without specific advertisement or promotion. Respondents to the module who submitted the pre-test and/or post-test were asked to identify themselves by level of training and/or health care occupation using a drag-down menu. Those who provided electronic mailing addresses were given their test results and explanations of any incorrect answers.
Spirometry interpretation testing. A series of 12 spirograms representing the entire range of material reviewed in the educational module was collected. For each, a brief clinical scenario was composed detailing a fictitious patient’s age and sex and the reason for ordering the patient’s spirogram. A series of nine possible interpretations accompanied the spiro-
Data analysis. Test scores were reported as the number correct out of a possible score of 12. The time
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S56
I
N
N
O
V
A
T
I
O
N
S
A
N
D
I
D
E
A
S
are useful for both those in training as well as primary caregivers in practice. Therefore, the change in preand post-test scores was examined for each trainee subgroup to determine how previous knowledge base influenced the effectiveness of the module. At all levels of training, post-test scores increased significantly (Fig. 2). The greatest gain in scores was observed among the medical students, who also manifested the lowest average pre-test scores as expected on the basis of their limited experience with the clinical use of spirometry (Fig. 2). Subgroups with higher pre-test scores and, presumably, more experience with interpreting spirometry, still showed significant improvement in post-test scores (Fig. 2). Similarly, nearly all individuals improved their post-test scores whether the pre-test score was above or below the median for their respective trainee subgroup (Fig. 3). Thus spirometry interpretive performance improved irrespective of training level or prior experience.
required to read the module and submit post-test results was derived from log-on records and recorded in whole minutes. Pre- and post-test scores were analyzed using paired, two-tailed t-tests (Microsoft Excel 5.0 for the Macintosh) with a P value of ⬍0.05 considered a significant difference between population means. To limit the influence of outlying values, data were also examined using median scores. RESULTS The spirometry module improved spirogram interpretation by trainees. The spirometry module was administered to 65 participants, including 20 thirdyear medical students, 17 interns, and 28 senior house staff. The test score results followed a normal distribution (Fig. 1) with scores improving from 6 ⫾ 0.3 (mean ⫾ SE) on the pre-test to 8 ⫾ 0.3 correct answers out of 12 on the post-test (P ⬍ 0.001). Furthermore, the median test score increased from 5 to 8 correct answers out of a possible 12, demonstrating that the increase in post-test mean score reflected overall group improvement and not simply individual high scores.
The spirometry interpretative skills improved across a wide range of topics. The observed improvement in spirometry interpretation skills could have resulted from learning a few novel facts or from a more global educational enrichment. Analysis of individual test answers showed that for nearly every question the percentage of correct responses increased (Fig. 4). The greatest gains appeared to
The spirometry module improves test performance at all levels of training. We are interested in developing educational tools for off-site teaching that
FIG. 1. The spirometry module improves spirogram interpretative skills. The spirometry module, with pre- and post-test assessments, was administered to 65 participants. Test scores are distributed normally with improvement from 6 ⴞ 0.3 correct responses on the pre-test (mean ⴞ SE) to 8 ⴞ 0.3 correct responses out of 12 total on the post-test (P F 0.001). Median score also increased from 5 to 8 correct responses, confirming that improvement occurred across much of the group.
FIG. 2. Spirometry interpretation skills improve at all levels of training. Test scores were subgrouped by participant training level. Data are displayed as median no. of correct responses out of a possible 12; n ⴝ no. of participants. Note that the greatest improvement in post-test scores was observed in the group with the lowest pre-test values (medical students); however, significant improvement was observed for all 3 subgroups (*P F .001, post-test vs. pre-test median).
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S57
I
N
N
O
V
A
T
I
O
N
S
A
N
D
I
D
E
A
S
involve learning the grading schema for airways obstruction, recognizing variable upper airway obstruction, and identifying uninterpretable tests. Overall, participation in the module improved spirometry interpretation across the entire range of disorders tested. The spirometry module was brief and simple to use. To model the experience of a remote user, our test group gained access to the spirometry module using the University of Iowa’s Internet provider. The time required to read the module and complete the post-test ranged from 25 to 30 min; those participants at advanced training levels required less time (medical students 30 ⫾ 2 min, senior house staff 25 ⫾ 2 min). During the test sessions, few problems were encountered using or accessing the module, and most participants did not require additional directions. Thus this self-directed educational tool was effective, time efficient, and user friendly.
FIG. 3. The spirometry module improved interpretive performance independent of prior experience. Pre- and post-test scores are shown for intern subgroup as no. of correct responses out of a possible 12, where A–Q indicate individual participants; arrowheads depict pre- and post-test median scores for entire intern group. All but 2 participants (A and C) had an improved post-test score after completing the spirometry module. Individual post-test scores improved whether the associated pre-test score was initially above or below the group median. Similar findings were observed for other groups tested (data not shown).
The spirometry module was effectively disseminated via the Internet. The spirometry module was incorporated within the Pulmonary Core Curriculum
FIG. 4. Computer-based tutorial produced broad improvement in spirometry reading skills. Analysis of submitted answers for individual test questions showed that the percentage of correct responses increased for nearly every question tested. The largest increase in correct answers appeared to involve the learning of classification schema for grading airways obstruction (questions 1, 4, 10, and 12), recognizing variable upper airway obstruction (UAO; question 11), and identifying uninterpretable studies (questions 2 and 9). mod, Moderate; r/o, rule out.
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S58
I
N
N
O
V
A
T
I
O
N
S
of the Virtual Hospital. The module was posted for 12 wk without specific advertisement. Despite this lack of solicitation, the site received ⬃4,000 ‘‘hits’’ representing the number of times the module was accessed. A pre-test was submitted by 122 participants who identified themselves as medical students (53%), house staff (12%), respiratory therapists (11%), and others such as community physicians, nurses, and non-health care workers. From those who offered information up front and/or responded to follow-up electronic mail, we were able to identify participants from Canada, the United Kingdom, Italy, Malaysia, Brazil, the Hawaiian Islands, and numerous sites within the continental United States. We received 17 completed pairs of pre- and post-tests. All 17 respondents increased their scores, with a median improvement of 2 correct responses (range 1 to 4). Thus, with the use of an existing electronic teaching network, the spirometry module was internationally displayed and accessed. For those motivated to submit pre- and post-tests, use of the module improved spirometry interpretation.
A
N
D
I
D
E
A
S
adhere to ATS guidelines (4). Thus there is an educational deficit among primary care providers in recognizing proper indications for and the correct interpretation of PFT that could negatively impact the delivery of efficient, cost-effective health care. In response to this educational need, we have designed a computer-based tutorial to teach spirometry interpretation that is brief, effective, and widely accessible via the Internet. The criteria and guidelines set forth by the ATS are emphasized in a user-friendly, multimedia format. Our analysis demonstrates that the spirometry module is educationally effective. Participants completing the module improved their ability to correctly interpret spirograms (Fig. 1). This improved performance was observed among all participants independent of level of training or prior experience in reading PFT (Figs. 2 and 3). Interpretative skills improved across the range of patterns covered by the module, demonstrating a broad-based increase in knowledge beyond the acquisition of a few novel facts (Fig. 4). Furthermore, the spirometry module can be completed with minimal time commitment (⬃30 min) and little supplementary instruction, which will hopefully encourage additional sessions as needed.
DISCUSSION As spirometry becomes more readily available outside of traditional pulmonary function laboratories, it will more frequently be ordered by caregivers who may never have been formally trained in proper interpretation. This lack of instruction combined with an emphasis on shifting diagnostic evaluations away from specialized centers could lead to both misdiagnoses as well as missed diagnoses. Indeed, the need for improved understanding of PFT indications and interpretation has been documented in other studies. A review of PFT ordering patterns by primary care internists and house staff at a community hospital found that up to two-thirds of the studies requested were not fully appropriate (16). Another study estimated that nonpulmonologists misinterpret spirometry up to one-third of the time with errors in disease classification, overreading of normal spirograms, and misreading of abnormal studies that illustrated restrictive defects or upper airway abnormalities (15). These authors suggest that standards for interpreting spirometry need to be more readily available to primary care physicians (15). Similarly, a recent Canadian study of PFT laboratories in British Columbia found marked interobserver variability in interpretation of standard spirograms. This variability was attributed primarily to failure to
The durability of the knowledge gained through participation in the spirometry module was not assessed. The effectiveness of this computer-based tutorial was not directly compared with that of more traditional printed texts or standard lectures on spirometry interpretation. Numerous other studies, however, have confirmed the effectiveness of similar educational tools for teaching radiograph interpretation, pulmonary auscultation, pediatric pulmonary diseases, and well-newborn care protocols (8, 10–13). Many of the participants in these studies preferred computer-based techniques over more traditional devices with similar educational efficacy. We envision computer-assisted learning serving as a welcome and effective adjunct to more traditional curricula. Certainly, computer-assisted learning has been integrated into the medical educational arena, and its potential applications are seemingly limitless (7, 12, 14, 23). However, the real challenge lies in disseminating these educational opportunities to students and practitioners who may not have immediate physical
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S59
I
N
N
O
V
A
T
I
O
N
S
A
N
D
I
D
E
A
S
ing medical education offering through the Virtual Hospital (http://www.vh.org/Providers/Simulations/ Spirometry/SpirometryHome.html).
access to the training center. Since computers entered the medical workplace, their use has become increasingly routine, in both the office and the hospital. After the spirometry module was posted on-line within the Virtual Hospital, off-site trainees and primary care practitioners from various countries around the world accessed the tutorial. Pre- and post-tests submitted via electronic mail were received, processed, and responded to without further user costs above those for Internet provider access. Thus an effective educational intervention was distributed internationally using technology that allows for central quality control and updating, around-the-clock access, and, if automated, immediate feedback to the participant.
In summary, we used a computer-based educational intervention to effectively teach spirometry interpretation to nonpulmonologists. With the use of the Virtual Hospital as a central repository, this information was distributed to trainees and primary caregivers all over the world. This teaching service is now provided to many off-campus referral centers and outreach clinics as part of a University of Iowa information network. Soon such tutorials may be linked to computerized pulmonary function reports, facilitating combined distribution of clinical data and relevant educational materials. Our preliminary study is not intended to prove that electronic educational tools are necessary or superior to current teaching practices. However, as time and resources for medical education continue to shrink, computer-aided instruction via the Internet offers promising, cost-effective adjuncts to traditional modes of education. In the future, we anticipate greater development and implementation of similar educational interventions.
Our study highlights several areas of concern that must be addressed for remotely accessible computerbased learning to realize its full potential. First, the quality of presented material must be assured. Although we incorporated the central features of the ATS recommendations for spirometry standardization into the module, clinical interpretative skills are subject to personal biases of the teacher. To maintain the quality of its material, the Virtual Hospital has initiated a peer review system akin to that used by standard published journals. All material is confidentially critiqued on-line and submitted by electronic mail to minimize response time; reviews are repeated with scheduled regularity to ensure that material remains current (Dr. Michael Peterson, Virtual Hospital Editorial Board, personal communication). Second, we received several comments noting great variability in the quality of displayed graphics and response times for viewing video images. Thus the usefulness of a multimedia educational tool may be limited by the equipment available to the end user. This limitation stresses the importance of striving to find applications that function across multiple-user platforms. Finally, although we were pleased to receive nearly 4,000 visits at the spirometry module site, completed test pairs represented ⬍1% of the total audience. To fully engage the target audience, additional carrot-and-stick incentives such as continuing medical education credits or special certification may be required (6, 9, 22). Toward this end, the spirometry module has recently been incorporated into a medical student minicourse along with other similar modules on acidbase interpretation and respiratory pathophysiology. In addition, the module is also available as a continu-
We thank Teresa Knutson-Choi, Nola Riley, and the Virtual Hospital support staff for expert technical assistance. This work was supported, in part, by the American Lung Association of Iowa and a grant from the National Library of Medicine. Address for reprint requests and other correspondence: T. J. Gross, Room C33-GH, Div. of Pulmonary, Critical Care, and Occupational Medicine, Univ. of Iowa Hospitals and Clinics, Iowa City, IA 52242 (E-mail:
[email protected]). Received 23 July 1998; accepted in final form 10 February 1999.
References 1. American Thoracic Society. Lung function testing: selection of reference values and interpretative strategies. Am. Rev. Respir. Dis. 144: 1202–1218, 1991. 2. American Thoracic Society. Standardization of spirometry, 1994 update. Am. J. Respir. Crit. Care Med. 152: 1107–1136, 1995. 3. Banks, D. E., M. L. Wang, L. McCabe, M. Billie, and J. Hankinson. Improvement in lung function measurements using a flow spirometer that emphasizes computer assessment of test quality. J. Occup. Environ. Med. 38: 279–283, 1996. 4. Broder, S. L., G. Copland, J. Berkowitz, and J. Road. Inter-observer variability of pulmonary function test interpretation (Abstract). Am. J. Respir. Crit. Care Med. 155: A71, 1997. 5. Case, S. M., and D. B. Swanson. Constructing written test questions for the basic and clinical sciences. [On-line] National Board of Medical Examiners. http://www.nbme.org/new.version/item.htm [1999, Jan. 28]
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S60
I
N
N
O
V
A
T
I
O
N
S
6. Chao, J. Continuing medical education software: a comparative review. J. Fam. Pract. 34: 598–604, 1992. 7. Chodorow, S. Educators must take the electronic revolution seriously. Acad. Med. 71: 221–226, 1996. 8. D’Alessandro, M. P., J. R. Galvin, W. E. Erkonen, M. A. Albanese, V. E. Michaelsen, J. S. Huntley, R. M. McBurney, and G. Easley. The instructional effectiveness of a radiology multimedia textbook (Hyperlung) versus a standard lecture. Invest. Radiol. 28: 643–648, 1993. 9. Davis, D. A., M. A. Thompson, A. D. Oxman, and B. Haynes. Evidence for the effectiveness of CME. JAMA 268: 1111–1117, 1992. 10. Desch, L. W., M. T. Esquivel, and S. K. Anderson. Comparison of a computer tutorial with other methods for teaching well-newborn care. Am. J. Dis. Child. 145: 1255–1258, 1991. 11. Galvin, J. R., M. P. D’Alessandro, W. E. Erkonen, T. A. Knutson, and D. L. Lacey. The Virtual Hospital: a new paradigm for lifelong learning in radiology. Radiographics 14: 875–879, 1994. 12. Glowniak, J. V. Medical resources on the Internet. Ann. Intern. Med. 123: 123–131, 1995. 13. Guntupalli, K. K., V. Bandi, C. Sirgi, C. Pope, A. Rios, and W. Eschenbacher. Usefulness of flow-volume loops in emergency center and ICU settings. Chest 111: 481–488, 1997. 14. Headrick, L., A. Kaufman, P. Stillman, L. Wilkerson, and R. Wighton. Teaching and learning methods for new generalist physicians. J. Gen. Intern. Med. 9: S42–S49, 1994. 15. Hnatiuk, O., L. Moores, T. Loughney, and K. Torrington. Evaluation of internists’ spirometric interpretations. J. Gen. Intern. Med. 11: 204–208, 1996.
A
N
D
I
D
E
A
S
16. Khatri, K., R. Kaufman, and W. Baigelman. Utilization of pulmonary function tests by primary care internists in a community hospital. Am. J. Med. Qual. 9: 49–53, 1994. 17. Kienzle, M. G., D. Curry, J. Galvin, E. Hoffman, E. Holtum, L. Shope, J. Torner, and D. Wakefield. Iowa’s national laboratory for the study of rural telemedicine: a description of a work in progress. Bull. Med. Libr. Assoc. 83: 37–41, 1995. 18. Mangione, S., L. Z. Nieman, and E. J. Gracely. Comparison of computer-based learning and seminar teaching of pulmonary auscultation to first-year medical students. Acad. Med. 67: S63–S65, 1992. 19. Rebuck, D. A., N. A. Hanania, A. D. D’Urzo, and K. R. Chapman. The accuracy of a handheld portable spirometer. Chest 109: 152–157, 1996. 20. Santer, D. M., V. E. Michaelsen, W. E., Erkonen, R. J. Winter, J. C. Woodhead, J. S. Gilmer, and M. P. D’Alessandro. A comparison of educational interventions: multimedia textbook, standard lecture, and printed textbook. Arch. Pediatr. Adolesc. Med. 149: 297–302, 1995. 21. Sestini, P., E. Rensoni, M. Rossi, V. Beltrami, and M. Vagliasindi. Mulitmedia presentation of lung sounds as a learning aid for medical students. Eur. Respir. J. 8: 783–788, 1995. 22. Thompson, W. L., J. P. Dyke, and E. Buonocore. Using the World-Wide Web to train and certify physicians in the safe use of fluoroscopy. AJR Am. J. Roentgenol. 166: 1263–1264, 1996. 23. Weiler, R. M. Creating a virtual materials and resources index for health education using the world wide web. J. Sch. Health 66: 205–209, 1996.
VOLUME 21 : NUMBER 1 – ADVANCES IN PHYSIOLOGY EDUCATION – JUNE 1999
S61
