Accuracy and Reproducibility of Telecytology Diagnosis of Cervical ...

Anatomic Pathology / TELECYTOLOGY FOR CERVICAL SMEARS

Accuracy and Reproducibility of Telecytology Diagnosis of Cervical Smears A Tool for Quality Assurance Programs Eung Seok Lee, MD,1,2 In Sun Kim, MD,2 Jong Sang Choi, MD,2 Bom Woo Yeom, MD,2 Han Kyeom Kim, MD,2 Jong Ho Han, MD,3 Mi Sook Lee, MD,3 and Anthony S-Y Leong, MB, BS, MD, FRCPA, FRCPath, FCAP, FASCP1 Key Words: Telecytology; Digital images; Cervical smears; Atypical squamous cells of undetermined significance; ASCUS; low-grade squamous intraepithelial lesion; LSIL; high-grade squamous intraepithelial lesion; HSIL; Squamous carcinoma; Adenocarcinoma DOI: 10.1309/7YTVAG4XNR48T75H

Abstract We randomly selected 50 cervical smears (benign, 14; atypical squamous cells of undetermined significance [ASCUS], 5; low-grade squamous intraepithelial lesion [LSIL], 10; high-grade squamous intraepithelial lesion (HSIL), 12; squamous cell carcinoma, 6; adenocarcinoma, 3) and captured 1,181 digital images (518 MB) at a maximum resolution of 1,600 × 1,200 pixels and transmitted them by e-mail. Diagnosis of glass slides and digital images was done independently in a double-blind manner by 3 pathologists and 3 cytotechnologists, commencing with the diagnosis of digital images followed by diagnosis of glass slides 3 months later. The procedure was repeated after 3 months. Diagnoses were recorded as benign, ASCUS or atypical glandular cells of undetermined significance, LSIL, HSIL, squamous cell carcinoma or adenocarcinoma, and “inadequate for diagnosis.” Diagnostic accuracy and interobserver reproducibility were analyzed using an intraclass correlation coefficient (ICC), which revealed good interobserver agreement for the first (0.72) and second (0.64) glass slide diagnoses and the first (0.72) and second (0.60) digital image diagnoses. The kappa values for intraobserver variation between first and second glass slide diagnoses and first and second digital image diagnoses showed moderate to excellent agreement. Digital images are suitable substitutes for glass slides; telecytology can be used as an alternative method for the cytologic diagnosis of cervical smears, particularly in quality assurance programs.

Quality assurance programs in cytology are one of the most important methods to maintain and improve the diagnostic acumen of cytotechnologists and cytopathologists, but there are difficulties in carrying out such programs. A long turnaround time for the circulation of glass slides is a major drawback, particularly when there are many participants and in widely spread institutions. The use of Kodachrome slides has been a partial but unsatisfactory solution because of costs and delays in preparation. Telepathology is a component of telemedicine and is simply defined as the practice of transmitting digital pathology images of microscopic or gross findings through telecommunication networks to remote viewing locations for diagnosis, storage, or education. 1 The concept of telepathology has been discussed for the past 10 years; however, publications in telecytology are much fewer than in telepathology. One study suggested telepathology as an alternative modality for quality assurance in breast histopathology.2 However, there is no previous study examining the use of telecytology as an alternative to the conventional microscopic examination of glass slides for quality assurance in cytology. This is the first study to examine the accuracy and reproducibility of the telecytology diagnosis of cervical smears.

Materials and Methods We randomly obtained 50 cervical smears (benign, 14; atypical squamous cells of undetermined significance [ASCUS], 5; low-grade squamous intraepithelial lesion, 10; high-grade squamous intraepithelial lesion, 12; squamous cell Am J Clin Pathol 2003;119:356-360

© American Society for Clinical Pathology 1

DOI: 10.1309/7YTVAG4XNR48T75H

356 1

Lee et al / TELECYTOLOGY FOR CERVICAL SMEARS

carcinoma, 6; and adenocarcinoma, 3) from the files of the Department of Anatomic Pathology, Anam Hospital, Korea University, Seoul. Four pathologists (E.S.L., I.S.K., J.S.C., and B.W.Y.) examined these cases, and consensus diagnoses were obtained. The most worrisome cells or groups in each case were selected by consensus opinion and marked. Digital images were captured at 4 magnifications (×40, ×100, ×200, ×400) for each marked area using an Olympus Camedia 2000 zoom digital camera (Olympus, Tokyo, Japan) mounted with a phototube adapter on a light microscope. The images were captured at a maximum resolution of 1,600 × 1,200 pixels, 24-bit true color (16.7 million color palette) in JPEG (Joint Photographers Experts Group) format. They were stored in a personal computer and on a CD-ROM disk and transmitted over the Internet by attachment to electronic mail (e-mail) from Anam Hospital to Samsung Medical Centre, Seoul, Korea. The maximum transmission rate via local area network and commercial network using ADSL (asymmetric digital subscriber line) was 250 to 1000 kilobits per second (kbps), which is 5 to 20 times faster than the maximum transmission rate of a 56.6kbps modem. IBM personal computers equipped with Pentium II 233-MHz microprocessors, 8-gigabyte hard disks, 64 MB of RAM with a 2-MB graphics card, and a 17inch color monitor with a maximal resolution of 1,280 × 1,024 pixels were used in this study. The number of images and selected fields for each case were recorded. The digital images were diagnosed independently by 3 pathologists (H.K.K., J.H.H., and M.S.L.) and 3 cytotechnologists in a double-blind manner. Diagnosis of glass slides followed 3 months later in a similar manner. Only marked areas were examined. The sequence was repeated after another 3-month interval. All but one of the observers had more than 3 years of cytology experience. None had previous experience with telecytology. Diagnoses were recorded in 5 categories: (1) benign, (2) ASCUS or atypical glandular cells of undetermined significance, (3) low-grade squamous intraepithelial lesion, (4) high-grade squamous intraepithelial lesion, and (5) squamous cell carcinoma or adenocarcinoma. A sixth category of “inadequate for diagnosis” was available. Additional information, including comments on adequacy of images, total time required for diagnosing all cases, problems encountered in diagnosing digital images, and whether there was a need for low-magnification digital images, was recorded by each observer. Interobserver variation was calculated by using an intraclass correlation coefficient (ICC), which determines the probability of agreement beyond that occurring by chance.3 Intraobserver variations in diagnoses between the first and second glass slide and the first and second digital image examinations were assessed by using the weighted kappa statistic.4 357 Am J Clin Pathol 2003;119:356-360 2 DOI: 10.1309/7YTVAG4XNR48T75H

Results We captured 1,181 digital images (518 MB) from the 50 cases in medium compression JPEG image files with 1,600 × 1,200-pixel resolution (0.40-0.45 MB per image). The mean number of selected fields and digital images for each case were 6 (range, 5-8) and 24 (range, 20-32), respectively. Email transmission was done en masse and was 100% successful with a total upload time of 5 hours and download time of 4 hours for the 518 MB of data from Anam Hospital to Samsung Medical Centre. There was no image distortion identifiable after the transmission process. ❚Table 1❚ gives a comparison of individual observer diagnoses in each diagnosis cycle compared with the consensus diagnoses. ICC values revealed that intraobserver agreement was good for the first glass slide diagnosis (0.72), first digital image diagnosis (0.72), second glass slide diagnosis (0.64), and second digital image diagnosis (0.62). Weighted kappa statistics for intraobserver variations between the first and second set of diagnoses made on glass slides and digital images showed values that ranged from moderate to excellent agreement ❚Table 2❚. The mean diagnostic time was 147.2 minutes (range, 125-160 minutes) for glass slides and 68.8 minutes (range, 58-75 minutes) for digital images. Five cases were recorded as having insufficient digital images for diagnosis. Low magnification (×40) of digital images was recorded as not necessary by all observers. The inability to focus at different levels to examine the architectural and cellular details of overlapping cell groups in glandular lesions was recorded as an impediment to diagnosis in digital images.

Discussion Digital imaging can be used in many areas of anatomic pathology, including the photography of gross and microscopic findings in both surgical and autopsy pathology. It is practical and cost-effective and provides many advantages over traditional pathology practices. Digital imaging also is the first step toward opening the door to many future applications and improved diagnostic, educational, and quality assurance activities. Most of the studies on telecytology have been confined to the study of breast aspiration biopsies. There is 1 published study on telecytology using video microscopy in cervical smears.5 However, in that study, video microscopy was used without using digitalization and image transmission. Our study is the first to examine the diagnostic accuracy and reproducibility of telecytology in cervical smears following digitalization and image transmission through the Internet. There are many advantages of using telecytology for quality assurance. Telecytology has the major advantage of © American Society for Clinical Pathology

Anatomic Pathology / ORIGINAL ARTICLE

❚Table 1❚ Consensus Diagnoses Compared With Observers’ Diagnoses Made on Glass Slides and Digital Images for Two Cycles* Consensus Diagnosis

Observer

Benign (n = 14)

ASCUS (n = 5)

LSIL (n = 10)

HSIL (n = 12)

SCC (n = 6)

Adenocarcinoma (n = 3)

1 A B C D

13 7 12 11

8 5 9 7

2 4 4 6

17 15 15 20

8 11 7 8

2 3 3 3

A B C D

9 8 7 6

8 6 8 10

2 8 4 7

19 16 21 18

9 10 8 7

3 2 2 2

A B C D

8 7 5 5

11 8 10 11

4 7 6 9

17 16 20 17

9 9 7 6

1 3 2 2

A B C D

6 7 7 9

8 10 10 17

6 8 5 5

17 16 10 11

11 7 15 7

2 2 3 1

A B C D

4 12 16 18

17 12 2 4

9 7 10 12

14 14 16 11

6 5 6 5

0 0 0 0

A B C D

12 11 14 14

4 5 3 5

5 7 5 10

20 16 19 12

6 8 7 6

3 3 2 3

2

3

4

5

6

ASCUS, atypical squamous cells of undetermined significance; HSIL. high-grade squamous intraepithelial lesion; LSIL, low-grade squamous intraepithelial lesion; SCC, squamous cell carcinoma. * A, First glass slide diagnosis; B, first digital image diagnosis; C, second glass slide diagnosis; D, second digital image diagnosis.

rapid turnaround time. Transmission of digital images through the Internet undoubtedly is faster than the conventional method of circulating glass slides, especially when the availability of cytologic smears is limited compared with that of histologic material. Slow turnaround of glass slides during quality assurance exercises is a problem. For the once-a-month assessment meeting of the Korea Cytology Society, glass slides were circulated among more than 50 institutions throughout the country. This exercise alone took least 3 months. Sometimes participants had to attend the meeting without viewing the slides. To circumvent this problem, appropriate selected digital images of the study cases are now posted on the Society’s Web site. The use of digital images also ensures the assessment of identical fields, avoiding the problem posed by differences in field selection. The main aim in quality assurance programs is to test participants’ ability to make the correct decision on a specific abnormal finding rather than the ability to screen an entire slide. Thus, telecytology circumvents the problem of field selection and assesses interpretation. It also eliminates the time that would be spent searching the slide for

❚Table 2❚ Intraobserver Variation Between Diagnoses Made on First and Second Examinations of Glass Slides and Digital Images* Observer 1 2 3 4 5 6 *

Glass Slide Diagnosis 0.80 (excellent) 0.80 (excellent) 0.73 (good) 0.74 (good) 0.59 (moderate) 0.75 (excellent)

Digital Image Diagnosis 0.68 (good) 0.80 (excellent) 0.72 (good) 0.57 (moderate) 0.68 (good) 0.76 (excellent)

Data are kappa values with an interpretation of the value in parenthesis.

abnormal cells. In the present study, the mean diagnostic time was reduced by more than half for digital images (68.8 minutes) compared with glass slides (147.2 minutes). Cost savings is another advantage. Telecytology reduces the expenses of postal or courier slide circulation and the cost and delays of Kodachrome slide preparation. Easy and continuous access to the case material from the Web site is yet another advantage over glass slides, which have to be Am J Clin Pathol 2003;119:356-360



358 3

Lee et al / TELECYTOLOGY FOR CERVICAL SMEARS

returned to the owner institution. After the quality assurance exercise, the digital images are still available for reference and teaching purposes. Telecytology also makes it possible to share cases beyond the circle of participating members. These advantages, together with the acceptable levels of diagnostic accuracy and reproducibility, strongly support the use of telecytology for quality assurance programs. Validation of accuracy is the first step in determining the usefulness of telecytology. Intraobserver reproducibility of telecytology diagnoses must be evaluated in addition to other factors such as cost-effectiveness and practicability of use in routine work. There are many reports of telepathology or telecytology that examine diagnostic accuracy. However, there is no previous study that examines the reproducibility of telecytology diagnoses. Our study shows that accuracy and intraobserver reproducibility of telecytology digital image diagnoses are as high as for conventional diagnoses made on glass slides. Accurate diagnosis of cervical smears is critical to management because treatment protocols differ. Raab et al5 reported that telecytology resulted in a tendency for underdiagnosing cervical smears largely because there is a reluctance to diagnose dysplastic lesions on the video monitor. We did not find that to be the case in our study, which was based on digital images. There is no single, universally accepted method for assessing agreement and discordance between observations, and we chose 2 parameters to assess interobserver and intraobserver agreement. The ICC was used to measure interobserver agreement. While closely related to the kappa statistic, which is a common method of measuring interobserver variability and has identical numeric parameters, ICC also has the advantage of analyzing data with multiple response levels when the observer’s agreement varies across a number of possible responses.3 A weighted kappa statistic was used to measure the intraobserver variation between first and second diagnoses for both glass slides and digital images.4 ICC and the kappa statistic have been used previously to measure interobserver and intraobserver variance in the diagnosis of ASCUS in cervical samples.6 Individual observer performance was variable. All observers had slightly higher diagnostic accuracy and intraobserver reproducibility with glass slides than with digital images with the exception of observer 5 (Table 1). This observer showed better performance with digital images (kappa, 0.68) compared with glass slides (kappa, 0.59) (Table 2). This suggested that observer 5 had good and reproducible diagnostic acumen when presented with specific lesions or fields as in the digital images but perhaps had difficulty maintaining concentration when scanning a whole slide. Observer 4 had fewer than 3 years of cytology experience and performed poorest with digital images (kappa, 0.57 compared with 0.74 for glass slide diagnosis). This suggested that observer 4 359 Am J Clin Pathol 2003;119:356-360 4 DOI: 10.1309/7YTVAG4XNR48T75H

tended to vary diagnostic criteria. The greater the experience, the more consistent are one’s diagnostic criteria, and this may have a major impact on accuracy and intraobserver reproducibility in telecytology diagnosis. In addition to cytologic experience, further telecytology experience will improve diagnostic accuracy and reproducibility in telecytology. Three factors can affect the diagnostic interpretation of digital images: adequate and appropriate sampling (field selection, number and microscopic magnification of images), image resolution, and color quality. Cytologic diagnosis is based on cytologic features such as nuclear/cytoplasmic ratio, chromatin pattern, nuclear pleomorphism, and cellular arrangement on a limited number of microscopic fields. In telepathology, one of the primary problems has been field selection (sampling error). Accuracy for telepathology has varied between 85% and 100%. Insufficient or inadequate sampling of images is the most common reason for low diagnostic accuracy. However, this can be minimized when an experienced pathologist who is familiar with the technology and trained to recognize and identify highly informative fields is available for field selection.7 With the development of total slide digitalization systems, this problem will be circumvented.8 In the present study, 5 cases were recorded as having insufficient fields and images for diagnosis. However, all 5 cases showed very few abnormal cells and were difficult to diagnose on glass slides. They showed low accuracy and intraobserver reproducibility in the diagnosis of glass slides. There were 3 cases that were difficult to interpret on digital images. All 3 cases were abnormal glandular lesions, and the inability to examine glandular lesions in 3 dimensions in digital images probably accounted for the failure to make the correct diagnosis. Magnification at ×100 was necessary to identify the background and architecture of cell clusters. However ×40 magnification was too low a resolution to permit interpretation of the digital images. Resolution is the ability to show fine details of visual information. The higher the resolution, the more visual information present. Low resolution and color depth were limiting factors in digital images of the past. However, the resolution and color depth of current digital cameras are more than adequate to make most histopathologic and cytologic diagnoses.9,10 Compression of the image file is another important factor that affects the quality of digital image, and lost visual information can never be recovered after image compression. JPEG is by far the most commonly used method of compressing image files. It maintains 24-bit color depth and is well suited for telepathology and telecytology. In the present study with high resolution (1,600 × 1,200) and 24-bit true color JPEG image files, the digital images after Internet transmission could be interpreted accurately and showed excellent cellular details for cytology diagnosis ❚Image 1❚, ❚Image 2❚, and ❚Image 3❚. © American Society for Clinical Pathology

Anatomic Pathology / ORIGINAL ARTICLE

❚Image 1❚ High-magnification digital image showing cellular details, including coarse chromatin, nuclear enlargement and irregularity in high-grade squamous intraepithelial lesion (Papanicolaou).

❚Image 2❚ Low-magnification digital image to show nests of malignant cells in a necrotic background in squamous cell carcinoma (Papanicolaou).

Digital images are a suitable substitute for glass slides in the diagnosis of cervical smears. Telecytology can be used as an alternative to replace the conventional glass slides used in quality assurance programs. From the 1Division of Anatomical Pathology, Hunter Area Pathology Service and Discipline of Anatomical Pathology, University of Newcastle, Newcastle, New South Wales, Australia; 2Department of Anatomic Pathology, Anam Hospital, Korea University, Seoul; and 3Department of Diagnostic Pathology, Samsung Medical Center, Seoul, Korea. Address reprint requests to Dr Leong: Hunter Area Pathology Service, Locked Bag 1, Hunter Region Mail Centre 2310, Australia.

References 1. Leong FJ. Practical applications of Internet resources for costeffective telepathology practice. Pathology. 2001;33:498-503. 2. Leong FJ, Graham AK, Schwarzmann P, et al. Clinical trial of telepathology as an alternative modality in breast histopathology quality assurance. Telemed J E Health. 2000;6:373-377. 3. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420-428. 4. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-174. 5. Raab SS, Zaleski MS, Thomas PA, et al. Telecytology: diagnostic accuracy in cervical-vaginal smears. Am J Clin Pathol. 1996;105:599-603. 6. Grenko RT, Abendroth CS, Frauenhoffer EE, et al. Variance in the interpretation of cervical biopsy specimens obtained for atypical squamous cells of undetermined significance. Am J Clin Pathol. 2000;114:735-740.

❚Image 3❚ Intermediate-magnification digital image showing a cluster of atypical cells in adenocarcinoma (Papanicolaou).

7. Weinberg RS, Allaert FA, Dusserre P, et al. Telepathology diagnosis by means of digital still images: an international validation study. Hum Pathol. 1996;27:111-118. 8. Leong FJ, McGee JO. Automated complete slide digitization: a medium for simultaneous viewing by multiple pathologists. J Pathol. 2001;195:508-514. 9. Halliday BE, Bhattacharyya AK, Graham AR, et al. Diagnostic accuracy of an international static imaging telepathology consultation service. Hum Pathol. 1997;28:17-21. 10. Weinstein MH, Epstein JI. Telepathology diagnosis of prostate needle biopsies. Hum Pathol. 1997;28:22-29.

Am J Clin Pathol 2003;119:356-360



360 5