Unsupervised segmentation for digital ... - Wiley Online Library

0 downloads 0 Views 763KB Size Report
of skin lesions, but fails when the lesion is brighter than the surrounding skin. Key words: segmentation – dermoscopy – hair recognition. – skin lesions – ...
Skin Research and Technology 2010; 16: 401–407 Printed in Singapore  All rights reserved doi: 10.1111/j.1600-0846.2010.00455.x

r 2010 John Wiley & Sons A/S Skin Research and Technology

Unsupervised segmentation for digital dermoscopic images Kajsa Møllersen1, Herbert M. Kirchesch2, Thomas G. Schopf1 and Fred Godtliebsen1,3 1

Norwegian Centre for Integrated Care and Telemedicine, University Hospital of North-Norway, Troms, Norway, 2Private Practice, Pulheim, Germany, and 3Department of Mathematics and Statistics, University of Troms, Troms, Norway

Background: Skin cancer is among the most common types of cancer. Melanoma is the most fatal of all skin cancer types. The only effective treatment is early excision. Recognising melanoma is challenging both for general physicians and for expert dermatologists. A computer-aided diagnostic system improving diagnostic accuracy would be of great importance. Segmenting the lesion from the skin is the first step in this process. Methods: The present segmentation algorithm uses a multiscale approach for density analysis. Only the skin mode is found by density analysis and then the location of the lesion mode is estimated. The density estimates are attained by Gaussian kernel smoothing with several bandwidths. A new algorithm for hair recognition based on morphological operations on binary images is incorporated into the segmentation algorithm.

Results: The algorithm provides correct segmentation for both unimodal and multimodal densities. The segmentation is totally unsupervised, with a digital image as the only input. The algorithm has been tested on an independent set of images collected in dermatological practice, and the segmentation is verified by three dermatologists. Conclusion: The present segmentation algorithm is fast and intuitive. It gives correct segmentation for most types of skin lesions, but fails when the lesion is brighter than the surrounding skin.

is one of the most common types of cancer for fair-skinned populations in Europe, North America and Australia. Among the various types of skin cancer, melanoma is the most fatal, accounting for approximately 90% of all skin cancer deaths (1). The only effective treatment is surgical excision of the lesion at an early stage. Discriminating between melanoma and harmless moles is a challenge both for general physicians (GPs) and for expert dermatologists. [(2), p. 1]. Dermoscopy (dermatoscopy, epiluminesence microscopy) is a technique for the examination of the upper layers of the skin using a glass plate and magnifying lenses. An immersion fluid between the skin and the dermoscope increases the translucency of the skin, thus facilitating visualization of moles and other skin lesions in great detail. The use of dermoscopy allows increased diagnostic accuracy for trained and experienced physicians, but it is still far from 100% accurate [(2), p. 7–10]. A computer-aided diagnostic (CAD) system for the early detection of melanoma, based on computer analysis of

digital dermoscopic images, can increase the diagnostic accuracy and reduce the need for training and experience. An image of a skin lesion contains the lesion, the surrounding skin and possibly artefacts such as hairs. Before performing an analysis on the lesion, the image has to be segmented into lesion and skin, and artefacts have to be recognised. A method based on morphological operations on binary images with the purpose of recognising hairs that possibly interrupt segmentation is presented. There are no attempts to replace the values of the hair pixels with the true (unknown) values of the underlying skin/lesion. A crucial part of the suggested segmentation method is based on density estimation of the pixel values. It takes advantage of an existing powerful scalespace method (SiZer) that allows for density analysis at several levels of resolution. Using scale-space for segmentation and automatic threshold selection for hair recognition, the use of sample-dependent parameters is avoided. Even the parameters of the hair recognition

S

KIN CANCER

Key words: segmentation – dermoscopy – hair recognition – skin lesions – melanoma

& 2010 John Wiley & Sons A/S Accepted for publication 04 June 2010

401

Møllersen et al.

algorithm are automatically adjusted according to the fixed spatial resolution of the images in a sample. A CAD system for melanoma detection is of special importance for GPs, because accuracy in skin lesion diagnosis is low. Although dermoscopy allows increased diagnostic accuracy for experienced physicians, it actually reduces diagnostic accuracy when used by inexperienced physicians (3). The equipment used in this study, a digital camera and a dermoscope, is affordable, available and ready to use without the need for calibration or training. This has some minor negative effects on the image quality, but these are necessary conditions for any product aiming at GPs. The skills needed to use the system are the same as those needed to use a digital camera. The reasonable price and the small size (the size of a compact camera) make the system accessible for GPs. In contrast to other segmentation and hair recognition algorithms, this method focuses on sample-independent parameters for a more general use.

Materials and Methods Camera, dermoscope and image format The skin lesions are photographed using a Ricoh GR digital camera (Ricoh Co. Ltd, Tokyo, Japan), recording 2460  3276 pixels with an 8 bit pixel depth. Attached to the camera lens is a Dermlite Pro II HR dermoscope (3 Gen LLC, San Juan Capistrano, CA, USA) with a diameter of 20 mm, consisting of a magnifying lens of  10 and 32 LEDs of polarised light with an output of approximately 18,000 lx (4). The dermoscope is placed in direct contact with the patient’s skin, resulting in a fixed distance between the lesion and the camera lens. Alcohol is used as a refraction fluid and is added between the skin and the dermoscope lens. The images are converted from raw format to tiff format in the sRGB colour space, using the white point set automatically by the camera for each image. The converted images have a 16 bit pixel depth and a spatial resolution of 0.011  0.011 mm per pixel. Samples The training sample consists of 137 images collected consecutively. The test sample consists of 11 images of melanomas collected consecutively and 80 images of benign lesions chosen randomly from an independent database of 649 images. Both samples are collected at Herbert Kirchesch’s

402

private practice in Cologne, Germany. The database consists of both melanocytic and non-melanocytic lesions, located on both the face and the body. Lesions larger than the dermoscope are not excluded. Pen marks, poor camera focus, insufficient refraction fluid and lack of contact between dermoscope and skin reduce the image quality. In spite of this, no images are excluded, because these segmentation obstacles occur in clinical practice.

Segmentation Algorithm The segmentation algorithm takes an image as input and returns the same image segmented into skin and lesion. The algorithm consists of four separate steps. (1) (2) (3) (4)

Illumination correction. Hair recognition. Density analysis. Post-processing.

In the following subsection, we describe the different steps in more detail. To make it easier for the reader to grasp the idea and the effect of each step, we also illustrate the effect by applying them to the observed lesion in Fig. 1(a). Illumination correction The dermoscopic image consists of an illuminated disc and a near-black background [see Fig. 1(a)]. The disc covers about one-third of the image and does not have the same position in all images. The 70th percentile is used as a threshold for creating a binary image from the luminance component in the IYQ colour space. In the resulting binary image, the brightest 30% of the pixels are white. The centre of mass of the largest object is taken as the centre of the disc. Because of imperfect image capturing, the circular dermoscope appears elliptical in the image. The minor axis length of this ellipse is used as radius in further processing. The non-homogeneous illumination causes the image to be darker towards the edges of the dermoscope. To correct this, an illumination correction matrix is generated on the basis of an image of a white colour standard. The mean of the pixel values of the circle segments in increasing distance from the centre is calculated for each colour layer in the RGB colour space. The mean pixel values decrease at an approximately con-

Dermoscopic image

Fig. 1. Segmentation steps: (a) input image; (b) the red layer of the general RGB illumination correction matrix with the image-specific correction matrix slightly off-centred; (c) the image after cropping and illumination correction; (d) detected artefact pixels; (e) binary image based on threshold and (f) final segmentation.

stant rate from the centre out to 9 mm from the centre. Newton’s interpolation is used to calculate the pixel values after 9 mm. The correction matrix consists of circle segments with the same values as the interpolated circle segment values from the white colour standard [see Fig. 1(b)]. The centre of illumination does not necessarily coincide with the centre of the disc. To estimate the centre of illumination, the 75th percentile of the luminance pixels situated inside the disc is used as a threshold to create a new binary image, where the brightest 25% of the pixels inside the illuminated disc are white. To remove noise, we use a structuring element (SE) defined by morphological operations. More precisely, a morphological opening with a 3  3 SE (see the first section) is applied, removing all objects that fit into the SE. The centre of the rectangle including all white pixels after the opening is regarded as the centre of illumination. Without the noiseremoving procedure, outlying white pixels may extend the bounding box and lead to an incorrect centre of illumination. Increasing the size or changing the shape of the SE has a minor impact on the location of the centre of illumination, but a larger SE requires more computational time. If

the centre of illumination has co-ordinates (i, j), then the image-specific correction matrix is cropped from the general correction matrix with left and right border (i radius, i1radius) and upper and lower border (j radius, j1radius). The lesion image is then scalar multiplied with the correction matrix. Figure 1(b) shows the red layer of the general RGB illumination correction matrix. The slightly off-centred circle shows the image-specific cropping. After illumination correction, the image appears to be more uniformly illuminated, as can be seen in Fig. 1(c). Hair recognition The presence of hairs and other artefacts can be an obstacle for correct segmentation. By recognising the pixels belonging to artefact objects, they can be excluded from the density analysis. A new approach to hair recognition by morphological operations is presented here. Morphological operations

There are two fundamental morphological operations: erosion and dilation. To carry out a morphological operation, a SE is defined. Morphological

403

Møllersen et al.

opening and closing are combinations of erosion and dilation [(5), p. 627–668]. An opening on a binary image removes all objects smaller than the SE. It breaks isthmuses that are thinner than the SE, and removes protrusions that fit into the SE. A closing on a binary image fills holes that are smaller than the SE. It fuses breaks and fills gulfs and gaps that fit into the SE. Grey scale opening and closing show similar performance. When an object smaller than the SE is removed by a grey scale opening, the new pixel values are those of the minimum of the neighbourhood (SE) for each pixel. For grey scale closing, the maximum pixel value is used. The top hat operation returns the image minus the opening of the image. Hair recognition algorithm

The hairs appear as long and thin objects that are generally darker than the skin. They are visible in all three colour channels, but appear with greater contrast to the skin in the red and blue channel. The red channel is used as an intensity image. The image is filtered with a 5  5 median filter. Otsu’s (6) threshold is used to create three binary images, with thresholds 0.85, 0.95 and 1.05 times Otsu’s threshold, respectively. A top hat operation with a 0.5-mm-long horizontal line as SE is applied to the binary image. All objects in the vertical direction thin enough to be a hair are preserved. An opening with a 0.25mm-long vertical line as SE is applied to remove noise. The procedure is applied to all three binary images separately. The resulting three images with vertical hair objects are combined by an OR operation. The top hat operation and the opening is repeated, but now the SEs are rotated 901 to detect hair objects in the horizontal direction. In each of the two binary images, the eccentricity and the major axis length for the ellipse having the same second central moments as each object are calculated. Objects longer than 1 mm and with an eccentricity 40.95 are regarded as hairs or fractions of hairs. The scales, visible in the upper left part of Fig. 1(c) are recognised as objects located minimum 5.5 mm from the centre, between 0.5 and 1.5 mm long, and with an eccentricity of minimum 0.95 of the corresponding ellipse. The scales are printed on the glass plate of the dermoscope, and the problem can be avoided by using a glass plate without print. The entire procedure is repeated for two rotations

404

of the image by 301 and 601. The result can be seen in Fig. 1(d).

Density analysis The RGB image is converted to an intensity image by principal component analysis (7), using the non-artefact pixels within a 0.9  radius distance from the centre of the illuminated disc, due to the poor lighting at the periphery. A binned kernel smoothing is performed, using a Gaussian kernel, 1001 bins and 11 bandwidths, ranging from 2  binwidth to pixel range. For each bandwidth, the 95% row-wise confidence intervals concerning significant slopes are calculated (8). Ideally, there will be two significant modes: skin and lesion. In most dermoscopic images, this is not the case; hence, another approach is needed. The skin mode is always present. By finding the location of the skin mode’s peak and left bound [see Fig. 2(a)], the pixels to the left of the bound (non-skin pixels) can be used to estimate the lesion mode’s peak. A peak is the bin where a significant slope changes direction from rising to falling. Bounds are where the slopes lose their significance or change direction from falling to rising. The skin mode is considered to be the rightmost mode whose right half covers more than 10% of the dermoscope area. The lesion pixels may form zero, one or several modes. If all bandwidths result in unimodal densities, the Sheather and Jones (9) plug-in method is used to calculate the optimal bandwidth. The skin mode’s right bound is found where the slope loses its significance in the SiZer methodology. A left bound estimate is found at equal distance, d, from the peak as the right bound [see Fig. 2(a)]. For multimodal densities, the skin mode’s left bound is found, where a significant slope changes its direction from falling to rising [see Fig. 2(b)]. The bandwidth resulting in the rightmost bound (for the left bound) is chosen. The left bound estimate for this bandwidth is calculated. If the estimate of the left bound lies to the right of where the slope changes direction, the estimate is used. The lesion mode peak is estimated as the population mean of all pixels situated to the left of the skin mode’s left bound. The global threshold is set to the arithmetic mean of the lesion mode peak and the skin mode peak.

Dermoscopic image

Fig. 2. Density analysis: (a) unimodal density and (b) multimodal density.

Post-processing The values of the hair pixels are replaced by the pixel values resulting from a morphological closing of the first principal component matrix with a 25-pixel radius disc as SE. The matrix is filtered with a 5  5 pixel median filter, and then morphologically opened using the same SE as for the closing. The global threshold is used to create a binary image. If the lesion covers o25% of the dermoscope area, all objects still in contact with the near-black surround after a morphological opening of the binary image are removed. A morphological opening with a 10-pixel radius disc is used to smooth the borders and break isthmuses. The largest object is considered to be the lesion.

Evaluation and Discussion The binary image resulting from the global thresholding and post-processing, as seen in Fig. 1(e), is the image segmented into lesion and skin. For evaluation of the segmentation, a mean filter is applied to the binary image, and then the image is thresholded again, resulting in a binary

image with a white background and a black continuous line representing the border between the lesion and the skin. This image is superimposed on the original image for evaluation [as seen in Fig. 1(f)]. Evaluation The 91 images from the test sample are evaluated by three dermatologists rating the segmentation on the scale as follows: 1 – good, 2 – acceptable, 3 – poor and 4 – bad. The evaluation method, often called subjective evaluation, is not optimal, due to the lack of measurable deviation from a gold standard. Several studies have shown that borders hand drawn by dermatologists differ significantly from dermatologist to dermatologist, and even the same dermatologist does not reproduce her own borders, when the same image is given two times with some time between, concluding that hand-drawn borders cannot be used as a gold standard [(10, 11), p. 171; (12)]. Another problem is the lack of a standardised measure of deviation. Celebi et al. (3) sum up four different deviation measures, and yet another measure is suggested by the authors. Subjective evaluation

405

Møllersen et al.

is used by Haeghen and Schmid (13–15). Both subjective evaluation and comparison of the images with a questionable ‘gold standard’ are sub-optimal evaluation methods, but provide a good indication of how an algorithm performs. The only objective measure, when the segmentation is part of a CAD system, is how the segmentation affects the final output diagnosis. In 85% of the images, the segmentation was evaluated as ‘good’ or ‘acceptable’ by at least two of the dermatologists. If one were to exclude images where the lesion is larger than the dermoscope, the fluid covers only part of the lesion, or the image is out of focus, 92% of the segmented images are evaluated as ‘good’ or ‘acceptable’ by at least two of the dermatologists. Note that only in very few cases do all three dermatologists give the same exact rating.

Discussion As can be seen from the results of the evaluation, the present segmentation algorithm performs well in most cases. An assumption made in the density analysis is that the lesion is darker than the skin. This is not always so, and in the cases where the lesion contains areas brighter than the skin near the lesion border, the segmentation algorithm fails (as seen in Fig. 3). An extension to the algorithm is needed to include these areas. The artefact recognition algorithm detects all hairs that are darker to slightly brighter than the lesion. Hairs that are considerably brighter than the lesion are not detected. For the present segmentation algorithm, this is not necessary, because these hairs will be regarded as part of the skin. The artefact recognition algorithm does not attempt to replace the hair pixel values by

any estimated value of the underlying unknown skin. This is one of the major advantages of the density analysis, as these data can simply be excluded. Other hair recognition algorithms based on morphological operations use the grey scale morphology, which requires much more computational time, in addition to a sampledependent threshold (12, 16). Density analysis is a fast and intuitive method for image segmentation. The problem of classical density analysis is that the result relies heavily on the parameters chosen, mainly bandwidth, and is often limited to multimodal densities (17–19). By performing a multiscale density analysis and including unimodal densities in the analysis, these problems can be avoided. The number of bins will affect the accuracy of the kernel smoothing, and 1001 bins were shown to produce the same results as non-binned smoothing. The shape of the kernel, chosen to be Gaussian here, has a minor effect on the resulting density estimates. Eight requirements for a systematic border detection study are suggested in (3). This study fulfils six of them. The results are not evaluated using borders determined by dermatologists, for the reasons discussed in the first section. Furthermore, the results are not compared with those of published border detection methods, due to the lack of a public dermoscopy image database. The segmentation algorithm presented provides a fast and robust segmentation for dermoscopic images. The artefact recognition and the density analysis lack sample-dependent parameters. Even the size of the SEs in the morphological operations is automatically adjusted to the spatial resolution of the images, in accordance with the fixed size of hairs and other artefacts.

Fig. 3. Parts of the lesion is as bright as the surrounding skin. The segmentation algorithm fails.

406

Dermoscopic image

Some segmentation algorithms use sample-dependent parameters (17, 19, 20, 21). Others restrict the segmentation to melanocytic lesions only, either by relying on blue colour bands or wavelengths [(22–24), p. 49; (25)], or by explicitly excluding non-melanocytic lesions from the study (26). The cheap and available equipment provides for a more general use of CAD systems for lesion diagnostics. The uncalibrated camera allows GPs to use the system without any previous training or follow-up. The simplicity of the algorithm requires only the access of a standard personal computer to be able to perform segmentation in real time. The algorithm requires no additional input from the user apart from the image itself. The algorithm, implemented in Matlab, takes less than 1 min from the time raw image is received till the time the image with the borders superimposed appears on the computer screen. By implementing the algorithm in a more efficient language (e.g. C11) with the skills of a trained programmer, the running time of the algorithm will potentially be reduced to a fraction of the present running time.

References 1. Cancer Registry of Norway. Cancer in Norway 2006. Technical report, Institute of Population-Based Cancer Research, Oslo, Norway 2006. 2. Stolz W, Braun-Falco O, Bilek P, Landthaler M, Burgdorf WHC, Cognetta AB. Color atlas of dermatoscopy, 2nd edn. Oxford: Blackwell Publishing, 2002. 3. Celebi ME, Iyatomi H, Schaefer G, Stoecker WV. Lesion border detection in dermoscopy images. Comput Med Imaging Graph 2009; 33: 148–153. 4. DermLite. Available at http://www.dermlite.com/ dl2hr.html (2009) and http://www.dermlite.com/ (accessed October 2009). 5. Gonzalez RC, Woods RE. Digital image processing, 3rd edn. Pearson Education International, London, UK 2008. 6. Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 1979; 9: 62–66. 7. Johnson RA, Wichern DW. Applied multivariate statistical analysis, 5th edn. Pearson Education International, London, UK 2002. 8. Hannig J, Marron JS. Advanced distribution theory for sizer. J Am Stat Assoc 2006; 101: 484–499. 9. Sheather SJ, Jones MC. A reliable data-based bandwidth selection method for kernel density estimation. J R Stat Soc Ser B Stat Methodol 1991; 53: 683–690. 10. Guillod J, Schmid-Saugeon P, Guggisberg D, Cerottini JP, Braun R, Krischer J, Saurat J-H, Murat K. Validation of segmentation techniques for digital dermoscopy. Skin Res Technol 2002; 8: 240–249. 11. Schmid P. Segmentation and symmetry measure for image analysis: application to digital dermatoscopy. PhD thesis, E´cole Polytechnique Fe´de´rale de Lausanne, Lausanne, 1999.

12. Schmid-Saugeon P, el Guillod J, Thiran J-P. Towards a computer-aided diagnosis system for pigmented skin lesions. Comput Med Imaging Graph 2003; 27: 65–78. 13. Haeghen YV. Development of a dermatological workstation: Preliminary results on lesion segmentation in cie L*A*B* color space. Proceedings of the International Conference on Color in Graphics and Image Processing, Saint Etinne, France 2000. 14. Schmid P. Lesion detection in dermatoscopic images using anisotropic diffusion and morphological flooding. Proceedings of the 6th International Conference on Image Processing (ICIP ’99), Vol. 3, 1999: 449–453. 15. Schmid P. Segmentation of digitized dermatoscopic images by two-dimensional color clustering. IEEE Trans Med Imaging 1999; 18: 164–171. 16. Lee T, Ng V, Gallagher R, Coldman A, McLean D. s Dullrazor : a software approach to hair removal images. Comput Biol Med 1997; 27: 533–543. 17. Lim YW, Lee SU. On the color image segmentation algorithm based on the thresholding and the fuzzy cmeans techniques. Pattern Recognit 1990; 23: 935–952. 18. Maglogiannis I, Zafiropoulos E, Kyranoudis C. Intelligent segmentation and classification of pigmented skin lesions in dermatological images. In: Antoniov G, Potamias G, Spyropoulos C and Plexousakis D, eds. Advances in artificial intelligence, Vol. 3955 of lecture notes in computer Science. Berlin: Springer, 2006: 214–223. 19. Zagrouba E, Barhoumi W. A preliminary approach for the automated recognition of malignant melanoma. Image Anal Stereol 2004; 23: 121–135. 20. Delgado Gomez D, Clemmensen LH, Ersbll BK, Carstensen JM. Precise acquisition and unsupervised segmentation of multi-spectral images. Comput Vis Image Underst 2007; 106: 183–193. 21. Fleming MG, Steger C, Zhang J, Gao J, Cognetta AB, Pollak I, Dyer CR. Techniques for a structural analysis of dermatoscopic imagery. Comput Med Imaging Graph 1998; 22: 375–389. 22. Elbaum M, Kopf AW, Rabinovitz HS et al. Automatic differentiation of melanoma from melanocytic nevi with multispectral digital dermoscopy: a feasibility study. J Am Acad Dermatol 2001; 44: 207–218. 23. Ganster H, Pinz A, ohrer RR, Wildling E, Binder M, Kittler H. Automated melanoma recognition. IEEE Trans Med Imaging 2001; 20: 233–239. 24. Lee TK. Measuring border irregularity and shape of cutaneous melanocytic lesions. PhD thesis, Simon Fraser University, Burnaby, Canada 2001. 25. Rajab MI, Woolfson MS, Morgan SP. Application of region-based segmentation and neural network edge detection to skin lesions. Comput Med Imaging Graph 2004; 28: 61–68. 26. Blum A, Luedtke H, Ellwagner U, Rassner RSG. Digital image analysis for diagnosis of cutaneous melanoma. Development of a highly effective computer algorithm based on analysis of 837 melanocytic lesions. Br J Dermatol 2004; 151: 1029–1038. Address: Kajsa Mllersen Norwegian Centre for Integrated Care and Telemedicine University Hospital of North-Norway PO Box 35 N-9038 Troms Norway Tel: 1479 778 3940 Fax: 1477 775 4098 e-mail: [email protected]

407