2014 Tenth International Conference on Signal-Image Technology & Internet-Based Systems
Multiclass Classification of Weld Defects in Radiographic Images Based on Support Vector Machines
Faiza Mekhalfa
Nafaa Nacereddine
Signal Processing and Imagery Division Welding and NDT Research Center, CSC Chéraga, Algeria
[email protected]
Signal Processing and Imagery Division Welding and NDT Research Center, CSC Chéraga, Algeria
[email protected]
in radiographic images. We review briefly some research works on automatic multiclass classification of weld defects. Jacobsen [4] used neural methods to discriminate between two defects and absence of defects. Aoki and Suga [5] applied multilayer perceptron artificial neural network (MLP-ANN) to identify five defect types. However, their work lacks of practical applications due to small database used. Wang and Liao [6] used MLP-ANN and fuzzy expert system to classify six types of defects. Liao [7] proposed a fuzzy K-nearest neighbor for classification of defects into six possible classes. Da Silva et al. [8] used two layer neural networks to identify six defect types and study the accuracy of defect classification. Nacereddine and Tridi [9], Mirapeix et al. [10] presented quantitative and qualitative results to classify four and three defects respectively using principal component analysis and artificial neural network. In Vilar et al. [11] a multilayer feedforward artificial neural network is employed to differentiate between five defects and false positives. Valavanis and Kosmopoulos [12] presented another approach for defect classification using geometric and texture features. Support vector machines SVMs are relatively new generation of techniques for classification problems. Based on the principle of structure minimum according to the statistical learning theory developed by Vapnik and Chervonenkis in 1964 [13], SVMs can solve practical problems encountered with traditional classifiers in the aspects of small training samples, nonlinearity, high dimension and local extreme values [14]. Originally, SVM was introduced for binary classification [15-17]. However, the major parts of practical applications involve multiclass classification, particularly in defect classification. Many approaches have been proposed by researchers to implement SVM for multiclass classification such as one versus one, one versus all, directed acyclic graph and error corrected output coding [18]. In this work, we develop an efficient defect classification system using multiclassifier SVM based on one versus all and one versus one approaches. A set of geometric features is defined to characterize defects in radiographic images. It consists of shape measures such as compactness, elongation, symmetry and the two first moments of Hu [19]. The extracted features are used as inputs to the designed SVM classifiers. The results of SVM classification are presented and compared with multilayer perceptron artificial neural networks (MLPANN) results [9].
Abstract—In this paper, we present through the experimental study the use of support vector machines (SVMs) in the automatic classification of weld defects in radiographic images. SVM is a machine learning tool used for classification and regression and it is well known for binary classification, but there are many approaches for multiclass classification, the most popular are one versus all and one versus one. The performance of the proposed classification system is evaluated using hundreds of radiographic images representing four types of defects. The experimental results show that the SVM classifier is an efficient automatic weld defect classification algorithm and can achieve high accuracy percent and is faster than multilayer perceptron artificial neural network (MLP-ANN). Keywords-classification of weld defects in radiographic images, support vector machine; one versus all approach; one versus one approach; multilayer perceptron artificial neural network
I.
INTRODUCTION
Radiography technique used in non-destructive testing is well-established for weld defect detection and identification. Traditionally, the film interpretation is evaluated by experts in radiography who inspect each film and detect the presence of possible flaws in order to be identified and measured. This task is difficult due to low dimensions of some defects and bad quality of radiographic films. In addition, the human interpretation is subjective and influenced by the experience and the knowledge of the inspector [1]. Computer aided analysis systems are developed to carry out automatic interpretation of radiographic images, in order to improve the accuracy and the objectivity of evaluation and make the inspection process more reliable and faster [2]. The automatic radiographic inspection system consists of five stages: film digitization, preprocessing, image segmentation, feature extraction and weld defects classification [3]. The digitization of radiographic films can be carried out using a digital scanner. Then image preprocessing techniques are used to reduce the noise effects and improve the contrast in raw radiographic images, to enable subsequent segmentation process. Feature extraction is necessary to obtain a set of features that can describe the characteristics of welding defects, and pattern classification methods are needed to analyze feature data and identify weld defect types. The defect classification stage is one of the most important steps in the automated inspection system. Many traditional classifiers are implemented to identify defects 978-1-4799-7978-3/14 $31.00 © 2014 IEEE DOI 10.1109/SITIS.2014.72
1
the defect region. Consequently, it is a particular of interest to represent this region by a binary function:
The paper is organized as follows: Section II describes the proposed methodology to develop the weld defect classification system based on SVM. In section III we present the experimental results to classify various flaws in radiographic images of welds where performance comparison between SVM and MLP-ANN classifiers is carried out, and we conclude our work in section IV. II.
f ( x, y ) 1 f ( x, y) 0
if ( x, y) interior (C ) otherwise
where (x,y) are the coordinates of the image pixel. D. Feature Extraction In this stage, the procedure generates an input vector called descriptor for each defect candidate which is then used as input to multiclass classifiers. The descriptor must be stable, concise, unique, accessible, and invariant to geometric transformation (translation, rotation, and scaling) [23]. In this work, a set of invariant geometric features which characterizes the defect shape is proposed. They are selected because of their shape discrimination capacities and their direct relationship to the weld defects type. The proposed descriptor contains shape measures derived from some geometric parameters, and the two first moments of Hu. So, for each weld defect, the geometric parameters (area (A), perimeter (P), centroid G( x , y ) , orientation angle (α), principal axes of inertia, width (W) and length (L) of the surrounding rectangle, partial areas (S1, S2, S3 and S4), semi-major and semi-minor axes (a,b) of the equivalent ellipse), are calculated (see Figure 1.); and the shape geometric features used in the classification problem are defined as follows Compactness: It measures the object shape that is calculated by:
METHODOLOGY
The proposed automatic system developed for defect classification consists of five major stages: Film digitization, preprocessing, image segmentation, feature extraction and defect classification. A. Film Digitization The digitization of radiographic films is an essential part of weld defect classification system. The majority of radiographic films used in this experiment were provided by the International Institute of Welding (IIW). These radiographic films are generally dark and their density is rather large. Hence, the choice of an appropriate resolution of scanning and an acceptable quality of digitizing plays an important role in the system performance. Here, we have used scanner Agfa Arcus II (800 dpi, 256 gray levels). The resulted radiographic images have a low contrast between the background and the weld defect regions containing some granularities and high noise due to the digitization process and the type of film used in industrial radiographic testing. B. Preprocessing After digitizing the films, it is common practice to adopt a preprocessing stage for the digitized radiographic images to reduce noise and improve contrast. In order to simplify the task, we could start by selecting the region of interest (ROI), which can be considered as the radiogram region where the interpreters suspect the presence of imperfections. The selection of the ROI saves the operator to make processing on the useless parts of the image, permitting reduction of the computing time and avoiding the computation to use irrelevant regions of the image, which can negatively influence the output results. After ROI selection, it is recommended to apply contrast enhancement, with the aim to improve the quality of radiographic images by highlighting the useful information while leaving the unimportant information intact [20].
Comp
4A P2
Elongation: Computes the ratio between the width to the length of the smaller rectangle that encloses the defect called as Minimal bounding box: Elong 1
W L
A L W
b a
(4)
Symmetry: Symmetry for binary image is defined as: Sym SymH SymV
2
(3)
Eccentricity: is obtained from the ratio of the minor axis to the major axis in the object equivalent ellipse as follows: Ecc 1
(2)
Rectangularity: is the ratio between the area of the defect and the area of minimal bounding box: Re ct
C. Image Segmentation Segmentation plays a fundamental role to extract useful features for further image analysis. It is a process in which the object of interest (weld defect) is extracted from the background of the image. Several image segmentation techniques have been developed in the literature [21]. In general, they involve edge detection, thresholding and clustering to segment the regions or detect the edges in the image. The extraction of defect contours from radiographic images has received enormous amount of attention recently. Because of image noise, low contrast, intensity inhomogeneity and blurry contours, a region based active contour model using adaptive B-spline is used here. The detail of this approach is given in [22]. As result, the boundary C of the defect is recovered and it suffices to consider the region inside C as the image foreground, i.e.
(1)
(5)
where SymH and SymV are respectively the horizontal and vertical symmetries given by:
E. Defect Classification Once a shape feature descriptor is calculated, the next step deals with the weld defect classification. Among multiclass classifiers found in the state of the art, support vector machine classifiers have received recently a great attention from the research community in pattern recognition.
Sym H inf( S 2 S 3 , S1 S 4 ) / sup(S 2 S 3 , S1 S 4 ) SymV inf( S 3 S 4 , S1 S 2 ) / sup(S 3 S 4 , S1 S 2 )
The invariant Hu moments: 1 20 02 2 2 (20 02 ) 411 ( 3 ) 2 (3 ) 2 30 12 21 03 3 4 (30 12 ) 2 (21 03 ) 2 2 2 5 (30 312 )(30 12 ) (30 12 ) 3(21 03 ) 2 2 (321 - 03 )(21 03 ) 3(30 12 ) (21 03 ) ( - ) ( ) 2 ( ) 2 20 02 30 12 21 03 6 411(30 12 )(21 03 ) 2 2 7 (321 30 )(30 12 ) (30 12 ) 3(21 03 ) (3 )( ) 3( ) 2 ( ) 2 12 30 21 03 30 12 21 03
Theory of Support Vector Machine Support vector machine (SVM) is promising classification method, originally developed for binary classification; the basic theory of SVM consists to draw an optimal hyperplane separating data points of different classes (see Figure 2.). Both separable and non-separable problems are handled by SVM in the linear and nonlinear cases. The idea behind SVM is to map the original data points from the input space to a high dimensional one, called feature space. The mapping is done by a suitable choice of Kernel function [25]. To implement SVM on image classification we are given a certain number p of training data, each data has two parts: the d-dimensional vector of image features and the corresponding labels of classes (either +1 or -1):
(6)
1 ( p q ) / 2 00
pq pq / , p q 2,3,... normalized central moments and
where
are
is the covariance matrix of the object, where n is the number of the object contour pixels. In this paper we consider only the two first invariant moments 1 and 2 which gives a description of the pixels repartition on the defect area. The normalized invariant moments ~i , i 1, ,7 were defined by [24]: ~
i 7
i 1
(8)
SVM maps the d-dimensional input vector x from the input space to the dh-dimensional feature space using a nonlinear function (.) : d d h . The separating hyperplane in the feature space is then defined as
n n 11 i ( xi x )( yi y ) 20 i ( xi x ) 2 n n 02 i ( yi y ) 2 11 i ( xi x )( yi y )
i log i / log 2 i
S ( xi , yi ) / xi d , yi 1,1, i 1, p
the
w. ( x) b 0 / w dh , b
(9)
The classifier should satisfy the condition of existence of w and b such that:
yi (w. ( xi ) b) 1
1/ 2
(10)
However, in practical applications data of both classes are overlapping, which makes a perfect linear separation impossible. Therefore, a restricted number of misclassifications should be tolerated around the margin. The resulting optimization problem for SVM where the violation of the constraints is penalized is given as:
(7)
N 1 2 min 2 w C i i 1 such that y ( w. ( x ) b) 1 , i i i ξ i 0 , i 1,..., p
(11)
where i is the relaxation factor considering classification error and C is the cost parameter that controls the tradeoff between allowing training errors and forcing strict margins (i.e. empirical risk minimization). Typically, the constrained optimization problem is referred as the primal optimization problem, which can be
Figure 1. Illustration of some geometric parameters [23].
3
written in the dual space by Lagrange multipliers αi ≥ 0. The solution should maximize the following expression: L( w, b, i ,
i
p 1 2 w C i 2 i 1
)
p
i
an advantage that the number of binary classifiers to construct equals number of classes. However, there are some drawbacks: first, for large training data sets and increasing class number, the memory requirement is very high; second, when the number of training samples in each class is equal, the ratio of training samples of one class to rest of the classes will be 1: (M-1), during the training phase. This ratio shows that training samples sizes will be unbalanced. Another approach consists in building a set of one versus one classifiers by dividing an M class problem into M(M -1)/2 binary problems. The learning phase of the classifiers is done using as training data only a subset of instances from the original data set which contain any of the two corresponding class labels. In the testing phase, every testing data tests M(M -1)/2 SVM classifiers and the class with the object samples that occurs the most is the predicting classification using voting method. The number of training data vectors required for each classifier is reduced than the previous method. Hence, this method is considered more symmetric. Moreover, the memory required is much smaller. However the main drawback of this method is the increase in the number of classifiers as the number of class increases.
(12)
( yi ( w. ( xi ) b) 1)
i 1
and the dual problem is given as p
L( )
i
i 1
subject to 0 !
1 p 2 i , j 1 i
i
j
! C et
y i y j K ( xi , x j )
i
yi 0
(13)
where the Kernel function K ( xi , x j ) corresponds to the inner product belonging to transformation space:
K ( xi , x j ) ( xi ). ( x j )
(14)
There are three kind of commonly used Kernel functions: linear kernel function, polynomial kernel function and radial basis function (RBF) [26]. Finally, the SVM classifier function can be written as p
f ( x) sign(
i
y i K ( x i . x j ) b)
III.
EXPERIMENTAL RESULTS
For this study, totally 344 radiographic images covering four types of defects (86 images for each defect type) were extracted from the base of the standard films provided by International Institute of Welding (IIW). They include crack (CR), lack of penetration (LP), porosity (PO) and solid inclusions (SI), the prior knowledge about these defect types were established by the human experts in the visual interpretation of the radiographic films. A sample of original radiographic images of each defect type and their ROIs are shown in Figure 3. Figure 4 shows Solid inclusion radiographic image, its ROI and the corresponding binary image. Then, a set of 344 feature vectors containing each seven components was randomly divided into training and testing subsets. The feature parameters [ ~1 ,~2 , Ecc, Comp, Rect, Elong, Sym] should be rearranged and considered as components of input vectors for our designed SVM classifiers. For multiclass classification based on SVM, we have adopted the two approaches: one.vs.all and one.vs.one with linear kernel function. SVM classifiers were trained by sequential minimal optimization (SMO) algorithm, which solves the SVM quadratic problem (QP) by decomposing it into QP sub problems and solving analytically the smallest possible optimization problem involving two Lagrange multipliers at each step [27]. The performance of SVM classifiers was compared to multilayer perceptron artificial neural network (MLPANN) [9]. The output layer of neural network classifier consists of four nodes corresponding to the four classes of defects and the number of neurons in the hidden layer is equal to 10. The training stage was carried out using backpropagation (BkProp) algorithm [28], with sigmoid function used to model the neuron activation. The learning rate of the steepest descent algorithm of BkProp is equal to 0.6 and the convergence error is taken equal to 10-6. All the algorithms were developed in Matlab environment and tested on Intel i7-2600 computer with a
(15)
i 1
Figure 2. Illustration of linear SVM.
SVM for multiclass classification Numerous approaches have been suggested to use SVM for multiclass classification, but there are two most popular approaches, one versus all and one versus one [18]. In one versus all method, classification functions are constructed between each class and the remaining classes. It divides an M class problem into M binary problems. The learning step of the classifiers is done by the whole training data, considering the data vectors from the particular class as positive and all others samples as negative. During the testing phase, data vectors are classified by finding margin from linear separating hyper plane. The final output is the class that corresponds to the SVM with the largest margin. This multiclass method has
4
3.4 GHz processor and 4 Gigabytes of RAM. The classification performance was evaluated by means of the total accuracy which expresses the fraction of cases that are correctly classified, the computation time and the confusion matrices. The results of the classification accuracies and the prediction times for the three classifiers with different numbers of input images in training and testing phases are shown in TABLE I and TABLE II respectively. TABLES III, IV and V give the confusion matrices, where the total data was divided in 80% for the training and 20% for the test.
TABLE II.
CLASSIFIERS
Method SVM one.vs.all SVM one.vs.one MLP-ANN
TABLE III. Classified /real CR LP PO SI TABLE IV. Classified /real CR LP PO SI TABLE V. Classified /real CR LP PO SI
Figure 4. Solid inclusion and its binarized version using the method in [22]. The initial and final contours are represented in red and cyan respectively.
TOTAL ACCURACY FOR SVMS AND MLP-ANN CLASSIFIERS
Method SVM one.vs.all SVM one.vs.one MLP-ANN
Training set percentage 20%
40%
60%
80%
93.98%
95.07%
95.33%
95.40%
95.59%
96.41%
96.73%
96.96%
92.27%
94.87%
95.58%
92.64%
Training set percentage 20%
40%
60%
80%
0.2224 s
0.2009 s
0.1754 s
0.1374 s
0.0407 s
0.0488 s
0.0590 s
0.0631 s
12.9630 s
12.0347 s
19.0000 s
7.61950 s
CONFUSION MATRIX FOR SVM (ONE.VS.ALL) CR
LP
PO
SI
0.9896 0.0513 0 0
0.0104 0.9487 0 0
0 0 0.9499 0.0722
0 0 0.0501 0.9278
CONFUSION MATRIX FOR SVM (ONE.VS.ONE) CR 0.9890 0.0352 0 0
LP
PO
0.0111 0.9648 0.0002 0
0 0 0.9735 0.0489
SI 0 0 0.0263 0.9511
CONFUSION MATRIX FOR MLP-ANN CR 0.8823 0 0 0
LP
PO
0.1765 1.000 0.0002 0
0 0 0.7059 0
SI 0 0 0.2941 1.0000
From TABLES I and II, we can say that SVM classifier with one.vs.one approach is promising in terms of overall accuracy and algorithm speed where, it presents the most outstanding performance compared to the other classifiers. For the various ratios of training data, the classification rate is high and remains quite high (around 96%) even with a few numbers of training samples. For the less performing SVM approach using the one.vs.all strategy, the recognition scores are comparable with MLP-ANN, except when 80% data are used in training, where SVM classifier achieves a higher score (95.40%) whereas, the MLP-ANN provides 92.64%. Moreover, as shown in execution times, the SVM one.vs.all algorithm is much faster than MLP-ANN and for this reason; it can be more practical, particularly, in real-time oriented applications. The confusion matrices given in TABLES III, IV and V present the correctness of each class and the percentage of confusion of a class with the others. For example, in TABLE III, 98.96% of data for CR class are correctly classified and 1.04% is mismatched with LP class. We notice that many cracks (CR) are classified as lack of penetration (LP) and vice versa, which is reasonable considering their visual similarity in radiographs. The accuracy for the classes LP and SI shown in TABLE V, obtained by MLP-ANN, is equal to 100%, i.e. is ideal. However, the classes CR and PO presented a large confusion with the other classes. This can be explained by the problem of overtraining in ANNs, i.e. when the proportion of the trained data compared to the tested one is too large.
Figure 3. A sample of radiographic images and their ROIs.
TABLE I.
COMPUTATIONAL TIME FOR SVMS AND MLP-ANN
5
IV.
CONCLUSIONS
[15] B. Boster, I. Guyon, and V. Vapnik, “A training algorithm for optimal margin classifiers”, Proceedings of the fifth annual ACM Workshop on Computational learning Theory, 1992, pp. 144-152. [16] V. Vapnik, The nature of statistical learning theory, Springer Verlag, 1995. [17] V. Vapnik, Statistical learning theory, Wiley, New york, 1998. [18] J.C. Platt, N. Cristianini, and J. Shawe-Taylor, “Large margin DAGs for multiclass classification”, In Advances in Neural Information Processing Systems, vol.12. MIT Press, pp. 547-553. [19] M.K. Hu, “Visual pattern recognition by moment invariants”, IRE Tran. On Information Theory, vol.8(2). 1962, pp. 179-187. [20] N. Nacereddine, L. Hamami and D. Ziou, “Thresholding techniques and their performance evaluation for weld defect detection in radiographic testing”, Machine Graphics Vision, vol. 15(3-4). 2006, pp. 557-566. [21] R.C. Gonzalez and R.E. Woods, Digital Image Processing, 3rd Edition, 2007. [22] N. Nacereddine, L. Hamami, D. Ziou, and A.B. Goumeidane, “Adaptive B-spline model based probabilistic active contour for weld defect detection in radiographic imaging”, Image processing & communications challenges 2, AISC 84, Springer Verlag, 2010, pp. 189-197. [23] N. Nacereddine, D. Ziou, L. Hamami, “Fusion based shape descriptor for weld defect radiographic image retrieval”, Int J Adv Manuf Technol , Springer Verlag , vol. 68 (9-12). 2013, pp. 28152832. [24] C.H. Tech and R.T. Chin, “On image analysis by the method of moments”, IEEE Trans PAMI, vol. 10 (4). 1979, pp. 496-513. [25] J. Luts, F. Onjeda, R. Van de Plas, B. de Moor, S.V. Huffel, and J.A.K. Suykens, “A tutorial on support vector machine-based methods for classification problems in chemometrics”, Analytica Chimica Acta, vol. 665. 2010, pp. 129-145. [26] C.J.C Burges, “A tutorial on support vector machines for pattern recognition”, Data Min. Knowl. Discov., vol.2. 1998, pp.121-167. [27] J. Platt, “ Fast training of support vector machines using squential minimal optimization”, In Advances in Kernel Methods: Support Vector Learning (B. Scholkopf, C.J.C. Burges, and A.J. Smola, eds.), MIT Press, 1999, pp. 185-208. [28] R. Rojas. The Backpropagation algorithm, Neural networks, Springer Verlag, 1996, pp. 151-184.
In this paper, two kinds of multiclass classification based on SVM have been developed and successfully used in weld defect classification system. Four types of defects were classified using SVM based on geometric shape features. The proposed method which adopts one.vs.one approach for multiclass classification is computationally effective and yield promising accuracy classification results and outperforms the multilayer perceptron artificial neural network classifier. The aim of the future works is to investigate the method in the large dataset of radiographic images and improve the classification accuracy by extracting other features. In addition, thanks of the reduced running time for SVM classifiers, they can be prone to real time applications, especially in digital radiography dedicated to nondestructive testing. REFERENCES [1] [2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13] [14]
G. Edward, Inspection of welded joints, In ASM Handbook welding, vol. 6. ASM International, 1993. C. Schwartz “Automatic evaluation of welded joints using image processing on radiographs”. International Conference, Proceeding of American Institue of Physics, vol. 657. 2003, pp. 689-694. D. Mery, R.R. da Silva, L.P. Calôba, and J.M.A. Rebello, “Pattern Recognition in the automatic inspection of aluminum casting”, International Symposium on Computed Tomography and Image Processing for Industrial Radiography, Berlin, June 2003, pp 223235. C. Jacobsen and U. Zscherpel, “Automated evaluation of digitized radiograph with neural methods”, In Proceedings of Computed Tomography for Industrial Application and Image Processing in Radiology, Berlin, March 1999, pp. 141-152. K. Aoki and Y. Suga, “Application of artificial neural network to discrimination of defect type in automatic radiographic testing of welds”, In ISI International, vol. 39(10). 1999, pp. 1081-1087. G. Wang and T.W. Liao, “Automatic identification of different types of welding defects in radiographic images”, NDT&E International, vol. 35(8). 2002, pp. 467- 476. T.W. Liao, “Classification of welding flaw types with fuzzy experts systems”, Expert Systems with Applications, vol. 25(1). 2003, pp. 101-111. R.R. Da Silva, M.H.S. Siqueira, M.P.V. de Souza, J.M.A. Rebello, and L.P. Caloba, “Estimated accuracy of classification of defects detected in welded joints by radiographic tests”, NDT&E International, vol. 38. 2005, pp. 335-343. N. Nacereddine and M. Tridi, “Computer-aided shape analysis and classification of weld defects in industrial radiography based invariant attributes and neural networks”, Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis ISPA, 2005, pp. 88-93. J. Mirapeix, P.B. Garcia-Allende, A. Cobo, O.M. Conde, and J.M. Lspez-Higuera, “Real time arc welding defect detection and classification with principal component analysis and artificial neural networks”, NDT&E International, vol. 40(4). 2007, pp. 315323. R. Vilar, J. Zapata, and R. Ruiz, “An automatic system of classification of weld defects in radiographic images”, NDT&E International, vol.42(5). 2009, pp. 467-476. I. Valavanis, and D. Kosmopoulos, “Multiclass defect detection and classification in weld radiographic images using geometric and texture features”, Expert Systems with Applications, vol. 37. 2010, pp. 7606-7614. V. Vapnik and A. Chervonenkis, “A note on one class of perceptrons”, Automation and Remote Control, vol. 25. 1964. N. Cristianin and J. Shawe-Taylor, Support vector machines, Cambridge University Press, 2000.
6
