A novel rotation-invariant template matching based on ... - IEEE Xplore

Proceedings of the 9th International Symposium on Mechatronics and its Applications (ISMA13), Amman, Jordan, April 9-11, 2013

A NOVEL ROTATION-INVARIANT TEMPLATE MATCHING BASED ON HOG AND AMDF FOR INDUSTRIAL LASER CUTTING APPLICATIONS Osman Arslan1, Berkan Demirci2, Halis Altun3, N.Serdar Tunaboylu4 Department of Compuıter Engineering, Mevlana University, Konya, TURKEY Department of Electrical & Electronics Engineering, Mevlana University, Konya, TURKEY 1,2,3

4

[email protected] [email protected] ABSTRACT

A new real-time rotation-invariant template matching is proposed for industrial laser cutting applications. The technique is based on Histogram of Oriented Gradients (HOG) algorithm to remove rotational angle before the template matching process. It exploits the HOG and the Average Magnitude Difference Function (AMDF) features for rotationinvariance. Since HOG features are robust against illumination effects, the proposed algorithm is suitable for harsh industrial environments. The final aim of this study is to devise a method to detect the contour of an object, which will automatically be cut from a sheet of material. 1.

INTRODUCTION

The laser cutting have been recently found a widespread industrial applications. In literature, there are many related studies on laser cutting [1-9]. Despite its widespread usage, laser cutting with machine-vision ability is rarely encountered in literature. An ability to automatically detect the contour of the object for laser cutting process will be beneficial and indispensable in some applications. In a recent study, ([10] Rao and Liu, 2011) a laser cutting machine with such an ability has been devised. In this study we propose a new approach based on the Histogram of Oriented Gradients (HOG) and the Average Magnitude Difference Function (AMDF) to develop a rotation and illuminationinvariant method to detect the contour of the object to be cut. The proposed method will be used to develop smart machinery for the footwear industry, which will utilize image processing and soft-computing techniques to identify the contour of a pattern to be cut. The HOG algorithm proposed by Shashua et all [11], Dalal and Triggers [12] and is widely used in object and pattern recognition due to its ability to exhibit superior performance in many applications under different conditions [13,14]. The technique exploits the geometric properties of the object. Detection of an object based on its geometric properties seems to be a fundamental sub-problem in many classification and recognition applications. There are plenty of works in the literature which uses geometric properties in many

978-1-4673-5016-7/13/$31.00 ©2013 IEEE

[email protected] [email protected]

applications such as hand gesture recognition [15], traffic sign recognition [16-19], human recognition [16][20], and recognition of several objects [21]. The final aim of this study is to develop an image processing module to integrate into a laser cutting process in footwear industry. The proposed module is a template matching which is based on the HOG and the AMDF algorithm. The algorithm, first of all, works on a raw image obtained directly from the camera without any preprocessing. Secondly the HOG features are extracted applying horizontal and vertical sobel operator on the raw image. The features are then normalized to deal with scaling problem. The AMDF is utilized to detect the rotational angle of the objects present in the image and remove the angle for a successful template matching. In the last step the template matching is performed to properly detect the contour of the object. The paper is organized as follows. Section 2 and 3 explain and elaborate the techniques, namely HOG and AMDF. Section 4 introduces the proposed template matching. Section 5 gives the results obtained by the proposed method and section 6 presents the results and draw conclusion. 2.

HOG FEATURES FOR ROTATION DETECTION

Despite there are lot of methods proposed in literature to detect the contour of an object, we propose to utilize a template matching based on HOG features and AMDF. HOG features exploit the characteristics of pixel orientation (θ) and magnitude values. It is proposed by Shashua [11] and Dalal [12]. The steps of extracting HOG features from given image could be listed as: First of all, a horizontal and vertical edge operators are applied on the image I to extract horizontal edge Ix and vertical edge, Iy separately as shown in Equation 1 and 2. In this step, horizontal and vertical Sobel filters, which are well-known edge operators in computer vision, namely Sx and Sy, are used to obtain Ix and Iy. In the second step, the magnitude of pixel gradient, Gxy, are calculated as given in Equation 3. Also the corresponding angle of the gradient is found by Equation 4. In the last step, a histogram

ISMA13-1


is constructed based on the angle of the gradients using Equation 5. The symbol in Equation 5 is the angle bin width and in our experiment it is selected as So there are 180 bins in the histogram.

I x  I  Sx

(1)

I y  I  Sy

(2)

G xy  I x2  I y2

(3)

θ xy = arctan h(i) 

Ix Iy

(4)

 θ   Gxy  i   xy   ( x , y )I    



(a)

(b) 800

600

600

400

400

200

200

Then magnitude and rotation angle (Gxy and θxy) of the the gradient at the pixel (x,y), is calculated from Ix and Iy using Equation 3 and 4. Finally a histogram is obtained using the gradients at specific angle which is determined by the number of bins. A further consideration would be to normalize the HOG values in order to obtain a scale-invariant algorithm. Normalization can be performed by dividing all bin values with the maximum of the bins which is formalized in equation (5) as follows

(5)

In our implementation of HOG, we do not implement local histogram regions, as defined in the original HOG features, instead we consider the whole image as a single region and the calculation of the gradient and the corresponding angle are performed accordingly. The implementation of the HOG algorithm requires a predefined number of bins, which represent the rotation angle of the pixel gradient to form a histogram. The number of bins is set as 180 in our experiment and a histogram is shown in Figure 1 for the object to be cut under this settings. As it is seen from the figure, a rotation of 30o corresponds to a shift in the histogram which can be exploited to obtain a rotation-invariant method.

800

will be detected using the proposed algorithm) and (b) the object with a 30o rotation (c) HOG of original image (d) HOG of the rotated image. The rotation cause the HOG feature shift to a position corresponding to 30o

ℎ𝑜𝑔𝑖 =

ℎ𝑜𝑔𝑖 𝑚𝑎𝑥(ℎ𝑜𝑔)

30

60

90

(c)

120

150

180

0 0

3.

AMDF AS A DISSIMILARITY

MEASURE

1

60

90 120 150 180

(d)

Figure 1. (a) original object (The contour of the object

OF

The AMDF is commonly used in speech processing to determine the periodicity of the waveform under consideration. Its common application is to find the pitch period of the speech waveform [22]. The AMDF is basically a quantity which indicates the dissimilarity between the speech waveform in a frame and its lagged version. If the similarity between two waveforms is high, the AMDF produces a minimum value at a lag which corresponding to the period. Therefore we may utilize the AMDF to measure the dissimilarity between two HOG features. A minimum value of AMDF indicates, in this case, that the similarity between two HOG features is increased at an amount of shift which corresponding to the rotation angle. Another alternative to measure similarity/dissimilarity between the HOG feature could be the Auto Correlation Function (ACF). However, compared to AMDF, ACF certainly introduce a heavy computational load due to huge number of multiplication used in its implementation. As a result, ACF might would be not suitable algorithm for the realtime implementation of the proposed algorithm. This is the constraint which led us to use AMDF as a measure of the dissimilarity/similarity between HOG features. AMDF is defined in (6) as follows 𝑓𝐴𝑀𝐷𝐹𝑖𝑗 (𝜏) = ∑𝑁 𝑖=1|𝑆𝑖 − 𝑆𝑗−𝜏 | 𝜏 = 0,1,

30

(5)

where N is the bin size. This operation makes it sure that the maximum value of the angle bins will be set to 1.

𝑁

0 0

𝑖 = 1, … 𝑁

𝑁−1

(6)

where N is the number of bins in the histogram, 𝑆𝑖 is the histogram of reference image, 𝑆𝑖−𝜏 the lagged version of histogram of the input image with a lag of 𝜏. The value of lag 𝜏, range from 1 to N-1, which corresponds 0

ISMA13-2


degree to 360 degree, respectively. In this study, a HOG feature vector of the template will be used as a reference. The reference HOG feature will be compared to the HOG feature extracted from the given input image. The minimum value obtained from AMDF is the indication of rotational angle. Figure 2 shows an example for two HOG features as given in Figure 1. It is seen that the minimum value of AMDF is correctly found the rotation imposed on the image. As the each bin width is 2o, the minimum occurs at the 15th bin, which corresponds to 30o. 180 160 140 120 100 80 60 40 20 0

0

20

40

60

80

100

120

140

160

180

Figure 2. The minimum of AMDF occurs at 15th bin, which corresponds to 30o in our experiment 4.

TEMPLATE MATCHING

Template matching [23] can be described as a technique to find a given object template in an image. There are numerous applications of template matching from the quality control in industrial application [24] to navigation system for mobile robots [25] or edge detection in image processing applications [26]. In our study we will use template matching to find the contour of a template for laser cutting application. As its simplicity we have used a method which is sometimes referred to as Linear Spatial Filtering as follows. 𝑅(𝑥, 𝑦) =

∑𝑥′,𝑦′ (𝑇( ′ , ′ )∙𝐼 ′( + ′ , + ′ )) √∑𝑥′,𝑦′ 𝑇( ′ , ′ )2 ∙∑𝑥′,𝑦′ 𝐼( + ′ , + )2

(7)

where I(𝑥, 𝑦) is a search image and (𝑥, 𝑦) represent the coordinates of each pixel in the search image; T(𝑥 , 𝑦 ). Block diagram of the proposed template matching method is given in Figure 3. After detecting the rotational angle, the template image will be rotated so that a match might be possible. If a match is found, a scalable vector graphics file will be created and it will be sent in order to direct the laser cutting machine.

ISMA13-3

Figure 3. Block diagram of the proposed method for template matching


6. 5.

REAL-TIME RECOGNITION OF GEOMETRICAL OBJECTS FOR AN INDUSTRIAL SYSTEM

In this section, the result obtained from the proposed new matching algorithm will be illustrated. As it is illustrated, the proposed template matching is able to detect the angle between the rotated image and the original temple in the region of interest. The effect of the rotation will be a shift in the HOG features. This rotation should be introduced (or in some case, remove) for a successful matching between the template and the image. After this correction phase, the template matching will be easily adaptable to locate the (x,y) position which provides the maximum correlation. An example is given in Figure 4-a where an image with objects illustrated. After removal of rotation angle, the correlation will provide the maximum as it is illustrated in Figure 4-b.

CONCLUSION

A new real-time rotation-invariant template matching is proposed for industrial laser cutting application. The technique utilizes the HOG and AMDF algorithms to remove the rotational angle even before the template matching process. It exploits the HOG feature and the AMDF for the rotation-invariance. The final aim of this study is to devise a method to detect the contour of an object, which will automatically be cut from a sheet of material. For this purpose a scalable vector graphic is generated. The solution to the problems encountered in object recognition for an industrial application are proposed in this study using the HOG. Eliminating the noisy gradients by the proposed method, the performance of the system is improved and furthermore an improvement in the speed is obtained for the creation of HOG identifiers by limiting the calculation area, leading to a system suitable for real-time applications. Further improvements are seemed to be achievable in more complex classifiers and a high quality camera system is employed.

7.

ACKNOWLEDGEMENT

This study has been carried out under the support of SAN-TEZ Project 01063.STZ.2011-2 sponsored by the Ministry of Science, Industry and Technology of Turkish Republic. (a)

(b) Figure 4 (a) Original image on which the proposed template matching will be applied (b) Correlation map which shows the highest correlation between the template and the image. The highest point (x,y) encircled by a red circle.

ISMA13-4


REFERENCES [1] Dubey A.K. and Yadava V, Laser beam machiningA review, International Journal of Machine Tools and Manufacture 48 (2008), 609-628. [2] Pandey A.K ve Dubey A.K, Intelligent Modelling of Laser cutting of thin sheet, International Journal of Modeling and Optimization, 1(2), 107-112, 2011. [3] Anuar A.B, Investıgatıon of Laser Cuttıng Parameters on Surface Qualıty of Staınless Steel, MSc, Universiti Teknologi Malaysia, (2010). [4] Hu J., Zhang Z., Luo J., Sheng X., “Simulation and experiment on standoff distance affecting gas flow in laser cutting”, Applied Mathematical Modelling 35 (2011) 895–902. [5] Ahn D.G., Byun K.W. and Kang M.C., “Thermal Characteristics in the Cutting of Inconel 718 Superalloy Using CW Nd:YAG Laser”, J.Mater. Sci. Technol., 26(4), 362-366, (2010). [6] Golnabi H., Bahar M., “Investigation of optimum condition in oxygen gas-assisted laser cutting”, Optics & Laser Technology 41 (2009) 454–460. [7] Leone C., Pagano N., Lopresto V., De Iorio I.; Solid state Nd:YAG laser cutting of CFRP sheet: influence of process parameters on kerf geometry and HAZ, INTERNATIONAL COMMITTEE ON COMPOSITE MATERIALS, ICCM-17 Edinburgh, Scotland July 27-31, 2009 (2009). [8] Karatas C., Keles O., Uslan I., Usta Y., “Laser cutting of steel sheets: Influence of workpiece thickness and beam waist position on kerf size and stria formation”, Journal of Materials Processing Technology 172 (2006) 22–29. [9] Zhang G.F ., Zhang B. , Deng Z.H., and Chen J.F., “An Experimental Study on Laser Cutting Mechanisms of Polycrystalline Diamond Compacts”, Annals of the CIRP Vol. 56/1/2007. [10] Rao H. Ve Liu N, Recognition and Positioning Technology of Intelligent Lazer Cutting Machine, Microcomputer_Information_2011, http://en.cnki.com.cn/Article_en/CJFDTOTALWJSJ201104034.htm [11] Shashua A., Gdalyahu Y., and Hayon G., “Pedestrian detection for driving assistance systems: Single-frame classification and system level performance”, In Proceedings of IEEE Intelligent Vehicles Symposium, 2004. [12] N. Dalal and B. Triggs., “Histograms of oriented gradients for human detection”, In C. Schmid, S. Soatto, and C. Tomasi, editors, International Conference on Computer Vision and Pattern Recognition, volume 2, pages 886–893, June 2005. [13] Bay, H., Tuytelaars, T., and Gool, L. V. (2006). Surf: “Speeded up robust features”, In Leonardis, 2006.

[14] Zhu Q., Mei-Chen Y., Kwang-Ting C., Avidan, S., “Fast Human Detection Using a Cascade of Histograms of Oriented Gradients”, Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on, 2006. [15] Ong E., Bowden R., “A Boosted Classifier Tree for Hand Shape Detection”, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition , 2004. [16] D.M. Gavrila, V. Philomin, “Real-Tıme Object Detection For 'Smart' Vehicles”, Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference ,1999. [17] Bahlmann C., Zhu Y., Ramesh V., Pellkofert M., Koehled T., “A System for Traffic Sign Detection, Tracking, and Recognition Using Color, Shape, and Motion Information”, IEEE, 2005. [18] Miura J., Kanda T., Shirai Y., “An Active Vision System for Real-Time Traffic Sign Recognition”, 2000 IEEE Intelligent Transportation Systems Conference Proceedings, October 2000. [19] Loy G., Barnes N., “Fast shape-based road sign detection for a driver assistance system, Intelligent Robots and Systems”, 2004. (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference, 28 Sept.-2 Oct. 2004. [20] A. Broggi, M. Bertozzi, A. Fascioli, M. Sechi, “Shape-based Pedestrian Detection”, IEEE Intelligent Vehicles Symp, 2000. [21] Rong-Chin L., Wen-Hsiang T., “PerspectıveTransformation-Invariant Generalized Hough Transform For Perspective Planar Shape Detection and Matching”, Pattern Recognition, Voh 30, No. 3, pp. 383-396, 1997. [22] G. Muhammad (2011):‘Extended Average Magnitude Difference Function Based Pitch Detection”, The International Arab Journal of Information Technology, Vol. 8, No. 2,197-208 [23] R. Brunelli, Template Matching Techniques in Computer Vision: Theory and Practice, Wiley, ISBN 978-0-470-51706-2, 2009 ([1] TM book) [24] Aksoy, M. S., O. Torkul, and I. H. Cedimoglu. "An industrial visual inspection system that uses inductive learning." Journal of Intelligent Manufacturing 15.4 (August 2004): 569(6). Expanded Academic ASAP. Thomson Gale. [25] Kyriacou, Theocharis, Guido Bugmann, and Stanislao Lauria. "Vision-based urban navigation procedures for verbally instructed robots." Robotics and Autonomous Systems 51.1 (April 30, 2005): 6980. Expanded Academic ASAP. Thomson Gale. [26] Wang, Ching Yang, Ph.D. "Edge Detection Using Template Matching (Image Processing, Threshold Logic, Analysis, Filters)". Duke University, 1985, 288 Pages; Aat 8523046.

ISMA13-5