1
REAL-TIME INSPECTION SYSTEM FOR PRINTED CIRCUIT BOARDS Nam-Hyeong Kim, Student Member, IEEE, Jae-Young Pyun, Student Member, IEEE, Kang-Sun Choi, Student Member, IEEE, Byeong-Doo Choi, and Sung-Jea Ko, Senior Member, IEEE Department of Electronics Engineering, Korea University, Seoul, Korea Email:
[email protected] ABSTRACT The PCB inspection system presented in this paper can detect defects including the breaks in the wires and short circuit and is significantly faster when compared to the existing techniques. The proposed inspection method is based on referential matching between the stored reference image and the test (observed) image. Block matching is performed to solve the misalignment in referential matching. In order to reduce the computational complexity, we implement the proposed algorithm using the Single Instruction Multiple Data (SIMD) instructions, so called SSE2.
while overcoming the weaknesses of each of the methods. These methods cover a large variety of defects compared to either referential or non-reference methods alone. In this paper, we present an advanced PCB inspection system based on the referential method which can effectively detect the defects of the PCB on the conveyor. To achieve the accurate alignment, we propose a fast image processing algorithm which can estimate both image shift and tilt. A shading correction method is also proposed to compensate for the non-constant intensity of the illumination source. II. THE REAL-TIME INSPECTION SYSTEM A. Proposed PCB inspection system
I. INTRODUCTION As the density of circuit patterns becomes finer, especially in the boards used for computers and wireless communication systems, pattern inspection by human takes a great deal of time and is not reliable due to inevitable human errors. Meanwhile, existing automatic optical inspection (AOI) systems have proved their efficiencies in detecting defects of the PCB. Existing automated inspection methods for the PCB can be classified as: (i) The referential methods perform a real point-to-point (or feature-to-feature) comparison. The reference image obtained from a golden board or CAD data is stored in an image database. A tested image is subtracted from the reference image to generate the difference image. Any parts of the pattern different from the corresponding parts of the reference pattern are considered defects. This method is very simple and computationally efficient. However, it requires particularly accurate alignment. (ii) The non-referential methods do not need a reference pattern. They are based on the idea that a pattern is defective if it does not conform with the design specification standards. These methods are also called design-rule verification methods or generic property verification methods. Applying the design-rule verification process directly to the image patterns is a time consuming process and needs a great computing power to meet user’s requirement of inspection time. (iii) The hybrid methods use both referential and nonreferential techniques. Thus, they exploit the strengths
Fig. 1 depicts the block diagram of the proposed PCB inspection system.
Frame Grabber "ideal" PCB
PCB to be inspected
The test image
The reference image
Alignment
Difference
Thresholding Thresholding
Dilation
Decision
Pentium 4
Fig. 1. Block diagram of proposed PCB inspection system.
When PCB’s are transported on the conveyor belt, the
2
shift of the PCB’s is inevitable. The mechanical guiding system can reduce the displacement within 2mm, about 4 pixels in a scanned image, which corresponds to ±0.2◦ tilt. In our method, an image processing algorithm is used to further reduce the translational and rotational displacement. The aligned test image is subtracted from the stored reference image and the resulting difference image is converted into a binary image using a thresholding algorithm. Finally, morphological operators are applied to the resulting binary image to detect the defects. Next, we describe each processing unit in detail. B. Alignment
start
angle = 0.0
Rotation
BM
angle = angle + 0.05 angle = 0.3
no
yes
For the proposed alignment method, we select the positions of five blocks of size 32 × 32 in the reference image, each of which includes a distinct feature pattern. To estimate the translational displacement between the reference PCB image and the test image, the full search block matching (FSBM) algorithm is used. The FSBM method looks for a best pair of indices (m, n) such that the error norm, the sum of absolute difference (SAD), is minimized ε = min kX − Y (m, n)k, m,n
(1)
P where ||x|| = k |xk | and the vectors X and Y (m, n) represent the pre-selected block and the possible candidate blocks in the scanned image, respectively. The proposed alignment algorithm determines the displacement vector using the FSBM with the five blocks as follows: 5 X (2) ε = min kXi − Yi (m, n)k, m,n
angle = -0.05
Rotation
BM
angle = angle - 0.05 angle = -0.3
no
yes
Compensate the test image
end
Fig. 2. Flowchart of the proposed alignment algorithm.
i=1
where the vectors Xi and Yi (m, n) represent the ith preselected block and the ith possible candidate blocks in the scanned image, respectively. Although the rotation is fairly small, the discrepancies between the stored reference image and the test (observed) image are fairly large. To account for orientation differences between these two images, these differences are compensated for before the reference image and the test image are compared. For the rotational alignment, the orientation difference minimizing the error norm in (2) is determined as the search area in the test image is rotated by 0.05◦ from 0.05◦ to 0.3◦ and by −0.05◦ from −0.05◦ to −0.3◦ . Fig. 2 illustrates the flowchart of the proposed alignment algorithm. Fig. 3 shows the result of the alignment. Fig. 3(a) shows the difference image between the reference and the test image compensated by the translational alignment method, where the misalignment is seen in the left lower region. Fig. 3(b) shows that the proposed translational and rotational alignment improves the matching performance. C. Thresholding - Binarization Thresholding is a method to convert a grayscale image into a binary image so that objects of interest are separated from the background. In the PCB inspection system, the
trace pattern of the PCB is segmented as objects of interest by the thresholding method. The thresholding algorithm involves simply setting all pixel values to a value of zero or one depending on whether the gray level at each pixel is greater than or less than a threshold. The difficulty with this method is to select an optimal threshold value. Especially when the area scan camera is used, it is very hard to acquire images of uniform intensity due to the characteristics of the lighting source and camera lens. Fig. 4 shows an example of the non-uniformity of illumination which was obtained by scanning a clean copper board before any manufacturing process. It is seen that the illumination (or brightness) even in the “clean copper” image varies with the position. This indicates that a fixed threshold value may result in unsatisfactory results. To solve this problem, we use an shading correction algorithm to compensate for the non-constant intensity of the light source as follows: Let mi and Ti be the average graylevel of pixels in region Ri and the optimal threshold value obtained by the method in [6]. Fig. 5(b) and 5(d) show region R2 in the “clean copper” image and the PCB reference image, respectively. We calculate mi ’s from the “clean copper” image and Ti ’s from the reference image. From the two sets {mi } and {Ti }, the minimum and the
3
reference image as tracking the boundary of the dilated candidate. To follow the boundary of the dilated candidate, we use 8 directional chain codes method. A cross point is defined as the change of intensity values along the boundary of the dilated candidate in the binary reference image from 255 (white, trace) to 0 (black, background) or from 0 to 255. If the number of cross points is greater than three with respect to a candidate, the dilated defect candidate is judged as a defect such as open and short. III. Experiments and Conclusion (a)
(b)
Fig. 3. Absolute difference between the reference and the test image. (a) With translational alignment only. (b) With translational and rotational alignment.
(a)
(b)
Fig. 4. The non-uniformity of illumination. (a) The image of a clean copper board. (b) The intensity contour line of (a).
maximum are selected. We denote the minimum and maximum of {mi } ( {Ti } ) by mmin and mmax ( Tmin and Tmax ). The “clean copper” image is transformed into a shade-corrected image using the intensity transformation, s = T (r), as shown in Fig. 5(f), where r and s are the intensity of pixels before and after processing, respectively. Each pixel value of the shade-corrected image is used as a threshold for the binarization of the corresponding pixel of the reference image. Fig. 6 shows the thresholding results. Figs. 6(a) and 6(b) are the result associated with a fixed threshold value. Figs. 6(c) and 6(d) show the results of the proposed thresholding method with shading correction. It is seen that the proposed method produces a better result in binarizing the image. D. Detecting defects Fig. 7 presents how to detect open and short errors in the test image. Our system uses a simple comparison method between the reference and the test image. Fig. 7(a) and 7(c) are a part of the reference image and the corresponding test image, respectively. The difference image between these two images, Fig. 7(d), is binarized as shown in Fig. 7(e), where the white object are defect candidate. The binary difference image is enlarged by the morphological dilation operation. In fig. 7(g), To determine whether the dilated candidate in Fig. 7(f) is a defect or not, the decision algorithm counts the number of cross points in the binary
4 pairs of the 1008 × 1018 area scan camera and 1.4Ghz Pentium 4 PC with SSE2 technology are deployed in parallel. The SSE2 for Intel’s Pentium 4 processor is designed to accelerate software performance of 128-bit SIMD doubleprecision floating point and integer applications over earlier processor generations. Each pair operates separately and a “host PC” gathers information including the type, the position, and the shape of defects from the 4 PC’s. To inspect a PCB of size 303 × 509mm2 , 4 test images are captured 7 successive times with the interval of 0.6 second. The use of the simple referential inspection method and SSE2 enable to realize the real-time inspection. Experimental result shows that the test image is grabbed and processed by the proposed algorithm within 0.6 second. It is expected that the proposed system can be effectively used. References [1] [2] [3]
[4]
[5]
[6]
M. Moganti, C.H.Dagli, and Shou Tsunekawa, “Automatic PCB Inspection Algorithms: A survey,” Computer Vision and Image Understanding, vol. 63, no. 2, pp. 287-313, Mar. 1996. M. Moganti, “A subpattern level inspection system for printed circuit boards,” Computer Vision and Image Understanding, vol. 70, no. 1, pp. 51-62, Apr. 1998. Y. Hara, N. Akiyama, and K. Karasaki, “Automatic Inspection System for Printed Circuit Boards,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. Pami-5, no. 6, pp. 623630, Nov. 1983. P. Szolgay, and K. T¨ om¨ ordi, “Analogic algoritms for optical detection of breaks and short circuits on the layouts of printed circuit boards using CNN,” International Joural of Circuit Theory and Applications, no. 6, pp. 103-116, Nov. 1999. Block-Matching In Motion Estimation Algorithms Using Pentium 4 Processor SSE2 Instructions, AP-940 ftp://download.intel.com/design/perftool/cbts/appnotes/sse2/ w me alg.pdf R. Jain, R. Kasturi, and B. G. Schunck, Machine Vision, McGraw-hill, New York, 1995.
4
R2
R3
R1
R5
R4
(a) m2
(b)
(c) (a)
(b)
(c)
(d)
T2
(d)
(e)
Fig. 6. Binarized images of two regions in a reference image. (a) R1 without shading correction. (b) R2 without shading correction. (c) R1 with shading correction. (d) R2 with shading correction.
s 255
s = T(r) Tmax Tmin
0
mmin
mmax
255 r
(f) Fig. 5. Shading correction. (a) Regions of interest. (b) An example of R2 region in the clean image. (c) The histogram and the mean value of (b). (d) An example of R2 region in the reference image. (e) The Histogram and the threshold value of (d). (f) Gray-level transformation function.
5
(a)
(b)
(c)
(d)
(e)
(f)
Cross point (from trace to background )
Cross point (from background to trace) (g) Fig. 7. Defect decision process. (a) A part of the reference image. (b) The corresponding part of binary reference image. (c) The corresponding part of test image. (d) Difference image. (e) Binary difference image. (f) Dilated image. (g) Overlaid binary reference image with (f).
Fig. 8. Automatic inspection system for printed circuit board.
