Spatial correction in dynamic photon emission by affine transformation matrix estimation 1
S. Chef1,2, S. Jacquir2, P. Perdu1, K. Sanchez1 and S. Binczak2
CNES, DCT/AQ/LE, Bpi 1414, 18 Avenue Edouard Belin, 31401 Toulouse, France Le2i UMR CNRS 6306, University of Burgundy, 9 Avenue Alain Savary, 21000 Dijon, France Phone: +33380399035 Email:
[email protected]
2
Abstract- Photon emission microscopy and Time Resolved Imaging have proved their efficiency for defect localization on VLSI. A common process to find defect candidate locations is to draw a comparison between acquisitions on a normally working device and a faulty one. In order to be accurate and meaningful, this method requires that the acquisition scene remains the same between the two parts. In practice, it can be difficult to set. In this paper, a method to correct position by affine matrix transformation is suggested. It is based on image features detection, description and matching and affine transformation estimation.
I. INTRODUCTION Defect localization often is the critical part of the failure analysis process. Optical techniques are tools of choice as invasiveness is minimized and few machining of the sample is required. Among these techniques, photon emission microscopy has been used for almost thirty years [1]. Around a decade and a half ago, a better understanding of the physical phenomena implied in the process and an improvement of the detection and acquisition hardware had made possible the recording of the positions and time of emitted photons during transistors' switching [2]. This is known as dynamic photon emission or Time Resolved Imaging (TRI). Regardless of the acquisition mode (static or dynamic), the common process used to highlight differences between databases acquired on a good and on a bad device relies on spatial comparison. In static mode, acquired photons are displayed in the two dimensions (x,y). A subtraction between the images is used to isolate potential faulty nodes. In dynamic mode, the photons characterized in the three dimensions (x,y,t) are projected in the (x,y) plan and the same operation is applied to find candidates. Between two samples, it can be difficult to ensure the exact same scene. Indeed, it requires that the two samples have the same orientation and the camera is located on the exact same region of the circuit. There also can be some changes along the z-axis. Camera should be set slightly closer or further from the component in order to get focus. A potential cause of this is the sample preparation that cannot be exactly the same for every part (substrate thickness, etc) or the mounting of the device on test board. As a result, the field of view can be a bit different. All of this leads to a lot of remaining pixels after the subtraction and finding the spot related to the defect can be challenging.
One solution to overcome the limitations of the spatial projection is to directly use a 3D process, like in [3]. Unfortunately, if the defect does not exhibit an emission with singular statistical properties, a comparison between databases shall also be considered. Once again, a correct positioning of photons is required so that databases are comparable. The difference between the two acquisition scenes can be interpreted as a geometrical transformation. Indeed, the difference of orientation between the two samples is a geometric rotation, camera position in the (x,y) plan is a translation. The change in the field of view is a scale modification. All of these transformations are linear and can be modeled by an affine transformation matrix. In this paper, a process to estimate this matrix using photon emission data is reported. Once it has been estimated, it becomes possible to make some corrections and have comparable data to find candidate locations for the defect. As long as the algorithm has been designed for 2D signals (i.e. images), it can be applied on both static and dynamic emission. The remaining of this paper is organized as follows: in the second section, the method is introduced in details. Some results are given in the third section. In the fourth section, the application boundaries are discussed. Finally, a conclusion finishes this paper. II. METHOD DESCRIPTION A. General Overview The natural process to find how an object has moved between two moments consists in finding some points of interest and see how they have changed in the same reference system. As photon emission is a stochastic phenomenon, it seems meaningless to try to compare the coordinates of single photons in both databases. On the other hand, emission should occur from the same nodes and for a long enough acquisition time, in the (x,y) projection, a node belonging to both databases should have similar properties in size and shape. As a consequence, images of photon emission have some specific interest points which might be detectable by some image processing algorithms. Once a specific point has been found for one image, we need to find its match in the second one to estimate the move parameters. It can be done by a characterization using its neighborhood. A vector of n dimensions named descriptor is built. Two points having similar vectors in the descriptor space
are more likely to be the same point of interest in the two images. After matching, the motion can be estimated and expressed as an affine transformation matrix. If this matrix is named T, its expression is the following : cos sin
0 0 , (1) 1 where A is the scale factor, r is the rotation angle and TX and TY are translation vector coordinates. Finally, the inverse transformation is applied to the image in order to restore the image. To sum-up here are the key steps of the process: 1) Features detection. 2) Descriptors building. 3) Features matching. 4) Motion estimation. 5) Inverse transformation
sin cos
B. Interest points detection The identification of matching points between images of a moving scene has drawn the attention of the image processing community for a long time. For instance Harris suggested an algorithm to detect and built descriptor of corners in [4] in 1988. In 2004, Lowe reported an algorithm to find scale space maxima and named Scale-Invariant Feature Transform (SIFT) [5]. One of the drawbacks of the method is its computational cost. In order to speed things up, a new algorithm called Speeded-Up Robust Features (SURF) has been introduced by Bay et al. in 2008 in [6]. A common approach to detect interest points is to build a scale space representation of an image and search for local maxima. Usually, this representation requires two steps: first, the image is convolved with a Gaussian filter of parameter and then, in a second time, is sub-sampled. The larger the , the coarser the result. Instead of doing so, the SURF algorithm computes the scale space transform using several tricks to accelerate the process. The Gaussian filter responses are first approximated with box-filters. The aim is to get results closed to the ones with Gaussians but at a shorter computation time because of the sparseness of the filter response. Another point is that the convolution operation is a costly one. The use of integral images helps to compute it faster. An integral image is defined as ∑ ∑ I i, j , where I(i,j) denotes a pixel of the , actual image. Considering these two tricks, for each pixel X = (x,y), a Hessian matrix H at scale s is computed : , , (2) , , , where Lij denotes the result of the convolution of the image in X with the filter in the direction ij. Finally, a blob map at each scale s is build using the determinant of the matrix H: ,
det
where w is a weight used to balance the determinant equation. Points of interest are found by searching for the blob map maxima at the different scales.
.
.
,
(3)
Fig. 1. Superposition over pattern of the two acquisitions.
C. Descriptor building and features matching For each key point identified, a descriptor of 64 or 128 components, depending on user's choice, is build from a neighborhood of 20 pixels. A smaller descriptor makes computation faster but brings a loss of precision. It is based on a 2D wavelet transform using Haar wavelet. Once again, the wavelet is chosen so that it allows a faster process thanks to integral images. Matching between images' points is achieved by searching for nearest neighbor in the descriptor space. Indexing structures like kd-tree can be considered to accelerate the process. D. Affine transformation : Motion Estimation Once matching has been done, a matrix of transformation (translation + rotation + scale factor) is found by using robust estimator such as Random Sample and Consensus (RANSAC) [7], M-Estimator Sample Based Consensus (MSAC) [8]. The second algorithm is an upgrade of the first one. RANSAC is based on the maximization of inliers (points that fit to the model). A set of points is randomly chosen and a first estimation of the parameters is performed. Then the model is applied to the data. Those which fit to the model are labeled as inliers. The process is repeated on a new random set including freshly labeled inliers and the model is updated. It continues until the number of inliers is maximized. The threshold for the minimum number of inliers can be difficult to set. Especially, if the potential number of inliers is low because there are only too few matching points, the estimation can be erroneous. Several algorithms tried to overcome the parameter selection issue. Among them, MSAC choose to penalize outlier according to one model and give a score to inliers according to how well they fit. There is a finer quantification of a model quality. At the end of the process, the estimated matrix has the following shape : ,
where the matrix coefficients are real values. The parameters of the transformation can be estimated as follow: arctan arctan , ̂ , 2 . 2 ̂ The hat in the expression above indicates an estimated value.
right sides are in single color (red on the left and cyan on the right), meaning that these parts of the picture have only been acquired for one of the devices. As a consequence, no advance study can be carried out for these locations. No comparison is possible as no data exist for both databases but only for one of them.
(a) (a)
(b)
Fig. 2. Superposition of the emission images of the two acquisitions before (left) and after correction(right). Circles and crosses indicates the location of interest points
III. APPLICATION AND RESULTS A. First Example The process is applied on TRI images. This kind of images can be complicated to deal with because of their low resolution. The images have been acquired using Hamamatsu TriPHEMOS on two microcontrollers built in 0,15 m technology. The superposition of the two emissions over pattern images are available in Fig. 1. Hereafter, left image will be referred as component A and the right one as component B. It can be seen that there is a small rotation of the device between the two acquisitions, in addition to a small translation. At first sight, a change of scale (parameter A in the matrix T expression in (1)) does not seem to occur so we can expect it to be close to one and be neglected. In order to highlight the differences, the two emission images have been superimposed in Fig. 2 (a). A preprocessing step composed of a median filtering was required. Without it, the results were absurd. The detected interest points have been marked with red circles for component A and green crosses for component B. Thirteen correspondences have been kept after SURF detection and matching. More than the half of them seems to indicate different transformation tensors. Superposition after inverse transform is available in Fig. 2 (b). Despite the high value of outliers, MSAC algorithm estimates a 22,2 translation vector ; a rotation angle 1,06 °; 36,7 a scale factor ~ 1. When there is overlapping of pixels of the same intensity, it appears in white in Fig. 2 (b). As it can be seen, the geometric transformation matrix found by the MSAC algorithm is correct as there is overlapping for most of the acquired area. Left and
(b)
Fig. 3. Superposition of images in the case of suspicious spots (circled in red) without (a) and with (b) spatial correction.
In order to quantify the correction quality, the mean square error (MSE) between the two images is computed. It is done before and after spatial correction. The MSE is defined as : MSE
,
,
,
(4)
where x and y are the dimensions of the images and A and B are the two images to compare. In this case, before spatial correction and with intensity normalization (meaning that I(x,y) is in [0;1]) and spatial correction, MSEB = 1,23 .10-2 a.u. whereas after transformation recovery, MSER = 9,0 .10-3 a.u. B. Second Example The databases have acquired on the same devices, with the same setup but at another location. As the center of rotation of the device was closer to this area, the spatial difference between the two databases was smaller. Superposition is available in Fig. 3 (a). This case is a bit more particular than the previous one as there is one spot that exist in one image and not on the other. It is circled in red in Fig. 3 (a) and (b). Even if the position of the two samples seems to be quite the same, because of a small misalignment, some differences exist at other locations. To discard any other potential difference, the process for spatial correction is applied. This time, good results have been achieved without any filtering. Twenty seven matching features have been found. The transformation matrix estimation gives the following 3,7 parameters: r = -0,038°; ; 1,0038 ~ 1 . 4,9 Once again, the values of the translation vector are given in pixels and the scale factor does not have any unit. The images superposition after spatial recovery is reported in Fig. 3 (b). The suspicious spot can better be seen. The mean square error computation gives the following results: MSEB = 4,4 .10-3 a.u.
and MSER = 2,9.10-3 a.u. Both scores have been calculated after images intensity normalization. Many differences between two acquisitions can exist due to the random nature of the photon emission process and the small changes of samples orientation. As a conclusion of this second application, even if a spot outcomes from the unprocessed images superposition, a spatial correction can help to discard the small meaningless differences and focus on the true defect related candidate spot. IV. DISCUSSION ON PREPROCESSING
Number of matching points
25
20
15
V. CONCLUSION 10
5
0
5
10
15
20 Filter Size
25
30
35
Fig. 4. Number of matching points as a function of the filter size for the images from Fig. 1 and 2.
0.06 0.05 0.04 MSER (a.u.)
resolution of the effective area of the TRI image in quite small (around 200x200 pixels) so the largest filter uses almost 25 % of the image, which is huge. As the figure shows, the larger the filter size, the smaller the number of matching points. Because the estimation has been performed with fewer points, recovery should be less reliable. A study to find an optimal filter size according to the mean square error has been carried out and results is reported in Fig. 5. Before any processing, the MSE value was of 1,23 .10-2 a.u. A minimum of 6,1 .10-3 a.u. is found for filter size of 16 pixels but the image after recovery looked like there was a shear transform, which is different from the simple rotation and translation considered in this paper. On the other hand, the recovered images corresponding to filter sizes with high MSE in Fig. 5 was effectively poorly recovered. As a conclusion, MSE can possibly be a metric to quantify how bad is the recovery but is not enough to quantify its goodness.
Spatial comparison is a key procedure to analyze data from photon emission and finding candidate spots for defect localization. Although it requires a good matching between the two scenes. Because it can be difficult to ensure when the acquisitions are not performed on the same sample, a method for spatial recovery based on interest points detection and characterization has been studied. Thanks to these features, an affine transformation matrix is estimated and applied to correct the images. It has been shown it can help to emphasize a suspicious spot, even if no correction was required at first sight. In order to grant success of the recovery, a preprocessing can be useful but the parameters remain empirically defined. In addition, the small number of detected features challenges the estimation. ACKNOWLEDGMENT
0.02
Authors would like to thank Renesas Electronics (databases), Hamamatsu Hpks for their technical support (TriPhemos) and the Regional Council of Burgundy for its financial support.
0.01
REFERENCES
0.03
0
5
10
15
20 Filter Size
25
30
35
Fig. 5. MSE after inverse transformation as a function of the filter size.
As it was the case for the first application example, some prior processing like filtering is sometimes required to get good results. One of the reason why is there are too many wrong matching between points. Filter like the median one acts like a low pass filter and the resulting image resolution is lowered. For the images used in Fig. 1, a study of the number of matching points found as a function of the filter size has been carried out. Results are available in Fig. 4. The size of the neighborhood used for the filtering is varying from 3 pixels to 33 pixels. The
[1] N. Khurana, C.L. Chiang, “Analysis of product hot electron problems by gated emission microscopy,” Proceedings of the 24th Annual IEEE Reliability Physics Symposium, pp. 189-194, 1986. [2] J.C. Tsang, J.A. Kash, “Picosecond hot electron light emission from submicron complementary metal-oxide-semiconductor circuits,” Applied Physics Letters, Vol. 70, No. 7, pp. 889-891, 1997. [3] S. Chef, P. Perdu, G. Bascoul, S. Jacquir, K. Sanchez, S. Binczak “News statistical post-processing approach for precise fault and defect localization in TRI database acquired on complex VLSI,” Proceedings of the 20th IEEE International Symposium on the Physical and Failure Analysis of Integrated Circuits, pp. 136-141, 2013. [4] C. Harris, M. Stephens, “A combined corner and edge detector,” Proceedings of the 1988 Alvey Vision Conference, pp. 23.1-23.6, 1988. [5] D.G. Lowe, “Distinctive image features from scale invariant keypoints,” International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004. [6] H. Bay, T. Tuytelaars, L. Van Gool, “Speeded-up robust features (SURF),” Computer vision and image understanding, Vol. 110, No. 3, pp. 346-359, 1997.
[7] M.A. Fischler, R.C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, Vol. 24, No. 6, pp. 381-395, 1981. [8] P.H. Torr, A. Zisseman, “Robust computation and parametization of multiple view relations,” Proceedings of the 6th International Conference on Computer Vision (ICCV), pp. 727-732, 1998.
