Surface signatures - IEEE Xplore

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE,

VOL. 24, NO. 8,

AUGUST 2002

1105

Surface Signatures: An Orientation Independent Free-Form Surface Representation Scheme for the Purpose of Objects Registration and Matching Sameh M. Yamany, Member, IEEE, and Aly A. Farag, Senior Member, IEEE AbstractÐThis paper introduces a new free-form surface representation scheme for the purpose of fast and accurate registration and matching. Accurate registration of surfaces is a common task in computer vision. The proposed representation scheme captures the surface curvature information (seen from certain points) and produces images, called ªsurface signatures,º at these points. Matching signatures of different surfaces enables the recovery of the transformation parameters between these surfaces. We propose using template matching to compare the signature images. To enable partial matching, another criterion, the overlap ratio is used. This representation scheme can be used as a global representation of the surface as well as a local one and performs near real-time registration. We show that the signature representation can be used to recover scaling transformation as well as matching objects in 3D scenes in the presence of clutter and occlusion. Applications presented include: free-form object matching, multimodal medical volumes registration, and dental teeth reconstruction from intraoral images. Index TermsÐSurface signatures, object registration, object matching, free-form surface representation.

æ 1

T

INTRODUCTION

HE registration process is an integral part of computer and robot vision systems and remains a topic of high interest in both fields. The importance of the registration problem in general comes from the fact that it is found in different applications, including surface matching [1], [2], 3D medical imaging [3], [4], pose estimation [5], object recognition [6], [7], [8], and data fusion [9], [10]. In order for any surface registration algorithm to perform accurately and efficiently, an appropriate representation scheme for the surface is needed. Most of the surface representation schemes found in literature have adopted some form of shape parameterization especially for the purpose of object recognition. One benefit of the parametric representation is that the shape of the object is defined everywhere, consequently, enabling the performance of high-level tasks such as visualization, segmentation, and shape analysis [11]. Moreover, this representation allows stable computation of geometric entities such as curvatures and normal directions. However, parametric representation are not suitable for presenting general shapes especially if the object is not of planar, cylindrical, or toroidal topology. Free-form surfaces, in general, may not have simple volumetric shapes that can be expressed in terms of parametric primitives. Dorai and Jain [6] have defined a free-form surface to be ªa smooth surface, such that the surface

. S.M. Yamany is with the Department of Systems and Biomedical Engineering, Cairo University, Egypt. E-mail: [email protected]. . A.A. Farag is with the Computer Vision and Image Processing Laboratory, CVIP Lab, R.m. 412 Lutz Hall, University of Louisville, Louisville, KY 40292. E-mail: [email protected]. Manuscript received 3 Nov. 2000; revised 2 Oct. 2001; accepted 3 Jan. 2002. Recommended for acceptance by Z. Zhang. For information on obtaining reprints of this article, please send e-mail to: [email protected], and reference IEEECS Log Number 113102.

normal is well defined and continuous almost everywhere, except at vertices, edges, and cusps.º Discontinuities in the surface normal or curvature and, consequently, in the surface depth, may be present anywhere in a free-form surface. Some representation schemes for free-form surfaces found in literature include the splash representation proposed by Stein and Medioni [12], the point signature by Chua and Jarvis [13], and COSMOS by Dorai and Jain [6]. Recently, Johnson and Hebert [14] introduced the spin image representation. Their surface representation comprises descriptive images associated with oriented points on the surface. Using a single point basis, the positions of the other points on the surface are described by two parameters. These parameters are accumulated for many points on the surface and result in an image at each oriented point which is invariant to rigid transformation. We introduced [1] another surface representation that uses the curvature information rather than the point density to create the signature image. Furthermore, we applied a selection process to select feature points on the surface to be used in the matching process. This reduction process solves the long registration time reported in the literature, especially for large surfaces. In this paper, we compare the sensitivity of both the point density and the curvature-based signatures in handling noisy data. The curvature-based signature image is generated by capturing the surface curvature information seen from each feature point. This image represents a signature of the surface at that point due to the fact that it is almost unique for each point location on the surface. Surface registration is then performed by matching signature images of different surfaces and, hence, finding corresponding points in each surface. For rigid registration, three point correspondences are enough to estimate the transformation parameters.

0162-8828/02/$17.00 ß 2002 IEEE

1106


VOL. 24,

NO. 8,

AUGUST 2002

Fig. 1. (a) Duality between triangulation and simplex mesh. (b) The circumsphere of radius R that includes the four points. (c) Cross section of the sphere and the calculation of the simplex angle.

This paper is organized as follows: The signature representation is described in Section 2. The points selection process is introduced in Section 3 and the matching process in Section 4. In Section 5, both the point density and the curvature-based representation are compared in terms of

handling noisy data. Matching signatures at different objects' scale is demonstrated in Section 6. Results and discussions are given in Section 7 and the paper concluded in Section 8.

Fig. 2. For each point P , we generate a signature image where the image axises are the distance d between P and each other point on the surface ~P and the vector from P to each other point. The image encodes the simplex . and the angle between the normal at P , U

YAMANY AND FARAG: SURFACE SIGNATURES: AN ORIENTATION INDEPENDENT FREE-FORM SURFACE REPRESENTATION SCHEME...

1107

Fig. 3. The figure shows three parametric objects (first row), a cylinder, a cone, and a torus. The theoretical SPS images corresponding to the three point locations shown in each object are demonstrated, respectively, in the following rows.

2

SURFACE SIGNATURE GENERATION

Our approach for fast registration is to establish a ªsurface signatureº for selected points on the surface, rather than just depending on the 3D coordinates of the points. The idea of obtaining a ªsignatureº at each surface point is not new [12], [13], [14]. The signature computed at each point encodes the surface curvature seen from this point using all

other points. This requires an accurate measure of the surface curvature at the point in focus. For parametric curves or surfaces, curvature measures can be obtained using the Frenet Frame values for the case of a curve or the Weingarten Map for the case of surfaces [15]. This requires the calculation of curve or surface derivative which is a complex operation and may introduce computa-

1108


VOL. 24,

NO. 8,

AUGUST 2002

Fig. 4. Examples of signature images taken at different point locations. Notice how the image features the curvature information. The dark intensity in the image represents a high curvature seen from the point while the light intensity represents a low curvature. Also, notice how different the image corresponding to a location from images of other locations is.

Fig. 5. (Top) Two examples of real objects, a statue and a speaker. (Bottom) Rendered views of the scanned 3D model of objects. The 3D statue model consists of 22,541 patches and the 3D speaker model consists of 16,150 patches.

tional errors to the representation scheme used. Moreover, such measures are hard to obtain for the case of unstructured free-form surfaces. Hebert et al. [16] used a simplex angle to describe changes in a simplex mesh

surface. We use the simplex angle to estimate the curvature value at points on a free-form surface. A free-form surface, in its general form, consists of unstructured triangular patches. There exists a dual form


1109

Fig. 6. The reduced set of points obtained for the statue and speaker models using (a) 0:66, (b) 0:1, (c) 0:3, and (d) 0:1.

consisting of unstructured simplex mesh as shown in Fig. 1a. A topological transformation associates a k-simplex mesh to k-triangulations or k-manifolds. This transformation works differently for vertices and edges located at the boundary of the triangulation from those located inside. The outcome of this transformation is a (k-p)-cell associated with a p-face of a k-triangulation [17]. In this work, we use a 2-simplex mesh form in the curvature calculation. Let P be a vertex of a 2-simplex mesh have three neighbors P1 ; P2 ; P3 . The three neighboring points define a ~P . They also lie on a circumscribe circle plane with normal U with radius r and the four points are circumscribed by a sphere with center O and radius R, as shown in Fig. 1b. The simplex angle shown in Fig. 1c is defined as [18]: sin

r ~P : ~1 U signPP R

sin : r

This is a polar implementation of the signature image and it can be easily converted into cartesian form. Fig. 3 shows examples of the effect of these two parameters using parametric objects. Also, we can notice that there is a

1

This definition is made with the assumption that the three neighbors are linearly independent, thus r 6 0. The simplex angle relates to the mean curvature H of the surface at the point P as follows: H

the surface relative to the point in study is unique. This is not true for surfaces of revolution (SOR). The signature image is generated as follows: As shown in Fig. 2, for each point P , defined by its 3D coordinates and ~P , each other point Pi on the surface can be the normal U related to P by two parameters: 1) the distance di k P ÿ Pi k and 2) the angle ! ~ ÿ1 UP :P ÿ Pi : i cos k P ÿ Pi k

2

The idea is to use this curvature measure and create a reduced representation of the surface at certain points. The reduced representation encodes the curvature values at all other points and creates an image. This image is called, a ªsignature image,º for this point because the change in curvature values with the distribution of all points forming

TABLE 1 A At Different , Their % from the Total M Points in the Model and the Time Taken to Obtain Them on an O2 SGI

1110


VOL. 24,

NO. 8,

AUGUST 2002

Fig. 7. (a) Case 1: Two telephone handsets with known transformation parameters. Notice how similar are the corresponding signature images. (b) Case 2: Part of a telephone handset, almost 50 percent of the original model, and example of the corresponding signature images. Partial matching is needed to establish the correspondence. However, this is not the case for all points, especially at the edge where the rest of the object is missing. Yet, the points are enough to estimate the transfomration parameters.

missing degree of freedom in this representation which is the cylindrical angular parameter. This parameter depends on the surface orientation which defies the purpose of having an orientation independent representation scheme. The size of the image depends on the object size but, for the sake of generalization, each object is normalized to its maximum length. At runtime matching, the scene image is normalized to the maximum length of the object in study. At each location in the image, the simplex angle i is encoded. Ignoring the cylindrical angular degree results in the case where the same pixel in the image can represent more than one 3D point on the surface. This usually occurs when the object has surfaces of revolution around the axis represented by the normal at the point P . These points have the same di and i and lie on the circle that has a radius di cosi and is distant by di sini from the point P along ~P . The average of their simplex angles is encoded the axis U in the corresponding pixel location.

Fig. 4 shows some signature images taken at different points on a statue and a phone handset. Each image uniquely defines the location of the point on the surface due to the encoded curvature information. In SOR, similar images can be obtained for different points. This can be expected as the registration of SOR objects is not unique and has infinite number of solutions.

3

SURFACE POINTS SELECTION

The concept of using special points for registration is not new. Thirion [3] used the same concept to register multimodal medical volumes and he used ªextremalº points on the volume edges (or ridges). Chua and Jarvis [13] used ªseedsº points in their matching approach. Stein and Medioni [12] used only highly structured regions in their approach. In many real-life objects, the majority of points forming the surface are of low curvature value. These points are redundant and do not serve as landmarks of the object. In


1111

TABLE 2 Comparison in Matching for Different Signature Image Size

threshold values, most of the landmarks of the object are still present in the set A. There are two cases, however, where the above analysis will fail. The first is when the surface is a plane or is a piecewise defined surface (e.g., a cube). In this case, for any > 0, the set A will be empty. This can be deduced from Fig. 1c when P falls in the plane formed by its neighbors. In this case, there exists no sphere circumscribing the four points (i.e., R 1), thus H 0. The second case is when the surface is part of a spherical, cylindrical, or toroidal shape. In this case, the curvature measure will be constant over the surface. Fortunately, in either case, these surfaces can be easily parameterized and the transformation parameters can be analytically recovered. Choosing a suitable threshold is still an unresolved issue in our representation since it depends on the nature of the object and its degree of complexity. For objects with many sharp edges, a large value of is enough to capture the object landmarks; however, for smoother objects, a large value of may not produce enough points to perfrom matching. A heuristic approach is currenlty used to define the value of for each case. This approach starts with a large value and according to the resulting set cardinality, this value is readjusted. The user can also visualize the selected points and can alter the value to capture most of the object's landmarks. Fig. 8. (Top) Clustering results for Case 1 in Fig. 7a where the correct registration parameters are known in advance. The image size used was 128 128. (Bottom) clustering resluts for Case 2 in Fig. 7b where partial matching is involved with an image size of 128 128.

this work, points of low curvature are eliminated and signature images are only generated for the set of remaining points. A test is also performed to eliminate spike points that have considerable higher curvature than its neighbors. These points are considered as noise. The simplex angle is used as a criterion to reduce the surface points and uses only a subset A S in the registration process, where S is the set of the simplex mesh points. A threshold is defined such that A contains the landmark regions of the surface. A fPi 2 Sj jsini j ; 0g:

3

Fig. 5 shows two examples of objects and their scanned models. Fig. 6 shows the reduced set of points A obtained for each model using different and Table 1 summarizes the values obtained. With low threshold values, more details about the object model are considered with considerable reduction in the set cardinality. Even with higher

4

SIGNATURE MATCHING

The next step in the registration process is to match corresponding signature images of two surfaces/objects or between a 3D scene and objects in a library. The ultimate goal of the matching process is to find at least three points correspondence to be able to calculate the transformation parameters. The benefit of using the signature images to find the correspondence is the use of image processing tools in matching, hence reducing the time taken to find accurate transformation. The developed matching engine should be simple based on the fact that the signature images of corresponding points should be identical in their content. Yet, due to the fact that 3D scanning sensors are noisy in nature and that the 3D scene may contain clutter or suffer from partial occlusion, a robust matching criteria is needed. One such criteria is template matching in which a measure defines how well a portion of an image matched a template. Let gi; j be one of our scene signature images and ti; j one of the library object (or original surface) signature templates and let D be the domain of definition of the template. Then, a measure of how well a portion of the scene image matches the template can be defined as [19]:

1112


VOL. 24,

NO. 8,

AUGUST 2002

Another more discriminating measure, based on the standard Euclidean distance, can be: En2

Fig. 9. Comparison of the correlation scores between the point density and the curvature-based signature representations. The object used is the statue corrupted with Gaussian noise at different SNR. The graph shows that both representations are robust to noise with the curvature based one having a better performance especially in low SNR.

Mm; n

X

X

jgi; j ÿ ti ÿ m; j ÿ nj:

4

j;i iÿm;jÿn2D

For surface signature matching, translation is not needed as the corresponding signature images have the same origin point at (0, 0), which means that only M0; 0 is calculated.

1 XX jgi; j ÿ ti; jj2 ; ND2 j;i 2D

5

where ND is the total number of pixels in the domain D. The domain D is defined over the template size. To enable partial matching, the matching measure is augmented by adding the overlap ratio O DDo , where Do is the domain of the overlapping pixels. Fig. 7 shows an example of two objects with known transformation parameters and another example where almost half of the object is missing. Fig. 8 shows the results of applying the matching measure En2 and the overlapping ratio O to determine the set of the best three candidates points correspondence. Table 2 shows that reducing the size of the signature image leads to a decrease in the number of correct points correspondence which means that more points are needed. Yet, the reduction in time with the smaller size is more suitable for real-time applications. It should be noticed that more reduction in the signature image size may lead to incorrect matching due to the averaging process. The end result of the matching process is a list of groups of likely three point correspondences that satisfy the geometric consistency constraint. The list is sorted such that correspondences that are far apart are at the top of the list. A rigid transformation is calculated for each group of correspondences and the verification is performed using a modified iterative closed point (ICP)

Fig. 10. Two sizes of the same object and their signature images at corresponding point locations.


1113

Fig. 11. (a) Two signature images at two different sizes. (b) The corresponding segmented images of the signatures in (a). (c) Another two examples of the same segmented signatures at smaller sizes.

technique [20]. This technique is faster than the original ICP [21] since it uses a grid transform and genetic algorithm to perform the search for the best fit. Groups are ranked according to their verification scores and the best group is refined using the modified ICP technique. More details on the recovery of transformation parameters using the correspondences groups and the modified ICP can be found in [22].

2.

5

3.

ERROR ANALYSIS

Matching signature images should produce perfect correlation for the same point on the same object independent from the object orientation in space. This would be true if the scanning process is noise-free, the scanned data has infinite resolution, and the signature image has continuous values. However, with the existence of scanning noise and the use of discrete sampling, three sources of errors in matching may arise: 1.

For the same object, the cloud of points resulting from two slightly different scanning procedures do not coincide exactly. This may result in slight variations in the relative distances and angles between the scanned points. Consequently, variations in the signature images for the same points may exist and this contributes to reduced correlation scores between them. However, due to the fact that the signature image is not continuous, these small variations can be tolerated by adjusting the image resolution. Also, using bilinear interpolation in the immediate pixel neighbors reduces the effect of such variations. Such a solution will slightly reduce the selectivity aspect of the signature image which results in a small increase in the number of points correspondences. However, geometrical consistency and the verification process are able to eliminate incorrect or less than perfect correspondences.

The variation in the scanned points coordinates and their relative distances and angles may also result in variations in normal orientations. As a consequence, the simplex angle and the value of the curvature at the same point will vary for different scans. This problem has small effect in the point density representation [14], yet it may have a damaging effect on the matching of signature images and establishing correspondences in the curvature-based representation. Another source of error due to the scanning process and the discrete nature of the signature images is the variation in density of scanned points with different scanner and/or object orientations. At smooth regions, this error is less frequent. But, at regions with high structural changes (typically regions near

Fig. 12. Result of matching a signature image at one point location with the corresponding signatures for the same point at different object sizes and also with signatures at another point location.

1114


VOL. 24,

NO. 8,

AUGUST 2002

Fig. 13. (a) Library object. (b) One signature image from the library. (c) The corresponding segmented signature. (d) Two 3D scenes with the object at different sizes. (e) Corresponding segmented scene signatures. (f) Matched object segmented signature at the exact scale.

discontinuities or high curvature regions), the density of scanned points may vary significantly with different object scan locations and orientations. For the point density representation, signature image at one point may vary significantly from the signature of the corresponding point on a different scan. However, for the curvature based representation, such error has a less damaging effect. This is due to the fact that point densities have no effect on this representation since it captures only the average curvature at these points. In an experiment to quantify the effect of the last two error sources on the correlation of signature images in both the point density and the curvature based representation, we injected Gaussian noise with different Signal to Noise Ratios (SNR) in the x, y, z components and the density of the scanned points of the statue object shown in Fig. 5.

Fig. 9 shows the correlation score between the signature images for corresponding points on the original and the corrupted objects. As demonstrated in the figure, both representations can handle noisy data efficiently with the curvature-based representation having a better performance than the point density representation. This is due to the fact that the corruption in the point density has no upper limit while the curvature value at any point is limited from ÿ to .

6

SIGNATURE MATCHING SCALE

AT

DIFFERENT OBJECTS'

Almost all the registration and pose estimation techniques reported in literature have ignored the scale dependency issue. This issue becomes important when trying to match a


1115

Fig. 14. The signature matching enabled fast recovery of the transformation parameter between these two models.

Fig. 15. Illustration of the effect of scene clutter and occlusion on the signature matching.

designed CAD model with real objects at different sizes. Also, scale independent matching is needed to overcome some of the problems associated with the type, location, and

orientation of the scanning sensor and the size of the reconstructed object. Different scanning sensors with different reconstruction algorithms (e.g., stereo, shape from

1116


VOL. 24,

NO. 8,

AUGUST 2002

Fig. 16. Examples of using the signature representation in object matching. A library of 10 objects is used. Some of these objects were scanned using a Cyberware 3030 laser scanner with a resolution of 1 mm. Others are obtained from CAD libraries.

X, etc.) may produce relatively different sizes of the same scanned scene. One reason for ignoring the scale recovery was that it is a very time consuming process especially for those who use local features to find correspondences (e.g., ICP [21], [2], COSMOS [6], point signatures [13]). In this section, we will show that the surface signature representation can be used to recover the scale while performing the matching. This makes the surface signature able to match objects under any similarity transformation. Since the signature image captures the curvature information of the object and since different sizes of the same object will have the same curvature information, it is

natural that the information coded in the image should be the same. The only difference will be that there will be a scaling effect in the d-axis in the image. Fig. 10 shows two sizes of the same object and the corresponding signature images at the same corresponding points. Both images were scaled to the same maximum distance which produced the scaling effect in the d-axis for the smaller size. Our first idea was to perform matching at different scale sizes. Yet, due to the fact that there is an averaging effect in the smaller size, matching corresponding pixels will produce larger error. Instead of matching the signature image pixelwise, the image is segmented into different curvature clusters. This


1117

Fig. 17. Examples of using the signature representation in object matching. A library of 10 objects is used. Some of these objects were scanned using a Cyberware 3030 laser scanner with a resolution 1mm. Others are obtained from CAD libraries.

segmentation removes the averaging effect and produces homogeneous regions for matching. The segmentation was performed using fuzzy thresholding where each pixel is assigned to a curvature cluster according to its membership value. Each signature image is segmented as follows: t0s i; j arg maxVk ti; j=s; k

k 2 1; c;

6

where t0s is the segmented output at scale factor s, Vk x is the fuzzy decision function, and c is the number of curvature clusters. The fuzzy decision function assigns the

pixel to belong to a certain curvature cluster if its fuzzy membership is larger than a certain value (in this work 0.7); otherwise, the value is computed from the average membership values of its neighboring pixels. This produces homogeneous segmented regions and reduces the effect of noisy pixels. Fuzzy membership functions were a priori defined to the segmentation procedure. Fig. 11 shows the result of segmenting signature images at different scale sizes. The matching criteria can be adjusted to include the scaling factor as follows:

1118


TABLE 3 Approximate Matching Time in Recognition on an O2 SGI

0

1 XX 1 En2 min@ jg0 i; j ÿ t0s i; jj2 A; s ND j;i 2D

7

where g0 is the segmented scene signature image and t0s is a scaled version of the segmented template image with scale factor s. We can store different sizes of the segmented template signature at each selected point location in the model object; however, this will increase the model signature library size and also these templates can be easily obtained from the original size during the matching. To limit the search space, an upper and lower limit on the scale factor is enforced and also the scale range is divided into predefined discrete levels. A more robust searching technique such as genetics algorithm could be used to find the optimal scale factor. Fig. 12 shows the results of matching signature images at different scale factors and compares them with matching a different location signature. The figure clearly demonstrates that best matching occurs at the correct scale factor only for the same signature. Although the scale range is less than 1, scaling can be performed on the scene signature as well as the model signature. This is needed when the object in the scene is larger than the model object in the library. Fig. 13 shows an example of matching a model object with two 3D scenes, each containing a different size of the object. The matching criteria was able to determine the correct object scale and point correspondences are established.

7

RESULTS

AND

DISCUSSIONS

We used the signature implementation in three applications. The first is object registration (an example of which is shown in Fig. 14) where two differently scanned objects are matched together. The signature registration was successful in recovering the transformation parameters. Also, the signature representation was used in matching objects in a 3D scene with their corresponding models in a library. The

VOL. 24,

NO. 8,

AUGUST 2002

proximity of the objects in the scene creates large amounts of clutter and occlusion. These contribute to extra and/or missing parts in the signature images. Using the signature polar representation, the effect of clutter, for many points, is only found in the third and/or fourth quadrant of the image as shown in Fig. 15. Examples of such application is shown in Figs. 16 and 17. Using the signature matching criterion, all of the models in the scene are simultaneously matched and localized in their correct scene positions. The models in the library and the 3D scenes are scanned using a Cyberware 3030 laser scanner with a resolution of 1mm. Some models (e.g., the duck, bell, and cup) were obtained from a CAD/CAM library. Table 3 shows the time needed to match objects in a scene using their signature templates. We compared the performance of our approach with the ICP and the spin image approaches. For the case of matching the statue object, it took 650 seconds using the ICP and 415 seconds using the spin image. Applying the feature points selection process with the spin image, it took 120 seconds to match the object. This is due to the fact that we needed more feature points to match the spin image compared to the points needed to match the signature image. The second application is multimodal medical image registration as shown in Fig. 18a. The dark surface represents the skin model reconstructed from the MR data and the light represents the skin model obtained from the CT. These models where obtained using a deformable contour algorithm that finds the outer contour in each slice and reconstructs a 3D mesh by connecting these contours. As the skin is modeled differently in the two image modalities, surface registration will only produce an initial registration. Other techniques, like maximizing the mutual information (MI) [23], can be used to enhance the result. The registration using signature and MI was much faster than using MI alone. The third application, shown in Fig. 18b, is in the dental teeth reconstruction [24]. The overall purpose of this system is to develop a model-based vision system for orthodontics to replace traditional approaches that can be used in diagnosis, treatment planning, surgical simulation, and for implant purposes. Image acquisition is obtained using intraoral video camera and range data are obtained using a 3D digitizer arm. A shape from shading technique is then applied to the intraoral images. The required accurate orthodontic measurements cannot be deduced from the resulting shape, hence, the need of some reference range data to be integrated with the shape from shading results. A neural network fusion algorithm [25] is used to integrate the shape from shading results and the range data. The output of the integration algorithm to each teeth segment image is a description of the teeth surface in this segment. The registration technique is then performed to register the surfaces from different views together.


1119

Fig. 18. Application of the signature representation for (a) multimodal medical volume registration and (b) teeth reconstruction from intraoral images (registration in dental application).

8

CONCLUSIONS

This paper proposed a new surface representation of freeform surfaces and objects. The proposed representation reduces the complexity of the registration and matching problems from the 3D space into the 2D image space. This was done by capturing the surface curvature information seen from feature points on the surface and encoding it into a signature image. Registration and matching were performed by matching corresponding signature images. The signature images were only generated for selected feature points. The results show a reduction in the registration and matching time compared to other known techniques, a major requirement for real-time applications. Applications included free-form object matching, multimodal medical volumes registration, and dental teeth reconstruction from intraoral images. Future research includes studying the

effect of shape deformation on the signature representation and modeling the change in the signature image with a deformation model. This will enable the signature representation to be used in nonrigid registration.

ACKNOWLEDGMENTS This work was supported in part by grants from the US National Science Foundation (EPS-9505674) and the DoD under contract: USNV N00014-97-11076.

REFERENCES [1] [2]

S.M. Yamany and A.A. Farag, ªFree-Form Surface Registration Using Surface Signatures,º Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1098±1104, Sept. 1999. Z. Zhang, ªIterative Point Matching for Registration of Free-Form Curves and Surfaces,º Int'l J. Computer Vision, vol. 13, no. 2, pp. 119±152, 1994.

1120

[3] [4] [5]

[6] [7] [8]

[9] [10] [11] [12] [13] [14] [15] [16]

[17] [18] [19] [20] [21] [22] [23] [24]

[25]


J.-P. Thirion, ªExtremal Points: Definition and Application to 3D Image Registration,º Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994. A. Gueziec and N. Ayache, ªSmoothing and Matching of 3D Space Curves,º Int'l J. Computer Vision, vol. 12, no. 1, pp. 79±104, 1994. S. Lavallee and R. Szeliski, ªRecovering the Position and Orientation of Free-Form Objects from Image Contours Using 3D Distance Maps,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, pp. 378±390, Apr. 1995. C. Dorai and A.K. Jain, ªCosmosÐA Representation Scheme for 3D Free-Form Objects,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, pp. 1115±1130, Oct. 1997. C.S. Chua and R. Jarvis, ª3D Free-Form Surface Registration and Object Recognition,º Int'l J. Computer Vision, vol. 17, pp. 77±99, 1996. A.E. Johnson and M. Hebert, ªUsing Spin Images for Efficient Object Recognition in Cluttered 3D Scenes,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 5, pp. 433±449, May 1999. R. Bergevin, D. Laurendeau, and D. Poussart, ªRegistering Range Views of Multipart Objects,º Computer Vision and Image Understanding, vol. 61, pp. 1±16, Jan. 1995. M. Soucy and D. Laurendeau, ªA General Surface Approach to the Integration of a Set of Range Views,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, pp. 344±358, Apr. 1995. K. Ikeuchi and Y. Sato, Modeling from Reality. Kluwer Academic Publishers, 2001. F. Stein and G. Medioni, ªStructural Indexing: Efficient 3-D Object Recognition,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 125±145, 1992. C.S. Chua and R. Jarvis, ªPoint Signatures: A New Representation for 3D Object Recognition,º Int'l J. Computer Vision, vol. 25, no. 1, pp. 63±85, 1997. A. Johnson and M. Hebert, ªSurface Matching for Object Recognition in Complex Three-Dimensional Scenes,º Image and Vision Computing, vol. 16, pp. 635±651, 1998. J. Opera, Differential Geometry and its Applications. Prentice Hall, 1997. M. Hebert, K. Ikeuchi, and H. Delingette, ªA Spherical Representation for Recognition of Free-Form Surfaces,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, p. 681, July 1995. H. Delingette, ªSimplex Meshes: A General Representation for 3D Shape Reconstruction,º Technical Report 2214, Unite de recherche INRIA Sophia-Antipolis, Mar. 1994. H. Delingette, M. Hebert, and K. Ikeuchi, ªShape Representation and Image Segmentation Using Deformable Surfaces,º Image and Vision Computing, vol. 10, pp. 132±144, Apr. 1992. R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis. John Wiley and Sons, 1973. S.M. Yamany, M.N. Ahmed, and A.A. Farag, ªA New GeneticBased Technique for Matching 3D Curves and Surfaces,º Pattern Recognition, vol. 32, no. 10, p. 1817, 1999. B.J. Besl and N.D. McKay, ªA Method for Registration of 3-D Shapes,º IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239±256, Feb. 1992. S.M. Yamany, ªSurface Point Signatures for Free Form Object Registration and Recognition,º PhD thesis, Computer Science and Eng. Dept., Univ. of Louisville, Dec. 1999. W.M. Wells, P. Viola, H. Atsumi, S. Nakajima, and R. Kikinis, ªMulti-Modal Volume Registration by Maximization of Mutual Information,º Medical Image Analysis, vol. 1, pp. 35±54, Mar. 1996. M. Ahmed, S.M. Yamany, E.E. Hemayed, S. Roberts, S. Ahmed, and A.A. Farag, ª3D Reconstruction of the Human Jaw from a Sequence of Images,º Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 646±653, 1997. M.G.-H. Mostafa, S.M. Yamany, and A.A. Farag, ªIntegrating Shape from Shading and Range Data Using Neural Networks,º Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR), June 1999.

VOL. 24,

NO. 8,

AUGUST 2002

Sameh M. Yamany received the BSc and MS degrees from Cairo University, Faculty of Engineering in 1991 and 1995, respectively. He received the PhD degree from University of Louisville, Kentucky, Speed School of Engineering in 1999. In 1999, he became an assistant professor in the computer science department at Old Dominion University, Virginia, where he concentrated on applications of computer vision in medical image analysis and telemedicine. His current research includes 3D registration, 3D compression, interactive remote medicine, medical image segementation and registration, and shape from X analysis. Dr. Yamany is a regular reviewer for a number of technical journals and national agencies including the US National Science Foundation and the National Institute of Health. He is a member of the IEEE and a member Phi Kappa Phi. Aly A. Farag was educated at Cairo University (BS degree in electrical engineering), Ohio State University, Columbus (MS degree in biomedical engineering), University of Michigan, Ann Arbor (MS degree in bioengineering), and Purdue University, Indiana (PhD degree in electrical engineering). Dr. Farag joined the University of Louisville, Kentucky in August 1990, where he is currently a professor of electrical and computer engineering. His research interests are concentrated in the fields of computer vision and medical imaging. Dr. Farag is the founder and director of the Computer Vision and Image Processing Laboratory (CVIP Lab) at the University of Louisville, which supports a group of more 20 graduate students and postdocs. His contribution has been mainly in the areas of active vision system design, volume registration, segmentation, and visualization, where he has authored or coauthored more than 80 technical articles in leading journals and international meetings in the fields of computer vision and medical imaging. Dr. Farag is an associate editor of IEEE Transactions on Image Processing. He is a regular reviewer for a number of technical journals and to national agencies including the US National Science Foundation and the National Institute of Health. He is a senior member of the IEEE and SME, and a member of Sigma Xi and Phi Kappa Phi.

. For more information on this or any other computing topic, please visit our Digital Library at http://computer.org/publications/dlib.