Registration vs. Reconstruction: Incorporating Structural ... - CiteSeerX

3 downloads 0 Views 2MB Size Report
Department of Computer Science and Engineering. §. Department of ... In biological samples, ductal structures such as blood vessels, lymph space, and .... Algorithm for Large Microscopy Images, Proceedings of the IEEE/NLM. Life Science ...
Registration vs. Reconstruction: Incorporating Structural Constraint in Building 3-D Models from 2-D Microscopy Images Lee Cooper† , Kun Huang† , Ashish Sharma† , Kishore Mosaliganti‡ , Tony Pan† Antony Trimboli§ , Michael Ostrowski§ † Department of Biomedical Informatics ‡ Department of Computer Science and Engineering § Department of Human Genetics

The Ohio State University A BSTRACT Registration is the key step for 3-D reconstruction of microanatomical structures from large number of microscopy images of biomedical samples. However, in most current approaches, 3-D structural information is not incorporated in the registration process. We present a novel approach by integrating structural constraints into the reconstruction pipeline. In stead of registering each image to its neighbors, we transform each image based on the information derived from its neighbors with incorporated structural constraint on ducts. The entire process is automatic and tested on a mouse mammary gland dataset with 160 images in the size of about 7500-by-600 pixels. I. I NTRODUCTION

Fig. 1. Left: original ductal structures sliced at different positions. Middle: the images for the ducts. Right: after the registration, the reconstructed ducts are column-like structures.

In biological samples, ductal structures such as blood vessels, lymph space, and mammary gland duct are commonly seen. It is generally assumed that these ductal structures wind through the tissues smoothly. Another advantage of focusing on these ductal structures is that they can be used as high-level features for matching in automatic registration. Microscopy image of histological sample usually contains numerous point (corner) features corresponding to various types of cellular structures, which makes it infeasible to use such point features for matching between two images. On the other hand, high-level features such as blood vessels can be used for fast and accurate registration for arbitrary rotation and translation between the images as shown by our previous works [2], [4]. This approach is especially useful in fast registration for large microscopy images datasets (e.g., 200 images of the size 4000-by-4000 pixels). The only assumption is that the nonrigid distortion of the sample is not significant and can be considered as local operations compared to the global rotation and translation (rigid transform). In this paper, we continue using this approach.

Automatic image registration is the technique to optimally align two images (2-D or 3-D) based on some cost function or evaluation criteria such as maximum mutual information (MMI) and minimum summed square difference [1], [5]. These techniques have also been applied in the case of reconstructing 3-D structures from stacks of 2-D images. For instance, given a series (e.g., 200) of microsocpy images of consecutive sections of mouse mammary gland, we want to reconstruct its 3-D model and specifically study the microanatomy of the ductal structures in the mammary gland. Due to the prevalence of soft tissues in the sample, the sections all have various distortion (e.g., shear and tear) and thus a nonrigid registration method is usually used in the traditional regiter-andreconstruction approach. A commonly asked question is how to evaluate/validate the results as there is no so-called ground truth available. This question actually implies a fundamental drawback of the traditional approach: the lack of structural constraint in the (optimization) process. As the registration is conducted over pairs of consecutive images, it only takes the consistency between two images into account but not the preservation of the original 3-D anatomical structure with the later being usually considered as a evaluation criteria instead of registration constraint. One consequence of such approach is illustrated in Figure 1 in which two ductal structures are “reconstructed” as two columns after this pairwise registration process even though “perfect” registration can be achieved between each pair of slides. In other words, such approach is more of a registration-for-registration approach than a registration-for-reconstruction one. In this paper, we present a different approach by putting the structural constraints into the processing pipeline. This implies that we need to integrate domain specific (biological) knowledge into our algorithm. In this paper, we begin with one of the most commonly accepted structural constraint: smoothness of ductal structures.

II. T HE R ECONSTRUCTION P IPELINE The reconstruction process is composed of three main stages: 1) Rigid registration. Fast rigid registration can be achieved using either optimization based approach such as MMI [3] or high-level feature based approach such as those in [2], [4]. In addition, specific methods can be applied to provide good initial estimate on registration. In the mammary gland example, the tissue samples have an enlongated shape (in Figure 2) and principal component analysis (PCA) is used to determine the principal directions of the sample in the images which are used for estimate of rotation angle between images. The purpose of this stage is to find the global rigid transformation between the images which facilitates the mathcing of corresponding high-level features by narrowing down the searching window.

57

2) Segmentation and tracking of high-level features. Here the high-level features are the ones that corresponding to ductal structures. In many cases, these high-level features can be easily segmentated via thresholding in the color space. For instance, blood vessels usually have distinct red color and mammary gland ducts are distinct dark structures embedded in the light-colored adipose tissues. After the ductal structures are segmented, their correspondences are located in the next image via feature matching by searching for regions with maximum normalized cross correlation. The centroid of these regions are then linked together. After this process is finished, trajectories for every ducts are generated. Due to the nonrigid distortion, these trajectories are usually very jaggy. 3) Trajectory smoothing and image transformation. The trajectory of each ductal structure is then smoothed. In this paper we begin with the most simple median filter for this purpose. However, more sphisticated methods such as spline fitting can also be applied. The key point here is that the smoothing generates the desired location of the high-level features in each image. Nonlinear transformations such the thin-plate spline can be applied to each image so that the features are moved to the new locations. Thus each image is not registered to any of its neighbors. Instead it is “registered” to a desired configuration of features. Nevertheless, the desired feature locations are derived from the neighbors of this image.

Fig. 3. Left: the trajectories for the high-level features. Right: smoothed trajectories after applying the median filter.

Fig. 4. Top left: volumetric rendering of the reconstructed mouse mammary gland. Top right: part of the mammary gland in 3-D. Bottom left: details of several reconstructed ducts. Bottom right: reconstruction without structural constraint. The ducts are reconstructed as columns.

IV. C ONCLUSION AND D ISCUSSION We present a novel approach in building 3-D models from stacks of microscopy images of histological samples. The key contribution is to integrating structural constraints into the reconstruction pipeline. In stead of registering each image to its neighbors, we transform each image based on the information derived from its neighbors with incorporated structural constraint on ducts. The entire process is automatic and fast. Despite these advantages, there are several technical challenges we need to solve. First, we are designing new PCA-based method to remove the systematic error induced by errorneous rigid registration. Second, we are developing new algorithm to incorporating more structural constraints such as taking the mechanical properties of the ducts into consideration. Third, we are looking for new ways of deriving the trajectories than using centroid of the regions as they are not the real corresponding points. Finally, we want to be able to automatically detect events such as merging and diverging of the ducts. R EFERENCES

III. R ESULTS We have implemented the above pipeline in Matlab to perform 3-D reconstruction of a set of 160 mouse mammary gland images as shown in Figure 2. Rigid registration is performed based on PCA and MMI. The region of interest are transformed into grayscale and the ducts are segmented via thresholding combined with a morphological erosion to remove cell membranes of the adipose tissue. The regions corresponding to the ducts and blood vessels are then separated from the background. Each duct is tracked as described in the previous section and the trajectories are shown in Figure 3. The trajectories are smoothed using a median filter with window size five. The entire segmentation, tracking, and smoothing process takes several minutes. Each image is transformed using the thin-plate spline method which moves the tracked features to the new locations generated by the smoothing step. Figure 4 shows several views of the reconstructed ducts in 3-D space. The volumetric rendering is generated using VolSuite, a volumetric rendering software developed at the Ohio Supercomputing Center. We also show the result without the tracking and smoothing process. As shown in Figure 4 Bottom right, the reconstructed ducts appear to be vertical columns as predicted.

[1] R. Baldock and J. Graham. Image processing and Analysis: a practical approach, Oxford University Press, Oxford, 2000. [2] J. Prescott, M. Clary, G. Wiet, T. Pan and K. Huang. Automatic registration of large set of microscopic images using high-level features, Proceedings of the IEEE International Symposium on Medical Imaging, Arlington, VA, April 2006. [3] R. Mosaliganti et. al. Registration and 3D visualization of large microscopy images, Proceedings of the SPIE Annual Medical Imaging Meetings, San Diego, CA, Feburary 2006. [4] K. Huang, L. Cooper, A. Sharma and T. Pan Fast Automatic Registration Algorithm for Large Microscopy Images, Proceedings of the IEEE/NLM Life Science Systems & Applications Workshop (LSSA), Bethesda, MD, July 2006. [5] D. Shen and C. Davatzikos HAMMER: Hierarchical Attribute Matching Mechanism for Elastic Registration, IEEE Trans. on Medical Imaging, 21(11):1421-1439, Nov 2002.

Fig. 2. An example of the mammary gland after the PCA-based rigid registration is performed.

58