5th Annual IEEE Conference on Automation Science and Engineering Bangalore, India, August 22-25, 2009
Automated 3D Geometric Reasoning in Computer Assisted Joint Reconstructive Surgery K. Subburaj, B. Ravi, and M. G. Agarwal
Abstract—Computer Assisted Orthopedic Surgery (CAOS) employing information and computer graphics technologies for preoperative planning, intraoperative navigation, and for guiding or performing surgical interventions, has received very little attention for bone tumor surgery applications. We have developed a CAOS system called OrthoSYS, driven by geometric reasoning algorithms to visualize tumor size, shape, and plan for resection according to the tumor’s spread, starting from a 3D model reconstructed from CT images. Anatomical landmarks on bone are automatically identified and labeled, useful for registering patient model with virtual model during surgery and also as a reference for tumor resection and prosthesis positioning. The thickness of bone stock remaining after tumor resection is automatically analyzed to choose the best modular stem and fix the prosthesis. A method for prosthesis components selection using fuzzy logic has been developed to assist the surgeons. The medial axis of the long bones and anatomical landmarks are used for positioning the prosthesis in virtual planning and verification in the intraoperative stage. A set of anatomical metrics have been developed to measure the effectiveness of the prosthetic replacement of bone.
I. INTRODUCTION
C
OMPUTER Assisted Orthopedic Surgery (CAOS) uses computer technology for preoperative planning, and for guiding or performing surgical interventions [1]. The system requires computed tomography (CT), magnetic resonance (MR), or fluoroscopy images for planning and guiding a surgery. Imageless systems use statistical shape models to create the anatomical models based on dimensions measured manually. Commercially available CAOS systems focus on spine, knee arthroplasty (distal femur), and hip surfacing (proximal femur) [2]. Surgeons have attempted to use the above systems for tumor surgery by customizing them [3] [4]. As per the authors‟ knowledge, there is no dedicated system for computer aided bone tumor surgery.
Manuscript received June 9, 2009. This work was supported by the Office of the Principal Scientific Advisor to the Govt. of India under Grant PSA/OrthoCAD/2006. K. Subburaj is with the Mechanical Engineering Department, Indian Institute of Technology Bombay, Powai, Mumbai 400076 INDIA (e-mail:
[email protected]). B. Ravi is with the Mechanical Engineering Department, Indian Institute of Technology Bombay, Powai, Mumbai 400076 INDIA (corresponding author phone: 91-22-25767510; fax: 91-22-25726875; e-mail:
[email protected]). M. G. Agarwal is with the Surgical Oncology Department, Tata Memorial Hospital, Dr. E Borges Road, Parel, Mumbai 400012 INDIA (email:
[email protected]).
978-1-4244-4579-0/09/$25.00 ©2009 IEEE
A. Joint Reconstructive Surgery Endoprosthetic reconstruction has replaced amputation as the treatment of choice for skeletally immature individuals (children) who have to undergo resection of juxta-articular tumor [5]. In 80% of these patients, tumors are found in extremities of the long bone, the knee being the commonest [6] (Fig. 1.a & 1.b). The anatomical and physiological nature of the human body makes the physical intervention highly variable and complex. Each patient offers a unique challenge in terms of the location, shape and extent of the bone tumor. Postoperative complications such as infection, mechanical failure, bone fracture, and aseptic loosening, lead to a high rate of failure [7] (Fig. 1c). Inappropriate surgical procedures following misdiagnosis affect recurrence and survival rates [8]. Other major causes of failures include increased fatigue loading on the bone and prosthesis. This could be a response to unconsidered bone deformities, an over- or under-sized prosthesis, or poor alignment in body. The major constraints associated with joint reconstructive surgery, which limit the usage of general CAOS systems, are: (i) prosthesis components are smaller than bone (loss of tissue due to biopsy contamination), (ii) most of the reference points are lost during massive tumor excision, and (iii) limited host bone is available for fixation (tumor spread; patients are mostly adolescents). B. Preoperative Planning The goals of preoperative planning in CAOS for joint reconstruction, which is the focus of this work, are: (i) enable a surgeon to understand the patient anatomy, extent of spread of the tumor, and the surgical challenges posed, (ii) help in the development of customized prostheses, alignment of prosthesis components, and evaluation of post-surgical outcomes, (iii) present the data (visualization, virtual execution of different surgical protocols before entering the operation theatre) in a way that enables fast and accurate decisions, (iv) assist the surgeon to execute the protocol effectively (reference and well planned steps), and (iv) guide medical robots or tools in computer aided surgery to perform precision cuts, bone shaping, or positioning of the implant. In joint reconstructive surgery, patient specific anatomical models are required to visualize the tumor size and shape, and plan for resection according to the spread of tumor [9]. This limits the options to adopt image based systems. Computed tomography has a prominent role, since it is well suited for orthopedics; the bone separates from the rest of the
367
Fig. 1. Tumor knee replacement (a) Tumor in MRI, (b) Endoprosthetic joint replacement, (c) Failed prosthesis in body.
tissue by its intensity. Usage of a virtual 3D anatomical model allows surgeon to visualize the extent of tumor and regional anatomical structures during preoperative planning [10]. This helps to improve the surgical outcome in terms of tension-free artery and wound closure, stability, decreased postoperative complications, and longevity of the prosthesis. C. Need for Automation in Preoperative Planning At present, surgeons select a prosthesis based on (i) reasonable track record of its use (5–10 years), (ii) prior experience of the surgeon with the prosthesis, and (iii) suitability to the patient anatomy. A tumor knee prosthesis typically comprises 6–12 different components, available in a wide range (totaling 100 or more) to cater to different size, gender, position (left/right), and condition (tumor position) of patients. This makes it difficult to select the right set of components in an intra operative setting. [11]. Successful preoperative planning can prevent use of under- and oversized prosthesis components and can identify extremes of size and bony deformity/morphology, which may require nonstandard implant sizes. Anatomical compatibility implies that the prosthesis has to fit into the anatomical space, and fulfill its function without interfering with the surrounding tissues. Anatomical landmarks are distinct regions or points on bones, with uniqueness in shape characteristics in their vicinity, used for registration of virtual model with the patient bone [12]. These landmarks are also used for referencing prosthesis components for positioning them in the body [13], and referencing the tumor resection cut. As of now, when patientspecific surgery planning is carried out, landmarks are identified manually and used for registration and other tasks. Variability associated with the identification and location of the anatomical landmarks has the potential to affect the surgical outcome [14]. Most of the landmarks used during the surgical procedure are manually palpable and geometrically recognizable. Geometric characteristics of landmarks can be explored for automatic recognition [15]. Another important factor is that the number of trained surgeons is limited even in dedicated cancer research hospitals (Mumbai, the commercial capital of India, a city
with a population of 13 million has fewer than 2,000 orthopedic surgeons and one cancer institute with only a few orthopedic oncologists) [5]. The lengthy and complicated limb salvaging procedures form a bottleneck despite their tremendous potential. The anatomical characteristic of bones can be described geometrically, which allow the use of computing methods to make certain major decisions. Integration of geometric reasoning and decision tools would help to reduce errors, and lead to better surgical plans. Careful patient-specific planning and evaluation should be performed before deciding upon limb salvage surgical treatment [8]. The above reasons have triggered a demand for methods and tools to automate surgical guidance. This task has been taken up for investigation and implemented as a software program. II. TUMOR KNEE JOINT RECONSTRUCTIVE SURGERY PLANNING SYSTEM
A. Functional Requirements The functional requirements of software for virtual surgery for orthopedic applications, including tumor knee replacement surgery, are identified as follows: 1) 3D Reconstruction and Visualization: Image (CT/MR) import, segmentation of various tissues (semiautomated/ interactive), re-slicing, reconstruction of 3D virtual models, basic geometric operation and tools. 2) Geometric Reasoning: Automatic anatomical landmark identification and labeling, anatomical dimensions extraction, automatic reference axes generation for positioning, and remaining bone stock analysis in terms of thickness and properties. 3) Surgery Planning: Prosthesis components selection, resection planning, prosthesis positioning, guidance by reference axes and anatomical landmarks, set of anatomical metrics to evaluate the surgical outcome. Accordingly, the architecture of a virtual surgery system named OrthoSYS has been designed (Fig. 2). B. System Modules The system is divided into five modules (Fig. 3): (i) the START module imports CT scan images in DICOM format, orients the image set, reduces noise in the images using filtering algorithms, analyzes distribution of density for segmentation purpose, and maps pixel intensity into different color patterns for visualizing tumor region, (ii) the MODEL module segments different tissues from image data, reconstructs 3D anatomical models from the segmented data, automatically identifies anatomical landmarks, evaluates the anatomical deformities (if any), and analyzes bone in terms of thickness, (iii) the IMPLANT module allows a resection to be planned based on oncological principles, extracts anatomical dimensions from 3D model, and selects the best prosthesis components suitable for the patient from database, and (iv) the SURGERY module defines the resection plane
368
Fig. 2. Virtual implantation system modules: schematic diagram.
considering the selected prosthesis components and tumor margin, resects the tumor affected region, positions the prosthesis components, and verifies the alignment of the limb after implanting the prosthesis in a virtual environment. (v) COMMON module contains tools such as linear scale, angle measurement, report generation by capturing output of each step of the planning, and context sensitive help. The algorithms are explained using a case study. C. Reconstruction of 3D Anatomical Models The steps of 3D anatomical model reconstruction start with the imported axial CT images in DICOM (Digital Imaging and Communications in Medicine) format. The ratio of pixel resolution in the image plane (x, y), to resolution along the normal (z) axis may not be equal (i.e. pixel size ≠ slice thickness; inter-slice gap ≠ slice thickness). A reslicing algorithm was devised to ensure isotropic data for further computation. These images are processed with filters to reduce the noise while preserving the edge information in the form of intensity gradients. Global thresholding of bony tissue, based on a range of Hounsfield Unit (HU) values (220 to 3000 HU), is carried out in line with local adaptive thresholding. Next, a 3D region growing algorithm groups the bone data from the thresholded regions. Due to the similarities (overlap) in bone and surrounding tissues, classification based only on thresholding is not feasible. 3D morphological operators (both manual and automatic) are used to close the discontinuities in outer (periosteal) and inner (endosteal) contours. These segmented data are rendered in two modes: volume and surface rendering (Fig. 3). D. Identification of Anatomical Landmarks Anatomical landmarks on the body surface are either isolated points or well defined regions exhibiting a conspicuous shape compared to their vicinity. We have used geometric invariant measures for identifying anatomical landmarks on reconstructed 3D bone model automatically. A meaningful region is defined as a set of connected facets having certain geometrical property (e.g. curvature, surface normal, etc.) which is similar across the region. The surface type of a point can be classified based on the signs of
Fig. 3. 3D reconstructed bone model from CT images.
Gaussian and Mean curvature into peak, ridge, pit and ravine. The algorithm for identification of surface region types is as follows: 1) Identify a seed facet to grow: An internal facet which satisfies the following criteria is identified as a seed facet and is assigned the region type corresponding to its vertex surface type. i. maximum to minimum side length of triangle is below a threshold value ii. it has not been assigned a region yet iii. at least one vertex is not a feature vertex iv. all three vertices are of the same surface type 2) Grow the region: In this step, adjacent triangles (i) whose common edge is not a feature edge, (ii) which have not yet been assigned a region, and (iii) whose three vertices are of the same surface type as the seed facet are absorbed. The facets absorbed in the region are assigned the region type of the seed facet. 3) Terminate growing: Region growing is terminated when there are no adjacent facets of same surface type. 4) Repeat steps 1 to 3 until no seed facets are available for growing. These anatomical landmarks are constrained in a spatial relationship that is the same for all of the knee models. In order to encode these constraints in a network graph, the edges pairs are characterized by the spatial adjacency of a landmark relative to its neighboring landmarks. Skeletal landmarks are automatically identified by an iterative process using this spatial adjacency network graph (Fig. 6). Fig. 5 shows landmarks of the femur bone and the identified anatomical landmark regions. E. Extraction of Anatomical Dimensions Major anatomical dimensions of the knee include mediolateral length (ML), anterio-posterior length (AP), inner diameter (ID), and outer diameter (OD) of femur: MLF, APF, ODF, and IDF and similarly for tibia: MLT, APT, ODT, and IDT. Other parameters include femur valgus angle (VAF), femur bone curvature (CF), and resection length (RLF) decided by the surgeon according to the spread of disease. Bone inner diameters (ID) are measured at three different
369
Fig. 6. Landmark recognition (a) landmarks (b) landmark regions.
Fig. 6. Assembly of selected prosthesis components, suitability factors, and verification in 3D bone model.
components and calculates the suitability factor for each one. Rules are compiled from surgeon experience. A set of anatomical metrics developed to evaluate the selected prosthesis components considering their suitability with respect to patient's anatomy, and classified with qualitative tag (Fig. 6).
Fig. 5. Spatial adjacency network graph of distal femur landmarks.
locations at equal intervals and then the minimum of them is taken. The APF is measured as the distance between the most anterior and posterior points of medial condyle. The MLF is measured between the MEF and LEF. MLT is given by the distance between anatomical landmarks LPT and MPT. F. Selection of Tumor Knee Prosthesis Components A semi-automated selection methodology, driven by extracted anatomical dimensions, has been implemented. This segregates prosthesis components with a qualitative tag: (1) „most suitable‟, (2) „probably suitable‟, (3) „not suitable‟. The surgeon can select knee prosthesis components from a limited number of possibilities without needing to select, order, obtain and check a complete knee prosthesis set. The standard modular tumor prosthesis components are classified into three major categories: small (S), medium (M), and Large (L) according to their sizes (ranges are decided by one of our morphometric studies of the knee of Indian individuals). The selection is performed in two steps: First, the components that are undersized and oversized are eliminated. The interrelationship between prosthesis components is considered in selection. This affects the flow of components selection. Then the geometric details of components are mapped with the measured anatomical parameters of the patient to form a fuzzy logic based decision-tree. This evaluates the suitability of the
G. Bone Stock Thickness Analysis Precise measurement of the thickness of remaining bone stock, where the implant is to be anchored, is vital especially in custom-designed prosthesis. Although the term thickness is commonly used (bone thickness, cartilage thickness, etc.), its intuitive definition (smallest dimension of a cross-section) is ambiguous for intricate models like bones. We have devised a definition of thickness at any point in bone and a method for computing it automatically. Interior thickness at a point Pi inside an object is defined as the minimum distance of Pi from the nearest surface Sj of the object (Fig. 7). Its value can be obtained by growing a sphere or by firing rays from the point along all directions towards the object surface. The shortest ray length (distance between point and surface) gives the thickness at that point. It can be written as Tint (Pi) = min (dist (Pi, Sj)). Let p,s Z , and ρ is a sequence of moves from P to S. 3
The number of moves in face, edge, and vertex is noted as f, g, and h respectively. The weighted distance (d) can be calculated as: (1) l(ρ)=fa+gb+hc d(p)=minla,b,c (ρ)
(2)
The values a, b, and c are the weights assigned to voxels. The actual distance of any cell from the closest boundary can be calculated as: (3) l (ρ)=V(f+ 2*g+ 3*h)
370
1, 2 , 3
Fig. 7. Internal thickness definition and visualization thickness distribution in tibial bone
H. Tumor Resection and Prosthesis Positioning The resection length is initially decided according to the spread of tumor. The reconstruction length = tumor affected region + tumor margin (≈30 mm) + tibial cut. The effective prosthesis length is usually higher than the required reconstruction length due to approximation of anatomical dimensions (e.g., valgus angle) and unconsidered deformities of tibia. Hence the resection length must be revised by considering selected prosthesis components set. Reference axes are generated from the identified anatomical landmarks and medial axis of the bone. A perpendicular cut is ensured by referencing the anatomical axis of the bone (i.e. medial axis). The resection length (tumor + oncological margin + extra length of the prosthesis) is measured from the distal femur extremity (landmark: MCF or LCF). The resection length is recalculated only if the extra length is less than 10 mm. If the extra length is greater than 10 mm, then, a customized space filler must be used. The final resection length = tumor affected region + margin (Fig. 8). The positioning of prosthesis components with 3D anatomical model is carried out with reference to anatomical landmarks and reference axes. The medial axes of both bone and prosthesis components are generated. Registration of components with the corresponding bone is carried out in two steps. Axial alignment is first ensured by registering anatomical axes of the femur and tibia bone, and axis of femoral and tibial stem. This step in alignment also acts as a guide during intra-operative navigation for positioning of the prosthesis. Rotational alignment of the prosthesis is ensured by referencing identified anatomical landmarks before tumor resection. The coordinate values of these landmarks are registered along with the corresponding points in prosthesis components. The ICP algorithm is used to minimize the error between the bone and prosthesis. Once the prosthesis is registered and superimposed on a patient‟s anatomical model, radial and axial alignment of the prosthesis is visually verified. Anatomical suitability of the prosthesis is verified by anatomical metrics. Alignment of the lower limb is evaluated after implantation by regenerating reference axes. This is to ensure that deviation in alignment of the long bones is within an acceptable range.
Fig. 8. Tumor resection and prosthesis positioning (a) resection plane, (b) bone stock after resection, (c) prosthesis alignment verification in bone model, and (d) virtually implanted prosthesis.
I. Anatomical Suitability Metrics Modular prosthetic components may require a compromise in conforming to anatomical shape and size, which might affects the functional outcome of the joint. A set of anatomical metrics has been developed to evaluate the suitability of selected prosthesis for the patient anatomy. (i) Curvature Deviation (δC): is the difference between the measured bone curvature and the obtained curvature after implanting a prosthesis with curved stem, with and without osteotomy (Fig. 9a). (ii) Geometric Deviation (δG): is the dimensional difference between the resected bone and selected prosthesis components (Fig. 9b). (iii) Knee Centre Shift: is the difference between articulation centre of the knee and the prosthesis after implantation. If the metric‟s value is higher than an acceptable range (in consultation with surgeons), then one may decide to go for a custom-made prosthesis (Fig. 10a) (iv) Reconstruction Length Deviation (δRCL): is the difference between actual reconstruction length and effective length of the prosthesis (Fig. 10b).
371
Fig. 9. Anatomical suitability metrics: (a) curvature deviation and (b) geometric deviation
Using the system developed in this work, surgeons can easily assess the difficulties and risks involved, and plan to optimize the surgical approach and to decrease surgical morbidity. This enables making decisions and verifying them before entering an operation theatre. Integrating 3D geometric reasoning algorithms for anatomical understanding with decision methods makes the system intelligent and fairly automatic. Although, the methods and database are focused on knee joint reconstruction surgery, they can be easily customized for other bone tumor surgery planning (hip, shoulder, elbow).
Fig. 10. Anatomical suitability metrics: (a) knee centre shift and (b) reconstruction length deviation.
REFERENCES
III. DISCUSSION
[1]
We have described a computer-aided joint reconstructive surgery (tumor) planning system, developed to address the gap in this area. Until now most tumor surgeries aimed at only survival of the patient. Now due to technological and surgical advances, it is necessary to consider the functional outcome of the surgery in terms of quality of life. To reduce human errors, geometry based methods were introduced in this work to automate the planning tasks. As CAOS in massive bone tumor resection and reconstruction is still maturing, our work fills an important gap in this area. Commercially available medical modeling systems, e.g. Mimics (Materialise Inc, Belgium), 3D Doctor (Able Software Corp., USA), Analyze (Mayo Clinic, USA), and Amira (Mercury Systems, GMBH), are designed for CAD engineers and not for surgeons, who actually use their service. These systems are largely focused on visualization capabilities rather than surgical planning and guidance of surgical tools. Intelligence in these systems is negligible as most of them use highly user interactive and data intensive methods. The system developed for computer-aided joint reconstructive surgery (tumor) planning has several unique capabilities. All planning procedures are performed in a 3D virtual environment (instead of a 2D method using radiographic images) which increases the understanding of anatomic structures of the patient and spread of the tumor. A dedicated geometric algorithm is used to automatically identify and label 3D anatomical landmarks, which will be used for referencing surgical tools. A novel adaptive fuzzy logic based decision-tree, driven by the extracted anatomical dimensions, for categorizing and selecting prosthesis components. The suitability metrics indicate a numerical measure to the surgeons to give more attention to error-prone steps during surgery and quantifying the postoperative posture (alignment of limb) with respect to the preoperative condition. Data generated at every stage are stored (coordinates of anatomical landmarks, tumor region, reference lines, anatomical dimensions, resection plane location and orientation, and position of prosthesis) and can be used to guide robots or execute computer assisted tools during surgery. Once the surgical plan is developed, it becomes a blueprint for surgical intervention.
[2]
[3]
[4]
[5] [6] [7]
[8] [9] [10] [11] [12] [13] [14]
[15]
372
R. H. Taylor, and D. Stoianovici, “Medical robotics in computerintegrated surgery," IEEE Trans. Robot. Automat., vol. 19, no. 5, pp. 765–781, 2003. J. Kowal, F. Langlotz, and L. -P. Nolte, “Basics of computer-assisted orthopaedic surgery,” in Navigation and MIS in orthopaedic surgery, J. B. Stiehl, W. H. Konermann, R. G. Haaker, and A. M. DiGioia, Eds. Heidelberg: Springer Verlag, 2007, pp. 2–8. K. Reijnders, M. H. Coppes, A. L. J. van Hulzen, J. P. Gravendeel, R. J. van Ginkel, and H. J. Hoekstra, “Image guided surgery: New technology for surgery of soft tissue and bone sarcomas,” Europ. J. Surg. Oncol., vol. 33, pp. 390–398, 2007. K. C. Wong, S. M. Kumta, K. H. Chiu, K. W. Cheung, K. S. Leung, P. Unwin, and M. C. M. Wong, “Computer assisted pelvic tumour resection and reconstruction with a custom-made prosthesis using an innovative adaptation and its validation,” Comput. Aided Surg., vol. 12, no. 4, pp. 225–232, 2007. M. G. Agarwal, C. Anchan, M. Shah, A. Puri, and S. Pai, “Limb salvage surgery for osteosarcoma: effective low-cost treatment,” Clin. Orthop. Rel. Res., vol. 459, pp. 82–91, 2007. M. Sundaram and E. M. Rohren, “Primary bone tumours,” in Imaging on Oncology, 2nd ed., J. Husband and R.H. Reznek, Eds., London: Taylor and Francis, pp. 515–536, 2004. I. W. Sim, L. F. Tse, E. T. Ek, G. J. Powell, and P. F. M. Choong, “Salvaging the limb salvage: Management of complications following endoprosthetic reconstruction for tumours around the knee,” Europ. J. Surg. Oncol., vol. 33, no. 6, pp. 796–802, 2007. D. G. Jeon, S. Y. Lee, and J. W. Kim, “Bone primary sarcomas undergone unplanned intralesional procedures” J. Surg. Oncol., vol. 94, no. 7, pp. 592–598, 2006. V. O. Lewis, “What‟s new in musculoskeletal oncology,” J. Bone Joint Surg. Am., vol. 89, no. 6, pp. 1399–1407, 2007. K. Subburaj, and B. Ravi, “High resolution medical models and geometric reasoning starting from CT/MR images,” in Proc. 10th IEEE Int. Conf. CAD/Graphics, Beijing, 2007, pp. 441–444. W. D. Bugbee, D. J. Ammeen, and G. A. Engh, “Does implant selection affect outcome of revision knee arthroplasty?,” J. Arthroplasty, vol. 16, no. 5, pp. 581–585, 2001. S. van Sint Jan, Colour atlas of skeletal landmark definitions: Guidelines for reproducible manual and virtual palpations, Churchill Livingstone/ Elsevier, 2007. M. Viceconti, D. Testi, M. Simeoni, and C. Zannoni, “An automated method to position prosthetic components within anatomical spaces,” Comput. Methods Prog. Biomed, vol. 70, no. 2, pp. 127–127, 2003. U. Della Croce, A. Cappozzo, and D. C. Kerrigan, "Pelvis and lower limb anatomical landmark calibration precision and its propagation to bone geometry and joint angles", Med. Biol. Eng. Comput., vol. 37, no. 1, pp. 155–161, 1999. K. Subburaj, B. Ravi, and M. G. Agarwal, "3D shape reasoning for identifying anatomical landmarks," Comput. Aided Des. Applications, vol. 5, no. 1–4, pp. 153–160, 2008.
