An immersive modeler for curved objects using implicit surfaces Masatoshi Matsumiyaa , Kiyoshi Kiyokawab , Haruo Takemuraa and Naokazu Yokoyaa a Graduate
school of Information Science, Nara Institute of Science and Technology, 8916-5 Takayama, Nara, Japan b Multimedia Communications Section, Communications Research Laboratory (CRL), Ministry of Posts and Telecommunications (MPT), 4-2-1 Nukui-Kitamachi, Koganei, Tokyo, Japan ABSTRACT Most of existing 3D-CAD and 3D-CG softwares have problems in intuitive manipulation and representation of 3D curved surfaces. This paper discusses design issues and a prototype system of an immersive modeler which enables a user to design intuitively and interactively 3D solid objects with curved surfaces by one’s hand and finger. Shape deformation is expressed by simple formulas without complex calculation because of skeletal implicit surfaces employed to represent smooth free-form surfaces. A polygonization algorithm that generates polygonal representation from implicit surfaces is developed to reduce the time required for rendering curved surfaces, since conventional graphics hardware is optimized for displaying polygons. The prototype system has shown that a user can design 3D solid objects composed of curved surfaces in a short time by deforming objects intuitively by one’s hand and finger in real-time. Keywords: virtual reality, interactive modeling, implicit surfaces, polygonization
1. INTRODUCTION Recently, images and animations using three dimensional (3D) computer graphics (CG) are widely used in various presentations, movies, commercial films and video games. In 3D-CG, 3D objects composed of free-form curved surfaces (curved objects) are well used to visualize natural objects, such as plants, animals and humans. As the needs for 3D-CG of natural objects increase, the needs for computer systems which enable designers to design curved objects with ease are increasing. Although there exist many 3D-CAD and 3D-CG softwares, which are called modelers or modeling software with the ability of designing curved objects, most of them have three problems as follows: 1. A designer usually designs, manipulates and edits curved objects with a keyboard and a mouse. Such a user interface is not intuitive and difficult for designing 3D curved objects, when they are compared to a designing task in real environments, such as clay work. 2. Curved objects are displayed on CRT displays. It is hard to understand 3D curved objects on 2D displays. 3. Most of the existing free-form deformation methods are indirect. Designers can not directly modify object surfaces. In many cases, curved objects are deformed by changing special parameters called control points, weights, and so on. Designers have to handle those parameters to deform objects. In addition, behavior of the deformation is difficult to be predicted and sometimes is unexpectable. In order to solve these problems, the following characteristics are required for solid modelers: 1. Manipulation methods similar to modeling activity in the real world. 2. Display methods with which users can intuitively look around the objects from various viewpoints for better understanding of their shape. Correspondence: Email:
[email protected]; WWW: http://yokoya.aist-nara.ac.jp/; Telephone: +81-743-72-5296; Fax: +81-743-72-5299
3. Deformation methods which come up to user’s expectations and are based on direct manipulation. Moreover, in order to design smooth curved surfaces, the modeler should have another characteristic as follows: 4. Curved objects are composed of continuous free-form curved surfaces. In previous studies, modelers with some of these four characteristics were developed. Galyean et al.1 developed a modeler which enables a user to sculpt a solid material represented by voxel data by using virtual sculpting tools. A user can design 3D objects with intuitive methods similar to real sclupting, because the sculpting tools are controlled by a 3D input device. However, 3D objects represented in voxel data do not have smooth curved surfaces. Wang et al.2 also developed a volume sculpting tool based on the concept of sculpting a voxel-based solid material. The manipulation method of this modeler has a sense of incompatibility compared with real sculpting, because a user has to manipulate 3D objects with 2D input and output devices, such as a CRT display and a mouse. A modeler developed by Noble at el.3 enables a user to deform parametric surfaces with direct manipulation. However, it is hard to predict the results of manipulation. Although there are a number of modelers that are related to our goals, no modeler has all of the four characteristics above.4–6 The goal of this study is to develop a modeler which has all of the four characteristics. First, in order to realize characteristics 1 and 2, we employ a direct manipulation method which enables a user to design curved objects by using one’s hand and finger in an immersive environment. An immersive modeler is a modeling software which enables a user to be immersed in a virtual environment by using a virtual reality (VR) techniques. It has a number of advantages: the 3D form is easy to be understood, modeling with direct manipulation is intuitive, and so on. Second, in order to realize characteristics 3 and 4, we newly developed a deformation method which enables a user to add or subtract one’s finger shapes to or from the objects. We employ implicit surfaces to represent smooth surfaces. Owing to the definition of implicit surfaces, the result of the deformation can be expressed by simple formulas without performing complex calculations such as collision detection during the deformation process. Finally, as for an immersive modeler, an image coupled with user’s head motion must be updated in real-time. It is necessary to make the time required for rendering surfaces little enough to realize interactive modeling. In this paper, we propose an efficient algorithm which generates polygonal representation of skeletal implicit surfaces using spatial partitioning and parallel processing, since conventional graphics hardware is optimized for displaying polygons. In the following sections of this paper, we respectively describe a definition and characteristics of implicit surfaces in Sec. 2, the details of the prototype modeler conformed to the above policy in Sec. 3, and examples of models created by using the modeler in Sec. 4.
2. SURFACE REPRESENTATION As described in Sec. 1, one of our goals is to enable a user to deform curved objects by finger shapes and maintain surface continuity during the deformation. Though polygonal approximation and parametric surfaces are often used to represent curved surfaces, collision detection and deformation processes are so costly that these models are not suitable for interactive modeling including immersive modeling. In our modeler, an implicit surface based on skeletons (skeletal implicit surfaces)7,8 is employed to represent smooth surfaces. Owing to the definition of implicit surfaces, the result of the deformation can be expressed by simple formulas without performing complex calculations such as collision detection during the deformation process. An implicit surface is defined by:
{P ∈ R3 |f (P ) = c},
(1)
where P is a point in space, and f (P ) is the value of the scalar field at P (the implicit value). A surface surrounds a solid whose points satisfy f (P ) ≥ c. Normal vectors are directed along the field’s gradient. In skeletal implicit surfaces, the implicit value f (P ) generated by a set of skeletal elements Si (i = 1 · · · n) with a set of associated field functions Fi is defined as follows: n X f (P ) = wi Fi (d(P, Si )), (2) i=1
where d(P, Si ) denotes the distance from P to Si , and wi is a weighting coefficient that can be positive or negative. Figure 1 illustrates examples of the effect of altering the weights in the equation (2). In Fig. 1(b), the weight on one of two blended spheres is gradually reduced. The effect of the weight change is similar to changing the size of the local coordinate system of the skeletal element.
(a)
(b)
(c)
Figure 1. Effects of changing the weight w in skeletal surfaces. By using a negative scalar weight, the skeletal element is subtracted from the field as shown in Fig. 1(c). The skeletons Si can be any geometric primitives: points, lines, curves, etc. Because of the blending properties, the alteration of the geometry of an individual skeletal element changes the surface in a smooth manner. To find the implicit value of a point in space, the contribution from each primitive is summed up according to the equation (2). A value contributed by a particular primitive is computed using a field function that is assigned as an attribute of the primitive. For example, a field function Fi can be defined as a graph sketched in Fig. 2. Deformation of objects is controlled by altering the slope at the control point (CP) and a radius of influence. Figure 3 shows the effect of altering the slope at CP. A soft blend is caused by a gradual slope as shown in Fig. 3(a); a hard blend, on the other hand, is caused by a sharp slope as shown in Fig. 3(b).
Figure 2. An example of a field function.
(a)
(b)
Figure 3. Effects of changing the slope of field function at control point CP.
Implicit surfaces have characteristics as follows: • A point satisfying f (P ) < c is outside the surface. A point satisfying f (P ) > c is inside the surface. A point satisfying f (P ) = c is on the surface. • An object deformed by collisions with other objects is approximately described by subtracting each field function.
3. AN IMMERSIVE MODELER FOR CURVED OBJECTS In this chapter, we describe a prototype immersive modeler which enables a user to deform curved objects by one’s hand and finger directly. In the following, an overview of the system, a deformation process and a rendering process are described in order.
3.1. Overview of the Prototype System Figure 4 shows an appearance of the system and Fig. 5 illustrates the system components. In the prototype modeler, a user designs a 3D form by hand in an immersive virtual environment. To immerse a user into the virtual environment, a user wears a glove device (Nissho Electronics Super Glove) and a head mounted display (OLYMPUS Mediamask) with 3D trackers (Polhemus 3SPACE Fastrak). The glove device detects angles of finger joints. The head mounted display presents computer generated stereoscopic images. The 3D trackers detect positions and orientations of the user’s hand and head.
Figure 4. An appearance of the system.
Figure 5. The system components.
In the prototype system, a user deforms a primitive object with three types of manipulations as follows. Finger subtraction This manipulation causes the forefinger shape to be subtracted from a primitive object. Moving the forefinger into the object causes a contacting part to be eliminated in succession as shown in Fig. 6. Traces of a finger subtraction are smooth, because a subtracted primitive object preserves continuous smooth surfaces.
Figure 6. Finger subtraction. Finger addition This manipulation causes the forefinger shape to be added to a primitive object. Moving the forefinger causes a locus to change curved surfaces as shown in Fig. 7. The created joint is smooth, because an added primitive object preserves continuous smooth surfaces.
Figure 7. Finger addition. Palm subtraction This manipulation causes the plane shape that is defined by a palm of a user to be subtracted from a primitive object. As shown in Fig. 8, the shape of the primitive object at the palm-side of hand remains, while the shape at the back-side of hand is removed from the primitive object.
Figure 8. Palm subtraction.
In execution of the modeler, a user selects current manipulation mode among three manipulation methods by pressing corresponding keys or by touching a switch in the virtual environment as shown in Fig. 9. The name of current mode is displayed on the switch; i.e.,“S Subtractive”, “A Additive”, or “P Plane” for the three modes respectively.
Figure 9. A virtual switch in a virtual environment. Figure 10 shows the data flow diagram of the prototype system. First, the deformation unit generates skeleton data using measured data with 3D trackers and a glove device. Then, the polygonization unit converts skeleton data into polygon data. Finally, the rendering unit renders and outputs polygon data to the HMD using OpenGL graphics libraries. The deformation unit and the polygonization unit are described in detail in Sec. 3.2 and Sec. 3.3, respectively.
Figure 10. Data flow diagram of the prototype system.
3.2. Deformation Unit The deformation unit deforms a curved object using skeletal implicit surface’s characteristics described in Sec. 2. In this section, the deformation processes of three types of manipulations are described. When the original equation (2) is used, deformation occurs inappropriately in a wide area around a finger. In order to limit the effect of the deformation within a vicinity of a user’s finger or palm, the equation (2) is heuristically modified into equation (3). Also, an Hermite curve shown in Fig. 11 is used as a field function F . f (P ) =
n X i=1
p wi Fi ( d(P, Si )).
(3)
Figure 11. Field function used in the prototype system. Here, we describe deformation processes of the three types of manipulations in detail. Suppose a primitive object consists of three point-skeletons for an example. The implicit value of this primitive object f1 (P ) is given by: p p p f1 (P ) = F ( d(P, P ointS1 )) + F ( d(P, P ointS2 )) + F ( d(P, P ointS3 )), (4) where P ointSi (i = 1 · · · 3) are point-skeletons and weights are w1 = w2 = w3 = 1.0. Finger subtraction A forefinger shape f2 (P ) is given by the equation (5). p f2 (P ) = w1 F ( d(P, LineS),
(5)
where LineS is a line-skeleton. A shape obtained by subtracting a forefinger shape from the primitive object is represented by the equation f1 (P ) − f2 (P ) as illustrated in Fig. 12. The deformed shape f (P ) is given by the equation (6). Subtraction is realized actually by addition of f2 (P ) with w1 = −1.0. f (P ) = =
f1 (P ) − f2 (P ) p p p p F ( d(P, P ointS1 )) + F ( d(P, P ointS2 )) + F ( d(P, P ointS3 )) − F ( d(P, LineS).
Figure 12. A sequence of finger subtraction.
(6)
Finger addition A shape obtained by adding a forefinger shape to the primitive object is represented by the equation f1 (P ) + f2 (P ) as shown in Fig. 13. The deformed shape f (P ) is given by the equation (7) where w1 = 1.0. f (P ) = =
f1 (P ) + f2 (P ) p p p p F ( d(P, P ointS1 )) + F ( d(P, P ointS2 )) + F ( d(P, P ointS3 )) + F ( d(P, LineS).
(7)
Figure 13. A sequence of finger addition. Palm subtraction A palm shape f2 (P ) is given by the equation (8). f2 (P ) = w1 F (
p
d(P, P laneS),
(8)
where P laneS is a plane skeleton. A shape resulting from subtracting a palm shape from the primitive object is represented by the equation f1 (P ) − f2 (P ) as in Fig. 14. A deformed shape f (P ) is given by the equation (9). Subtraction is realized as addition of f2 (P ) with w1 = −1.0. f (P ) = =
f1 (P ) − f2 (P ) p p p p F ( d(P, P ointS1 )) + F ( d(P, P ointS2 )) + F ( d(P, P ointS3 )) − F ( d(P, P laneS).
(9)
Figure 14. A sequence of palm subtraction. The deformation unit generates skeleton data. Skeleton data consist of geometry of skeletons, field functions and weighting coefficients mentioned above. Even after the finger or the palm is removed, the deformation is preserved by keeping those skeleton data sampled whenever the hand is moved up to a constant distance. In the next section, the polygonization method for curved objects that are composed of skeleton data is described.
3.3. Polygonization for Fast Rendering As for an immersive modeler, an image coupled with the deformation of curved objects and user’s head motion must be updated in real-time. Although implicit surfaces are suitable for rendering using ray tracing methods, ray tracing is difficult to run in real-time.9,10 To reduce the time required for rendering them quick enough for interactive modeling, we have developed an efficient algorithm for polygonization of implicit surfaces, because polygons can be quickly rendered by using a conventional graphics hardware. There are several algorithms for the polygonization of implicit surfaces.11,12 However, none of them is suitable for interactive modeling system, because they cannot keep up with real-time operation as amount of the deformation increases. In our polygonization algorithm, polygon patches are made from vertices which are intersection points between the curved object surfaces and voxels’ edges. Details of the algorithm are described below. Pre-processing 1. Two voxel spaces (voxel space - a set of voxels) are created that surround a curved object. One is used to obtain intersection points between curved surfaces and voxels’ edges (the conversion-voxel space). The other is used to preserve skeleton data and to calculate the implicit values described in Sec. 2 (the data-voxel space). Figure 15 shows two-dimensional illustrations of voxel spaces.
Figure 15. Two voxel spaces. Left is the conversion-voxel space and right is the data-voxel space. 2. All curved surfaces (skeleton data) are converted to polygon patches using conversion-voxels which intersect curved surfaces in the conversion-voxel space. The algorithm for the conversion is described below in detail. Processing on the fly 1. Each skeleton data generated by the deformation unit is preserved in a corresponding data-voxel which intersects the skeleton in the data-voxel space. 2. The following process is repeated for each conversion-voxel in the conversion-area which is the neighboring area of a forefinger as shown in Fig. 15(a). (a) Each voxel is decomposed into six tetrahedra as shown in Fig. 16
Figure 16. Decomposition of a conversion-voxel. (b) Implicit values for vertices of tetrahedra are calculated by using skeleton data that are stored in data-voxels intersecting conversion-area as shown in Fig. 15(b). (c) Vertices are examined whether they are inside or outside a curved object based on implicit values as shown in Fig. 17. (d) If one vertex of an edge is inside a curved object and the other is outside, an intersection point exists on the edge. This intersection point is determined by binary sectioning.13 Binary sectioning is a simple and reliable method to compute accurate surface vertex locations. Given one point inside the surface and one point outside, the subdivision converges to a surface vertex by computing a midpoint and recurring on the subsegment that connects the midpoint and the signed endpoint. The intermediate value theorem guarantees the existence of a root for a continuous function including the equation (3). (e) Triangle polygons are generated by connecting calculated intersections. At the same time, surface normal is calculated for each intersection.
Figure 17. Intersection points between tetrahedra and a curved surface. Constant processing speed for real-time processing is realized by limiting an area of conversion. Moreover, the data-voxel space reduces the amount of computation without making accuracy worse, because no skeleton affects outside the radius of influence. The algorithm is implemented using parallel processing. The polygonization method described above has the problem that skeleton data still keeps increasing while modeling. Therefore, in order to prevent useless skeleton data increase, the number of skeleton data stored in a
data-voxel is limited. A voxel which stores the maximum number of skeleton data can not keep any more skeleton data. Thus, the prototype modeler has three parameters: 1)size of conversion-voxels, 2)size of data-voxels and 3)the maximum number of skeleton data per a voxel. These parameters are determined by considering complexity of the modeling and the performance of a computer.
4. EXAMPLES OF MODELING As examples of modeling, a teacup, fish with water plants and 3D freehand characters are designed as shown in Fig. 18. It took about 20, 20 and 5 minutes for modeling each design, respectively. This results that a user of the prototype system can design complex and smooth curved objects in a short time. The frame rate during the modeling is about 10 frames per second using four MIPS R10000(195MHz) CPUs (three CPUs for polygonization, one for processing other units and unit control). Paremeters mentioned in Sec. 3.3 are set to 1)120x120x120 for conversion-voxels, 2)48x48x48 for data-voxels and 3)10 for the maximum number of skeleton data per a voxel.
Figure 18. Examples of modeling. A teacup (left), fish with water plants (middle), and 3D freehand characters (right).
5. CONCLUSION In this paper, we first discussed four characteristics that are required for a modeler for curved objects and then described a prototype modeler satisfying all of these requirements. Four characteristics are 1)manipulation methods which are similar to modeling activities in the real world, 2)intuitive and easy way to view curved objects from various viewpoints, 3)direct deformation method which comes up to user’s expectation and 4)curved objects which are composed of continuous free-form curved surfaces. In order to satisfy these requirements, we employed a direct manipulation method such that a user can design curved objects by one’s hand and finger. Then, we employed skeletal implicit surfaces for surface representation and implemented three deformation methods (finger subtraction, finger addition and palm subtraction) by adding or subtracting appropriate equations. The prototype modeler was implemented using various algorithmic techniques. A number of experimental designs have shown that a user of the prototype system can actually design curved objects in a short time. Although our models lack the precision that is required for accurate product manufacturing, they are adequate as first-pass model designs or for applications where model precision is not greatly important, such as shown in Sec. 4. The present work provides designers or artists with a new modeling tool in immersive virtual environments. As future work, we will 1)use precise spatial input devices, 2)develop an automatic parameter adjustment method and 3)evaluate the method by various users. As the magnetic tracker that is used in the prototype modeler has imprecise and noisy data, it is difficult to perform delicate manipulations. The use of precise input devices will increase precision of the model. In addition, gesture recognition and haptic interfaces will be studied.14 As for the second issue, parameters described in Sec. 3.3 are empirically determined in the present implementation. Therefore,
it will be useful to examine an automatic parameter setting method which maximizes the precision of a modeling at a constant frame rate. As the last issue, the evaluation of the prototype modeler by designers and artists will show the feasibility of the system.
REFERENCES 1. T. A. Galyean and J. F. Hughes, “Sculpting: An interactive volumetric modeling technique,” in Proc. ACM SIGGRAPH ’91, pp. 267–274, 1991. 2. S. W. Wang and A. E. Kaufman, “Volume sculpting,” in Proc. ACM Interactive 3D Graphics, pp. 151–156, 1995. 3. R. A. Noble and G. J. Clapworthy, “Sculpting and animating in a desktop vr environment,” in Proc. IEEE Computer Graphics International, pp. 187–195, 1998. 4. S. Coquillart, “Extended free-form deformation: A sculpturing tool for 3d geometric modeling,” in Proc. ACM SIGGRAPH ’90, pp. 187–196, 1990. 5. J. P. Y. Wong, R. W. H. Lau, and L. Ma, “Virtual 3d sculpturing with a parametric hand surface,” in Proc. IEEE Computer Graphics International, pp. 178–186, 1998. 6. Y. H. Chai, G. R. Luecke, and J. C. Edwards, “Virtual clay modeling using the isu exoskeleton,” in Proc. IEEE VRAIS ’98, pp. 76–80, 1998. 7. B. Wyvill, “Animation and special effects,” in Introduction to Implicit Surfaces, J. Bloomenthal, ed., pp. 242– 269, Morgan Kaufmann Publishers, 1997. 8. M. P. Cani-Gascuel and M. Desbrun, “Animation of deformable models using implicit surfaces,” IEEE Trans. on Visualization and Computer Graphics 3(1), pp. 39–50, 1997. 9. J. C. Hart, “Sphere tracing: A geometric method for the antialiased ray tracing of implicit surfaces,” The Visual Computer 12(10), pp. 527–545, 1996. 10. G. Wyvill, “Ray tracing implicit surfaces,” in Introduction to Implicit Surfaces, J. Bloomenthal, ed., pp. 166–195, Morgan Kaufmann Publishers, 1997. 11. M. Desbrun, N. Tsingos, and M. P. Gascuel, “Adaptive sampling of implicit surfaces for interactive modeling and animation,” in Proc. Implicit Surfaces ’95, pp. 171–185, 1995. 12. J. Bloomenthal, “An implicit surface polygonizer,” in Graphics Gems IV, P. Heckbert, ed., pp. 324–349, Academic Press, 1994. 13. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, 2nd ed., Cambridge University Press, 1993. 14. T. V. Thompson II, D. E. Johnson, and E. Cohen, “Direct haptic rendering of sculptured models,” in Proc. ACM Interactive 3D Graphics, pp. 167–176, 1997.
