the correspondence map between the vertices and trian- gles of the ..... ing point is close to the focal point or close to a caustics and then only slight changes of ...
Efficient rendering of multiresolution meshes with guaranteed image quality Reinhard Klein and Andreas Schilling Wilhelm-Schickard-Institut, GRIS, Universit¨at T¨ubingen, Germany
Abstract
4, 5, 13, 14, 16]. The generation of a MRM of an object
Recently, output-sensitive rendering algorithms have gained a lot interest. While some years ago the number of triangles describing a scene was smaller than the number of pixels on the screne, this relation has changed to the opposite. The size of the models has become significantly larger while the screen resolution remained more or less constant. One way to get output-sensitivity, is the view-dependent use of multiresolution models. The key idea is to determine the lowest resolution for each part of the object that is necessary to get good images that are visually indistinguishable from rendering the original mesh. To take care of the illumination of the objects there are approaches that use cones of normal vectors to perform illumination dependent selective refinement. This allows the correct rendering of highlights and transitions between lit and unlit areas of an object. The drawback of these approaches is that using only the normal cone for the detection gives only a very poor criteria to detect the problematic cases. Therefore, too much refinement has to be performed which in most cases prevents real time update rates between successive frames. In this paper we describe a new approach for better estimation of normal deviations between different levels of detail. This allows for accurate lighting with a minimum number of triangles.
1 Introduction and previous work Multiresolution modeling (MRM) for simplicial geometric models (any dimension, non-orientable, non-manifold, non-regular) has become very popular. Multiresolution models provide the basis to handle, visualize and transmit over the network and edit very large datasets [15, 2, 9, 1, 1 Universit¨ at T¨ubingen, Auf der Morgenstelle 10 / C9, 72076 T¨ubingen, Germany. E-mail: andreas reinhard @gris.uni-tuebingen.de http:==www.gris.uni-tuebingen.de
f
j
g
Figure 1: Example of a hierarchy of simplification operations. On the top the original triangulation is shown. The simplification algorithm performs the simplification operations in the sequence OP1, OP2, OP3, OP4. Since OP2 and OP3 can be performed independently from each other (as their regions do not overlap) they can be swapped in the sequence. This is shown in the right branch of the figure.
Figure 2: The acyclic directed graph of the simplification operations shown in Figure 1. Since the operations OP2 and OP3 depend only on OP1 they can be performed in arbitrary order. OP4 depends on both, OP2 and OP3. Therefore, both operations have to preceed OP4.
generally involves a sequence of local simplification operations like vertex removal, edge collapse, triangle collapse and vertex clustering. This sequence of local simplification operations defines a sequence of coarser and coarser approximations of the original model, the MRM. In order to be able to refine a coarse approximation of the model, the inverse local simplification operations are needed and the MRM must contain the necessary information. Selective refinement becomes possible if the sequence defined by the simplification process is changed. This is only possible under certain circumstances which guarantee that the parts of the mesh on which the operation is performed are present in the current mesh. This requirement automatically defines a hierarchy of modification operations and the corresponding triangles. This hierarchy can be considered as a directed acyclic graph of modification operations or the associated triangles [12]. Figure 1 shows an example. Figure 2 shows the corresponding acyclic directed graph. For illumination depended selective refinement the surface normals are essential. For the decision if the mesh at a certain location has to be refined further, the normals contained in the actual coarse approximation cannot be used. Information about the normals in the corresponding area of the original model (which can be totally different) is required.
Figure 3: To determine the normal cone for the light grey triangle in the triangulation the hierarchy shown in Figure 2 is travesed up to the finest level (OP1). Then the normalcone is calculated from the normals of all predecessor triangles in the hierarchy. The resulting normal cone is far too large.
Therefore, in previous work the variation of the normals in a certain area is represented by a cone of normals represented by a vector together with an angle. The normal cone conservatively bounds all the normals of all the triangles in the corresponding area. This corresponding area is defined by several authors [17, 5, 10] as all the triangles in the original mesh which are descendents of the area being considered in the simplification hierarchy. However, the dependencies introduced by the simplification hierarchy are not related to the geometry of the
Figure 4: The upper two pictures show the illumination of a car bonnet without and with adaptive triangulation. The lower two pictures show the corresponding triangulations. Data courtesy of Volkswagen AG. relevant area. Therefore, in general the area considered for the calculation of the normal cone comprises in many cases more than twice the actual needed area, see Figure 3. This leads to much larger normal cones than required and therefore to unnecessary refinements. Therefore, we suggested in [6] not to use the hierarchy to determine the corresponding area but instead to use a geometric correspondence. Despite of this improvement there still remains a general problem when a normal cone is used: The cone of normals does not tell anything about the distribu-
tion of the normals in the relevant area. Figure 4 shows the result of lighting dependent simplification using normal cones. Note that in the area of the highlight the car bonnet is refined down to triangles of pixel size although this would be unnecessary especially if Phong-interpolation is used. If we knew that the normals over the area of a triangle can be sufficiently well approximated by a linear interpolation of the vertex normal (which occurs very often) the refinement could be avoided in the following cases:
always, if Phong interpolation is used, if Gouraud interpolation is used and a linear brightness distribution can be assumed.
In the remainder of the paper, we will introduce a suitable measure to be stored within the MRM, that allows us to detect these cases during the refinement process at rendering time and to obtain substantially smaller approximating meshes for the same high image quality.
2 The simplification algorithm Figure 5: Alias problems due to sampling of normals. The mesh simplification algorithm always starts with the finest triangulation in 3D space approximating the original model surface. In the following this starting triangulation is denoted with M , indicating that it contains M vertices. The simplification algorithm successively simplifies the starting triangulation by removing vertices from the current triangulation and retriangulating the resulting hole or by collapsing edges or triangles. This is done until no further vertices can be removed from the simplified triangulation without exceeding a predefined distance between the original triangulation and the simplified one. If after a single simplification step the simplified triangulation contains m vertices it is denoted by m : The order in which the simplification steps are performed is determined by a priority queue. For every single vertex vi of the simplified mesh an error value i is computed and stored. This value describes the potential error, that is the modified one-sided Hausdorff distance that would occur between the simplified and the original mesh if the simplification step was performed [9]. At the beginning of the algorithm this error value is computed for every vertex, edge or triangle in the original mesh. The vertices, edges or triangles of the original mesh are then sorted into a priority queue according to their potential error. Afterwards, in each simplification step we successively eliminate the vertex, edge or triangle with the smallest potential error on top of the priority queue. If a simplification step is actually performed the potential errors of the neighbor elements are recomputed and the priority queue is updated.
2.1
Generating information about the normals
2.1.1 The normals in a simplified triangulation During the simplification process the normals at the vertices must be recalculated to avoid alias problems due to undersampling, see Figure 5. After a simplification step the normals at the vertices vi incident to the modified area are given by
ni = P 1A
X
j j j
nj Aj ;
where j enumerates all triangles incident to vertex vi :. 1 Note that as a consequence of this filtering procedure, the normals at the vertices of a given triangle may change during the simplification process. 2.1.2 Determination of deviation angles To get information about the deviation of the normals, the correspondence map between the vertices and triangles of the original triangulation and the triangles of the simplified triangulation used for the distance calculations [9] is exploited. For a given triangle (vk1 ; vk2 ; vk3 ) of the simplified triangulation this map delivers all vertices 1 In order to model sharp edges several normals at a vertex are necessary. These cases can be handled in a similar way taking into account only the respective incident triangles.
ni nij1
2.1.3 Deviation of normals due to normal filtering
As described above, normal filtering is performed by recalculating the normals at the vertices of a simplified triVk2 angulation based on the adjacent triangles. This filtering does not only occur during the simplification process but also during the extraction of selectively refined meshes from the MRM. During the recalculation of these normals, different constellations of adjacent triangles must vk1 v’i be taken into account. The deviation between the normal vector used to determine the deviation angles (see previv'ij1 v'ij2 ous paragraph)and an actual normal vector varies. To get a vk3 conservative estimation for this deviation after the simpliFigure 6: Determination of deviation angles by projecting fication process all possible normals have to be computed. This can be done with a simple recursive algorithm operoriginal vertices on plane of triangle. ating on the directed graph of the hierarchy.
nij2
vi
3
vi 2 M
The extraction algorithm
of the original triangulation that are nearer to
than to any other triangle of the simplified triangulation. Each of these vertices corresponding to is projected onto the triangle : In addition to these vertices we determine all vertices which are connected to one of the vertices vi via an edge and project them onto the plane defined by : The projected vertices are denoted with v 0 : They are connected by the same edges as their counterparts in the original triangulation. For each projected triangle ij of which at least one vertex does not belong to ; the two intersection points vij0 1 and vij0 2 of their projected edges with the edges of triangle are determined and corresponding normal vectors nij1 and nij2 of these points in the original mesh are interpolated, see Figure 6. In the next step for each of these vertices vi0 ; and intersec0 the barycentric coordinates with respect to tion points vij k are computed, and using these coordinates, linearly interpolated normals n i based on the current normals at the vertices vk1 ; vk2 ; vk3 are computed. In a last step the deviation angles i between the original normals ni and corresponding interpolated normals n i in the vertices and intersection points are computed. After computing all these angles a linear deviation function f is computed, such that i f (vi0 ): The function f is defined by its values 1 ; 2 ; 3 at the vertices vk1 ; vk2 ; vk3 :
Figure 7: Simplified model of a sphere. The goal of the multiresolution model is to minimize the number of polygons that are sent to the rendering pipeline, without affecting the quality of the resulting image. To achieve this, the multiresolution model allows the extraction of selectively refined models either by coarsening or refining the current triangulation. The decision whether a given triangle may remain in the triangulation or if its surrounding area must be refined further or if a coarser triangulation of its surrounding area is possible, is the most important step in this process. Figures 7 to 10 serve as illustrations for the different criteria for this decision. For an implementation this can be reduced to the single question of whether a given triangle fulfills the quality requirements or not [8]. To answer this question we have to investigate certain characteristics of that part
of the original surface that corresponds to the triangle in 3.3 question. In the following we call this part of the original surface corresponding surface for simplicity. The flow diagram of the algorithm we use for this decision is shown in Figure 11.
Controlling the shading errors
3.1 Handling backfacing surface parts The first question in the algorithm is to decide if a triangle (vi1 ; vi2 ; vi3 ) and its corresponding part on the original surface is on the backside with respect to the current viewing position. It can easily be answered in a conservative Figure 9: Geometrically refined model with highlight: way using the normals ni1 ; ni2 ; ni3 at the vertices of the shading artefacts! triangle together with the corresponding deviation angles i1 ; i2 ; i3 . Let ei1 ; ei2 ; ei3 be the vectors pointing from the vertices to the observer then we get the following criteria. If
6 (,eij ; nij ) < =2 , ij ; j = 1; 2; 3; then the surface is backfacing and the triangle need not be refined any more. In the other cases we proceed with checking the geometric and shading errors.
3.2 Controlling the geometric error
Figure 10: Illumination dependent selective refinement. In the following we assume that the Blinn-Phong shading model is used. For one light source it is given by:
8< L k + L k (n l) a a d d +Ls ks (n h)m ; L=: La ka
if (n l) > 0; else,
where ka ; kd and ks are the ambient, diffuse and specular material coefficients, La ; Ld ; Ls the corresponding luminaces, l is the direction from the surface point to the light e Figure 8: Refined model: screen space error < 1/2 pixel. source, and h = jjll+ +ejj the so called halfway vector between the vector to the light source l and the vector to the To control the geometric error the distance (Hausdorff- observer e. distance) between original and simplified triangulation [9] must be converted to a screen space error. Several ap3.3.1 Only ambient light contributions proaches to this problem have been published and can be used [7, 17, 5, 10]. A further significant reduction can be The first test determines for a given triangle if the corachieved by avoiding the refinement outside of the view responding surface faces away from the light source and frustum. However, refinement outside the view frustum is therefore, gets only (constant) ambient lighting. To perinevitable to support fast changes of the viewing direction form this test we determine if the normal cones at the vertices defined by the normals ni1 ; ni2 ; ni3 at the vertices in VR-applications.
Backside?
Yes
Do not refine
No
Projected Triangle Smaller than pixel?
6 (,lij ; nik ) < =2 , ij ; j; k = 1; 2; 3; Yes
Do not refine
No
Refine
No
Yes
No
Lin. approx. of normals sufficient?
Yes
Do not refine
Yes
Linear Phong Interpolation sufficient?
Refine
No
α*dL/dα small enough? Yes
Phong interpolation used?
only the constant ambient term has to be considered and the triangle is not refined further 2 . 3.3.2 Is linear Phong interpolation sufficient?
Geometry ok?
Facing away from light source
and the corresponding deviation angles i1 ; i2 ; i3 are facing away from the direction to the light source. Let li1 ; li2 ; li3 be the vectors pointing from the vertices to the light source then we get the following criteria. If
Yes
Do not refine
No
In the next test we determine if linear Phong interpolation on the given triangle adequately describes the illumination of the corresponding surface. As a criteria we assume that it can be tolerated if the correct color in a pixel is replaced by a color that would be correct on a point, that is located in a one pixel-wide neighborhood of the pixel itself. In other words, the deviation angle , introduced by the varying normals on the corresponding surface must be less than the minimal deviation between Phong interpolated normals of two points that are one (screen-) pixels apart3 . Let nx ; ny respectively, be the partial derivatives of n in the directions of (screen-) x and y and cos( ) nx + sin( ) ny the derivative of n in the direction of (cos( ); sin( )). Now, we have to guarantee that a deviation of the normal by an angle in an arbitrary direction is always less than the minimum absolute deviation value d determined by nx and ny . To get d (and with that ) we could solve the corresponding eigenvalue problem, see Figure 12. To avoid this computation we use the simple conservative bound for d (and therefore for tan( )) based on a projection in the normal vector space, see Figure 12. If nx ny 0 we get the following formula
tan( ) jjnx ,(nx(ny ,nx)) ((n , nny), (nnx , n )) jj: y x y x Otherwise, nx is replaced by ,nx : For sake of simplicity, for nx and ny in this calculation the Phong increments can be used. Up to this point the angular deviations are con-
Refine
No
Brightness sufficiently linear?
Yes
2 Although
Do not refine
Figure 11: Flow diagram of the extraction algorithm.
cases can be constructed where these nine tests are required, they can normally be reduced to three tests ( lij ; nij ) < =2 ij ; j = 1; 2; 3 with only marginal effects on the resulting illumination 3 To be correct the deviation of the halfway vector should also be considered. However, we neglect the deviation of this vector in a one pixelwide neighbourhood.
,
6
,
n
n+∂n/∂x
α
Figure 12: Finding the smallest deviation in directional deviation space. trolled. Therefore, the resulting meshes cannot only be used for Phong-illumination but also in ray-tracing applications, or with environment mapping which is of special interest in the CAD-industry to visualize the quality and smoothness of free-form surfaces. If linear interpolation of the normals is not sufficient, there still may be cases where refinement may be unnecessary, since the gradient of the intensity may be so small that in larger neighborhoods of the pixel or even on the whole triangle the differences of intensities cannot be recognized by an observer. This cases are detected in a subsequent test. For this purpose let us assume that we use a Phong model. Than the original normals deviate from the interpolated normals at most by an angle of : Now we have to find an upper bound for the changes in the brightness that might be caused by a variation of the normals by an angle of in arbitrary direction. To end up d this we consider the derivatives @L @ of the diffuse term @L and @ of the specular term, where cos() = (n l) and cos = (n h): Then the changes that can be caused by an angle are bounded by
@L () + max @L ()) ; (max @ @
where min ,
