Photorealistic ray tracing to visualize automobile ... - OSA Publishing

Photorealistic ray tracing to visualize automobile side mirror reflective scenes Hocheol Lee,1,* Kyuman Kim,1Gang Lee,2 Sungkoo Lee,2 and Jingu Kim2 1

Department of Mechanical Engineering, Hanbat National University, Daejeon, 305-719 South Korea 2 R&D Institute, Bullsone Co. Ltd., Incheon, 406-840 South Korea * [email protected]

Abstract: We describe an interactive visualization procedure for determining the optimal surface of a special automobile side mirror, thereby removing the blind spot, without the need for feedback from the error-prone manufacturing process. If the horizontally progressive curvature distributions are set to the semi-mathematical expression for a free-form surface, the surface point set can then be derived through numerical integration. This is then converted to a NURBS surface while retaining the surface curvature. Then, reflective scenes from the driving environment can be virtually realized using photorealistic ray tracing, in order to evaluate how these reflected images would appear to drivers. ©2014 Optical Society of America OCIS codes: (080.2740) Geometric optical design; (080.4228) Nonspherical mirror surfaces; (110.1758) Computational imaging; (220.0220) Optical design and fabrication.

References and links 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

J. F. Morgan and M. Blanco, “Synthesis study of light vehicle non-planar mirror research,” NHTSA DOT HS 811–328, 47 (2010). J. Luoma, M. J. Flannagan, and M. Sivak, “Effects of nonplanar driver-side mirrors on lane-change crashes,” Transp. Hum. Factors 2(3), 279–289 (2000). S. M. O'Day, “Binocular disparity in ashperical mirrors,” SAE Technical Paper 980918 (1998). R. A. Hicks, “Controlling a ray bundle with a free-form reflector,” Opt. Lett. 33(15), 1672–1674 (2008). J. Kuwana, M. Itoh, and T. Inagaki, “Dynamic side-view mirror: assisting situation awareness in blind spots,” in Proceedings of IEEE Conferences on Intelligent Vehicles Symposium (IV) (Gold Coast, Australia, 2013), pp. 455–460. T. Ehlgen, T. Pajdla, and D. Ammon, “Eliminating blind spot for assisted driving,” IEEE Trans. Intell. Transp. Syst. 9(4), 657–665 (2008). H. Lee, D. Kim, and S. Yi, “Horizontally progressive mirror for blind spot detection in automobiles,” Opt. Lett. 38(3), 317–319 (2013). C. Fowler, “Recent trends in progressive power lenses,” Ophthalmic Physiol. Opt. 18(2), 234–237 (1998). J. Loos, G. Greiner, and H. Seidel, “A variational approach to the progressive lens design,” Comput. Aided Des. 30(8), 595–602 (1998). J. Wang, R. Gulliver, and F. Santosa, “Analysis of a variational approach to progressive design,” SIAM J. Appl. Math. 64(1), 277–296 (2003). L. Qin, L. Qian, and J. Yu, “Simulation method for evaluating progressive addition lenses,” Appl. Opt. 52(18), 4273–4278 (2013). W. Jiang, W. Bao, Q. Tang, and H. Wang, “A variational-difference numerical method for designing progressive-addition lenses,” Comput. Aided Des. 48, 17–27 (2014). A. Selenow, E. A. Bauer, S. R. Ali, L. W. Spencer, and K. J. Ciuffreda, “Assessing visual performance with progressive addition lenses,” Optom. Vis. Sci. 79(8), 502–505 (2002). Y. Han, K. J. Ciuffreda, A. Selenow, and S. R. Ali, “Dynamic interactions of eye and head movements when reading with single-vision and progressive lenses in a simulated computer-based environment,” Invest. Ophthalmol. Vis. Sci. 44(4), 1534–1545 (2003). J. E. Sheedy, “Progressive addition lenses--matching the specific lens to patient needs,” Optometry 75(2), 83– 102 (2004). H. Lee, G. Lee, S. Lee, and J. Kim, “Slumping process of the horizontally progressive type of automobile side mirror,” in Classical Optics 2014, OSA Technical Digest (online) (Optical Society of America, 2014), paperOW2B.2. Y. Chen and A. Y. Yi, “Design and fabrication of freeform glass concentrating mirrors using a high volume thermal slumping process,” Sol. Energy Mater. Sol. Cells 95(7), 1654–1664 (2011).

#221024 - $15.00 USD Received 14 Aug 2014; revised 28 Sep 2014; accepted 5 Oct 2014; published 14 Oct 2014 (C) 2014 OSA 20 October 2014 | Vol. 22, No. 21 | DOI:10.1364/OE.22.025729 | OPTICS EXPRESS 25729

18. A. S. Glassner, An Introduction to Ray Tracing (Morgan Kaufman, 1989). 19. T. Ma, J. Yu, P. Liang, and C. Wang, “Design of a freeform varifocal panoramic optical system with specified annular center of field of view,” Opt. Express 19(5), 3843–3853 (2011). 20. A. Davis, “Raytrace assisted analytical formulation of Fresnel lens transmission efficiency,” Proc. SPIE 7429, 74290D (2009). 21. J. C. Halimeh, T. Ergin, J. Mueller, N. Stenger, and M. Wegener, “Photorealistic images of carpet cloaks,” Opt. Express 17(22), 19328–19336 (2009). 22. J. C. Halimeh and M. Wegener, “Photorealistic rendering of unidirectional free-space invisibility cloaks,” Opt. Express 21(8), 9457–9472 (2013). 23. A. J. Danner, “Visualizing invisibility: Metamaterials-based optical devices in natural environments,” Opt. Express 18(4), 3332–3337 (2010). 24. F. Mauch, M. Gronle, W. Lyda, and W. Osten, “Open-source graphics processing unit–accelerated ray tracer for optical simulation,” Opt. Eng. 52(5), 053004 (2013). 25. A. Brückner, J. Duparré, R. Leitel, P. Dannberg, A. Bräuer, and A. Tünnermann, “Thin wafer-level camera lenses inspired by insect compound eyes,” Opt. Express 18(24), 24379–24394 (2010). 26. T. Nakamura, R. Horisaki, and J. Tanida, “Computational superposition compound eye imaging for extended depth-of-field and field-of-view,” Opt. Express 20(25), 27482–27495 (2012). 27. C. Elster, J. Gerhardt, P. Thomsen-Schmidt, M. Schulz, and I. Weingärtner, “Reconstructing surface profiles from curvature measurements,” Optik (Stuttg.) 113(4), 154–158 (2002). 28. D. W. Kim, B. C. Kim, C. Zhao, C. Oh, and J. H. Burge, “Algorithms for surface reconstruction from curvature data for freeform aspherics,” Proc. SPIE 8838, 88380B (2013). 29. C. Zhao and J. H. Burge, “Orthonormal curvature polynomials over a unit circle: basis set derived from curvatures of Zernike polynomials,” Opt. Express 21(25), 31430–31443 (2013). 30. http://www.plm.automation.siemens.com/products/nx/. 31. L. Piegl and W. Tiller, The NURBS Book (Springer, 1997), Chap. 4.

1. Introduction The driver’s blind spot in an automobile can potentially conceal nearby vehicles and is an issue of some concern as it has been known to cause crashes [1,2]. As a means of eliminating or reducing the effect of the blind spot, a number of side mirror designs have been proposed. For example, the widely commercialized European aspheric mirror can provide a larger angle side-view, under the binocular disparity limit [3], and a free-form mirror design technique has been suggested [4], which focuses on the excessive field view angle only, regardless of current side mirror safety regulations. These normally require a spherical convex mirror, above the specific radius of curvature, or a flat surface for distant vision in the inner zone. Outside the inner zone, control of the outer surface for better viewing of nearby objects is currently being permitted in the case of the European aspheric mirror, but this is not allowed on the driver’s side in the USA. As an alternative to driver awareness of blind spots, a dynamic side-view mirror has been proposed, which changes the mirror angle dynamically to track vehicles entering the blind spot [5], thus allowing drivers to detect a collision risk by observing the vehicle directly through the mirror. Similarly, a camera-based system has been introduced that supplies drivers with optical images of their environment [6], while Volvo have developed a blind spot information system that provides warning signals. However, these signals are limited by the capabilities of the vision sensor. In our previous research, we proposed the horizontally progressive mirror [7], inspired by the ophthalmic progressive addition lens [8], which can enlarge the rear view angle of the automobile side mirror. The outer zone of the mirror surface can be designed as a cylindrical surface, while the driver’s inner zone remains a spherical convex surface. Only the curvature of the intermediate zone connecting the two zones is progressively changed. The magnification of the car image in the outer zone does not vary and a stable image can be shown to the driver, which is different to that provided by the European aspheric mirror. As regards ophthalmic progressive lenses, versatile design techniques to provide a more comfortable reading zone have been published [9–12]. To achieve this, the mean power and peak astigmatism are first set and then blended, and the lens surface maps are then determined using a numerical computational method. However, the wearer’s visual adaptation and the dynamic interaction of the different types of progressive addition lenses are still important [13,14]. The optimal parameters of the mean power and astigmatic distribution are determined


by matching the power map to the individual wearers’ needs [15]. Similarly, in terms of automobile free-form side mirrors, the surface design procedure used to select the curvature distribution should consider the driver’s vision, including the optimum rear-view angle, vehicle image shrinkage, image distortion, and binocular disparity. However, the slumping process used to fabricate automobile mirrors results in manufacturing errors, causing deviations from the curvature design [16]. Therefore, curvature compensation is required in order to satisfy the specified curvature distributions [17]. From a designer’s perspective, the selection of the optimal design using successfully fabricated mirrors is a time-consuming process. However, the use of virtual reflective image formation may be helpful here, as the tedious manufacturing process can be omitted. For these purposes, photorealistic ray tracing could be a candidate method, and is known as an established technique for obtaining virtual images based on geometrical optics [18]. Ray tracing has conventionally been used to design optimal optics layouts by minimizing optical aberrations [19], and is also useful in the formulation of optical performance features, such as the transmittance efficiency of the Fresnel lens [20]. Through home-built dedicated raytracing software, photorealistic images of invisibility cloaks and devices have been visualized in order to determine an intuitive impression of their performance [21,22] or to simulate the real environments in which they would actually function [23]. As support for these approaches to optical design and simulation, modern graphics processing units for accelerated ray tracing techniques are still being developed [24]. Therefore, computational imaging, which is based on cooperative optical design and image processing, has been applied to realize compound eye imaging [25,26]. In this paper, we present an intuitive design procedure for an automobile side mirror of free-form curvature by visualizing the reflective scene in the driving environment virtually, which requires an auxiliary optimal design tool to simplify the tedious feedback process of the mirror design, manufacturing, and evaluation. In addition, a new surface curvature distribution is examined, with the aim of providing a more comfortable rear view image to drivers. 2. Mirror surface map generation 2.1 Previous work Previously, we demonstrated new side mirrors yielding a stable vehicle image by inserting a horizontally progressive zone, between the two outer zones (used for the far and near views, using the slumping process [7]. The surface of the progressive zone was smoothly blended by numerically modeling the connection between the two surfaces, which are known as the spherical inner and cylindrical outer surfaces. However, as a result of manufacturing errors during the slumping process used to bend the mirror plate, the design width of the progressive zone could not be acquired easily. Therefore, we had difficulty in obtaining an exact reflective image using this surface design. In addition, it was difficult to determine if the original mirror design specification was optimal with respect to the tolerable bend of the traffic lane on the outer zone, which is traded-off against the large rear-view angle. 2.2 New surface map design An optimal mirror surface curvature design should generate the best reflective image in addition to compensating for manufacturing defects. In this study, the vertical radius of curvature will vary along the horizontal direction, which had a fixed value in our previous work [7]. This is expected to improve the reflective view between cases with the same horizontal rear-view angle. On the other hand, the optical power map of the curvature distribution directly affects the rear view angle, vehicle image size and horizontal shrinkage, and lane distortion. When the two principal curvatures, Kmax and Kmin, are determined from the


two principal curvature radii, Rx and Ry, at a given position, the mean curvature, Km, and the curvature difference, Ka, are expressed as

( K max + K min )

1 1 1  1 1 (1) . −  +  ; K a = ( K max − K min ) = 2 2  Rx Ry  Rx Ry Here, Km is the optical power, indicating the magnification of the image (vehicle) size, while Ka represents the astigmatism and gives the ratio of the horizontal and vertical dimensions of the image. A given position, z(x), and the radius of curvature, R(x), are mathematically related according to Km =

=

R ( x) =

  dz  2  1 +      dx  

3

2

(2) . d2z dx 2 From the surface curvature data, it is possible to reconstruct the surface profiles using the numerically integrated solution of Eq. (2) [27–29]. In this study, the two directional components of the radius of curvature R, Rx and Ry, illustrated in Fig. 1 below, are mathematically decoupled from each other, as expressed in Eq. (3).

Fig. 1. Two principal components of the radii of curvature of the mirror surface. Rx varies along the horizontal direction only. (a) Rx. (b) Ry.

 Rx   R1 ( xi )  R ( xi , y j ) =   =   , i = 1,..., m; j = 1,..., n.  Ry   R2 ( xi , y j ) 

(3)

The generation of the point set of the surface map through a separated numerical integration procedure is conducted as follows. First, Eq. (2) can be incorporated into Eq. (4) to yield the surface profile, z(xi, yc), along the horizontal x-direction at a specific vertical position, yc, with g(xi, yc) being the first derivative of z(xi, yc).  z ( xi , yc ) = g ( xi , yc ), =

3 2 2 1  1 + { g ( xi , yc )}  ,  R1 ( xi ) 

(4)

= f ( x i , g ( xi , yc ) ) .


By applying the Euler method to obtain the first derivative term of the Taylor series expansion, Eq. (4) can be written in the numerically integrated form given in Eq. (5). Here, h is the integration interval, while the boundary conditions at the profile can be arbitrary values at yc. g ( xi +1 , yc ) = g ( xi , yc ) + f ( xi , g ( xi , yc ) ) h,

(5)

z ( xi +1 , yc ) = z ( xi , yc ) + g ( xi , yc ) h.

Now, z(xi, yc) can be used as the new initial boundary condition in order to obtain the vertical surface points. The surface point z(xm, yi) at a specific position, xm, has a vertical curvature distribution, R2(xm, yj), along the vertical y-direction, and can be expressed as  z ( xm , y j ) = p ( xm , y j ), =

{

}

1 1 + p x , y 2  ( m j )  R2 ( xm , y j ) 

(

3

2

(6)

,

)

= q x m , p ( xm , y j ) .

Here, we will limit R2(xm, yj) to R2(xm), which is therefore constant in the vertical y-direction, so that it is only horizontally progressive. Therefore, the point set of all z(xi, yi) can be fully solved, since z(xm, yi) can be simply acquired as a circle of radius R2(xm). 3. Visualization with ray-traced rendering 3.1 NURBS surface conversion The point set for the surface map can be fit to the non-uniform rational B-splines (NURBS) surface using NX 6.0 software [30], as shown in Fig. 2. The NURBS expression is defined as follows [31]. It has degree p in the u-direction and degree q in the v-direction, and a bivariate vector-valued piecewise rational function of the form n

S ( u, v ) =

m

 N i =0 j =0 n m

i, p

(u ) N j , q (v) wi , j Pi , j

 Ni , p (u ) N j ,q (v)wi , j

,

0 ≤ u , v ≤ 1,

(7)

i =0 j =0

Here, the {Pi,j} form a bidirectional control net, the {wi,j} are the weights, and {Ni,p(u)} and {Nj,q(v)} are the nonrational B-spline base functions on the knot vectors U = {0, ... , 0, u p +1 , ... , ur − p −1 , 1, ... ,1}; V = {0, ... , 0, vq +1 , ... , vs − q −1 , 1, ... ,1},     p +1

q +1

q +1

(8)

q +1

where r = n + p + 1 and s = m + q + 1. Also, the conversion requires surface patches in the u- and v-directions, the number of which is adjusted through consideration of a minimized fitting error. Three surface patches are sufficient to express a smooth surface in the vertical direction. In addition, in the direction horizontal to the surface, the degree, p, of each patch and the patch number should be large in order to satisfy the minimum conversion error. However, large values require excessive surface conversion processing time.


Fig. 2. NURBS surface acquisition from the point set. The number of degrees and patches are adjusted until the desired conversion accuracy and curvature deviation is obtained.

3.2 Ray-traced rendering configuration As an application of this process, the real scene on a road was captured using a conventional high-resolution camera as a reference image source. It included the yellow straight lanes and the surrounding vehicles on the road, as shown in Fig. 3.

Fig. 3. Reference image source.

The reference image was projected to the mirror surfaces and virtually visualized by the ray-tracing module in the NX 6.0 software [30]. A schematic diagram of the ray-tracing apparatus used to generate the reflective images is given in Fig. 4. It is configured at 8.2 m backwards from the viewing position, as the ray reference and the distance are adjustable according to the mirror size. The distance between the viewing position and the center of the mirror surface is always set to 762 mm, based on the driver’s seat position in the average automobile.


Fig. 4. Schematic diagram of the ray-traced rendering method used to obtain the reflected images on the mirror surface from the reference image.

The viewing position is horizontally aligned to the right side of the mirror on the driver’s side, not to its center, in order to obtain real oblique images from the mirrors. 3.3 Visualization examples and comparison The reflection images were visualized for three different spherical convex surfaces of a mirror of dimensions 250 × 380 mm, and then cut to the shape of a commercial truck side mirror, as shown in Figs. 5(a)-5(c). The spherical radii of curvature were 1,250 mm, 1,050 mm, and 950 mm, respectively.

Fig. 5. Virtual reflection images from three spherical convex surface mirrors on the driver’s side. Radius of curvature of (a) 1,250 mm, (b) 1,050 mm, and (c) 950 mm.

The mirror with radius of curvature of 950 mm in Fig. 5(c) has a wider rear-view angle and can therefore reveal a nearby green vehicle. In addition, as the radius of curvature decreases, the vehicles in the far distance appear smaller and all images show slight distortion, as apparent in the bend of the yellow lanes and the curved gray column. To avoid the problem of excessive vehicle shrinkage in the far distance and to obtain wider rear-view angles at the same time, three progressive mirrors were designed and visualized, and the images obtained using each mirror are displayed in Figs. 6(a)-6(c). The radius of curvature of all three progressive mirrors was set to 1,250 mm along the horizontal direction on the right side and decreased progressively to 400 mm on the left side, which resulted in the same horizontal #221024 - $15.00 USD Received 14 Aug 2014; revised 28 Sep 2014; accepted 5 Oct 2014; published 14 Oct 2014 (C) 2014 OSA 20 October 2014 | Vol. 22, No. 21 | DOI:10.1364/OE.22.025729 | OPTICS EXPRESS 25735

rear-view angle as the mirror with a spherical radius of curvature of 950 mm featured in Fig. 5(c). However, the mirrors were constructed with three varying radii of curvature in the vertical direction (Ry). They varied progressively from the right to the left side along the horizontal direction, but were constant in the vertical direction. For the mirror used to generate the image shown in Fig. 6(a), Ry decreases from 1,150 mm to 1,050 mm, while it decreases from 1,250 mm to 1,150 mm across the mirror used for Fig. 6(b). In Fig. 6(c), the mirror Ry increases uniquely from 1,250 mm to 1,350 mm. The vehicle shapes in the far distance and the rear-view angles were approximately the same in both Figs. 6(b) and 6(c).

Fig. 6. Virtual view of horizontally progressive mirrors on the driver’s side. The three Rx distributions are identical and decrease continuously from 1,250 mm on the right side to 400 mm on the left side. The three Ry distributions all differ. (a) Ry is 1,150 mm on the right side and 1,050 mm on the left side. (b) Ry is 1250 mm on the right side and 1,150 mm on the left side. (c) Ry is 1,250 mm on the right side and 1,350 mm on the left side.

The design parameters and specifications are listed in Table 1. Horizontal fields of view (H-FOV) of each image are all the same as 44.8°. Vertical fields of view (V-FOV) of each image are 49.4°, 46.8°, and 42.8°, respectively. The yellow traffic lanes are all bent in the near distance. Yellow lane straightness is defined as the ratio of the vertical deviation of the upper curved yellow line on the mirror left side to the vertical traverse of a straight line, which is supposed to be formed only in the case of no difference between the horizontal and vertical curvature distribution. Therefore, all three straightness were acquired from the image analysis of Figs. 6(a)-6(c). In Fig. 6(c), the largest straightness is 40% while the V-FOV is the smallest value. And the gray column in Fig. 6(c) appears straighter than in Figs. 6(a) and 6(b) because V-FOV from the large vertical radius of curvature makes is the smaller. We can expect that, of the three images, the reflective scene in Fig. 6(c) provides a more comfortable image vertically to drivers with the same horizontal rear view angle, even though the vertical field of view in the near distance is smaller. Finally, two surface design of Figs. 6(a) and 6(c) are being manufactured for field test on the road environment.


Table 1. Performance evaluation with the visualized scenes for blind spot area control Specification Horizontal field of view(degree) Horizontal radius of curvature on the right side (mm) Horizontal radius of curvature on the left side (mm) Distance from the mirror to the driver’s eye (mm) Mirror width (mm) Mirror height on the left side (mm) Vertical field of view on the left side (degree)

Figure 6(a)

49.4

Figure 6(b) 44.8 1,250 400 762 230 270 46.8

Figure 6(c)

42.8

Vertical radius of curvature on the right side (mm)

1,150

1,250

1,250

Vertical radius of curvature on the left side (mm) Yellow lane straightness (%)

1,050 27

1,150 30

1,350 40

On the other hand, the latest progresses are summarized again for comparison in Table 2. Table 2. The comparison of the proposed work with the latest research progress of referenced papers Paper

Objective

Methodology

Visualization

Research progress

Proposed work

Optical surface design procedure of the free form side mirror for the automobiles

NURBS expression by numerical integration from the surface curvatures and its visualization

Yes

Ref. 5

Dynamic side mirror for the automobiles

Side mirror yaw angle control system to avoid collision

Driving simulator environment

Ref. 7

New automobile side mirror development (Our previous work)

Two circle blending by the cubic Bézier curve and its slumping

No

Ref. 11

Evaluation of the progressive addition lens of the free form surfaces

Direct free form surface height calculation and its contour power map conversion

No

An interactive optical surface design and its visualization before manufacturing step by visualization Demonstration of the dynamic mirror view control to detect the following vehicles Demonstration of an wider angle mirror concept inspired by the ophthalmic lens Nonoptical approach to simulate the progressive addition lens with the two freeform surfaces

Ref. 12

Progressive addition lens design method

NURBS expression by error function optimization to get surface curvatures

No

A numerical optimization method to get free form surfaces

Ref. 22

Visualization of invisibility cloaks Open-source graphic processing unit

Photorealistic rendering

Yes

Hardware and software to accelerate ray tracing

System development

Virtual demonstration of invisibility cloaks GPU and OptiX accelerator for ray tracing in optics Surface reconstruction by Zernike expression to measure optical surface Zernike polynomial to represent the free form optical surface

Ref. 24 Ref. 28

Algorithm for free form surface reconstruction

Curvature measurement and its Zernike polynomial matching

No

Ref. 29

Zernike polynomial expression of the free form surface

Zernike polynomial composition for curvature distribution

No

Final surface decision of free form shapes such as the ophthalmic lens, automobile side mirror depends upon the human responses, even though the surface data could be calculated by the direct numerical integration of the curvatures or surface numerical optimization to evaluate the wanted curvature distribution. Therefore, photorealistic rendering technique in optics can be a promising tool to realize the real scenery virtually, which not acquired easily before its manufacture according to the accelerated ray tracing hardware and software


environment. Also it can be an intuitive and interactive method to the optical designers, especially in the field of progressive lens, free form mirror, and more invisible cloaks. 4. Conclusion We have tried to demonstrate the usefulness of a visualization procedure during the vehicle mirror design process. In this work, the numerical integration of the surface curvature distribution and the following photorealistic ray tracing were done to compare the reflective scenes of the optical mirror surfaces. Three images, as visualization examples, were compared with the one horizontal FOV of 44.8° and the different vertical FOV of 49.4°, 46.8°, and 42.8°. And the lane straightness of 27%, 30%, and 40% were also examined. This technique allows us to determine how the reflection view of the free-form side mirror of the automobile appears to the driver in a driving environment. It is also beneficial as regards the design of automobile side mirrors, in that it allows the complex manufacturing implementation to be omitted. Therefore, it enables designers to evaluate the optimal design parameters in terms of rear-view performance and comfort, based on human response to the surface curvature distribution. Acknowledgments This research was supported by the Basic Science Research Program through the National Research Foundation of Korea (NRF), funded by the Ministry of Education (NRF2013R1A1A2007517).
