Correctly making panoramic imagery and the

Correctly making panoramic imagery and the meaning of optical center R. Barry Johnson Department of Physics Alabama A&M University Normal, AL 35762 ABSTRACT The production and viewing of panoramic scenes have fascinated people for over a millennium beginning with the camera obscura. With the advent of photography in the mid-1800s, techniques were developed to create panoramic scenes far larger than could be realized by a single camera. Making a panoramic scene from two or more images taken using a film camera was found to be challenging. Various advanced techniques for making panoramic images were developed during the 20th century and required specialized cameras and film processing. In recent years, digital cameras and powerful software programs have become readily available to aid in making panoramic imagery although often strange artifacts appears in the composite image. This is generally due to improperly locating the rotation axis of the camera. Interestingly, there is significant argument about where the axis of rotation should be for making proper panoramic imagery. The most common “answer” is the rotation axis should be about the second or rear nodal point, which will be shown in this paper to be incorrect. A second “answer” is to rotate the camera about its optical center. This is also incorrect; however, what actually constitutes the optical center of a lens and its applications will be briefly discussed. Examples of correctly and incorrectly produced panoramic image will be presented. Keywords: optical center, cardinal points, nodal imaging, panoramic, panorama, Gauss, Listing, nodal planes

1. INTRODUCTION The concept of cardinal points was introduced in the 1840’s and has continued to be of importance in the design and analysis of rotationally-symmetric lens systems. In this paper, a historical review of the origin of cardinal points, a discussion of the concept of optical center, and an explanation of about what axis a camera should be rotated when making panoramic images are presented. Perhaps one surprising conclusion is that the optical center is the seventh cardinal point from which all other of the cardinal points can be referenced since its location is independent of wavelength. Another conclusion is that the wide-field cameras should not be rotated about the second nodal point, but about the entrance pupil when capturing images for constructing panoramic images.

2. CARDINAL POINTS In 1841, Prof. Carl Friedrich Gauss (1777-1855) published his famous treatise on optics (Dioptrische Untersuchungen) in which he demonstrated that, so far as paraxial rays are concerned, a lens of any degree of complexity can be replaced by two principal points and two focal points, where the distances from the principal points to their respective focal points being the focal lengths of the lens. This was also the first time a formal or mathematical definition of focal length had been established. Gauss realized that imagery of a symmetrical lens system could be expressed by a series expansion where the first-order provided the ideal or stigmatic image behavior and the third and higher orders were the aberrations. He left the computation of the aberrations to others. Professor J. B. Listing (1908-1882) was one of eight of Gauss’ doctoral students. He received his degree in 1834. As a consequence of German politics, Prof. Weber and seven other professors at Göttingen were fired and Gauss was asked to recommend a replacement for Weber. Gauss suggested three possible candidates with Listing in third place. When the first two turned down the offer, Listing was appointed in 1839 despite never having published a paper. As professor of physics, Listing could choose his area for research. Although recognized as a versatile scientist, yet he chose another area to the ones which he had already worked in, and began to study the optics of the human eye. He published Beiträge

Current Developments in Lens Design and Optical Engineering IX, edited by Pantazis Z. Mouroulis, Warren J. Smith, R. Barry Johnson, Proc. of SPIE Vol. 7060, 70600F, (2008) 0277-786X/08/$18 · doi: 10.1117/12.805489 Proc. of SPIE Vol. 7060 70600F-1 Downloaded from SPIE Digital Library on 17 Nov 2011 to 76.29.243.168. Terms of Use: http://spiedl.org/terms

zur physiologischen Optik in 1845, which became a classic. In this work, Listing introduced the concept of nodal points in a lens system.

I Fig. 1. Basic optical system showing cardinal points where n < n’. Symbols are explained in the text.

Listing needed a means to describe a simple model of the eye. He determined that conjugate points having unit angular magnification exist and named them Knotenpunkte (or nodal points as they became known in 1860’s). This means that a nodal ray directed towards first nodal point will appear to exit the second nodal point without angular deviation (although spatial translation is typical). Listing also derived the imaging equations using nodal points and nodal planes. and N1 N 2 are equal, where P1 and P2 are the principal points, N1 and N2 are the He also proved that the distance PP 1 2 nodal points, F1 and F2 are the focal points, n and n’ are the refractive index of the object and image space, respectively. Listing’s method of using the nodal points and focal points to develop an imaging equation is actually an alternative to Gauss’ imaging equation. He determined that the nodal planes have lateral magnification of n / n′ and that the angular magnification of the principal points is also n / n′ . The validity of the nodal-planes lateral magnification is readily shown using Joseph Lagrange’s (1736-1808) theorem with an object of height h placed at N1, viz., h′n′θ = hnθ which implies the conjugate magnification is m = h′ h = n n′ . It can be further shown that that the anterior (front) and posterior (rear) focal lengths can be given as f = F1 P1 = F2 N 2 and f ′ = F1 N1 = P2 F2 . Listing evidently realized that his ray tracing of the eye would be simplified by using a mixture of Gauss’ and his formulations. Consequently, he used the unit lateralmagnification between the principal planes and the unit angular-magnification between the nodal points, thereby mitigating the need to scale heights or angles by n / n′ . It is also interesting to recall that Joseph Maximilian Petzval (1807-1891) Petzval was a well-known applied physicist and engineer when he accepted the Chair of Mathematics at the University of Vienna in 1837. In August 1839, Louis Daguerre disclosed his process for the photographic process to the Paris Academy of Sciences which created the requirement for a low F-number and wide field-of-view lens having good image quality. The camera he used had the Chevalier lens which was a very-slow achromatic doublet (F/15) and was considered to be unsatisfactory for portraiture. Petzval’s colleague Etingshausen, who had attended the meeting in Paris, proposed that Petzval design a proper lens for photography to which Petzval “enthusiastically accepted.” There appears to be a complete lack of evidence that Petzval, or anyone that he knew, had had any prior experience designing lenses or practical optics! Petzval secured the patronage of the Archduke Ludwig. The Archduke ordered that two artillery corporals and eight gunners skilled in computing be placed at his disposal. With this computational help, Petzval incredibly succeeded to complete two lens designs within six months. Both lenses were “double lenses” comprised of the same F/5 cemented doublet achromatic first element and an airspaced doublet second element that differentiated the two types of lens. The first was his well-known portrait objective (F/3.6), which was extraordinarily fast at that time. The F/8.7 wide-field lens, called the “Orthoskop,” produced images over a significanlty greater field-of-view than the portrait objective and was claimed to be free of distortion. The first lens was completed in May 1840, before Gauss even published his first-order theory! This lens was produced is significant quantities. It was several years before the second lens was fabricated. Few people recognize that Joseph Petzval not only developed third-order aberration theory in a form useful for practical optical design, but that he also extended it into higher orders as far as the ninth order! In addition, what isn’t said is that it is likely that Petzval “invented” a form of first-order optics similar to Gauss.1

Proc. of SPIE Vol. 7060 70600F-2 Downloaded from SPIE Digital Library on 17 Nov 2011 to 76.29.243.168. Terms of Use: http://spiedl.org/terms

It should also be noted that there are negative- or anti-principal planes and negative- or anti-nodal points, although these were apparently not discussed by either Gauss or Listing. Negative- or anti-principal planes are conjugate planes having negative unit lateral magnification. Negative- or anti-nodal points are conjugate points having negative unit angular magnification. These are located at ± the distance “f” from F1 and F2 . An example application is a lens used at unit magnification (m = -1) where the object is at 2f from the first principal point and the image is 2f from the second principal point. Further, it is noted that the cardinal points are intrinsic characteristic of the lens system and do not depend upon the aperture stop location or entrance and exit pupil positions. The entrance pupil and exit pupil locations depend upon the lens elements before and after the aperture stop, respectively. Consequently, imaging of the aperture stop by each of these two lens subgroups can be accomplished by using their respective cardinal points to locate the pupils.

3. OPTICAL CENTER

Fig. 2. The optical center is where the nodal ray crosses the optical axis.

The optical center of a thick lens, shown in Fig. 2, is defined to be the axial crossing point of a nodal ray aimed at N1 which also appears to exit the lens angularly undeviated from N2. A remarkable characteristic of the optical center is that it is spatially stationary with wavelength.2 Although no proof is provided in Ref. 2 as is almost no discussion, it is easy to show that the stationarity statement is true. It is noted that the drawing in Ref. 2 is taken from Ref. 4 without attribution, and that Ref. 4 cites Schroeder3. It isn’t clear who is the optical center concept originator, although Schroeder is the oldest paper found so far. The following is Johnson’s derivation and expansion of this concept. The optical center “OC” location of a single thick lens with n = n′ (Fig. 2) can be determined by realizing that the ratio of the height at each surface is equal to the ratio of the respective radii, i.e., y1 r1 = . y2 r2 Letting the distance from the first surface vertex to the OC be t1 and the distance from the second surface vertex to the OC be t2 , then the lens thickness t is given by t = t1 − t 2 . It is also evident that y1 y2

=

t1 t2

.

Solving for t1 in terms of radii, the optical center is located at t1 =

tr1 r1 − r2

=

t 1−

r2

=

t c1

.

1− r1 c2 The location of the optical center can occur before, between, or after the nodal points. For example, for a symmetrical biconvex lens ( r1 = − r2 ), the optical center lies exactly at the center of the lens and between the nodal points. For a lens


having r1 = 20, r2 = 5, t = 8 and N = 1.5 , both nodal points and the optical center are located behind the lens in the order N 1, N2, and OC . In the first example, a nodal ray transversing the lens physically crosses the optical axis at the optical center while in the second example it does not. In the second case, back projecting the transversing nodal ray locates the intersection with the optical axis. Also, when the radii have equal value and sign, the optical center is located at infinity. The optical center is conjugate with the nodal points; however, while the nodal points are related by unit angular magnification, the nodal-point to optical-center magnification ( mOC ) is not necessarily unity. In general, mOC is the ratio of the nodal ray slope angles at the first nodal point and the optical center. For a single thick lens with n = n′ , the magnification mOC can be readily shown to be given by mOC =

Noted that as t → 0 , mOC →

1

N

r1 − r2 N ( r1 − r2 ) − t ( N − 1)

.

and as t → r1 − r2 , mOC → 1 for all N .

All non-afocal rotationally symmetric lenses have an optical center just as they possess the cardinal points. Although a general proof is not given, it should be evident that is a valid statement. Should the aperture stop be located at the optical center, then the entrance pupil will be located at the first nodal point and the exit pupil will be located at the second nodal point with unity pupil magnification. This statement is true whether the lens is of symmetrical or unsymmetrical design. 2ND NOOPL POINT

EXIT PUPIL

OPIIOAL IENTEA

Ii a

ENTAPAOE PUPIL

P PA A T U A A XTU P

(a)

(b)

Fig. 3. (a) Double Gauss lens and (b) location of the lens’ nodal points, optical center, aperture stop, and entrance and exit pupils.

Arguably, the optical center can be considered the seventh cardinal point as it is conjugate with the nodal points and, hence, is related to the other cardinal points in a like manner. The optical center is spatially stationary with wavelength while the other six cardinal points are a function of wavelength for lenses containing dispersive materials. Fig. 3(a) depicts a typical Double Gauss lens showing the beam passages for three field angles. A fan of rays is shown in Fig. 3(b) directed towards the first nodal point. This bundle of nodal rays comes to a focus somewhat behind the aperture stop and then diverges. The exiting nodal rays appear to come from the second nodal point. The location where the bundle focuses is the optical center of this lens. In addition, the locations of the aperture stop, entrance, and exit pupils are shown. As mentioned previously, the nodal points and optical center are conjugates. So if one directs a fan of nodal rays at an angle such that they would appears to focus at height hN on the first nodal plane, then it will appear that the exiting fan from the lens originates from a “source” at the same height hN on the second nodal plane. In addition, the fan actually focuses on the optical center plane at height hOC = hN (u N uOC ) . Nodal imaging is no different than pupil imaging or object-image imaging in that the first-order properties are valid only for the paraxial region; however, cardinal points and their imaging equations provide very valuable insight into any


centered lens system. Furthermore, nodal and optical center images typically suffer aberrations not unlike those seen in pupil imaging. As mentioned above, should the aperture stop be placed at the optical center, then the entrance and exit pupils will be located at the respective nodal points and have unity pupil magnification. It was observed that the image quality of the Double Guass lens used for evaluation improved overall when the aperture stop was moved to the location of the optical center. 0mm

mm

1.7mm

= 100mm

Fig. 4. Rotation of lens about its optical center to displace image with possible application as an integrated optical micrometer.

A couple of additional applications of the optical center are simultaneous measurement of the chromatic variation of focal length of a lens and as an optical micrometer. Typically flat plats are tilted to create an optical micrometer in an autocollimator or theodolite. An alternative may be to rotate the eyepiece in autocollimator or theodolite about its optical center thereby combining the functionality of the traditional eyepiece and tilting flat plates. Fig. 4 shows the Double Gauss lens (not really an eyepiece of course) being rotated about its optical center and the image displacement for three rotational angles. The focus of the lens is upon the autocollimator or theodolite reticule.

4. PANORAMIC IMAGING Photographers have been performing panoramic imaging for about a century. The three most common camera forms are the so-called stovepipe, moving-film, and multiple lens cameras. The stovepipe camera comprises a lens and a slit fixed together that rotate about N2 with stationary curved film having a radius = f. The total scanned field-of-view (FOV) can be up to about 120º although the slit keeps the horizontal FOV very narrow. The aspect ratio is ≅1:2.3. The moving film camera is similar to the stovepipe camera except that the film moves with the slit and lens. The film mechanism is synchronized with the lens rotation. This allows the FOV to be a full 360º and to realize no distortion if the object is appropriately curved. The multiple-lens camera typically comprises 4 to 8 lenses carefully aligned. The film is curved and the FOV is limited compared to the other two approaches. Image blending can be a problem. Panoramic imaging using digital cameras has become popular in recent years. A question that seems to plague the photographic community is “About what axis should the camera be rotated?” A number of “answers” are offered and argued in the literature (books, journals, forums, and blogs). A summary of these “answers” are as follows: • About first Nodal Point (uncommon) • About second Nodal Point (most common) • About “the” Nodal Point (uncommonly common) • About Optical Center (except they meant N2) • About the Entrance Pupil (modestly uncommon) • About the Image Plane (rare, but often what people do because they don’t have required special equipment) So what is the issue? It is parallax induced errors! A panoramic image is made by software stitching of two or more overlapping digital camera images, each having significant FOV (10-40º being common). Parallax errors can occur when objects are both near and far from the camera. If the objects are all very distant, then the rotation axis location is not too important. As will be demonstrated in an example below, parallax errors can cause often bizarre artifacts in the


composite image generated by the software. Typical programs used to create stitched panoramas are Adobe Photoshop CS2 and CS3, and PTGuiPro.

Distani

Distani

Near Near

(a)

(b)

Distant

NeaF

NeaF

Distant

(c)

(d)

Fig. 5. Lens rotated about its (a) optical center, (b) second nodal point, (c) front vertex, and (d) rear vertex. Principal rays for both a very distant object and a near object are designated.

Fig. 5 shows the Double Gauss lens when rotated about various points. The rays shown are the principal rays from the distant and near objects to illustrate the parallax issue. As is evident by inspection of Fig. 5, the distant image is higher on the detector plane than the near image when the lens is rotated about the optical center, second nodal point, and front vertex. When the lens is rotated about the rear vertex, the images reverse position. Fro this lens, there is clearly a rotation axis position between the optical center and the rear vertex where the parallax errors are mitigated.

Fig. 6. Lens rotated about the entrance pupil.

Fig. 6 shows that there is no parallax error when the lens is rotated about the entrance pupil. For those skilled in geometrical optics, this should be evident. For the classical panoramic cameras, rotation about the second nodal point is correct since the image formed by the lens is stationary on the film and parallax isn’t an issue due to the slit limiting the scanned direction instantaneous FOV. These slit-based cameras require significantly longer exposure time than does the digital camera form or multi-lenses camera. To properly make panoramic images using a digital camera, it is helpful to have appropriate attachments for the camera. Really Right Stuff manufactures both single and dual axis nodal slides that can allow proper positioning of the camera.5 In the optical shop, a standard technique for determining the focal length of a lens and the location of the principal points is to rotate the lens on a nodal slide as the lens is repositioned until the image of a point source remains spatially stationary. A similar technique can be used to locate the entrance pupil position. The target source projector needs to include two source where one is projected as being at infinity and the other appears near to the lens under test. Such a source is relatively simple to construct. When viewing these sources, the lens rotated on a nodal slide as the lens is


repositioned until the observed separation of the image of the two point sources remains constant. The image does move as the lens is rotated; it is the separation that remains constant (if the sources are aligned along the optical axis, then the images will be superimposed when the lens is rotated about the entrance pupil). The exit pupil can be located by reversing the lens.

\I

Fig. 7. Overlapping images taken with the camera being rotated about its entrance pupil.

Fig. 8. Images in Fig. 7 stitched together using Photoshop CS3. No parallax induced error seen.

Fig. 7 shows three overlapping images taken with the camera being rotated about its entrance pupil. Fig. 8 shows the result when these three images were stitched together using Photoshop CS3. Notice the relationship of the near candle and the door frame in the background. This is an example of a properly made panoramic image.

——

Fig. 9. Overlapping images taken with the camera being rotated about its image plane.


Fig. 10. Left and right images in Fig. 9 stitched together using Photoshop CS3. Arrow shows a parallax induced artifact.

(a)

(b)

Fig. 11. (a) Images in Fig. 9 stitched together using Photoshop CS3. Arrow shows primary parallax induced artifact. (b) Close-up of (a) so that both parallax induced artifacts can be observed.

CONCLUSIONS The optical center has been arguably shown to be the seventh cardinal point and has seemingly been ignored for perhaps a century or more. Possible applications of the optical center include forming unit-magnification entrance and exit pupils at the respective nodal points, construction of an integrated optical micrometer, and simultaneous measurement of the chromatic variation in focal length of a lens. There are likely other applications of the optical center that will evolve. It has been shown that for making panoramic images using a digital camera, the correct rotation axis is at the entrance pupil (for both horizontal axis and vertical axis). Since this mitigates the parallax error, it was also shown that this technique can be used in the optical laboratory/shop to locate the position of the entrance pupil.

REFERENCES [1] [2] [3] [4] [5]

Andrew Rakich and Raymond Wilson, “Evidence supporting the primacy of Joseph Petzval in the discovery of aberration coefficients and their application to lens design,” Proc. SPIE, 6668, 66680B-1-13 (2007). F. Jenkins and H. White, Fundamentals of Optics, Third Edition, McGraw-Hill, Ney York, 72 (1957). H. Schroeder, "Notiz betreffend die Gaussischen Hauptpunkte," Astron. Nachrichten, 111,187-188 (1885). H. Erfle, "Die optische Abbildung durch Kugelflaechen", Chapter III in S. Czapski und O. Eppenstein, Grundzuege der Theorie der Optischen Instrumente nach Abbe, 3rd ed., H. Erfle and H. Boegehold, Eds., Leipzig:Barth,.72-134 (1924). Really Right Stuff, 205 Higuera St., San Luis Obispo, CA 93401 USA
