dimensionnelle d'une Scbne a partir d'une Cambra en Mouvement: de I'Injuence de la Precision. PhD thesis, UniversitC Claude Bernard, Lyon I, October.
What Accuracy for 3 Measurements with Cameras? Giannoula FLOROU and Roger MOHR GRAVIR-INPG-INRIA 655 avenue de 1’Europe F-38330 Montbonnot FRANCE Cedex E-mail: f lorou@macedonia . uom.gr mohr@imag . fr
Abstract
Methods that use geometric properties. They use objects whose images have some characteristic that are invariant to the actual position of the object in space and can be used to calibrate some of the internal camera parameters [4].
We estimate the internal and external parameters of the camera, and simultaneously the distortion S parameters. Our aim is the selection of the best distortion model, using stiatistical test for the importance of distortion parameters. Also we examine the accuracy in camera parameter estimation and 3 0 reconstruction, in relation with the noise in the image. We answer the question “until which level of noise, is it possible to obtain a good camera parameter estimation, and from there a good reconstruction?’’ Experiments are evalueted on simulated data and finally a test is performed with real data.
Methods that don’t require known calibration points but they require camera motion and more than one image 171. Methods that use known world points. This is the standard calibration method and it works with well designed calibration frame [9]. We will use such a method in section 3. The difference of our approach, is the distortion parameters estimation done simultaneously with the camera calibration.
1. Introduction
Works about the importance and precision of distortionparameters are refered in [2].
1.1. The problem
2. Modeling the Camera We have one or more images, taken by the same camera, of an objet in the 3D space. We know the 3D coordinates of some points in the objet, but we do not know the position of the camera in the 3D coordinates system. Camera calibration is the estimation of the parameters i n the model for the transform between the world (3D) and camera coordinate (2D) frame. It is a key issue for deriving metric information from images. We assume that the 2D points (points in image) are known, with an accuracy modeled as noise in the image. We want to find the relationship between image COordinates and 3D coordinates, i.e. the complete model for the transformation between the world and camera coordinate frames. Our aim is the link of the accuracy in the estimation of camera parameters and the quality of reconstruction.
2.1. The pin hole model The standard camera model is the pin hole model of perspective projection. Given the position of a point in 3D world coordinates, the model predicts the position of the point’s image in 2D pixel coordinates. Calibration data for the model consists of 3D ( 2 ,y, z ) world coordinates of a feature point and corresponding 2D ( U , U) coordinates (in pixels) of the feature point in one or more images. The image plane is parallel to the (X,Y) plane and the Z axis coincides with the optical axis. Then the transform between the world and camera coordinate frames, is represented by a 3x4 matrix M, and it can be writen in homogeneous coordinates, as:
1.2. Related works
Many methods were proposed for this calibration process: the reader is refered to [ 101 for a review. Methods can be divided into the following categories:
1015-4651/96 $5.00 0 1996 IEEE Proceedings of ICPR ’96
354
where: M= n
where: n the number of points, u t ,vi are the predict coordinates of the point i in image frame (pixel), U : , v: are the mesured coordinates of the point i in image frame (pixel). As such a minimization is non linear problem, we use a non linear optimisation program based on LeverbergMarquardt algorithm [8]. Parameter estimation is done by minimising (3). This can be performed using the full model given by equations (2). From ( 2 ) , we can estimate the distortion model and the camera parameters, simultaneously. Such a method requires however a non linear minimization and therefore it relies on an initial estimate which should be close enough to the final solution. We use the Faugeras-Toscani [5] method for computing the matrix M and then, using QR decomposition of this matrix, we have an initial estimate for the camera parameters, Assuming no distortion, the distortion parameters are set to zero. After this step, Leverberg-Marquardt algorithm is used to find the optimal solution minimizing the imagecoordinate error (cost function (3)).
The model (without distortion) for this transormation, has 10parameters. Four internal for image scaling constants ( a , , ~ , ) and the image centre position (uo,w0), and six external parameters for the camera coordinate system's position and orientation relative to the world system. The external parameters are : e l , e 2 , e3 the rotation angles if we parametrise the rotation matrix R between world coordinate system and camera coordinate system, with Euler angles. 2 1 , t 2 , t 3 the translation components for the transform between the world and camera coordinate frames.
2.2. Modeling the distortion Distortion provide systematic correlated errors in the image position. It is usually modeled with 5 parameters [4], [?I. These parameters are:
kl, k z , IC3 for radial lens distortion coefficients
PI, P2 for decentering distortion coefficients The formula is U
PI,
+
/cliir2
=a,
+ +
+ + +
+ +
+ +
+ + +
4. Experiments with simulated data
+ kzlir4 + /c3iir6 + P~(z? + r 2 ) + 2 ~ z t i 0
R 2 1 ~ R22y R 2 3 Z t 2 R ~ I XR32 * y R 3 3 2 t 3 + k 1 o r Z+ k20r4 + k 3 m 6 + 2 ~ ~ +a p2(202 a +r2) U = a,,
V,
+ +
R I Ix Rizy R I ~ Zt l R31-7: R ~ Z YR 3 3 ~ t 3
For the synthetic data, they are 60 point in 3D space. The 60 points were randomly scatted in a sphere of radius 14 unit. The cameras were given random orientations and were placed at various distances from the centre of the sphere with a mean distance 70 units. The focal length is 15" and the scene size is almost 45 units. These 3D points are projected in the image, and theirs locations are perturbed with different gaussien noises with mean 0 and standard deviation of 0.05, 0.1, 0.2 or 1.O pixel. Furthemore, we added to each point ( U , v), radial distortion (6,, 6,) obtained from the following formulaes:
(2)
There are alternative non parametric methods for correcting the distortion [3] They correct the distortion of points in the image, before the proccess of calibration, and they use a simple geometric property: straight lines in the space have to be projected as straight lines in the images. In our estimation, we do not know relations between points, and we estimate the model for camera parameters and distortion parameters, simultaneously. As a result this provide a complete camera model for the transformation between the world (3D) and camera coordinate (2D) frames.
where ( u o ,U,) is the coordinates of the principal point, r 2 is the distance between the principal point and ( U , v), and kl is the distortion parameter. For each level of noise 10 simulated images were generated. From them the parameters and the mean and standard deviation for each parameter were estimated, from the results.
3. Estimation method We estimate tlhe model by a standard least suare fitting which minimizes the error in the image location of points with cost function:
355
Figure 1. Estimation of uo with and without model for data with maximum distortion 1.0 (left) and 5.0 (right) pixels
Table 1. Estimation of camera parameters without distortion model for synthetic data having a maximum distortion of 5.0 pixels.
between nombre of parameters in the model and accuracy of the estimation. For this reason, we use statistical tests.
We simulated camera with radial distortion 0.5, 1.0 or 5.0 pixel at the image corner (cf. equations (4)). We have estimated the camera model without distortion parameters. The results of camera parameters estimation, are summurized in the table 1. Notice that the camera parameters are not correctly estimated. The confidence interval of parameters (mean values & 3 standard deviation) doesn’t include the true value. This because the distortion has an influence in the model. Distortion parameters and camera parameters are also highly correlated. Introducing a model including the distortion parameters, the estimations are slightly better. However, we can find smaller errors with smaller standard deviation for the parameters u o , v O , without using model of distortion, when the noise (1.O pixel) is larger than the distortion 1 .O pixel at the image corner. Figure 1 displays the estimation for u 0 with its standard deviation, in the case of small ( I pixel maximum) and median distortion ( 5 pixels maximum). Considering the case of small distortion, it has to be noticed that, when estimating uowith a model withoutdistortion parameters, the result is better than with distortion parameters, as the noise reaches at r = 1 pixel (cf. left part of figure 1). The reason is that distortion effects are absorbed by the noise. But when the distortion is larger,observing the right part of figure 1 , we see that the parameter estimation using distortion model is much better.
5. Selection of right camera model As we have seen, we need to select the parameters of distortion, which are significant. For this, we propose to validate them, using statistical tests. We suppose that errors are normally distributed with mean 0. We are interested in finding the importance of the parameters of distortion in the model. For every parameter of distortion we make the folowing hypothesis: Ho: we considere that the parameter i is not significant and is set to 0 HI : we considere that the parameter i is significant In order to validate these hypothesis, we compute the confidence regions. A confidence region is a region of parameter values that contains a certain percentage of the total probability distribution. For example: there is a confidence level 95% that the true parameter values fall within this region around the measured value. The more the region is large, the more this percentage is large too. Regions are designed to have the smallest area (volume), for a given level of Confidence. In case of gaussien distribution they are limited by confidence ellipses. If we have only one estimation of parameters (real data) we can find the confidence intervals, for every parameter in the model, using the obtained variance-covariance matrix C, which is provided by the Leverberg-Marquardt algorithm, as an additional result. For the parameter A,, the confidence interval CY% is given by the formula [ l ] :
4.2. Number of distortion parameters in the
model It is well known that radial distortion is the major distortion source; the question is to derive how much parameters are needed as we can have a model for distortion with 1,2,3, 4 or 5 parameters. If we have more parameters in the model, we obtain smaller error of estimation, but larger confidence interval. We will discuss this in section 7. We need to select
356
where: bX, xi is the value of y distribution for the significant level a% f the value of objelctive function with the estimated parameters n the number of points J is the variance of the residuals C the variance-covariance matrix (DO)-' and D is the jacobi matrix of the estimated parameters In the results we computed the 6 X i value for a = 90% and we can see if the parameters of the distortion model are significant or not. If this value is bigger than the estimated value (that is 0 is included in the confidence region), the parameter is not significant (there is a probability only 10% that it is not equal to 0).
reconstruction error is very small for angle 40 or 45 degrees, if the noise is 0.05,O. 1 or 0.2 pixels.
=a&?;
7. Real data
Figure 2. images of the calibrated reference frame
6. Quality of reconstructed points
In figure 2, we see two images of a calibration frame. We find the locations of 3D points in the image with subpixel accuracy 0.05 pixel [3]. The results with real data are seen in the table 2, for real images (figure 2) with distortion. We don't know the true values of parameters. In order to validate the results, we compute only the reconstruction error, since we know the 3D points coordinates. We use the half part of the data to estimate the parameters, and the other half part to reconstruct the scene. Thus we have no influence of estimation process in the reconstruction. We estimate first, a model without parameters of distortion. Secondly we consider the 5 distortion parameters, and finally only the significant distortion parameters are put into the model. We did these estimations separately, for each image and also simultaneously for two images. In the estimation with both images, the parameter estimation is more accurate, as it relies on more data. In table 2 we have two images taken with a lens having distortion. Remember that there is a * before a number if the correspondant parameter is considered as being signifiacant. In practice, we consider only parameters for which the dX, is close to its estimated value. For the model without parameters, the reconstruction error is 0.70 If we use model with 5 parameters of distortion, the error is only 0.55 lop3 and if we use model with only IC1 , P I ,Pz parameters, the erUsing the distortion model, we divide error ror is 0.56 by a factor close to 2. We have to select the model with kl, P I ,P2.where we have the smaller reconstruction error, and the smaller confidence interval too.
If we estimate the parameters for at least two cameras, we can reconstruct the 3D points by triangulation [6]. We investigate here the accuracy of the 3D reconstruction from the accuracy of the camera parameters. We assume that only camera parameters are source of errors; no error is assumed in the image measurements for these steps. So when we refer to noise, this is the noise with which camera parameters were estimated. After correcting the distortion at every point, we find the line which is supposed that join the point in the image (u,v) with the correspondant 3D point (X,Y,Z). The relative error of reconstruction is the mean of the difference of distance between two 3D points and the two points 3D reconstructed. We find the mean distance for all couple of points, and we divide it by the maximun distance. The formula is:
max
'>J
Jm (5)
where: Pi,Pj are the known 3D points Pil Pi are the recontructed 3D points, and d,(Pi - P i ) is the: euclidean distance between the 3D points P; Pj . The reconstruc:tiondepends on the relative position of the two cameras. If the angle between the two camera axis is too small (we call this angle, rotation angle), it is well known that the reconstrwtion degenarates. We made experiments using rotation angle between the two camera position, of 10, 20,30, and 45 degrees. The Z-axis coincides with the optical axis and we rotate: the camera around the Y-axis. The camera is also translated in order to observe the same scene. The
8. Conclusion We have described the process of camera calibration and distortion model estimation, when we want to find an accur-
357
6A, Wllh
I
paclmeicr
324
23 286 IZY
-1hll Y
IO89.2 I9 3 1W5 I
34.5
23.5
6X,
47.6 -la53 29
m a g l&Z 6A,
“e2
2x1.’) 18.5
71 220 I1 8 239 3 Y5
7.54 e-ON .Y 01 c-OX -4 16 e-I18 -7.61 e-UN *563 e m
0
0
11
n
0
I)
n n
0 0
(1
n
0 56
0 0 (1
0
056
n
n 72
0
0
n
Table 2. Estimation of parameters and relative reconstruction error for real data ate model for the transformation world-image. If we use a model with many distortion parameters, we have a big confidence interval for every parameter in the camera and distortion model. When we want accuracy in the model estimation, we need to diminuer this interval. So, we made statistical test for the significance of parameters, to find only the important distortion parameters in the model. If the distortion is much smaller than the noise in the image, the use of distortion model, causes an error in the camera parameters estimation, because the effect of noise is more important than the effect of distortion. In this case, it is better to use a camera model without distortion. If we want parameters estimation with accuracy smaller than 0.1 units, we need to have image location with accuracy until 0.1 pixel. If the distortion is greater, it is worth to introduce distortion model, particulary if images measurements are accurate (with subpixel accuracy). We have seen that using a good distortion model (having only the important parameters), we have a better camera parameters estimation and a better image location after correction of distortion. Hence we can obtain a good accuracy in the reconstruction. For real data with large distortion, the gain in 3D reconstruction is better about a factor 2 or more, if we use the right distortion model. Also, we have seen that the z-axis rotation angle e l , is estimated well, because is not correlated with the others camera parameters. But the other camera parameters are correlated and a small error in camera parameters estimation has an large influence in the 3D reconstruction error.
System. PhD thesis, ETH-Zurich, 1992. [3]
P. Brand. Reconstruction Tridimensionnelle d’une Scbne a partir d’une Cambra en Mouvement: de I’Injuence de la Precision. PhD thesis, UniversitC Claude Bernard, Lyon I, October 1995. ftp://ftp.imag.fr/pub/MOVYtheses/brand.ps.gz.
[4] D.C. Brown. Close-range camera calibration. Photo-
grammetric Engineering, 37(8):855-866, 1971.
[5] O.D. Faugeras and G. Toscani. Camera calibration for 3D computer vision. In Proceedings of International Workshop on Machine Vision and Machine Intelligence, Tokyo, Japan, 1987.
[6] R. Hartley and P. Sturm. Triangulation. In Proceedings of ARPAImage Understanding Workshop, Monterey, California, pages 957-966, November 1994.
[71 R.I. Hartley. Self-calibration from multiple views with a rotating camera. In Proceedings of the 3rd European Conference on Computer Vision, Stockholm, Sweden, pages 471-478. Springer-Verlag, 1994.
S.A. Teuk[BI W.H. Press, B.P. Flannery, olsky, and W.T. Vetterling. Numerical Recipes in C. Cambridge University Press, 1988. [9] C.C. Slama, editor. Manual of Photogrammetry,fourth edition. American Society of Photogrammetry and Remote Sensing, Falls Church, Virginia, USA, 1980. [lo] G.P. Stein. Accurate internal camera calibration using rotation, with analysis of sources of error. In E. Grimson, editor, Proceedings of the 5th International Conference on Computer Vision, Cambridge, Massachusetts, USA, pages 230-236. IEEEComputer Society Press, June 1995.
References [ 11 A. Antoniadis, J.Berruer, and R.Carmona. Regression non lineaire et applications, volume 1 of economie et
statistiques avancees. Economika, 1992. [ 2 ] H.A. Beyer. Geometric and Radiometric Analysis of a CCD-Camera Based Photogrammetric Close-Range
358
