Maximum Likelihood Robust Regression with Known and Unknown Residual Models Sami Brandt Helsinki University of Technology, Laboratory of Computational Engineering P.O. Box 9400, 02015 HUT, Finland
[email protected] Abstract The first part of this paper is mathematical. We will prove formally that by considering independent residual models for “good” residuals and outliers and minimizing the sum of squared observed residuals weighted by the estimated posterior probabilities to be correct is consistent estimator for the true parameter values. The assumptions made are similar to assumptions needed for consistence of the ordinary maximum likelihood estimator. The result hence proves the optimality of the robust estimator in the sense that the ML estimate for the model is asymptotically achieved regardless of outliers in the data. The second part of the paper considers robust estimation in cases where the parametric form of the outlier distribution is unknown. We propose that the total residual distribution could be modeled by mixture of Gaussians, and the minimum description length principle used in the model selection, i.e., determining the number of kernels. The third part of the paper considers some experiments. We show how the proposed robust estimator can be used in image geometry estimation problems such as estimating the fundamental matrix.
1 Introduction Robust estimation is a central problem in many research communities including computer vision. The unsolved difficulty has been in dealing with outliers, the rogue observations that are not explicable by the underlying model. The classical approach in regression diagnostics [12] is first computing a least-squares fit, then removing the points whose residuals exceed a pre-defined threshold, and iterating until the outliers are removed. An improved way is to use some influence measures to pinpoint potential outliers of which a good example is [14]. Robust statistics has the same goal as regression diagnostics, but the procedure goes in the opposite order since after the good robust fit the outliers are to be identified if necessary. One of the best known methods is the least median of squares (LMedS). [11] M-estimators [8] are proposed to replace the square function in the LS by another symmetric, positive definite function that obtains unique minimum
at zero. In the computer vision community, some popular methods are based on RANSAC [6, 15] where the idea is to randomly sample minimal configurations and to select the one that maximizes the number or some other quantity of residuals that fall inside some predefined threshold. The known procedures have following problems: 1. In robust statistics the residual form is fixed in advance. Therefore it is not adaptive to the contaminating residuals that however may be in some sense independent from the relevant ones. 2. Methods in robust statistics assume symmetric residual distribution. There is no reason for this in general since the contaminants may be totally asymmetrically distributed and that causes the estimates to be biased. 3. Even if the original “good” residuals alone were normally distributed, the residual after the outlier removal is not, since there will be false rejections and false retentions. 4. The classification of residual values to good and false is an ill-posed problem since the false observations can also have small residual values. This is due to that the residuals distributions of the good and contaminating observations residual overlap in general. To cope with the problems above we must define an outlier in a slightly different way than they are usually considered. In this paper outliers are understood as the residuals of false observations. False observations are the contaminating data that are generated by another stochastic process we are not intended to model. Accordingly, this definition allows that the outliers generally have a distribution that overlaps the residual of the good observations and being an outlier does not imply a large residual value. In addition, there is neither a reason to assume that the outlier distribution would be symmetric with respect to the correct distribution. From this view point have introduced a robust estimator [3] that solves the problems described above. This paper continues that work by generalizing and proving the optimality of the approach.
2 Outlier Free Regression Model First we define a model in a general way such that the setting covers both linear and nonlinear regression.
be an observation and the Definition 1 Let set of model parameters where and are metric spaces. The model is the functional for which holds: the model explains the observation perfectly with parameters .
If and given the parameters understood as a hyper surface in .
, a model can be
)* +-,
Definition 2 Let the null space of the model with respect to the first argument be and a distance function in the observation space . The residual corresponding to the observation is the signed distance between the observation and the null space
#! "%$'&( ./
0 2 14:8 95=-7 ? @%)*A +-, (1) where 1B C-3ED
