Semi and Nonparametric Models in Econometrics - Part I ... - Crest

Semi and Nonparametric Models in Econometrics

Semi and Nonparametric Models in Econometrics Part I: quantile regression

Xavier D’Haultfœuille CREST-INSEE

Semi and Nonparametric Models in Econometrics

Outline Model and motivation Inference in quantile regressions Additional properties Quantile regression in practice Quantile IV Quantile restrictions in nonlinear models

Semi and Nonparametric Models in Econometrics Model and motivation



Prologue: quantiles I

The τ -th quantile (τ ∈ (0, 1)) of a random variable U is defined by qτ (U) = inf{x/FU (x) ≥ τ }, where FU denotes the distribution function of U. Note that when FU is strictly increasing, qτ (U) = FU−1 (τ ). Otherwise, qτ (U) satisfies for instance:

 

q(U)

q(U)


Prologue: quantiles I

The quantile function τ 7→ qτ (U) is an increasing, left continuous function which satisfy, for all a > 0 and b: qτ (aU + b) = aqτ (U) + b.

I

Caution: qτ (U + V ) 6= qτ (U) + qτ (V ) in general.

I

Conditional quantiles are simply defined as:

(1)

qτ (Y |X ) = inf{u/FY |X (u|X ) ≥ τ }. I

Similarly to conditional expectations, conditional quantiles are random variables (as they depend on the random variable X ).


The model

I

Let Y ∈ R and X ∈ Rp , we consider here a model of the form Y = X 0 βτ + ετ , qτ (ετ |X ) = 0. Equivalently, we have qτ (Y |X ) = X 0 βτ .

I

This model is similar to the standard linear regression, except that we replace the conditional expectation E (Y |X ) by a conditional quantile.


First motivation: measuring heterogenous effects I

The effect of a variable may not be the same for everybody. We ignore this fact in standard linear regression by focusing on average effects.

I

However, such heterogeneity may be important for policy reasons. We may also want to test for homogeneity.

I

Consider for instance the “location-scale” model: Y = X 0 β + (X 0 γ)ε, where ε is independent of X and we suppose X 0 γ ≥ 0. Then, by (1): qτ (Y |X ) = X 0 (β + γqτ (ε)) . In other terms, βτ = β + γqτ (ε).


First motivation: measuring heterogenous effects I

In the location scale model, βOLS = β but running OLS we miss the fact that the effect of X differs according to quantiles of the Quantile Regression unobserved variable ε.

I

Example of the Engel Curves: 2000

4

•

1500

•

1000

•

500

Food Expenditure

• •

•

•• • • • •• •• •• ••• ••• •• • • • • • • • ••• • • • • • • • ••••• •••••••• • • • • • • •• • ••••• •••••• • • • • •• ••••••• • •••• ••••• •• •••••••• • • •••••••••••••••••••••••• •• • • •••••••••••••••• •• ••••• ••• 1000

2000

•

•

• •

•

3000

4000

5000

Household Income

Data taken from Engel’s (1857) and Koenker and Hallock (2001). Seven estimated quantile regression lines for

Engel Curves for Food: This gure plots data taken from Ernst Engel's 1857 study of the dependence of households' food ex-

Figure 2.2. The median is indicated by the dashed line while the OLS estimate is the dotted line. different values of quantiles.


Second motivation: robustness to outliers and to heavy tails I

We want to draw inference on a variable Y ∗ but observe, instead of Y ∗ , “contaminated” data Y = CX 0 α + (1 − C )Y ∗ , where C = 1 if data are contaminated, 0 otherwise (C is unobserved). We suppose that p = P(C = 1) is small but X 0 α is large.

I

Consider first a linear model E (Y ∗ |X ) = X 0 β. Then, instead of β, OLS estimate (1 − p)β + pα. The bias p(α − β) may be large even if p is small.

I

Now consider the quantile model qτ (Y ∗ |X ) = X 0 βτ . τ τ . In this case, qτ (Y |X ) = X 0 β 1−p so instead of βτ , we estimate β 1−p It is independent of α and will typically be close to βτ . If some components of βτ are independent of τ (homogenous effects), the contamination does not affect their estimation.


Second motivation: robustness to outliers and to heavy tails I

In a similar vein, consider a linear model Y = X 0 β + ε, X ⊥ ⊥ ε.

I

If ε is symmetric around zero, we can estimate β with OLS or median regression but we may prefer to estimate it with median regression if ε has heavy tails.

I

Indeed, if E (|ε|) = ∞ (examples ?), OLS are inconsistent whereas the median is always defined. One can show that estimates using median regression are consistent.

I

Useful in finance, insurance...

Semi and Nonparametric Models in Econometrics Inference



The check functions I

It is easy to estimate the τ -th quantile of a random variable Y : we simply consider the order statistic Y(1) < ... < Y(n) and estimate qτ (Y ) by bτ (Y ) = Y(dnτ e) , q where dnτ e ≥ nτ > dnτ e − 1.

I

It does not seem obvious, however, to generalize this to quantile regression.

I

The key observation is the following property:

Proposition Consider the check function ρτ (u) = (τ − 1{u < 0})u. Then: qτ (Y ) ∈ arg min E [ρτ (Y − a)] . a


The check functions Proof: suppose for simplicity that Y admits a density fY . Then we have Z a E [ρτ (Y − a)] = τ (E (Y ) − a) − (y − a)fY (y )dy . −∞

This function is differentiable, with ∂E [ρτ (Y − a)] = −τ − (a − a)fY (a) + ∂a

Z

a

fY (y )dy = FY (a) − τ. −∞

This function is increasing, thus a 7→ E [ρτ (Y − a)] is convex and reaches its minimum at qτ (Y )


The check functions I

The minimum need not be unique (there may be several solutions to FY (a) = τ ). When Y is not continuous, there may be no solution to FY (a) = τ but we can still show that qτ (Y ) is the minimum of E [ρτ (Y − a)].

I

The τ -th quantile minimizes the risk associated with the (asymmetric) loss function ρτ (.). This is similar to the expectation which minimizes the risk corresponding to the L2 -loss : E (Y ) = arg min E (Y − a)2 . a

I

Similarly to conditional expectation, we can extend the reasoning to conditional quantiles. We have qτ (Y |X = x) = arg min E [ρτ (Y − a)|X = x] . a

X

Thus,integrating over P , (x 7→ qτ (Y |X = x)) = arg min E [ρτ (Y − h(X ))] . h(.)


Definition of the estimator I

Suppose that qτ (Y |X ) = X 0 βτ . We have, by the preceding argument, βτ ∈ arg min E ρτ (Y − X 0 β) . (2) β

I

We use this property to define the quantile regression estimators. Suppose that we observe a sample (Yi , Xi )i=1...n of i.i.d. data, we let n 1X b ρτ (Yi − Xi0 β). (3) βτ ∈ arg min β n i=1

I

N.B.: when τ = 1/2 (median), this is equivalent to minimizing n

1X |Yi − Xi0 β|. n i=1

The corresponding solution is called the least absolute deviations (LAD) estimator.


Identification I

Before proving consistency of the estimator, we have to prove identification of βτ by (2). In other words, is βτ the unique minimizer of β 7→ E [ρτ (Y − X 0 β)]?

I

One can show that this holds if the residuals are continuously distributed conditional on X and the matrix E fετ |X (0)XX 0 is positive definite (very similar to the rank condition in linear regression).

I

N.B.: this fails to hold when fετ |X (0) = 0, which is logical because as mentioned before in the unconditional case, the minimizer of (2) is not unique when the d.f. is flat at τ .


Consistency I

Achieving consistency of βbτ is not as easy as with OLS because we have no explicit form of the estimator.

I

We may use the special feature of ρτ , or use general consistency theorems on M-estimators defined as n 1X ψ(Ui , θ). (4) θb = arg min θ n i=1

Theorem

(van der Vaart, 1998, Theorem 5.7) Let Θ denote the set of parameters θ and suppose that for all δ > 0: n 1 X P sup ψ(Ui , θ) − E (ψ(U1 , θ)) −→ 0, (5) θ∈Θ n i=1

inf θ/d(θ,θ0 )≥δ

E (ψ(U1 , θ))

>

E (ψ(U1 , θ0 )).

Then any sequence of estimators θbn defined by (4) converges in probability to θ0 .

(6)


Consistency I

Here Ui = (Yi , Xi ) and ψ(U, θ) = ρτ (Y − X 0 θ).

I

Condition (6) is a “well-separated” minimum condition, which is typically satisfied in our case under the identification condition above and if we restrict Θ to be compact.

I

The first condition is the most challenging. By the law of large numbers, we have pointwise convergence but not, a priori, uniform convergence. To achieve this, we may use Glivenko-Cantelli theorems.

I

The idea behind is that if the set of functions (ψ(., θ))θ∈Θ is not “too large”, one can approximate the supremum by a maximum over a finite subset of Θ and applies the law of large numbers to each of the elements of this subset.


Consistency Example: the standard Glivenko-Cantelli theorem. Let us consider the functions ψ(x, t) = 1{x ≤ t}. Then, if Y1 is continuous: n 1 X P sup ψ(Yi , t) − E (ψ(Y1 , t)) −→ 0. n t∈R i=1

N.B.: letting Fn denote the empirical d.f. of Y , this can be written in a more usual way as P sup |Fn (t) − F (t)| −→ 0. t∈R

Proof: fix δ > 0 and consider t0 = −∞ < ... < tK = ∞ such that F (tk ) − F (tk−1 ) < δ. Then for all t ∈ [tk−1 , tk ], Fn (t) − F (t) ≤ Fn (tk ) − F (tk−1 ) ≤ Fn (tk ) − F (tk ) + δ Similarly, Fn (t) − F (t) ≥ Fn (tk−1 ) − F (tk−1 ) − δ. Thus, |Fn (t) − F (t)| ≤ max{|Fn (tk ) − F (tk )|, |Fn (tk−1 ) − F (tk−1 )|} + δ.


Consistency As a result, sup |Fn (t) − F (t)| ≤ t∈R

max

i∈{0,...,K }

|Fn (ti ) − F (ti )| + δ.

By the weak law of large numbers, the maximum tends to zero. The result follows This proof can be generalized to classes of functions different from (1{. ≤ t})t∈R . A δ-bracket in Lr is a set of functions f with l ≤ f ≤ u, 1/r R < δ . For a where l and u are two functions satisfying |u − l|r dF given class of functions F, define the bracketing number N[ ] (δ, F, Lr ) as the minimum number of δ-brackets needed to cover F.

Proposition (van der Vaart, 1998, Theorem 19.4) Suppose that for all δ > 0, N[ ] (δ, F, L1 ) < ∞. Then n 1 X P sup f (Xi ) − E (f (X1 )) −→ 0. f ∈F n i=1


Consistency The proposition applies to many cases, see van der Vaart (1998), chapter 19, for examples. In particular, it holds with parametric families satisfying |ψ(Ui , θ1 ) − ψ(Ui , θ2 )| ≤ m(Ui )||θ1 − θ2 ||, E (m(U1 )) < ∞.

(7)

In quantile regression, |ρτ (Y − X 0 β1 ) − ρτ (Y − X 0 β2 )|

≤

max(τ, 1 − τ )|X 0 (β1 − β2 )|

≤

||X || × ||β1 − β2 ||.

Thus (7) holds provided that E (||X ||) < ∞. This establishes consistency of βbτ since we can then apply the theorem above.


Asymptotic normality I

We now investigate the asymptotic distribution of βbτ .

I

The usual method for smooth M-estimator is to use a Taylor expansion. The first order condition writes as n 1 X ∂ψ b = 0. (Ui , θ) n ∂θ

(8)

i=1

b we get Then expanding around θ, " # n n 1 X ∂2ψ 1 X ∂ψ b 0 )+oP (||θ−θ b 0 ||). 0= (Ui , θ0 )+ (Ui , θ0 ) (θ−θ n ∂θ n ∂θ∂θ0 i=1

i=1

√ Hence, provided that one can show that ||θb − θ0 || = OP (1/ n), we have " n # n √ 1 X ∂2ψ 1 X ∂ψ b √ (U , θ ) n( θ − θ ) = (Ui , θ0 ) + oP (1). i 0 0 n ∂θ∂θ0 ∂θ n i=1

i=1



By the weak law of large numbers, the central limit theorem and Slutski’s lemma, we get: √ L n θb − θ0 −→ N (0, J −1 HJ −1 ), h 2 i ∂ ψ where J = E ∂θ∂θ and H = V ( ∂ψ 0 (Ui , θ0 ) ∂θ (Ui , θ0 )). This kind of variance is often called a “sandwich formula”.

I

N.B.: in the maximum likelihood case, −J = H = I0 , the Fisher information matrix, and the formula simplifies.

I

In quantile regression, this method cannot be applied since the derivative of ρτ (for u 6= 0) is the step function ρ0τ (u) = τ − 1{u < 0} for which no Taylor expansion is available.

I

The first order condition(8) may not hold exactly either. However, 0 can be replaced by oP √1n , which will be sufficient subsequently.


Asymptotic normality Two key ideas for these kinds of situations: I

θ 7→ Q(θ) I

∂ψ ∂θ h(Ui , θ) is not i differentiable at θ0 , ∂ψ = E ∂θ (Ui , θ) is usually (continuously)

Even if θ 7→

differentiable.

Starting from (8), we then write: n √ 1 X ∂ψ b b b − Q(θ0 ) 0 = √ (Ui , θ) − Q(θ) + n Q(θ) n i=1 ∂θ √ b + Q 0 (θ) e n(θb − θ0 ). = Gn (θ) (9) h i ∂ψ b and Gn (θ) = √1 Pn where θe ∈ (θ0 , θ) i=1 ∂θ (Ui , θ) − Q(θ) . Gn is n

a stochastic process (i.e., a random function) which is called the empirical process. √ b I To show asymptotic normality of n(θ − θ0 ), it suffices to show b converges to a normal distribution. that Gn (θ)



By the central limit theorem, for any fixed θ, Gn (θ) converges to a normal distribution. Here however, θb is random.

I

The idea is to extend “simple” central limit theorem to convergence of the whole process Gn to a continuous gaussian process G . This is achieved through Donsker theorems.

I

Such theorems may be seen as uniform CLT, just as Glivenko-Cantelli were uniform LLN. Under such conditions, we can L b −→ prove that Gn (θ) G (θ0 ), a normal variable.

I

As previously, Donsker theorems can be obtained when the class of functions F is not too large. For instance:

Proposition (van der Vaart, Theorem 19.5) Gn , as a process indexed by f ∈ F, converges to a continuous gaussian process if Z 1q ln N[ ] (δ, F, L2 )dδ < ∞. 0



Like previously, many classes of functions satisfy the bracketing integral condition. In parametric classes where (7) holds, for instance, one can show that for δ small enough, N[ ] (δ, F, L2 ) ≤

K . δd

Thus the bracketing integral is finite and one can apply the previous theorem. I

Coming back to (9), we have, under the bracketing integral condition, √ ∂ψ L 0 −1 0 −1 b n θ − θ0 −→ N 0, Q (θ0 ) V (Ui , θ0 ) Q (θ0 ) ∂θ



Application to the quantile regression: the bracketing integral condition is satisfied, thus it suffices to check the differentiability of Q(β) at βτ . Here, ∂ψ/∂θ(Ui , θ) = − (τ − 1{Y − X 0 θ < 0}) X . Thus, −Q(β)

= τ E (X ) − E [1{ετ < X 0 (β − βτ )}X ] = τ E (X ) − E Fετ |X (X 0 (β − βτ )|X )X

I

Thus, provided that ετ admits a density conditional on X at 0, Q(.) is differentiable and Q 0 (βτ ) = E fετ |X (0|X )XX 0 .

I

Besides, ∂ψ V (Ui , θ0 ) ∂θ

= E {V [(τ − 1{Y − X 0 βτ < 0}) X |X ]} = τ (1 − τ )E [XX 0 ] .



Finally, we get:

−1 −1 √ L n βbτ − βτ −→ N 0, τ (1 − τ )E fετ |X (0|X )XX 0 E XX 0 E fετ |X (0|X )XX 0 . I

Remark 1: if Y = X 0 β + ε where ε is independent of X (location model), ετ = ε − qτ (ε) and the asymptotic variance Vas reduces to Vas =

τ (1 − τ ) −1 E [XX 0 ] . fε (qτ (ε))2

This formula is similar to the one for the OLS estimator, except that σ 2 is replaced by τ (1 − τ )/fε (qτ (ε))2 . In general, as we let τ → 1 or 0, fε (qτ )2 becomes very small and thus βbτ becomes imprecise. This is logical since data are often more dispersed at the tails.



bτ , in Remark 2: this result applies in particular to simple quantiles q which case we have: √ τ (1 − τ ) L n (b qτ − qτ ) −→ N 0, 2 . fY (qτ )

I

Remark 3: we can also generalize it to parameters (βτ1 , ..., βτm ) corresponding to different quantiles: m √ L n βbτk − βτk −→ N (0, V ) , (10) k=1

where V is a m × m block-matrix, whose (k, l) block Vk,l satisfies Vk,l = [τk ∧ τl − τk τl ] H(τk )−1 E [XX 0 ] H(τl )−1 and as before, H(τ ) = E fετ |X (0)XX 0 .


Confidence intervals and testing

I

This result is useful to build confidence intervals or test assumptions on βτ .

I

However, to obtain estimators of the asymptotic variance, one has to estimate fετ |X (0|X ), which is a difficult task. Alternative solutions have thus been proposed for inference:

I

I I I

using rank tests (not presented here); using bootstrap or, more generally, resampling methods; making finite sample inference.


Confidence intervals and testing: asymptotic variance estimation I

In the location model, Vas = τ (1 − τ )E (XX 0 )−1 /fε (qτ (ε)), and the only problem is the denominator. Note that 1 fε (qτ (ε))

=

1

=

∂Fε−1 (τ ) ∂τ

fε (Fε−1 (τ )) F −1 (τ + h) − Fε−1 (τ − h) . = lim ε h→0 2h

I

Thus we can estimate this term by, e.g., (Fbε−1 (τ + hn ) − Fbε−1 (τ − hn ))/2hn , where hn → 0.

I

This is (roughly) the estimator provided by default in Stata. However, the corresponding variance estimator is inconsistent in general when ε is not independent of X .



In this general case, the main difficulty is to estimate J = E (fετ |X (0|X )XX 0 ]. A simple solution for that purpose, proposed by Powell (1991), relies on the following idea: 1{|ετ | ≤ h} XX 0 . J = lim E h→0 2h

I

Letting εbiτ = Yi − Xi0 βbτ , we thus may estimate J by Jb =

n 1 X 1{|b εiτ | ≤ hn }Xi Xi0 . 2nhn

(11)

i=1

I

As often in statistics, hn must be chosen so as to balance the bias and variance of Jb (for consistency, we must have hn → 0 and nhn → ∞).

I

Note that we can replace the uniform kernel 1{|u| ≤ 1}/2 in (11) by any other density function.



With a consistent estimator of Vas in hand, we can easily make inference on βτ .

I

Confidence interval on βτ : q q bas , βbτ + z1−α/2 V bas , ICα = βbτ − z1−α/2 V where z1−α/2 is the 1 − α/2-th quantile of the N (0, 1) distribution.

I

The Wald statistic test of g (βτ ) = 0 writes T = ng (βbτ )0

−1 ∂g ∂g b (βτ )Vas (βτ ) g (βbτ ), ∂β 0 ∂β

and it tends to a χ2dim(g ) under the null hypothesis.


Confidence intervals and testing: bootstrap

I

The previous approach requires to choose a smoothing parameter hn , and results may be sensitive to this choice.

I

Alternatively, we can use bootstrap by implementing the algorithm: For b = 1 to B: - Draw with replacement a sample of size n from the initial ∗ , ..., k ∗ ) denote the sample (Yi , Xi )i=1...n . Let (kb1 bn corresponding indices of thePobservations; - Compute βbτ∗b = arg minβ nj=1 ρτ (Ykbj∗ − Xk0 ∗ β). bj


Confidence intervals and testing: bootstrap I

Then we can estimate the asymptotic variance by ∗ Vas =

B 1 X b∗ b 2. (βτ b − β) B b=1

I

I

Confidence intervals or hypothesis testing may be conducted as before, using the normal approximation. Alternatively (percentile bootstrap), you can compute the empirical quantiles qu∗ of (βbτ∗1 , ..., βbτ∗B ) and then define a confidence interval as ∗ ∗ , q1−α/2 ]. IC1−α = [qα/2

I

N.B.: there are other resampling methods specialized for the quantile regression, see Koenker (1994), Parzen et al. (1994) and He and Hu (2002).


Confidence intervals and testing: finite sample inference I

Simple yet very recently developed idea (Chernozhukov et al., 2009, Coudin and Dufour, 2009): if βτ = β0 , then Bi (β0 ) = 1{Yi − Xi0 β0 ≤ 0} is such that Bi (β0 )|Xi ∼ Be(τ ).

I

As a result, for all g (.) and positive definite Wn , under the hypothesis βτ = β0 , the distribution of !0 ! n n 1 X 1 X Tn (β0 ) = √ (τ − Bi (β0 ))g (Xi ) Wn √ (τ − Bi (β0 ))g (Xi ) n i=1 n i=1 is known (theoretically at least). Letting z1−α denote its (1 − α)-th quantile, we reject the null hypothesis if Tn (β0 ) > z1−α .

I

In practice, the distribution of Tn (β0 ) under the null can be approximated by simulations.


Confidence intervals and testing: finite sample inference I

We can then define a confidence region by inverting the test: CR1−α = {β/Tn (β) ≤ z1−α }. Indeed, letting βτ denote the true parameter, Pr(CR1−α 3 βτ )

=

Pr (Tn (βτ ) ≤ z1−α )

≥ 1 − α. I

This is a general procedure to build confidence regions from a test.

I

To obtain confidence interval on a real-valued parameter ψ(βτ ), we let IC1−α = {ψ(β), β ∈ CR1−α }. This is known as the projection method (see, e.g., Dufour and Taamouti). Corresponding confidence intervals are conservative.

I

The computation of such confidence regions / intervals may be demanding. See Chernozhukov et al. (2009) for MCMC methods which (partially) alleviate this issue.


Testing homogeneity of effects I

As mentioned before, an interesting property of quantile regression is that it allows for heterogeneity of effects of X across the distribution of Y . A byproduct is that they also provide tests for the homogeneity hypothesis.

I

Let X = (1, X−1 ) and βτ = (β1τ , β−1τ ) and T denote a set included in [0, 1], the test formally writes as β−1τ = β

∀t ∈ T .

This may be seen as testing for the location model Y = X 0 β + ε, with ε ⊥ ⊥ X. I

If the set T is finite, we can use (10) to implement such a test. If the set is infinite, this is far more complex and can be achieved using the convergence of τ 7→ βbτ as a process (see Koenker and Xiao, 2002).

Semi and Nonparametric Models in Econometrics Additional properties



Interpretation as a random coefficient model I

Consider the following random coefficient model: Y = X 0 βU ,

U|X ∼ U[0, 1].

(12)

Suppose also that for all x, τ 7→ x 0 βτ is strictly increasing. Then: P(Y ≤ X 0 βτ |X ) = P(X 0 βU ≤ X 0 βτ |X ) = P(U ≤ τ |X ) = τ. In other words, qτ (Y |X ) = X 0 βτ . I

This is useful to simulate models satisfying the linear restriction for all quantiles.

I

This also shows that assuming quantile regression, we consider random coefficient model with a unique underlying random variable, which may be interpreted as the ranking on an unobserved variable.


On the monotonicity assumption

I

If we assume qτ (Y |X ) = X 0 βτ for all τ , then for all x in the support of X , τ 7→ x 0 βτ should be increasing. Note that except when βτ = (ατ , β) (i.e., under a location model), this cannot be true when the support of X (apart from the constant) is Rp .

I

Even if this support is not Rp , the estimated functions τ 7→ x 0 βbτ may not satisfy this requirement. We can prove (see Koenker, 2005) that τ 7→ x 0 βbτ is increasing but this does not always hold for x 6= x.

I

If this is not the case, then this may be due to I I

misspecification, i.e. qτ (Y |X ) 6= X 0 βτ ; finite sample errors.


On the monotonicity assumption I

Recently, Chernozhukov et al. (2009) have proposed an elegant solution to this issue.

I

If there is no misspecification, under the random coefficient representation (12): FY |X (y |x) = P(x 0 βU ≤ y ) =

Z

1

1{x 0 βu ≤ y }du.

(13)

0

I

Their ideas is to use (13) to estimate the conditional distribution function: Z 1 b FY |X (y |x) = 1{x 0 βbu ≤ y }du. 0

I

Even if u 7→ x 0 βbu is not monotonic, FbY |X (y |x) will be a proper distribution function.


On the monotonicity assumption I

Then define the “rearranged quantile estimator” as bτ∗ (Y |X ) = inf{y /FbY |X (y |x) ≥ τ }. q This function is increasing as a “standard” quantile function of a distribution function.

I

Chernozhukov et al. (2009) prove that: I

I

bτ∗ (Y |X ) is always closer to the true quantile function than q x 0 βbu (even under misspecification). bτ∗ (Y |X ) and x 0 βbu have the If there is no misspecification, q same limit distribution so that all inference based on bτ∗ (Y |X ). asymptotic properties of βbu also applies to q


On the monotonicity assumption 8

5

1.0

(taken from Chernozhukov et al., 2009) Q−1 F

0

0.0

1

0.2

2

0.4

3

0.6

4

0.8

Q Q*

0.0

0.2

0.4

0.6 u

0.8

1.0

0

1

2

3

4

y

Figure 1. Left: The pseudo-quantile function Q and the rearranged quantile function Q∗ . Right: The pseudo-distribution function Q−1 and the distribution function F induced by Q.

5


On the monotonicity assumption

19

(taken from Chernozhukov et al., 2009) 20000

B − Rearranged curves

5000

10000

15000

Non−veterans Veterans

0

5000

10000

Annual earnings

15000

Non−veterans Veterans

0

Annual earnings

20000

A − Original curves

0.2

0.4

0.6

Quantile index

0.8

0.2

0.4

0.6

0.8

Quantile index

Figure 4. Chernozhukov and Hansen’s estimates of the structural quantile functions of earnings for veterans (left panel), and their rearrangements (right panel).

Semi and Nonparametric Models in Econometrics Quantile regression in practice



Computation of βbτ . I

There is no explicit solution to (3) so one has to solve the program numerically.

I

An issue is the non differentiability of the objective function. Standard algorithms such as the Newton-Raphson cannot be used here.

I

The key idea is to reformulate (3) as a linear programming problem: min

(β,u,v )∈Rp ×R2n +

τ 10 u + (1 − τ )10 v

s.t. Xβ + u − v − Y = 0,

where X = (X1 , ..., Xn )0 , Y = (Y1 , ..., Yn )0 and 1 is a n-vector of 1. I

Such linear programming problems can be efficiently solved by simplex methods (for small n) or interior point methods (large n).


Computation of βbτ . I

Simplex method: consider a linear programming problem of the form min c 0 x

x∈Rn

s.t. x ∈ S = {u/Au ≥ b, Bu = d},

(14)

where c ∈ Rn , A and B are two matrices and “≥” is considered elementwise. I

Then one can show that (i) S is a convex polyhedron and (ii) if solutions exist, then they are vertices of S.

I

Basically, the simplex method consists of going from one vertex to another, choosing each time the steepest descent.

I

Interior point methods: consider (14) with A = In and b = 0, the idea is to replace (14) by minn c 0 x − µ

x∈R

n X

ln xk

s.t. B x = d.

(15)

k=1

(15) can be solved easily with a Newton method. Then let µ → 0.


Software programs I

SAS: proc quantreg.

proc quantreg data=(dataset) algorithm=(choice of algo.) ci= (method for performing confidence intervals); class (qualitative variables); model (y) = (x) /quantile = (list of quantiles or ALL); run; I

By default, the simplex method is used. One should switch to an interior point method (by letting algorithm=interior) for n ≥ 1000.

I

By default, the confidence intervals are computed by inverting rank-score tests when n ≤ 5000 and p ≤ 20, and resampling method otherwise (N.B.: the latter provide more robust standard error estimates).


Software programs I

Stata: command qreg: qreg depvar indepvars , quantile(choice of quantile)

I

The standard errors estimated by qreg are valid for the location model only. To obtain better standard errors estimates (given with the bootstrap), you should use bsqreg instead.

I

To obtain simultaneously several quantile regressions, use sqreg: sqreg depvar indepvars , quantiles(choice of quantiles)

N.B: The standard errors estimates provided by sqreg are computed with the bootstrap.


Software programs I

A very complete R package has been developed by R. Koenker: quantreg.

library(quantreg) rq(y ~ x1 + x2, tau = (single quantile or vector of quantiles), data=(dataset), method=("br" or "fn")) I

To obtain inference on all quantiles put tau = -1 (or any number outside [0, 1]).

I

method ="br" corresponds to the Simplex (default), while ”fn” is an interior point method.

I

a tutorial is available at Roger Koenker’s webpage.


An example I

I look at the impact of various factors on birth weight, following Abreveya (2001). Indeed, a low birth weight is often associated with subsequent health problems, and is also related to educational attainment and labor market outcomes.

I

Quantile regression provides a more complete story than just running a probit on the dummy variable (birth weight < arbitrary threshold).

I

The analysis is based on exhaustive 2001 US data on birth certificates. I restrict the sample to singleton births with mothers black or white, between the ages of 18 and 45, resident in the US (roughly 2.9 million observations).

I

Apart from the gender, information on the mother is available: marital status, age, being black or white, education, date of the first prenatal visit, being a smoker or not, number of cigarettes smoked per day...


An example I

SAS code: ods graphics on; proc quantreg data=birth weights ci=sparsity/iid alg=interior(tolerance=1e-4); model birth weight = boy married black age age2 high school some college college prenatal second prenatal third no prenatal smoker nb cigarettes /quantile= 0.05 to 0.95 by 0.05 plot quantplot; run; ods graphics off;

I

Stata code: sqreg birth weigh boy married black age age2 high school some college prenatal second prenatal third no prenatal smoker nb cigarettes, quantiles(0.05 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 0.95)

I

Stata is quite long here (1 hour for a single quantile with 20 bootstrap replications). To run SAS on large databases like this one, you may have to increase the available memory.


An example Quantile and Objective Function Quantile Objective Function Predicted Value at Mean

0.1 31108564.261 2727.4037

Parameter Estimates

Parameter Intercept boy married black age age2 high_school some_college college prenatal_second prenatal_third no_prenatal smoker nb_cigarettes

DF Estimate 1 1 1 1 1 1 1 1 1 1 1 1 1 1

2150.419 83.8925 64.9045 -251.465 38.3584 -0.6657 6.5725 36.6800 76.1075 -4.1840 22.2022 -472.532 -156.928 -5.8266

Standard Error 41.9615 3.8034 4.9650 5.4947 3.0443 0.0523 5.7090 6.4022 6.7700 5.9940 12.2669 19.1648 10.6564 0.8140

95% Confidence Limits 2068.176 76.4380 55.1734 -262.234 32.3916 -0.7682 -4.6170 24.1319 62.8384 -15.9321 -1.8405 -510.095 -177.815 -7.4221

2232.662 91.3471 74.6357 -240.696 44.3251 -0.5631 17.7620 49.2281 89.3765 7.5641 46.2449 -434.970 -136.042 -4.2311

t Value Pr > |t| 51.25 22.06 13.07 -45.77 12.60 -12.73 1.15 5.73 11.24 -0.70 1.81 -24.66 -14.73 -7.16