On images of quantum representations of mapping class groups

arXiv:0907.0568v2 [math.GT] 14 Sep 2009

On images of quantum representations of mapping class groups∗ Louis Funar

Toshitake Kohno

Institut Fourier BP 74, UMR 5582 University of Grenoble I 38402 Saint-Martin-d’H`eres cedex, France e-mail: [email protected]

IPMU, Graduate School of Mathematical Sciences The University of Tokyo 3-8-1 Komaba, Meguro-Ku, Tokyo 153-8914 Japan e-mail: [email protected]

September 15, 2009

Abstract We answer conjectures of Squier concerning groups related to the kernels of Burau’s representations of the braid groups at roots of unity. In particular, one finds that the image of the braid group on 3 strands is a finite extension of a triangle group, using geometric methods. The second part of this paper is devoted to applications of these results to qualitative characterizations of the images of quantum representations of the mapping class groups. On one hand they are large since the image of any Johnson subgroup contains a free non-abelian subgroup. On the other hand they are not so large since they are of infinite index into the group of unitary matrices with cyclotomic integers entries. 2000 MSC Classification: 57 M 07, 20 F 36, 20 F 38, 57 N 05. Keywords: Mapping class group, Dehn twist, Temperley-Lieb algebra, triangle group, braid, Burau representation, Johnson filtration, quantum representation.

1

Introduction and statements

The first part of the present paper is devoted to the the study of groups related to the kernels of the Burau representation of braid groups at roots of unity. We consider the two conjectures stated by Squier in [25] concerning these kernels. These conjectures were part of an approach to the faithfulness of the Burau representations and were overlooked over the years because of the counterexamples found by Moody, Long, Paton and Bigelow (see [19, 17, 2]) for braids on n ≥ 5 strands. the other one is false basically because of non-faithfulness of the Burau representation for n ≥ 5. However we can show that it fails also when n = 3, for other reasons. Specifically let Bn denote the braid group in n strings with the standard generators g1 , g2 , . . . , gn . Squier was interested to compare the kernel of the Burau representation at a k-th root of unity q to the normal subgroup Bn [k] of Bn generated by gjk , 1 ≤ j ≤ n − 1. Our first result answers conjecture C2 from [25]: Theorem 1.1. The intersection of Bn [k] over all k is trivial. The proof uses the asymptotic faithfulness of quantum representations of mapping class groups, due to Andersen ([1]) and independently to Freedman, Walker and Wang ([7]). The other conjecture stated in [25] is that Bn [k] is the kernel of the Burau representation. This is false basically because of non-faithfulness of the Burau representation for n ≥ 5 (see Proposition 2.5). However we can show that it fails also when n = 3, but for other reasons. ∗

Preprint available electronically at http://www-fourier.ujf-grenoble.fr/∼funar

1

The main body of the first part is then devoted to the complete description of the image of the Burau representation of B3 . There is no general approach to problems of this kind, but hopefully the B3 can be settled completely. We can state our main result in this direction: Theorem 1.2. Assume that q is a primitive n-th root of unity and g1 , g2 are the standard generators of B3 . 1. If n = 2k then β−q (B3 ) has the presentation with generators g1 , g2 and relations Braid relation: Power relations: Center relation:

g1 g2 g1 = g2 g1 g2 , g12k = g22k = (g12 g22 )k = 1, (g1 g2 )3lcm(3,k)gcd(2,k) = 1

2. If n = 2k + 1 then β−q (B3 ) has the presentation with generators g1 , g2 and relations Braid relation: Power relations: Extra relations: Center relation:

g1 g2 g1 = g2 g1 g2 , g12k+1 = g22k+1 = (g12 g22 )2k+1 = 1, (g1−2 g22k )2 = (g12k g22k−2 )3 = 1, (g1 g2 )6lcm(3,2k+1) = 1

The proof of this algebraic statement has a strong geometric flavor. In fact, another key ingredient is Squier’s theorem concerning the unitarizability of Burau’s representation (see [25]). The nondegenerate Hermitian form invariant by the braid group is not always positive definite. First, we find whether it does, so that the representation can be unitarized (i.e. it can be conjugate into U (2)) and whether it does not. But still in the latter situation the representation can be complex-unitarized (namely is conjugate within U (1, 1)). We will focus then on the complex-unitary case. We show that that the image of some free subgroup of P B3 by the Burau representation is a subgroup of P U (1, 1) generated by three rotations in the hyperbolic plane. Here the hyperbolic plane is identified to the unit disk of the complex projective line CP 1 . Geometric arguments due to Knapp, Mostow and Deraux (see [12, 20, 6]) show that the image is a triangular group and thus we can give an explicit presentation of it. Then we can describe the whole image of B3 . In particular, we obtain a description of the kernel of the Burau representation of B3 at roots of unity. The second part of this paper consists of applications of these results to study the images of the mapping class groups by quantum representations at roots of unity. Specifically we want to prove that the image of the mapping class group by quantum representations is large in a sense and not so large, from another point of view. The first part is not purely technical preparation for the second one. Finding the images of the Burau representation seems to be a difficult problem interesting by itself (see e.g. [5]). Some results of the same type are already known. In [8] we proved that the images are infinite and non-abelian (for all but finitely many explicit cases) using earlier results of Jones who proved in [14] that the same holds true for the braid group representations factorizing through the Temperley-Lieb algebra at roots of unity. Masbaum found then in [18] explicit elements of infinite order. General arguments concerning Lie groups show actually that the image should contain a free non-abelian group. Further Larsen and Wang showed (see [15]) that the image
of the quantum representations is dense in the projective unitary of mapping class groups at the smallest root of unity exp 2πi 4r group for any prime r ≥ 5. We will consider in this paper the Johnson filtration by the subgroups of the mapping class group which act trivially on the k-th nilpotent quotient of the fundamental group of the surface, for 2

k ∈ Z+ . As it is well-known the Johnson filtration comes within the framework of finite type invariants of 3-manifolds (see e.g. [9]). Our first result is then shows that the image is large in the following sense (see also Proposition 4.1 for a more precise statement): Theorem 1.3. The image of a Johnson subgroup by the SO(3) quantum representation at a D-th root of unity contains a free non-abelian group if the genus is at least 4 and D ≥ 11. The idea of proof for this theorem is to embed a pure braid group within the mapping class group and to show that its image is large. Namely, to a 4-holed sphere suitably embedded in the surface we associate a pure braid subgroup P B3 on 3-strands embedded in the mapping class group. The quantum representation contains a particular sub-representation of P B3 which can be identified with the Burau representation (see [8]) of B3 restricted at P B3 . One way to obtain elements of the Johnson filtration is to consider elements of the lower central series of P B3 and extend them to all of the surface by identity. Thus it would suffice to understand the image of the lower central series by the Burau representation at roots of unity in order to find large subgroups in the image. This method which consists in analyzing the contribution of the small subsurfaces (usually holed spheres) to various subgroups of the mapping class groups was also used in an unpublished paper by T.Oda and J.Levine (see [16]) for lower bounds of the ranks of the gradings of the Johnson filtration. On the other hand Gilmer and Masbaum proved in [10] that the mapping class group preserves a certain free lattice defined over the cyclotomic integers OD in the conformal blocks associated to the SO(3)-TQFT. Specifically they considered D to be either an odd prime congruent to −1 mod 4 or 4 times a prime congruent to 1 mod 4 and OD be the integer ring of D-th roots of unity. We call such D to be basically odd primes. Thus there exists a free OD lattice Sg,D in the space of conformal blocks for the genus g surfaces and a non-degenerate hermitian OD -valued form on Sg,D such that (a central extension of) the mapping class group acts on Sg,D and preserves the hermitian form. Therefore the image of the mapping class group consists of unitary matrices (with respect to the hermitian form) with entries from OD . The group U (OD ) of such matrices is a lattice in the product U of unitary groups associated to the Hermitian form and all Galois embeddings of D-th roots of unity in C. The natural question is to compare the image ρD (Mg ) and this (projectivized) lattice P U (OD ). Our main result shows that the image is not so large, with respect to the whole lattice: Theorem 1.4. The image ρD (Mg ) is not a lattice in a higher rank semi-simple Lie group, for g ≥ 3 and D ≥ 11. In particular, when D is basically an odd prime the image ρD (Mg ) is of infinite index in P U (OD ), for g ≥ 3. The idea of the proof is to show that ρD (Mg ) has infinite normal subgroups of infinite index. The technique used for the proof of Theorem 1.3 can be used to quite a lot of subgroups of mapping class groups (having the intersection with a braid subgroup large enough) and so we can prove that their images are infinite. We are able to prove that for the subgroup generated by the F -th powers of Dehn twists (where F divides D and F < D) and also for the second derived subgroup [[Kg , Kg ], [Kg , Kg ]] of the Johnson subgroup Kg . But sometimes the same method work also for quotients like the metabelian quotient of Kg if we can get control on the intersections with the smaller braid group and solve the analogous problem for the braid group. In our case for instance the metabelian quotient of the triangular group (coming out from the Burau representation) is large enough to find that the image of [[Kg , Kg ], [Kg , Kg ]] is of infinite index in that of Kg . By a celebrated theorem of Margulis this cannot happen for a lattice in a higher rank group, in 3

particular a lattice in P U. This will imply that the discrete subgroup ρD (Mg ) is of infinite index in the lattice P U (OD ). Although the proof of Theorem 1.3 does not use the full strength of our results from the first part – a shorter proof can be obtained by using the Tits alternative instead of the explicit description of the image of the Burau representation of B3 , along the same lines of reasoning – they seem essential for the second theorem. Acknowledgements. The first author was partially supported by the ANR Repsurf:ANR-06BLAN-0311. We are grateful to Martin Deraux for discussions concerning this paper. The second author is partially supported by Grant-in-Aid for Scientific Research 20340010, Japan Society for Promotion of Science, and by World Premier International Research Center Initiative, MEXT, Japan. A part of this work was accomplished while the second author was staying at Institut Fourier in Grenoble. He would like to thank Institut Fourier for hospitality.

2 2.1

Braid group representations Jones and Burau’s representations at roots of unity

We start with the following classical definition. Definition 2.1. The Temperley-Lieb algebra Aτ,n is the C∗ -algebra generated by the projectors 1, e1 , . . . , en e∗j = e2j = ej ei ej = ej ei , if |i − j| ≥ 2 ej ej+1 ej = ej ej−1 ej = τ ej According to Wenzl ([27]) there exist such
unitary projectors ej , 1 ≤ j ≤ n, for every natural n if and only if τ −1 ≥ 4 or τ −1 = 4 cos2 πn , for some natural n ≥ 3. However, for given n one could find projectors e1 , . . . , en as above for τ −1 = 4 cos2 (α), where the angle α belongs to some non-trivial arc of the unit circle. Another definition of the Temperley-Lieb algebra (which is equivalent to the former, at least when τ verifies the previous conditions) is as a quotient of the Hecke algebra: Definition 2.2. The Temperley-Lieb algebra An (q) is the quotient of the group algebra CBn of the braid group Bn by the relations: (gi − q)(gi + 1) = 0 1 + gi + gi+1 + gi gi+1 + gi+1 gi + gi gi+1 gi = 0 where gi are the standard generators of the braid group Bn . The quotient obtained using only the first relation is the Hecke algebra Hn (q). It is known that An (q) is isomorphic to Aτ,n where τ −1 = 2 + q + q −1 , and in particular when q is
2πi the root of unity q = exp n . We suppose from now on that τ −1 = 2 + q + q −1 .

We will analyze the case where n = 3 and q is a root of unity, and more generally for |q| = 1. Then Aτ,3 and An (3) are
nontrivial
and well-defined for all q with |q| = 1 belonging to the arc of circle 2πi to exp joining exp − 2πi 3 3 . We will recover this result below, in a slightly different context. Further Aτ,3 is semi-simple and splits as M2 (C) ⊕ C. The natural representation of B3 → Aτ,3 sends gi into qei − (1 − ei ). It is known to be unitary when τ −1 ≥ 4 (see [14]).

Proposition 2.1. Let q = exp(iα). 4

1. Completely reducible representations ρ of B3 into GL(2, C) which factor through A3 (q) are equivalent, for q 6= −1 to the following: !   1 − q+1 −(q + 1)c q 0 , ρ(g2 ) = ρ(g1 ) = q2 0 −1 −ǫ(q + 1)cr 2 q+1 where |c|2 |q + 1|2 = |q + q + 1|, r ∈ R and ǫ ∈ {−1, 1}. If q = −1 the representations are abelian. 2π 2 2. We have ǫ = 1 if and only if α ∈ [− 2π 3 , 3 ]. Moreover, the representation is unitary iff r = 1 and ǫ = 1. 4π 2 3. If α ∈ [ 2π 3 , 3 ] the ǫ = −1 and the representation lies in U (1, 1) iff r = 1.   q 0 Proof. We can choose the image of g1 to be ρ(g1 ) = since completely reducible 20 −1 dimensional representations are diagonalizable. Since g2 is conjugate to g1 in B3 we have ρ(g2 ) = U ρ(g1 )U −1 , where, without loss of generality we can suppose U ∈ SL(2, C). We discard from now on q = −1 since then the representation is abelian, as ρ(g1 ) is scalar.     a b qad + bc −(q + 1)ab Set U = , where ad − bc = 1. Then ρ(g2 ) = . Then ρ factors c d (q + 1)cd −qbc − ad through A3 (q), namely the second identity of Definition 2.2 is satisfied, if and only if (q 6= −1):

qad + bc = −

1 q+1

2

q +q+1 q This gives the solutions d = (q+1) 2 a , and c = − (q+1)2 b , (and if q is a primitive root of unity of order 3 there is an additional solution b = 0, arbitrary c) and so ! 1 − q+1 −(q + 1)ab 2 +q+1)q ρ(g2 ) = q2 − (q(q+1) 3 ab q+1

This matrix is the claimed one, by choosing ǫ to be the sign of the real number q + q + 1, the c |q+q+1| from the statement is ab and so r 2 = |q+1| 4 |ab|2 .   u v 2 , where |u|2 + ǫ|v|2 = 1. In this case the Finally r = 1 is equivalent to ρ(g2 ) = −ǫv u matrices are either in U (2) or in U (1, 1), depending on ǫ. Notice that representations coming from complex c with different arguments are conjugate. The representations ρ of B3 that arise as above for r 2 = 1 will be called Jones representations of B3 (at q). Proposition 2.2. Let B3 → U (2) be a unitary Jones representation at q = exp(2πiα), for α ∈ 2π ∗ [− 2π 3 , 3 ]. Identify U (2)/C to SO(3). The image of the the generators g1 and g2 into SO(3) are two rotations g1 and g2 of angle π + α whose axes form an angle θ with cos θ =

cos α 1 + cos α

Proof. We have first to normalize unitary matrices to elements of SU (2). This corresponds to replacing ρ(gj ) by σj = λρ(gj ), λ2 q = −1. 5

 w + ix y + iz with real The set of anti-hermitian 2-by-2 matrices, namely matrices A = −y + iz w − ix w, x, y, z is identified with the space H of quaternions w + ix + jy + kz. Under this identification SU (2) corresponds to the sphere of unit quaternions. In particular any element of SU (2) acts by conjugacy on H. Let R3 ⊂ H be the vector subspace given by the equation w = 0. Then R3 is SU (2) conjugacy invariant and the linear transformation induced by A ∈ SU (2) on R3 is an orthogonal matrix. This is the map Q : SU (2) → SO(3). 

A simple computation shows that Q



w + ix y + iz −y + iz w − ix





 1 − 2(y 2 + z 2 ) 2(xy − wz) 2(xz + wy) =  2(xy + wz) 1 − 2(x2 + z 2 ) 2(yz − wx)  2(xz − wy) 2(yz + wx) 1 − 2(x2 + y 2 )

This shows that Q(σ1 ) is the rotation of angle π +α around the axis i ∈ R3 in the space of imaginary quaternions. Instead of giving the cumbersome computation of Q(σ2 ) observe that Q(σ2 ) is also a rotation of angle π + α since it is conjugate to it. Let N be the axis of rotation. If N = ui + vj + wk then cos θ = u. A simple computation shows that the matrix of the rotation of angle π + α around the axis N has a matrix whose first entry on the diagonal reads u2 + (1 − u2 ) cos(π + α). This gives u2 + (1 − u2 ) cos(π + α) = 1 − 2(y 2 + z 2 ) where y 2 + z 2 = | − λ(q + 1)c|2 = 1 −

1 |q + 1|2

cos α This gives |u| = 1+cos α . Identifying one more term in the matrix of Q(σ2 ) yields the sign of u. We omit the details.

Remark 2.1. In [24] the authors consider the structure of groups generated by two rotations of finite order for which axes form an angle which is an integral part of π. Their result is that there are only few new relations. However the proposition above shows that we cannot apply these results to our situation. It seems quite hard just to find those α for which axes verify this condition. Definition 2.3. The (reduced) Burau representation β : Bn → GL(n − 1, Z[q, q −1 ]) is defined on the standard generators   −q 1 βq (g1 ) = ⊕ 1n−3 0 1   1 0 0 βq (gj ) = 1j−1 ⊕  q −q 1  ⊕ 1n−j−4 , for 2 ≤ j ≤ n − 1 0 0 1 Jones already observed in [14] that the following holds true for the smallest roots of unity: Proposition 2.3. The Jones representation of B3 at q 6= −1 is conjugate to the tensor product of 2π the parity representation and the Burau representation at q, for all q = exp(2πα), α ∈ [− 2π 3 , 3 ].

Proof. Recall that the parity representation σ : B3 → {−1, 1} ⊂ C∗ is given by σ(gj ) = −1. The Burau representations for n = 3 is given by     −q 1 1 0 βq (g1 ) = , βq (g2 ) = 0 1 q −q 6



 1 a (q+1)a Take then V = , for q 6= −1, where a is given by ǫ(q + 1)c = qa2 . We verify easily 1 0 a that V ρ(gj )V −1 = σ ⊗ βq (gj ). Remark 2.2. The definition of Aτ,3 in terms of orthogonal projections has a unitary flavor and this 2π 2π explains why it is well defined on the unit circle only for q = exp( 2π i α), where α ∈ [− 3 , 3 ]. In fact also the Burau representation (which is defined for any complex q 6= 0) it is unitarizable only 2π when q = exp(2πiα) with α ∈ [− 2π 3 , 3 ]. However, by proposition 2.1 the Burau representation is 2π 4π complex unitary when α ∈ [ 3 , 3 ], which means that it can be conjugated into U (1, 1).

2.2

Two conjectures of Squier and proof of Theorem 1.1

This section is devoted to the study of the kernels of the Jones and Burau’s representations at roots of unity. Motivation comes from the following conjectures of Squier from [25]: Conjecture 2.1 (Squier). The kernel of the Burau representation β−q for a primitive k-th root of unity q is the normal subgroup Bn [k] of Bn generated by gjk , 1 ≤ j ≤ n − 1. The second conjecture of Squier, which is related to the former one, is: Conjecture 2.2 (Squier). The intersection of Bn [k] over all k is trivial. Proposition 2.4. Conjecture 2.2 holds true for every n ≥ 3. Proof. Let Σ0,n+1 be a disk with n holes. The (pure) mapping class group M (Σ0,n+1 ) is the group g of framed pure braids P B n and fits into the exact sequence: g 1 → Zn → P B n → P Bn → 1

where Zn is generated by the Dehn twists along the n holes boundary curves. It is well-known that g the exact sequence above splits i.e. there exists a section P Bn ֒→ P Bn.

The extended mapping class group M ∗ (Σ0,n+1 ) is the group of mapping classes of homeomorphisms of the disk with n holes that fix pointwise the boundary of the disk but are allowed to permute the remaining boundary components, (which are suitably parameterized). Thus M ∗ (Σ0,n+1 ) is the group of framed braids on n strands and we have then a similar exact sequence: 1 → Zn → M ∗ (Σ0,n+1 ) → Bn → 1 ∗ denote the corresponding Let g1 , . . . , gn−1 denote the standard generators of Bn and g1∗ , . . . , gn−1 ∗ mapping classes in M (Σ0,n+1 ). −1 Recall (see [3]) that the following elements Aij = gi gi+1 · · · gj−1 gj2 gj−1 · · · gi−1 form a generators ∗ · · · g ∗ g ∗ 2 g ∗ −1 · · · g ∗ −1 then the image of A into system for P Bn . If we denote by A∗ij = gi∗ gi+1 ij j−1 j j−1 i ∗ g P B n by the natural section is the product of Aij with a number of commuting Dehn twists along boundary circles. We need the correction factors in order to recover the framing information in g P Bn.

Recall now that Bn acts on the set SCC(Σ0,n+1 ) of isotopy classes of simple closed curves of Σ0,n+1 . This is induced from the action of Bn on the fundamental group of the n-th punctured disk, which is isomorphic to π1 (Σ0,n+1 ). We can represent Aij as the Dehn twist along a simple closed curve γij on Σ0,n+1 . Lemma 2.1. Let γ ∈ SCC(Σ0,n+1 ). Then there exist some i, j and ϕ ∈ Bn such that ϕ(γij ) = γ. 7

Proof. The set of surfaces obtained by cutting along one of the curves γij exhausts all homeomorphism types of subsurfaces of Σ0,n+1 obtained by cutting along simple closed curves. So, given two curves whose complements are homeomorphic we can find an element of the extended mapping class group M ∗ (Σ0,n+1 ) of the surface sending one curve into the other. Observe that we need the extended mapping class group (i.e. framed braids) since arbitrary homeomorphisms need not preserve the boundary components. This extended mapping class group is the group of framed braids. However simple closed curve can be isotoped so that they do not touch the boundary and thus we can use the action of genuine braids (i.e. of elements of Bn ) instead of framed braids. g Dehn twists along simple closed curves γ on Σ0,n will be called Dehn twists of P B n . Their images into P Bn will be called Dehn twist of P Bn .

Set P Bn [k] be the group generated by the k-th powers of Dehn twists of P Bn . One key observation is that:

Lemma 2.2. The normal subgroup P Bn [k] of P Bn generated by the set of k-th powers Akij of the generators Aij (for all i ≤ j) is the group Bn [2k]. Proof. Any element of Bn [2k] is a product of conjugates of gj2k and thus it is a pure braid. Thus Bn [2k] ⊂ P Bn . It is clear that Akij ∈ Bn [2k]. Further if Tγk is a power of a Dehn twist along the curve γ then there exists ϕ ∈ Bn such that ϕ(γij ) = γ, by the previous lemma. Thus Tγk = ϕAkij ϕ−1 . Since Bn [k] is a normal subgroup of Bn and Akij ∈ Bn [k] we find that Tγk ∈ Bn [k], and thus P Bn [k] ⊂ Bn [2k]. Consider now an arbitrary conjugate of ggj2k g−1 , g ∈ Bn . Then gj2 is a Dehn twist Tγj−1j and hence k ggj2k g−1 = Tg(γ . Thus this conjugate belongs to P Bn [k]. In particular every element of Bn [2k] j−1j ) belongs P Bn [k]. This proves the claim. The next step is to embed M (Σ0,n+1 ) into the mapping class group Mn+1 of the closed orientable surface of genus n + 1, by adding a one-holed torus to each boundary component (see e.g. [23] for the injectivity claim). Consider then the quantum representation ρk of Mn+1 at a k-th root of unity. According to [1, 7] we have ∩∞ k=2 ker ρk = 1. Since the image by ρk of a k-th power of a Dehn twist along a curve in Σn+1 is trivial it follows that the subgroup P Bn [k] is contained in ker ρk , namely Bn [2k] ⊂ ker ρk . As a consequence ∩k∈ZBn [2k] = 1. This proves the proposition. Proposition 2.5. The conjecture 2.1 is false for n ≥ 5, for all but finitely many q of even order. Proof. One knows by results of Bigelow ([2]), Moody ([19]), Long and Paton ([17]) that for n ≥ 5 the (generic i.e. for formal q) the Burau representation β into GL(n − 1, Z[q, q −1 ]) is not faithful. Let a ∈ Bn be such a non-trivial element in the kernel of β. Suppose that the conjecture 2.1 is true for infinitely many primitive roots of unity q of even order. Then a should belong to the intersection of kernels of all βq , over all roots of unity q. By the previous proposition we have ∩∞ k=2 Bn [2k] = 1. If ker βq = Bn [2k] for infinitely many roots of unity q of even order 2k, it follows from the previous proposition that a ∈ ∩∞ k=2 Bn [2k] = 1, which is a contradiction. Remark 2.3. The proof of the asymptotic faithfulness in [7] is given for one primitive root of unity q of given order. However this works as well for any other primitive root of unity, by using a Galois conjugate. 8

3

The image of Burau’s representation of B3 at roots of unity

3.1

Discrete subgroups of P U(1, 1)

In this section we will be concerned with Squier’s conjecture for B3 . To be precise we will consider the Burau representation β−q at the root −q, instead of q, for reasons that will appear later. Notice that for q = 1 then the arguments below will break down and the Burau representation β−1 is not conjugate to the Jones representation which is abelian. However, adirect inspection shows    1 1 1 0 that β−1 (B3 ) is the subgroup of GL(2, Z) generated by and . This is the 0 1 −1 1 well-known group SL(2, Z). From now on we will consider q 6= 1. Let us denote by A = β−q (g12 ) and B = β−q (g22 ) and C = β−q ((g1 g2 g1 )2 ). As it is well-known P B3 is isomorphic to the direct product F2 × Z, where F2 is freely generated by g12 and g22 and the factor Z is the center of B3 generated by (g1 g2 g1 )2 . It is a simple check that  2      q 1+q 1 0 −q 3 0 A= , B= , C= 0 1 −q − q 2 q 2 0 −q 3 Recall that P SL(2, Z) is the quotient of B3 by its center. Since C is a scalar matrix the homomorphism β−q : B3 → GL(2, C) factors to a homomorphism β −q : P SL(2, Z) → P GL(2, C). The free     1 2 1 0 subgroup of P SL(2, Z) generated by and is the isomorphic image of F2 ⊂ P B3 . 0 1 2 1 The fundamental object we are interested in is the subgroup Γ−q of P GL(2, C) = P U (1, 1) generated by A and B. We observed above that, when q = exp(iα), then 1. if α ∈ [ π3 , 5π 3 ] then β−q is unitary and the image belongs to U (2); 2. if α ∈ [− π3 , π3 ] then β−q is can be conjugated into U (1, 1). Our aim now is to find whether the group of β−q (B3 ) is discrete. If the image can be conjugated to be contained in U (2) (and then after renormalization to SU (2)) the discreteness is equivalent to the finiteness of the image. The finiteness of the image was completely characterized by Jones in [14] for all Bn (actually for the Jones representation, but using proposition 2.3 the same holds, when n = 3, for the Burau representation). He studied the case of the smallest root of unity for each order, but Galois conjugates of q yield isomorphic groups so that the discussion in [14] is complete: the only cases where the image of β−q (B3 ) is finite is when −q is a primitive root of unity of order 1, 2, 3, 4, 6 or 10. It appears that the complex unitary case is even more interesting since we can find infinite discrete subgroups of SU (1, 1). Our first result in this direction is: Proposition 3.1. The
group Γ−q , is conjugate to a discrete subgroup of P U (1, 1) if and only if , for integer k or else Γ−q is finite and so −q is a primitive root of unity of either q = exp ±2πi k order 1, 2, 3, 4, 6 or 10. Proof. We will consider the action of Γ−q on the projective line CP 1 . However, this action is unitarizable and hence we can conjugate it to action. Let then V be the matrix from   a simpler 1 a (1−q)a , for −q 6= −1, where a is given by the proof of proposition 2.3, namely V = 1 0 a

9

ǫ(1 − q)c = −qa2 and ǫ is the sign of the real number 1 − q − q. Denote the conjugate V −1 ZV by Z. We have then ! !  2  1+q 3 q 2 +q 5 1+q 2 q 2 −q 3 q 0 2 2 2 2 1−q 1−q (1−q) a (1−q) a , AB = A= ,B= 3 3 2 0 1 −q(1 + q)a2 − q+q −q(1 + q)a − q+q 1−q 1−q where the product AB =



−q 2 − q 2 − q q 2 + q 3 −q − q 2 q2



.

We observed above that, when q = exp(iα) then 1. if α ∈ [ π3 , 5π 3 ] then β−q is unitary and its image is contained in U (2); 2. if α ∈ [− π3 , π3 ] then β−q can be conjugated into U (1, 1). 2 We will consider then that α = 2π k for some integer k ≥ 3 (and discard then q = 1). The case 2π −1 when α = − k is completely similar. In particular V β−q (B3 )V belongs to a copy of U (1, 1).

Recall that U (1, 1) is a subgroup of GL(2, C) which keeps invariant (and hence acts on) the unit disk D ⊂ C. The action of P U (1, 1) on D is conjugate to the action of the isomorphic group P SL(2, R) on the upper half plane. The former is simply the action by isometries on the disk model of the hyperbolic plane. The key point of our argument is the existence of a fundamental domain for the action of Γ−q on D. We will look to the fixed points of the isometries A, B, AB on the hyperbolic plane D. We have the following list: 1. A has fixed point set {0, ∞}, and thus a unique fixed point in D, namely its center O. 2

q −q+1 1 2. B has fixed point set {− (1−q)a 2 , − q(1−q)a2 } and thus a unique fixed point in D, namely 2

q −q+1 1 P = − q(1−q)a 2 . In fact, if cos(α + π) ∈ [− 2 , −1], then

2 p p q − q + 1 = |1 − q||a|2 = |1 − q + q 2 | = 1 + 2 cos(α + π) ∈ [0, 1] q(1 − q)a2 2

q −q+1 q 3. AB has fixed point set {− (1−q)a 2 , − (1−q)a2 } and thus an unique fixed point in D, namely 2

q −q+1 Q = − (1−q)a 2.

Therefore A, B and AB are elliptic elements of P U (1, 1). Actually all of them are rotations of the same angle 2α: 1. A(z) = q 2 z and hence A is the counterclockwise rotation of angle 2α around O; 2. B is conjugate to A and thus is either a rotation of angle 2α around P or else of angle −2α. 3. AB has the (distinct since q 2 6= 1) eigenvalues −q 3 and −q and so is diagonalizable. Therefore AB is either the rotation of angle 2α around Q or else the rotation of angle −2α. \ Consider now the geodesic triangle ∆ = OP Q in D. The angle P OQ at O equals α since Q = qP . Since the argument of q is acute it follows that the orientation of the arc P Q is counterclockwise. Moreover, this shows that d(O, P ) = d(O, Q), where d denotes the hyperbolic distance in D and \ \. hence the angles OP Q = OQP

10

2

Let us introduce the element D = β−q (g2 ), which verifies D 2 = B. Then D = B. We can compute next ! 2 D=

1 1−q

−qa2

q −q+1 (1−q)2 a2 q2 1−q

We know that D is a rotation of angle α (or −α) around P since is conjugated to β−q (g1 ). We can check that D(Q) = 0 and hence D is the counterclockwise rotation of angle α around P and d(P, Q) = d(P, O). Thus all angles of the triangles are equal to α. This also shows that B is the counterclockwise rotation of angle 2α. Since both A and B are counterclockwise rotations of angle 2α it follows that AB is also the counterclockwise rotation of angle 2α.

3.2

Triangular groups as images of the pure braid group P3

We will use now a result of Knapp from [12], later rediscovered by Mostow (see [20]) and Deraux ([6], Theorem 7.1): Lemma 3.1. The three rotations of angle 2α in D around the vertices of an equilateral hyperbolic triangle ∆ of angles α generate a discrete subgroup of P U (1, 1) if and only if α = 2π n , with n ∈ Z+ , n ≥ 4 if n even and n ≥ 7 if n odd. Proof. This is a particular case treated in ([6], Theorem 7.1), but also in [12]. This ends the proof. Before to proceed we make a short digression on triangle groups. Let ∆ be a triangle in the π π π 1 hyperbolic plane of angles m , n , p , so that m + n1 + 1p < 1. The extended triangle group ∆∗ (m, n, p) is the group of isometries of the hyperbolic plane generated by the three reflections R1 , R2 , R3 in the sides of ∆. It is well-known that a presentation of ∆∗ (m, n, p) is given by ∆∗ (m, n, p) = hR1 , R2 , R3 ; R12 = R22 = R32 = 1, (R1 R2 )m = (R2 R3 )n = (R3 R1 )p = 1i The second type of relations have a simple geometric meaning. In fact, the product of the reflections in two adjacent sides is a rotation by the angle which is twice the angle between those sides. The subgroup ∆(m, n, p) generated by the rotations a = R1 R2 , b = R2 R3 , c = R3 R1 is a normal subgroup of index 2, which coincides with the subgroup of isometries preserving the orientation. One calls ∆(m, n, p) the triangular (or triangle) group associated to ∆ (also called von Dyck groups). Moreover, the triangle group has the following presentation: ∆(m, n, p) = ha, b, c ; am = bn = cp = 1, abc = 1i Observe that ∆(m, n, p) makes sense also when m, n or p are negative integers, by interpreting the associated generators as clockwise rotations. The triangle ∆ is a fundamental domain for the action of ∆∗ (m, n, p) on the hyperbolic plane. Thus a fundamental domain for ∆(m, n, p) consists of the union ∆∗ of ∆ with the reflection of ∆ in one of its sides. Proposition 3.2. Let m < k be such that gcd(m, k) = 1. Then the group Γ− exp( ±2mπi ) is a 2k triangular group with presentation Γ− exp( ±2mπi ) = hA, B; Ak = B k = (AB)k = 1i 2k

11

Proof. Denote by ∆( απ , απ , απ ) the group generated by the rotations of angle 2α around vertices of the triangle ∆ of angles α, even if α is not an integer part of π. We saw above that Γ−q is isomorphic to ∆( απ , απ , απ ). π π π ∗ When α = 2π 2k the group ∆( α , α , α ) is a triangular group, namely it has the rhombus ∆ as fundamental domain for its action on D. In particular Γ−q is the triangular group with the given presentation.

For the general case of α = 2πm 2k where q is a primitive 2k-th root of unity
the situation is however and inducing an autoquite similar. There is a Galois conjugation sending −q into − exp ±2πi 2k morphism of P GL(2, C). Although this automorphism does not preserve the discreteness it is an isomorphism of Γ−q onto Γ− exp( ±2πi ) . This settles the claim. 2k

If n is odd n = 2k + 1 then the group Γ−q is a quotient of the triangular group associated to ∆ which embeds into the group associated to some sub-triangle ∆′ of ∆. Proposition 3.3. Let m < k be such that gcd(m, 2k + 1) = 1. Then the group Γ− exp( ±2mπi ) is 2k+1

isomorphic to the triangular group ∆(2, 3, 2k + 1) and has the following presentation (in terms of our generators A, B): Γ− exp( ±2mπi ) = hA, B; A2k+1 = B 2k+1 = (AB)2k+1 = 1, (A−1 B k )2 = 1, (B k Ak−1 )3 = 1i 2k+1

Proof. It suffices to consider the case when m = 1, as above. The discreteness proof of ([6], Theorem 2k+1 2k+1 7.1) shows that the group ∆( 2k+1 2 , 2 , 2 ), which is generated by the rotations a, b, c around vertices of the triangle ∆ embeds into the triangle group associated to a smaller triangle ∆′ . One constructs ∆′ by considering all geodesics of ∆ joining a vertex and the midpoint of its opposite side. The three median geodesics pass through the barycenter of ∆ and subdivide ∆ into 6 equal triangles. We can take for ∆′ any one of the 6 triangles of the subdivision. It is immediate that ∆′ π , π2 and π3 so that the associated triangle group is ∆(2, 3, 2k + 1). The latter group has angles 2k+1 has presentation ∆(2, 3, 2k + 1) = hα, u, v ; α2k+1 = u3 = v 2 = αuv = 1i where the generators are the rotations of double angle around the vertices. 2k+1 2k+1 Lemma 3.2. The natural embedding of ∆( 2k+1 2 , 2 , 2 ) into ∆(2, 3, 2k + 1) is an isomorphism.

Proof. A simple geometric computation shows that a = α2 , b = vα2 v = u2 α2 u, c = uα2 u2 2k+1 2k+1 Therefore α = ak+1 ∈ ∆( 2k+1 2 , 2 , 2 ).

Introducing this in the relation αuv = 1 we obtain ak+1 uv = 1, and thus u = ak v. The relation u3 2k+1 2k+1 reads now ak (vak v)ak v = 1 and replacing bk = vak v we find v = ak bk ak ∈ ∆( 2k+1 2 , 2 , 2 ). 2k+1 2k+1 2k+1 2k+1 2k+1 Further u = ak v = a−1 bk ak ∈ ∆( 2k+1 2 , 2 , 2 ). This means that ∆( 2 , 2 , 2 ) is actually ∆(2, 3, 2k + 1), as claimed.

It suffices now to find a presentation of ∆(2, 3, 2k + 1) that uses the generators A = a, B = b. It is not difficult to show that the group with the presentation of the statement is isomorphic to ∆(2, 3, 2k + 1), the inverse homomorphism sending α into ak+1 , u into A−1 B k Ak and v into Ak B k Ak .

12

3.3

Proof of Theorem 1.2

Proposition 3.4. Assume that q is a primitive n-th root of unity and g1 , g2 are the standard generators of B3 . 1. If n = 2k then β−q (B3 ) has the presentation with generators g1 , g2 and relations Braid relation: Power relations: Center relation:

g1 g2 g1 = g2 g1 g2 , g12k = g22k = (g12 g22 )k = 1, (g1 g2 )3lcm(3,k)gcd(2,k) = 1

2. If n = 2k + 1 then β−q (B3 ) has the presentation with generators g1 , g2 and relations Braid relation: Power relations: Extra relations: Center relation:

g1 g2 g1 = g2 g1 g2 , g12k+1 = g22k+1 = (g12 g22 )2k+1 = 1, (g1−2 g22k )2 = (g12k g22k−2 )3 = 1, (g1 g2 )6lcm(3,2k+1) = 1

Proof. Recall that P B3 is the direct product of the free group generated by A = g12 , B = g22 with the center of B3 , which is generated by (g1 g2 )3 . The image of (g1 g2 )3 by β−q is the scalar matrix −q 3 1. Now the order of −q 3 is r = lcm(3, k)gcd(2, k) if q is a primitive 2k-th root of unity and respectively r = 6lcm(3, 2k + 1) when q is a primitive 2k + 1-th root of unity. This proves the center relations. We have then P B3 ∩ ker β−q = ker β−q ∩ F (g12 , g22 ) × h(g1 g2 )3r i where n is the order of −q. Lemma 3.3. If n = 2k + 1 the inclusion P B3 ⊂ B3 induces an isomorphism P B3 B3 → P B3 ∩ ker β−q ker β−q Equivalently, we have an exact sequence 1 → P B3 ∩ ker β−q → ker β−q → S3 → 1 Proof. The induced map is clearly an injection. Observe next that g12k+1 , g22k+1 ∈ ker β−q and thus for every x ∈ B3 there exists some η ∈ ker β−q such that ηx ∈ P B3 . Thus the image of the class ηx is x and this shows that the induced homomorphism is a surjection. The second assertion follows. In particular if we add the elements g12k+1 , g22k+1 to the set of normal generators of P B3 ∩ ker β−q we obtain a set of normal generators of ker β−q . This proves the claim concerning odd n. Lemma 3.4. If n = 2k, k ≥ 4 then ker β−q ⊂ P B3 . Thus the inclusion P B3 ⊂ B3 induces an exact sequence P B3 B3 1→ → → S3 → 1 P B3 ∩ ker β−q ker β−q Proof. It suffices to consider to show that β−q (g) 6∈ β−q (P B3 ) for g ∈ {g1 , g2 , g1 g2 , g2 g1 , g1 g2 g1 }. Since none of β−q (g), for g as above is a scalar matrix it is equivalent to show that β−q (g) 6∈ β−q (hg12 , g22 i) = hA, Bi. We will conjugate everything and work instead with A, B. The triangular group generated by A and B has fundamental domain the rhombus ∆∗ consisting of ∆ union its reflection Rj ∆. We call the common edge of the two triangle of the rhombus the diagonal.

13

The image of the gi is the rotation of angle α around a vertex of the triangle ∆. If it belongs to ∆(k, k, k) then it should act as an automorphism of the tessellation with copies of ∆∗ . When the vertex fixed by gi lies on the diagonal of ∆∗ then a rotation of angle α sends the rhombus onto an overlapping rhombus (having one triangle in common) and thus it cannot be an automorphism of the tessellation. This argument does not work when the vertex is opposite to diagonal. However, let us color the triangle ∆ in white and Rj ∆ in black. Continue this way by coloring all triangles in black and white so that adjacent triangles have different colors. It is easy to see that the rotations of angle 2α (and hence all of the group ∆(k, k, k)) send white triangles into white triangles. But the rotation of angle α around a vertex opposite to the diagonal sends a white triangle into a black one. This shows that the image of the gi does not belong to ∆(k, k, k). The last cases are quite similar. The images of g1 g2 (and g2 g1 ) sends ∆∗ into an overlapping rhombus having one triangle in common. While the image of g1 g2 g1 does not preserve the black and white coloring. This proves the lemma. It follows that the ker β−q is generated by the same generators as ker β−q ∩ P B3 and hence the relations are the claimed ones. Remark 3.1. If gcd(n, 3) = 1 then the central relation is a consequence of the other relations. Specifically, we have (g1 g2 )3m = g12m g2 (g12 g22 )m g2−1 , by using g1 g2 g12 g2 = g2 g12 g2 g1 . It seems to be independent when 3 divides n. Corollary 3.1. The inclusion B3 [2k] ⊂ ker β−q is strict, showing a failure of Squier’s conjecture 2.1 of a different nature. Proof. In fact the free group F (g12 , g22 ) is a normal subgroup of B3 . Moreover B3 [2k] ∩ F (g12 , g22 ) is the normal subgroup generated by g12k and g22k while ker β−q ∩ F (g12 , g22 ) is generated by the three power relations. But one verifies easily that element (ab)k does not belong to the normal subgroup generated by ak and bk in the free group F (a, b), when k > 2. This proves the claim. Remark 3.2. It is likely that (g12 g22 )k 6∈ Bn [2k] for any number n of strands.

4

Johnson subgroups and proof of Theorem 1.3

For a group G we denote by G(k) the lower central series defined by G(1) = G, G(k+1) = [G, G(k) ], k ≥ 1 An interesting family of subgroups of the mapping class group is the set of higher Johnson groups, defined as follows. We set π = π1 (Σg ). We set Ig (k) to be the group of mapping classes of homeomorphisms whose action by outer automorphisms on π/π(k+1) is trivial. Thus Ig (0) = Mg , Ig (1) is the Torelli group, while Ig (2) is the group generated by the Dehn twists along separating simple closed curves and considered by Johnson and Morita (see e.g. [13, 21]), which is often denoted Kg . Proposition 4.1. For g ≥ 3 we have the following chain of normal groups of finite index ρD (Ig (3)) ⊂ ρD (Kg ) ⊂ ρD (Tg ) ⊂ ρD (Mg )

14

ρ (M )

Proof. There is a surjective homomorphism f1 : Sp(2g, Z) → ρDD (Tgg) . The image of the D-th power of a Dehn twist in ρD (Mg ) is trivial. On the other hand the image of a Dehn twist in Sp(2g, Z) is a transvection and taking all Dehn twist one obtains a generator system for Sp(2g, Z). Using the congruence subgroup property for Sp(2g, Z) (when g ≥ 2) the image of D-th powers of Dehn twists in Sp(2g, Z) generate the congruence subgroup Sp(2g, Z)[D] = ker(Sp(2g, Z) → Sp(2g, Z/DZ)). Since the mapping class group is generated by Dehn twists the homomorphism f1 should factor Sp(2g,Z) through Sp(2g,Z)[D] = Sp(2g, Z/DZ)). In particular the image is finite. T

By the work of Johnson one knows that Kgg is a finitely generated abelian group A isomorphic V to 3 H ⊕ F , where F is a finite group and H is the homology of the surface. Thus there is a ρ (T ) surjective homomorphism f2 : A → ρDD(Kgg ) . The group Tg is generated by Dehn twists Tγ along

bounding simple closed curves γ and elements of the form Tγ Tδ−1 , where γ, δ are disjoints simple closed curves cobounding a subsurface (of genus 1 actually). Now TγD and (Tγ Tδ−1 )D = TγD Tδ−D ) ρD (Tg ) ρD (Kg ) . But the classes of Tγ (Tγ Tδ−1 )D will generate the subgroup

have trivial images image in

and Tγ Tδ−1 generate also A and hence

the classes of TγD and

DA of multiples of D. This shows that

f2 factors through A/DA which is a finite group. In particular

ρD (Tg ) ρD (Kg )

is finite.

Kg Ig (3)

Eventually is also a finitely generated abelian group A3 (the image of the third Johnson homomorphism). Since Kg is generated by the Dehn twists along bounding simple closed curves ρ (Kg ) the previous argument shows that ρDD(Ig (3)) is the image of an onto homomorphism from A3 /DA3 and hence finite. Remark 4.1. A natural question is whether ρD (Ig (k + 1)) is of finite index into ρF (Ig (k)), for every k. The arguments above break down at k = 3 since we don’t know convenient generators of the higher Johnson groups which are products of commuting Dehn twists. More specifically we have Ig (k) by the Johnson homomorphism. Here to know the image of the group Mg [D] ∩ Ig (k) into Ig (k+1) Mg [D] is the normal subgroup generated by D-th powers of Dehn twists. If the image is a lattice ρ (Ig (k)) Ig (k) then we can deduce as above that ρDD(Ig (k+1)) is finite. in Ig (k+1) The main result of this subsection is the following more precise restatement of Theorem 1.3: Proposition 4.2. For every D ≥ 7, g ≥ 4 (and D 6∈ {1, 2, 3, 4, 6, 10} for g = 2, 3) and arbitrary k, the image ρD (Ig (k)) of the quantum SO(3)-representation (at a primitive D-th root of unity) contains a free non-abelian group. Proof. The proof is similar to that from [8], where one proved that the image of the quantum representation ρD at level D is infinite in the given range. Consider g ≥ 4. Let us embed Σ0,4 into Σg by means of curves c1 , c2 , c3 , c4 as in the figure below. Then the curves a and b which are surrounding two of the holes of Σ0,4 are separating.

c3

c2

c1 a

b

c4

The subgroup generated by the Dehn twists along a and b is a free subgroup of Mg . The pure braid group P B3 embeds into M0,4 which further embeds into Mg when g ≥ 4. Then the group generated by the Dehn twists a and b is identified with the subgroup generated by g12 and g22 into 15

P B3 . Moreover, P B3 has a natural action on a sub-block of the conformal block associated to Σg as in [8], which is isomorphic to the restriction of the Burau representation at D-th roots of unity at P B3 . Notice that the two Dehn twists above are from Kg . Proposition 4.3. The above embedding of P B3 into Mg sends P B3 (k) into Ig (k). Proof. Choose the base point p of the fundamental group π1 (Σg ) on the circle c4 that separates the subsurfaces Σ3,1 and Σg−3,1 . Let ϕ be a homeomorphism of Σ0,4 that is identity on the boundary and whose mapping class b belongs to P B3 ⊂ M0,4 . Consider its extension ϕ e to Σg by identity outside Σ0,4 . Its mapping class B in Mg is the image of b into Mg . In order to understand the action of B on π1 (Σg ) we introduce three kinds of loops based at p: 1. Loops of type I are those included in Σg−3,1 . 2. Loops of type II are those contained in Σ0,4 . 3. Begin by fixing three simple arcs λ1 , λ2 , λ3 embedded in Σ0,4 joining p to the three other boundary components c1 , c2 andc3 respectively. Loops of type III are of the form λ−1 i xλi , where x is some loop based at the endpoint of λi and contained in the 1-holed torus bounded by ci . Thus loops of type III generate π1 (Σ3,1 , p). Now, the action of B on homotopy classes of loops of type I is trivial. The action of B on homotopy classes of loops of type II is completely described by the action of b ∈ P B3 on π1 (Σ0,4 , p). Specifically, let A : B3 → Aut(F3 ) be the Artin representation (see [3]). Here F3 is the free group on three generators x1 , x2 , x3 which is identified with the fundamental group of the 3-holed disk Σ0,4 . (k)

Lemma 4.1. If b ∈ P B3

then A(b)(xi ) = li (b)−1 xi li (b), where li (b) ∈ F3 (k) .

Proof. This is folklore. Moreover, the statement is valid for any number n of strands instead of 3. Here is a short proof avoiding heavy computations. It is known that the set P Bn,k of those pure braids b for which the length m Milnor invariants of their Artin closures vanish for all m ≤ k is a normal subgroup P Bn,k of Bn . Further the central series of subgroups P Bn,k satisfies (see e.g. [22]) [P Bn,k , P Bn,m ] ⊂ P Bn,k+m, for all n, k, m, and hence, we have P Bn (k) ⊂ P Bn,k . Now, if b is a pure braid then A(b)(xi ) = li (b)−1 xi li (b), where li (b) is the so-called longitude of the ith strand. The longitude is uniquely determined by its linking number with the strand, for instance if we ask that the total exponent of xi into li (b) to be zero. Next we can interpret Milnor invariants as coefficients of the Magnus expansion of the longitudes. In particular, this correspondence shows that b ∈ P Bn,k if and only if li (b) ∈ Fn (k) . This proves the claim. The action of B on homotopy classes of loops of type III can be described in a similar way. Let a −1 homotopy class a of this kind be represented by a loop λ−1 i xλi . Then λi ϕ(λ) is a loop contained in Σ0,4 , whose homotopy class ζi = ζi (b) depends only on b and λi . Then it is easy to see that B(a) = ζi−1 aζi Let now yi , zi be standard homotopy classes of loops based at a point of ci which generate the fundamental group of the holed torus bounded by ci , so that {y1 , z1 , y2 , z2 , y3 , z3 } generate π1 (Σ3,1 , p), the free group F6 of rank 6.

16

(k)

Lemma 4.2. If b ∈ P B3

then ζi (b) ∈ F6 (2k) .

Proof. It suffices to observe that ζi (b) is actually the i-th longitude li (b) of the braid b, expressed now in the generators yi , zi instead of the generators xi . We also know that xi = [yi , zi ]. Let then ζ : F3 → F6 be the group homomorphism given on the generators by ζ(xi ) = [yi , zi ]. Then ζi (b) = ζ(li (b)). Eventually if li (b) ∈ F3(k) then ζ(li (b)) ∈ F6(2k) and the claim follows. Therefore the class of B belongs to Ig (k), since its action on every generator of π1 (Σg ) is a conjugation by an element of π1 (Σg )(k) . The group ρD (P B3 ) contains the image of the Burau representation into SO(3) and hence a free non-abelian subgroup. We claim that ρD (P B3 (k) ) also contains a free non-abelian group for every k. This follows from the results in the previous section. In fact a finitely generated subgroup of SO(3) is either finite or abelian or else dense in SO(3). If the group is dense then it contains a free non-abelian subgroup. Moreover solvable groups of SU (2) (and hence of SO(3)) are abelian. The finite subgroups of SO(3) are well-known. They are the following: cyclic groups, dihedral groups, tetrahedral group (automorphisms of the regular tetrahedron), the octahedral group (the group of automorphisms of the regular octahedron) and the icosahedral group (the group of automorphisms of the regular icosahedron or dodecahedron). All but the last one are actually solvable groups. The icosahedral group is isomorphic to the alternating group A5 and it is well-known that it is simple (and thus non-solvable). As a side remark this group appeared first in relation with the non-solvability of the quintic equation in Felix Klein monography [11]. Now the subgroup ρD (P B3(k) ) contains Γq (k) = βq (P B3(k) ), (when k ≥ 2). Lemma 4.3. The group Γq (k) is non-solvable and thus non-abelian for any k. Moreover, Γq (k) cannot be A5 , for any k. Proof. If Γq (k) were solvable then Γq would be solvable. But one knows that Γq is not solvable. In fact any infinite triangle group has a finite index subgroup which is a surface groups of genus at least 2. Therefore each term of the lower central series of that surface group embeds into the corresponding term of the lower central series of Γq , so that the later is non-trivial. Since the lower central series of a surface group of genus at least 2 consists only of infinite groups it follows that no term can be isomorphic to the finite group A5 either. The previous lemma shows that βq (P B3 (k) ) is neither finite nor abelian, so that it is dense in SO(3) and hence it contains a free non-abelian group. Thus ρD (Ig (k)) contains a free non-abelian subgroup. Remark 4.2. Here is another proof for k = 1 and those D having a prime factor larger than 5. The image ρD (Mg ) into P U (N (D, g)) is dense in P SU (N (D, g)), if D = 2r, and r ≥ 5 prime (see [15]). In particular the image is Zariski dense. By Tits’ alternative (see [26]) the image is either solvable or else it contains a free non-abelian subgroup. However if the image were solvable then its Zariski closure would be a solvable Lie group. Thus the image should contain a free non-abelian subgroup. If D ≡ 0(mod 4) is not prime but has a prime factor r ≥ 5 then the claim for D follows from that for 2r. Using the strong version of Tits’ theorem due to Breuillard and Gelander (see [4]) there exists a free non-abelian subgroup of Mg /Mg [D] whose image in P SU (N (D, g)) is dense. Here Mg [D] denotes the (normal) subgroup generated by D-th powers of Dehn twists.

17

Corollary 4.1. For any k the quotient group Ig (k)/Mg [D] ∩ Ig (k), and in particular Kg /Kg [D], (for g ≥ 3 and D ≡ 0(mod 2) and D 6= 2, 3, 4, 6, respectively D 6= 10 for g ≤ 3) contains a free non-abelian subgroup. Here Kg [D] is the (normal) subgroup generated by the D-th powers of Dehn twist along bounding simple closed curves.

5

Intermediary normal subgroups and proof of Theorem 1.4

We start by showing that there are several families of intermediary normal subgroups, at least when D is not a prime. Proposition 5.1. Assume that D has a divisor F < D verifying D 6= 2, 3, 4, 6, respectively D 6= 10 for g = 3. Then ρD (Mg [F ]) is infinite and of infinite index in ρD (Mg ). Proof. For the sake of simplicity assume that F and D are even. We show first that ρD (Mg [F ]) is infinite. We have ρD (Mg [F ]) ⊃ ρD (P B3 [F ]), and thus it suffices to show that the image of the group generated by σ12F and σ22F is infinite. The image of this group by the Burau representation βq (for q D = 1) is the subgroup IF (D) of ∆(D, D, D) generated by aF , bF . Lemma 5.1. The element [aF , bF ] is of infinite order in ∆(D, D, D) and in particular IF (D) is infinite. Proof. The element cF F = [aF , bF ] belongs to the commutator subgroup ∆(D, D, D)(2) . We will actually show that the image of cF F is of infinite order in H1 (∆(D, D, D)(2) ). In order to do that let us describe the abelianization of ∆(D, D, D)(2) . We introduce the elements cij = [ai , bj ] ∈ ∆(D, D, D)(2) , for all 1 ≤ i, j ≤ D. Lemma 5.2. The group H1 (∆(D, D, D)(2) ) is the quotient of the free abelian group generated by cij , for 1 ≤ i, j ≤ D − 1, by the relation D−1 X

cii − ci+1i = 0

i=1

Proof. The kernel K of the abelianization homomorphism Z/DZ ∗ Z/DZ → Z/DZ × Z/DZ is the free group generated by the commutators. If we denote by e a and eb the generators of the two cyclic i j groups Z/DZ then K is freely generated by [e a , eb ], where 1 ≤ i, j ≤ D − 1. The group ∆(D, D, D) is the quotient of Z/DZ ∗ Z/DZ by the relation (e ae b)D , which belongs to K. Therefore we can (2) identify ∆(D, D, D) and the quotient of K by one additional relation. Further, one has the following identity in the free group: (e aeb)D e a−De b−D = [e a, eb][eb, e a2 ][e a2 , eb2 ] · · · [e aD−1 , ebD−1 ][ebD−1 , e aD ][e aD , ebD ]

This proves that the abelianization H1 (∆(D, D, D)(2) ) is generated by cij with one relation, as stated. From this description we obtain that the element cF F is of infinite order in H1 (∆(D, D, D)(2) ) and hence of infinite order in ∆(D, D, D)(2) . This proves that ρD (Mg [F ]) is infinite. Let us show now that ρD (Mg [F ]) is of infinite index in ρD (Mg ). Let consider σ12 , σ22 i be the subgroup of P B3 which is embedded into Mg as in the previous section.

18

Lemma 5.3. We have Mg [F ] ∩ hσ12 , σ22 i = hσ12F , σ22F , (σ12 σ22 )F i. Consequently the map P B3 → Mg induces an injection Mg hσ12 , σ22 i ֒→ 2F 2F 2 2 F M hσ1 , σ2 , (σ1 σ2 ) i g [F ] Proof. We know that P B3 ∩ Mg [F ] ⊂ P B3 ∩ ker ρD = ker ρF |P B3 . Since ρF |P B3 contains the Burau representation βqF (at qFF = 1) we infer ker ρF |P B3 ⊂ ker βqF . But we identified the kernel of Burau representation on P B3 in section 3.3. and the lemma follows. This lemma implies that

ρD (Mg ) ρD hσ12 , σ22 i) ֒→ 2F 2F 2 2 F ρ ρD (hσ1 , σ2 , (σ1 σ2 ) i) D (Mg [F ])

Further, since βqD is a sub-representation of ρD we have also an obvious injection βqD (hσ12 , σ22 i) ρD hσ12 , σ22 i) ֒→ βqD (hσ12F , σ22F , (σ12 σ22 )F i) ρD (hσ12D , σ22D , (σ12 σ22 )D i) But the left term of the last inclusion is the quotient of ∆(D, D, D) by the normal subgroup generated by aF , bF , (ab)F , namely the group ∆(F, F, F ). In particular it is infinite as soon as F ≥ 4. The proof for odd D or F is similar. We skip the details. For a group G we denote by G(k) the derived central series defined by G(1) = G, G(k+1) = [G(k) , G(k) ], k ≥ 1 We can prove now: Proposition 5.2. The group ρD ([[Kg , Kg ], [Kg , Kg ]]) is infinite and has infinite index in ρD (Kg ). Proof. We consider the embedding of M0,4 into Mg from the figure below.

c1

c2 a

c3 b

c4

This induces an embedding P B3 ⊂ Kg as in [16]. This embedding further induces a homomorphism (k)

(k)

τ (k) :

P B3

(k+1)

P B3

→

Kg

(k+1)

Kg

It is known from [16] that τ (1) is an injective homomorphism. We want to analyze the higher (2) homomorphism τ (2) . Recall that P B3 = Z × F2 so that P B3 = [F2 , F2 ] is the free group generated by the commutators dij = [ai , bj ], where i, j ∈ Z \ {0}. Let then dij denote the class of dij into (2)

P B3

(3)

P B3

(2)

= H1 (P B3 ). We are not able to show that τ (2) is injective. However a weaker statement

will suffice for our purposes: Lemma 5.4. The image τ (2) (hd11 i) is non-trivial. 19

Proof. Observe that the image of d11 into the graded group associated to the lower central series P B3 (2) is non-zero and actually generates the group. We have then a commutative diagram P B3 (3)

P B3 (2) P B3 (3)

→

Kg (2) Kg (3)

↑

↑

(k)

P B3

(k+1)

P B3

(k)

→

Kg

(k+1)

Kg

The vertical arrows are surjective. A result of Oda and Levine (see [16], Theorem 7) shows that the top homomorphism between lower central series quotients is an embedding. This shows that Kg the image of τ (2) (d11 ) into Kg (2) is non-trivial and hence the claim follows. (3)

In order to complete the proof it is enough to show that the image subgroup hρD (d11 )i into the ρ (P B3 (2) ) quotient group ρD ) is infinite. This is a consequence of the following: D (P B3 (3)

Lemma 5.5. The image of the group hβqD (d11 )i into the quotient group

βqD (P B3 (2) ) βqD (P B3 (3) )

is infinite.

Proof. The quotient group under scrutiny is H1 (∆(D, D, D)(2) ) (assuming D even). The image of d11 in the quotient is the element c11 arising in the proof of Lemma 5.1, and hence of infinite order if D ≥ 4. This proves the fact that ρD ([[Kg , Kg ], [Kg , Kg ]]) is of infinite index. Eventually ρD ([[Kg , Kg ], [Kg , Kg ]]) is infinite if g ≥ 4 and D ≥ 10 because ρD (Kg ) has a free non-abelian subgroup. The proposition follows. Remark 5.1. A more delicate question is whether ρD ([Kg , Kg ]) is of finite index in ρD (Kg ). Our method cannot answer this question since the abelianization of the triangular group is finite. Nevertheless, if the answer to this question were negative then we would obtain that H1 (Kg ) is infinitely generated. We are able now to prove Theorem 1.4, which we restate here in more precise terms: Proposition 5.3. The group ρD (Mg ) is not a lattice in a higher rank semi-simple Lie group for g ≥ 3 and D ≥ 11. In particular, when D is basically prime then ρD (Mg ) is of infinite index in P U (OD ). Proof. By a theorem of Margulis any normal subgroup in a lattice of rank at least 2 is either finite or else of finite index. In particular Proposition 5.2 shows that ρD (Mg ) is not such a lattice. Let now U be the complex unitary group associated to the Hermitian form on the conformal blocks. The group U has rank at least 2, when the genus g ≥ 3. In fact one observes first that there exists at least one primitive D-th root of unity (D ≥ 10) for which the hermitian form is not positive definite for g ≥ 2. Then the structure of conformal blocks shows then that in higher genus we have large sub-blocks on which the hermitian form is negative (respectively positive) definite. Thus the rank is at least 2. In particular, ρD (Mg ) is not a lattice in P U. Now ρD (Mg ) is contained in the lattice P U (OD ). Since finite index subgroups in a lattice are lattices it follows that ρD (Mg ) is of infinite index in U (OD ).

20

References [1] J.E.Andersen, Asymptotic faithfulness of the quantum SU(n) representations of the mapping class groups, Ann. of Math. 163(2006), 347–368. [2] S.Bigelow, The Burau representation is not faithful for n = 5, Geom. Topol. 3(1999), 397–404. [3] J.Birman, Braids, Links, and Mapping Class Groups, Ann. Math. Studies, Princeton Univ. Press, 1969. [4] E.Breuillard and T.Gelander, On dense free subgroups of Lie groups, J. Algebra 261(2003), 448–467. [5] D.Cooper and D.D. Long, A presentation for the image of Burau(4) ⊗ Z2 , Invent. Math. 127(1997), 535–570. [6] M.Deraux, On the universal cover of certain exotic K¨ ahler surfaces of negative curvature, Math. Annalen 329(2004), 653-683. [7] M.H.Freedman, K.Walker and Zhenghan Wang, Quantum SU(2) faithfully detects mapping class groups modulo center, Geom. Topol. 6(2002), 523–539. [8] L.Funar, On the TQFT representations of the mapping class groups, Pacific J. Math. 188(2)(1999), 251–274. [9] S.Garoufalidis and J.Levine, Finite type 3-manifold invariants and the structure of the Torelli group. I, Invent. Math. 131(1998), 541–594. [10] P.Gilmer and G.Masbaum, Integral lattices in TQFT, Ann. Sci. cole Norm. Sup. (4) 40(2007), 815–844. [11] F.Klein, Lectures on the icosahedron and the solution of equation of the fifth degree, 1888. [12] A.W.Knapp, Doubly generated Fuchsian groups, Michigan J. Math. 15(1969), 289-304. [13] D.Johnson, The structure of the Torelli group. II. A characterization of the group generated by twists on bounding curves, Topology 24(1985), 113–126. [14] V.F.R.Jones, Braid groups, Hecke algebras and type II1 factors, in Geometric methods in operator algebras, Proc. US-Japan Seminar, H.Araki, E.G.Effros Eds., 1986, 242-271. [15] M.Larsen and Zhenghan Wang, Density of the SO(3) TQFT representation of mapping class groups, Comm. Math. Phys. 260(2005), 641–658. [16] J.Levine, Pure braids, a new subgroup of the mapping class group and finite-type invariants, Tel Aviv Topology Conference: Rothenberg Festschrift (1998), 137–157, Contemp. Math., 231, Amer. Math. Soc., Providence, RI, 1999. [17] D.D.Long and M.Paton, The Burau representation is not faithful for n ≥ 6, Topology 32(1993), 439–447. [18] G.Masbaum, An element of infinite order in TQFT-representations of mapping class groups, Lowdimensional topology (Funchal, 1998), 137–139, Contemp. Math., 233, Amer. Math. Soc., Providence, RI, 1999. [19] J.A. Moody, The faithfulness question for the Burau representation, Proc. Amer. Math. Soc. 119(1993), no. 2, 671–679. [20] G.D.Mostow, On discontinuous action of monodromy groups on the complex n-balls, J.A.M.S. 1(1988), 555-586. [21] S.Morita, On the structure of the Torelli group and the Casson invariant, Topology 30(1991), 603–621. [22] T.Ohkawa, The pure braid groups and the Milnor µ-invariants of links, Hiroshima Math. J. 12(1982), 485–489. [23] L.Paris and D.Rolfsen, Geometric subgroups of mapping class groups, J. Reine Angew. Math. 521(2000), 47–83. [24] C.Radin and L. Sadun, On 2-generator subgroups of SO(3), Trans. Amer. Math. Soc. 351(1999), 4469– 4480. [25] C.Squier, The Burau repersentation is unitary, Proc. Amer. Math. Soc. 90(1984), 199–202. [26] J.Tits, Free subgroups in linear groups, J. Algebra 20(1972), 250–270. [27] H.Wenzl, On sequences of projections, C.R.Math. Rep. Acad. Sci. Canada 9(1987), 5–9.

21