---
abstract: 'Let $C$ be a complex smooth projective algebraic curve endowed with an action of a finite group $G$ such that the quotient curve has genus at least $3$. We prove that if the $G$-curve $C$ is very general for these properties, then the natural map from the group algebra ${{\mathbb Q}}G$ to the algebra of ${{\mathbb Q}}$-endomorphisms of its Jacobian is an isomorphism. We use this to obtain (topological) properties regarding certain virtual linear representations of a mapping class group. For example, we show that the connected component of the Zariski closure of such a representation acts ${{\mathbb Q}}$-irreducibly in a $G$-isogeny space of $H^1(C; {{\mathbb Q}})$ and with image often a ${{\mathbb Q}}$-almost simple group.'
address:
- 'Departamento de Matemática, UFMG, Belo Horizonte - MG (Brasil)'
- 'Yau Mathematical Sciences Center, Tsinghua University Beijing (China) and Mathematisch Instituut, Universiteit Utrecht (Nederland)'
author:
- Marco Boggi
- Eduard Looijenga
title: Curves with prescribed symmetry and associated representations of mapping class groups
---

[^1]

Introduction {#introduction .unnumbered}
============

A classical theorem of Hurwitz (1886) asserts that for a *very general* complex projective smooth curve $C$ of genus $\ge 2$, the endomorphism ring of its Jacobian $J(C)$ is as small as possible, namely ${{\mathbb Z}}$. Lefschetz [@lefschetz] proved in 1928 that this is still true if we restrict ourselves to hyperelliptic curves and we refer to Ciliberto’s survey [@Ciliberto] for what other results of that type were known before 1989. As of then the theorem of Lefschetz has been refined and generalized in several directions. For example, Ciliberto, van der Geer and Teixidor i Bigas [@CGT] studied (and to some extent characterized) the locus of curves for which the endomorphism ring of the Jacobian is strictly larger than ${{\mathbb Z}}$, while Zarhin addressed this problem over other base fields and also considered in [@zahrin2004] and [@zahrin2012] curves with a specified automorphism of a particular type other than a hyperelliptic involution: he proved that for a general such curve the endomorphism ring of its Jacobian is indeed as small as it could possibly be.

In the present paper we obtain a natural generalization of these results, at least over the base field ${{\mathbb C}}$, by proving such a minimality property for curves endowed with an action of a given (but arbitrary) finite group. More precisely, we prove that for a very general complex smooth projective curve $C$ with automorphism group $G$ whose quotient curve has genus at least $3$, the natural map from ${{\mathbb Q}}G$ to the rational endomorphism algebra of the Jacobian of $C$ is an isomorphism (Corollary \[endo2\]). We also obtain a result of this kind in the hyperelliptic case (Theorem \[endo\] and Theorem \[theorem:GN\]).

Although we believe that this has an interest of its own, our motivation has been of a more topological nature, since it has implications for certain virtual linear representations of the (pointed) mapping class groups, as we shall now explain. Let $S$ be a closed oriented connected surface of genus $\ge 2$ endowed with an action of a finite group $G$ by orientation preserving diffeomorphisms. Then $G$ embeds in the mapping class group ${\operatorname{Mod}}(S)$ of $S$. The centralizer ${\operatorname{Mod}}(S)^G$ of $G$ in ${\operatorname{Mod}}(S)$ maps to a finite index subgroup of the mapping class group ${\operatorname{Mod}}(S_G)$ of the orbifold surface $S_G$ (these are mapping classes of the underlying surface which take orbifold points to orbifold points of the same type), with kernel the center $Z(G)$ of $G$. Thus ${\operatorname{Mod}}(S)^G$ and ${\operatorname{Mod}}(S_G)$ have in common a subgroup of finite index so that the natural representation of ${\operatorname{Mod}}(S)^G$ on $H^1(S,{{\mathbb Q}})$ can be regarded as a virtual linear representation of the mapping class group ${\operatorname{Mod}}(S_G)$. This representation takes its values in the centralizer of $G$ in the symplectic group ${\operatorname{Sp}}(H^1(S,{{\mathbb Q}}))$ and respects the isotypical decomposition of $H^1(S,{{\mathbb Q}})$ into irreducible ${{\mathbb Q}}G$-modules. Then the above minimality property implies that when $S_G$ has genus $\ge 3$, a ${{\mathbb Q}}G$-isotypical summand of $H^1(S,{{\mathbb Q}})$ is essentially also isotypical for the representation of ${\operatorname{Mod}}(S)^G$ (to be more precise, isotypical for the *connected monodromy group*, that is, the identity component of the Zariski closure of the image of this representation). In particular, the subspace of $H^1(S,{{\mathbb Q}})$ spanned by the finite ${\operatorname{Mod}}(S)^G$-orbits is a sum of ${{\mathbb Q}}G$-isotypical summands. This subspace comes with a natural polarizable Hodge structure which is independent of any complex structure on $S$, but let us add Putman and Wieland conjecture that this subspace is trivial.

Our main applications are through the use of Theorem \[topology1\]. For example, Corollary \[cor:primmon\] states that in case a ${{\mathbb Q}}G$-isotypical summand is not fixed by all the $G$-invariant multi-Dehn twists, and the genus of the quotient curve is at least $3$, then the connected monodromy group acts irreducibly on that summand. Theorem \[topology1\] gives rise to a number of questions, which we collected in \[quest:opology1\]. The paper also ends in this spirit: with a conjecture (\[conj:liegeneration\]) and a question (\[quest:arithmetic2\]).

We thank Ben Moonen for drawing our attention to a paper by Masa-Hiko Saito and we thank Tyakal Venkataramana for helpful consultations.

*Conventions*. If a group $G$ acts properly discretely on a smooth object $S$ in a category where this makes sense, then $S_G$ denotes the associated orbifold object.

When $G$ is a finite group and $k$ is a field of characteristic zero, then we denote by $X(k G)$ the set of irreducible characters of $k G$. For every $\chi\in X(k G)$, we let $V_\chi $ be a finitely generated (irreducible) $k G$-module with that character. Is $H$ is a $k G$-module of finite rank, then we write $H[\chi]$ for the $V_\chi$ isogeny space ${\operatorname{Hom}}_{kG}(V_\chi,H)$. The *$\chi$-isotypical summand* of $H$ is the image of the evaluation map $V_\chi\otimes_k H[\chi]\to H$ and denoted by $H_\chi$. We indeed have $H=\oplus_{\chi\in X(k G)} H_\chi$.

We write ${{\mathbb K}}$ for the Hamilton quaternions, considered as a division ring containing ${{\mathbb R}}$.

Endomorphisms of Jacobians of $G$-curves {#endomorphisms}
========================================

A central role in this paper is played by a moduli stack whose definition we will give in a moment. We do this in some detail, as it involves some subtleties. More on this can be found in the paper by Collas and Maugeais [@cm] which also addresses the question over which number field such a stack is defined.

Let $S$ be a fixed closed oriented surface of genus $g\ge 2$, $G$ a finite group and $\phi$ a monomorphism of $G$ in the mapping class group ${\operatorname{Mod}}(S)$ of $S$. By a theorem of Nielsen-Kerckhoff, we can lift $\phi$ to a monomorphism $\Phi$ of $G$ to a group of orientation preserving diffeomorphisms of $S$. This implies that $S$ admits a complex structure also invariant under $G$ so that we obtain a smooth projective complex curve $C$. We consider the $G$-curves which so arise. To be precise, we consider pairs $(C,G{\hookrightarrow}{\operatorname{Aut}}(C))$, where $C$ is a smooth projective complex algebraic curve $C$ of genus $\geq 2$ and $G{\hookrightarrow}{\operatorname{Aut}}(C)$ a monomorphism of groups such that there exists an orientation preserving $G$-equivariant homeomorphism of $C$ onto $S$. Observe that the last property only depends on the ${\operatorname{Mod}}(S)$-conjugacy class of $\phi$. So if we precompose $\phi$ with an automorphism $\sigma$ of $G$, then we get an isomorphic object if and only if there exists an orientation preserving diffeomorphism $h$ of $S$ such that $h \phi (g)h^{-1}=\phi\sigma(g) $ for all $g\in G$. We denote a pair as above simply by $(C,G)$ (which is admittedly somewhat of an abuse of notation, as it hides the datum of the ${\operatorname{In}}({\operatorname{Mod}}(S))$-orbit of $\phi$) and we will often refer to it as a *$G$-curve* of topological type $\phi$.

The $G$-curves of topological type $\phi$ are parameterized by a D-M stack ${{\mathscr M}}^\phi$, which is defined as follows. Recall that the space of isotopy classes of complex structures on $S$ compatible with the given orientation is parameterized by a connected complex manifold, the Teichmüller space ${\operatorname{Teich}}(S)$, which supports a holomorphic family of curves ${{\mathscr C}}_{{\operatorname{Teich}}(S)}/{\operatorname{Teich}}(S)$. The mapping class group ${\operatorname{Mod}}(S)$ acts on this family. We let $G$ act on ${\operatorname{Teich}}(S)$ via $\phi$ (or what amounts to the same, via the natural map $G\to {\operatorname{Mod}}(S)$) and then its fixed point locus ${\operatorname{Teich}}(S)^G$ ([^2]) parameterizes the curves of topological type $\phi$. Since the $G$-invariant complex structures on $S$ are in bijective correspondence with the complex structures on $S_G$, we may identify ${\operatorname{Teich}}(S)^G$ with the Teichmüller space ${\operatorname{Teich}}(S_G)$ of the orbifold $S_G$.

The centralizer of $G$ in ${\operatorname{Mod}}(S)$, ${\operatorname{Mod}}(S)^G$, meets ${\operatorname{Mod}}(S)$ in the center $Z(G)$ of $G$ and $G\cdot{\operatorname{Mod}}(S)^G$ is a subgroup of the normalizer of $G$ in ${\operatorname{Mod}}(S)$. The center $Z(G)$ acts trivially on ${\operatorname{Teich}}(S)^G$ and the quotient ${\operatorname{Mod}}(S)^G/Z(G)$ can be identified with a finite index subgroup of the mapping class group ${\operatorname{Mod}}(S_G)$ (this is the group of mapping classes of the underlying surface which respect the type of the orbifold points). Note that an element of ${\operatorname{Mod}}(S_G)$ lies in this image if and only if it lifts to a mapping class of $S$ which commutes with the $G$-action.

The group $G\cdot{\operatorname{Mod}}(S)^G\subset {\operatorname{Mod}}(S)$ acts on the restriction ${{\mathscr C}}_{{\operatorname{Teich}}(S)^G}/{\operatorname{Teich}}(S)^G$ of ${{\mathscr C}}(S)/{\operatorname{Teich}}(S)$ to ${\operatorname{Teich}}(S)^G$ and since $G$ acts trivially on ${\operatorname{Teich}}(S)^G$, this turns the restriction ${{\mathscr C}}_{{\operatorname{Teich}}(S)^G}/{\operatorname{Teich}}(S)^G$ into a $G$-curve over ${\operatorname{Teich}}(S)^G$. It is universal as a marked $G$-curve, the marking being given by an equivariant isotopy class of diffeomorphisms with $(S,G)$ (so $\phi$ is here still being suppressed in the notation). If we divide out this universal marked $G$-curve by the action of ${\operatorname{Mod}}(S)^G$, that is, if we pass to the stack or orbifold quotient, then we obtain a smooth complex D-M stack, denoted ${{\mathscr M}}^\phi$, which parameterizes the $G$-curves of topological type $\phi$. It is characterized by the property that if $Y$ is a complex variety (or scheme for that matter), then a smooth $G$-curve over $Y$ of topological type $\phi$ gives rise to a (stack) morphism $Y\to {{\mathscr M}}^\phi$, and that thus is obtained a bijection between the isomorphism classes of smooth $G$-curves over $Y$ of topological type $\phi$ and the morphisms from $Y$ to ${{\mathscr M}}^\phi$. Beware that if we ignore the stack structure by passing to the underlying variety, the fiber over every closed point is a quotient of a $G$-curve by a subgroup of $G$ which contains $Z(G)$.

It is clear that ${{\mathscr M}}^\phi$ only depends on the ${\operatorname{In}}({\operatorname{Mod}}(S))$-orbit of $\phi$. Since ${\operatorname{Teich}}(S)^G$ is a simply-connected manifold, ${{\mathscr M}}^\phi$ is irreducible and the fundamental group of the underlying analytic orbifold is ${\operatorname{Mod}}(S)^G$. Precomposition with an automorphism $\rho$ of $G$ may lead to a different D-M complex stack ${{\mathscr M}}^{\phi\rho}$ to which we shall refer as the *$\rho$-twist* of ${{\mathscr M}}^\phi$. If however, $\rho$ is inner and defined by conjugation with some $g\in G$, say, then $g: C\to C$ makes the $G$-curve isomorphic to its $\rho$-twist. So the union ${{\mathscr M}}^{[G]}$ of ${\operatorname{Aut}}(G)$-twists (denoted by Collas-Maugeais in [@cm] by ${{\mathscr M}}_{g}[G]$, where $g$ is the genus of $S$) comes with an action of ${\operatorname{Aut}}(G)$, which permutes its irreducible components through ${\operatorname{Out}}(G)$. If we pass to the stack quotient with respect to this ${\operatorname{Aut}}(G)$-action, we obtain a D-M stack ${{\mathscr M}}^{(G)}$ which parameterizes curves $C$ endowed with a subgroup of ${\operatorname{Aut}}(C)$ isomorphic to $G$ (but with no such isomorphism specified).

The forgetful morphism ${{\mathscr M}}^{\phi}\to {{\mathscr M}}(S)$ is finite and the universal $G$-curve ${{\mathscr C}}_{{{\mathscr M}}^\phi}/{{\mathscr M}}^\phi$ fits in a forgetful cartesian diagram which maps to the universal curve ${{\mathscr C}}_{{{\mathscr M}}(S)}/{{\mathscr M}}(S)$. Note that we have a factorization ${{\mathscr M}}^{\phi}\to{{\mathscr M}}^{(G)}\to {{\mathscr M}}(S)$. In fact, ${{\mathscr M}}^{(G)}$ can be regarded as the normalization of the image of ${{\mathscr M}}^{\phi}\to {{\mathscr M}}(S)$ (this expresses the fact that a curve $C$ whose automorphism group contains a copy of $G$, will in general have only one such copy) and ${{\mathscr M}}^{\phi}\to{{\mathscr M}}^{(G)}$ will be étale.

The following observation is important for what follows. For every $G$-curve $(C,G)$ of type $\phi$ (so defining a point of ${{\mathscr M}}^\phi$), $G$ acts on $H^0(C,\Omega_C)$ but the character of this action will be independent of the point we choose, simply because ${{\mathscr M}}^\phi$ is irreducible. In other words, this only depends on $\phi$, so that this character may be regarded as a topological invariant. We are therefore justified in denoting it by $\chi^\phi$. It is clear that for every $\rho\in {\operatorname{Aut}}(G)$, $\chi^{\phi\sigma}=\rho^*\chi^{\phi}$.

Let us say that a geometric point $\{x\}\to {{\mathscr M}}^\phi$ is a *very general point* of ${{\mathscr M}}^\phi$ if it is not contained in a given countable union of closed proper substacks of ${{\mathscr M}}^\phi$.

\[endo\]Let $G$ be a finite group and $\phi : G\to {\operatorname{Mod}}(S)$ a monomorphism such that the centralizer of the image of $\phi$ does not contain a hyperelliptic involution. Assume that the associated moduli stack ${{\mathscr M}}^\phi$ has positive dimension and that for a $G$-curve $C$ representing a point of ${{\mathscr M}}^\phi$ the action of $G$ on $H^0(C, \Omega_C)$ satisfies the following two properties:

1.  every irreducible symplectic representation occurring in $H^0(C, \Omega_C)$ occurs with multiplicity $\ge 3$ and

2.  every non-self-dual representation occurring in $H^0(C, \Omega_C)$ occurs along with its dual and both do so with multiplicity $\ge 2$.

Then the natural homomorphism ${{\mathbb Q}}G\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is surjective, provided the $G$-curve $C$ represents a very general member of ${{\mathscr M}}^\phi$.

\[rem:verygeneral=dominant\] We can avoid the notion of a very general point by using the following more concrete (and equivalent) formulation of the conclusion of this theorem: if ${{\mathscr C}}/Y$ is a $G$-curve over a complex variety $Y$ of type $\phi$ such that the associated morphism $Y\to{{\mathscr M}}^\phi$ is dominant, then ${{\mathbb Q}}G\to {\operatorname{End}}_{{\mathbb Q}}(J({{\mathscr C}}/Y))$ is onto. This is essentially what we shall prove and it is also the formulation through which we will use it.

\[notinjective\]If the genus of the quotient curve is zero, then the natural homomorphism ${{\mathbb Q}}G\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is not injective, as the trivial $G$-module is not represented in $H^0(C,\Omega_C)$. But as we will see, the Chevalley-Weil formula implies that this map is injective when the genus of $C_G$ is at least $2$. This formula will also show that the conditions we imposed on the $G$-action on $H^0(C, \Omega_C)$ are topological, in the sense that they only depend on how $G$ acts on $\phi$ (as we should expect).

The assumption that ${{\mathscr M}}^\phi$ has positive dimension translates into asking that the moduli space of the quotient orbifold curve (so endowed with the finite subset representing irregular $G$-orbits) is not rigid. This simply amounts to demanding that the regular $G$-orbit space of $C$ has negative Euler characteristic, but is not a $3$-punctured sphere. It is also equivalent to $H^0(C, \Omega_C^{\otimes 2})^G$ being nonzero.

The hypothesis that $G$ does not contain a hyperelliptic involution in its center is needed in order to ensure that some point of ${{\mathscr M}}^\phi$ is defined by a nonhyperelliptic curve (this will actually imply that every point of ${{\mathscr M}}^\phi$ has that property).

Observe that if a nontrivial automorphism of $C$ acts trivially on its canonical image in $\check{{\mathbb P}}(H^0(C,\Omega_C))$, then $C$ is hyperelliptic and this automorphism is the hyperelliptic involution. The latter acts as $-1$ on $H^0(C,\Omega_C)$, and so $G$ always acts faithfully on $H^1(C,{{\mathbb C}})$.

The hypothesis on the $G$-module structure of $H^0(C,\Omega_C)$ comes from the use of the following lemma in the proof.

\[lemma:quadratic\] Let $F$ be a finite dimensional ${{\mathbb C}}G$-module with the property that

1.  every irreducible symplectic representation occurring in $F$ occurs with multiplicity $\ge 3$ and

2.  every non-self-dual representation occurring in $F$ occurs along with its dual and both do so with multiplicity $\ge 2$.

Let $L$ be a compact Lie subgroup of ${\operatorname{GL}}(F)$ which contains the image of $G$ and has the property that $({\operatorname{Sym}}^2F)^L= ({\operatorname{Sym}}^2F)^G$. Then $L$ is contained in the image of ${{\mathbb R}}G$ in ${\operatorname{End}}(F)$.

The proof uses the decomposition of ${{\mathbb R}}G$ as an ${{\mathbb R}}$-algebra in terms of the decomposition of ${{\mathbb C}}G$ as a ${{\mathbb C}}$-algebra and so we need to recall how this comes about. Remember that $X({{\mathbb C}}G)$ stands for the set of characters of irreducible ${{\mathbb C}}G$-modules, that for every $\chi \in X({{\mathbb C}}G)$ we have chosen a representative ${{\mathbb C}}G$-module $V_\chi$, and that the obvious map $$\textstyle {{\mathbb C}}G\to \prod_{\chi\in X({{\mathbb C}}G)} {\operatorname{End}}(V_\chi)$$ is an isomorphism of ${{\mathbb C}}$-algebras. ‘Taking the dual’ defines an involution $\chi\mapsto \chi^*$ in $X({{\mathbb C}}G)$. Then for all $\chi, \mu \in X({{\mathbb C}}G)$, the space of $G$-invariant bilinear forms $V_\chi\times V_{\mu}\to {{\mathbb C}}$ is nonzero unless $\mu=\chi^*$, in which case it is generated by a nondegenerate pairing $\alpha_\chi: V_\chi\times V_{\chi^*}\to {{\mathbb C}}$. The fixed point set of this involution decomposes into the orthogonal characters $X({{\mathbb C}}G)_+$ (the $\chi$ for which $\alpha_\chi$ is symmetric) and the symplectic characters $X({{\mathbb C}}G)_-$ (the $\chi$ for which $\alpha_\chi$ is anti-symmetric). Let $X({{\mathbb C}}G)_o\subset X({{\mathbb C}}G)$ be a system of representatives for the free orbits, so that $X({{\mathbb C}}G)$ is the disjoint union of $X({{\mathbb C}}G)_+$, $X({{\mathbb C}}G)_-$, $X({{\mathbb C}}G)_o$ and $X({{\mathbb C}}G)_o^*$. Choose a $G$-invariant inner product $h_\chi$ in $V_\chi$. When $\chi \in X({{\mathbb C}}G)_\varepsilon$ with $\varepsilon=\pm$, let $c_\chi$ be the anti-linear automorphism of $V_\chi$ characterized by the property that $\alpha_\chi(x,y)=h_\chi(c_\chi(x), y)$. Note that $c_\chi$ is $G$-equivariant. Hence so is $c_\chi^2$. As $c_\chi^2$ is also ${{\mathbb C}}$-linear, Schur’s lemma (applied to the irreducible ${{\mathbb C}}G$-module $V_\chi$) implies that $c_\chi^2$ must be a scalar, $\lambda$, say. The identity $$\lambda h_\chi(x, x)=h_\chi(c_\chi^2(x), x)=\alpha_\chi(c_\chi(x),x)=\varepsilon \alpha_\chi(x, c_\chi(x))=\varepsilon h_\chi(c_\chi(x), c_\chi(x))$$ then shows that $\lambda$ is real and has the sign $\varepsilon$. Upon replacing $\alpha_\chi$ by $|\lambda|^{-1/2}\alpha_\chi$, we arrange that $c_\chi^2=\varepsilon 1$. In the orthogonal case ($c_\chi^2=1$), the fixed point space of $c_\chi$ is a real form $V_\chi({{\mathbb R}})$ of $V_\chi$ and so ${{\mathbb R}}G$ maps to ${\operatorname{End}}_{{\mathbb R}}(V_\chi({{\mathbb R}}))$. In the symplectic case ($c_\chi^2=-1$), $c_\chi$ endows $V_\chi$ with the structure of a right ${{\mathbb K}}$-module (recall that ${{\mathbb K}}$ denotes the Hamilton quaternions: here we let $c_\chi$ act as $j$), which extends the ${{\mathbb C}}$-vector space structure, so that ${{\mathbb R}}G$ maps to ${\operatorname{End}}_{{\mathbb K}}(V_\chi)$. It is known that the disjoint union of $\{V_\chi({{\mathbb R}})\}_{\chi\in X({{\mathbb C}}G)_+}$, $\{V_\chi\}_{\chi\in X({{\mathbb C}}G)_-}$ and $\{V_\chi\}_{\chi\in X({{\mathbb C}}G)_o}$ represent the distinct elements of $X({{\mathbb R}}G)$ (complexification as a ${{\mathbb R}}G$-module reproduces resp. $V_\chi$, $V_\chi\oplus V_\chi$ and $V_\chi\oplus V_{\chi*}$) and that the obvious ${{\mathbb R}}$-algebra homomorphism $$\label{eqn:rdec}
\textstyle {{\mathbb R}}G\to  \big(\prod_{\chi\in X({{\mathbb C}}G)_+} {\operatorname{End}}_{{\mathbb R}}(V_\chi({{\mathbb R}})) \big)  \times \big(\prod_{\chi\in X({{\mathbb C}}G)_-} {\operatorname{End}}_{{\mathbb K}}(V_\chi)\big)  
\times \big(\prod_{\chi\in X({{\mathbb C}}G)_o} {\operatorname{End}}(V_\chi)\big).$$ is an isomorphism (see for instance [@Serre], §13.2).

For $\chi\in X({{\mathbb C}}G)$ we put $F[\chi]:={\operatorname{Hom}}_{{{\mathbb C}}G}(V_\chi, F)$, so that the obvious evaluation map $\oplus_{\chi\in X({{\mathbb C}}G)} V_\chi\otimes F[\chi]\to F$ is an isomorphism of $G$-modules. Via this isomorphism, the image of ${{\mathbb C}}G$ in ${\operatorname{End}}(F)$ is identified with the product of the ${\operatorname{End}}(V_\chi)$ taken over all $\chi\in X({{\mathbb C}}G)$ with $F[\chi]\not=0$. The generators $\alpha_\chi$ determine an isomorphism $$({\operatorname{Sym}}^2 F)^G
\cong \big(\underset{\chi\in X({{\mathbb C}}G)_+}\oplus {\operatorname{Sym}}^2 F[\chi]\big)  \oplus \big(\underset{\chi\in X({{\mathbb C}}G)_-}\oplus  \wedge^2 F[\chi]\big)  
\oplus \big(\underset{\chi\in X({{\mathbb C}}G)_o} \oplus F[\chi]\otimes F[\chi^*]\big).$$ We are given that this space is also equal to $({\operatorname{Sym}}^2 F)^L$. We claim that for every $\chi\in X({{\mathbb C}}G)$ and every line $\ell \subset F[\chi]$, $L$ preserves $V_\chi\otimes\ell$. To see this, note that with any $q\in {\operatorname{Sym}}^2 F$ is associated a subspace $F_q\subset F$, namely the smallest subspace of $F$ such that $q\in {\operatorname{Sym}}^2F_q$ (so that $q$ is nondegenerate, when regarded as a quadratic form on the dual of $F_q$). It is clear that any linear transformation of $F$ which fixes $q$, leaves $F_q$ invariant. In our case this means that $L$ leaves invariant every subspace $F_q$ with $q\in ({\operatorname{Sym}}^2 F)^G$. Among the nonzero subspaces $F_q$ we thus obtain are those of the form $V_\chi\otimes \ell$ with $\chi\in X({{\mathbb C}}G)_+$ and $\ell$ any line in $F[\chi]$, $V_\chi\otimes P$ with $\chi \in X({{\mathbb C}}G)_-$ and $P$ any plane in $F[\chi]$, and $(V_\chi\otimes \ell)\oplus (V_{\chi^*}\otimes \ell')$ with $\chi\in X({{\mathbb C}}G)_o$ and $\ell\subset F[\chi]$ and $\ell'\subset F[\chi^*]$ arbitrary lines. Then our assumptions (i) and (ii) imply that for every $\chi\in X({{\mathbb C}}G)$ and every line $\ell$ in $F[\chi]$, $V_\chi\otimes\ell$ is an intersection of such subspaces (and hence preserved by $L$). It follows that $L$ acts trivially on each ${{\mathbb P}}(F[\chi])$, or rather, that $L$ is contained in $\prod_{\chi\in X({{\mathbb C}}G)} {\operatorname{GL}}(V_\chi)\otimes \mathbf{1}_{F[\chi]}$, so that $L$ is at least contained in the image of ${{\mathbb C}}G$ in ${\operatorname{End}}(F)$. If $F[\chi]=0$, then the factor ${\operatorname{GL}}(V_\chi)\otimes \mathbf{1}_{F[\chi]}$ is of course trivial, but otherwise $L$ will act on $V_\chi$ in such a way that it preserves $\alpha_\chi$.

Assume that $F[\chi]\not=0$. Since $L$ is compact, it leaves invariant an inner product in $V_\chi$. Since this inner product is also left invariant by $G$, we may assume that this is the one we used to define the $c_\chi$ above in case $\chi\in X({{\mathbb R}}G)_\pm$. So then $L$ preserves $c_\chi$ when defined. Hence in all cases $L$ lands in the corresponding factor of the decomposition given by Equation (\[eqn:rdec\]) so that it is contained in the image of ${{\mathbb R}}G$ in ${\operatorname{End}}(F)$.

\[rem:\] This lemma can fail when a symplectic representation occurs with multiplicity $2$. To see this, let $V$ be an irreducible ${{\mathbb R}}G$-module which is of quaternionic type. This means that the ${{\mathbb R}}$-algebra $k:={\operatorname{End}}_{{{\mathbb R}}G}(V)$ is isomorphic to ${{\mathbb K}}$ and the image of ${{\mathbb R}}G$ in ${\operatorname{End}}_{{\mathbb R}}(V)$ is equal to ${\operatorname{End}}_k(V)$. Let $s: V\times V\to {{\mathbb R}}$ be a $G$-invariant positive definite quadratic form. Then $L:={\operatorname{O}}(V, s)$ is clearly not contained in the image of ${{\mathbb R}}G$. The ${{\mathbb C}}G$-module $F:={{\mathbb C}}\otimes_{{\mathbb R}}V$ is a direct sum of two copies of a symplectic irreducible ${{\mathbb C}}G$-module, but both $G$ and $L$ have the same subspace of invariants in ${\operatorname{Sym}}^2(V_{{\mathbb C}})$, namely the line spanned by the tensor determined by $s$.

Before we begin the proof of Theorem \[endo\], we recall that for a polarized abelian variety $A$, the polarization determines in the algebra ${\operatorname{End}}_{{\mathbb Q}}(A)$ an anti-involution (the Rosati involution) $b\mapsto b^\dagger$ which is positive in the sense that for every nonzero $b\in {\operatorname{End}}_{{\mathbb Q}}(A)$, ${\operatorname{tr}}(bb^\dagger)> 0$. This remains so after tensoring with ${{\mathbb R}}$. We recall a few facts about such algebras.

Given a finite dimensional ${{\mathbb R}}$-algebra ${{\mathscr B}}$ with a positive anti-involution $\dagger$, then by the classification of such algebras, ${{\mathscr B}}$ naturally decomposes into a product of ordinary matrix algebras with coefficients ${{\mathbb R}}$, ${{\mathbb C}}$, or ${{\mathbb K}}$, and $\dagger$ is then in each factor equivalent to taking the conjugate transpose. So the set of $f\in {{\mathscr B}}$ with $ff^\dagger=1$ is a subgroup $L ({{\mathscr B}})$ of the group of units in ${{\mathscr B}}$. It is a finite extension of a product of orthogonal groups, unitary groups and unitary quaternion groups, so in particular, compact. Moreover, it generates ${{\mathscr B}}$ as an ${{\mathbb R}}$-algebra. Given a finite dimensional ${{\mathscr B}}$-module $F$, then we say that an element of ${\operatorname{Sym}}^2F$ is ${{\mathscr B}}$-invariant if for all $b\in {{\mathscr B}}$, it is killed by $b\otimes 1-1\otimes b^\dagger$. Since $L({{\mathscr B}})$ generates ${{\mathscr B}}$ as an algebra, this is equivalent to this element being $L({{\mathscr B}})$-invariant in the usual sense. There is similar notion of covariance, but in view the reductive nature of ${{\mathscr B}}$ it is not worthwhile to make the distinction.

This situation shows up when we consider a polarized abelian variety $A$, endowed with a homomorphism ${{\mathscr B}}\to  {\operatorname{End}}_{{\mathbb R}}(A)$ of algebras with involution. If we put $F:=H^0(A,\Omega_A)$, then ${\operatorname{End}}_{{\mathbb R}}(A):={{\mathbb R}}\otimes {\operatorname{End}}(A)$ (and hence ${{\mathscr B}}$) acts on $F$. As is well-known, the space of first order deformations of $A$ as a polarized variety can be identified with the complex dual of ${\operatorname{Sym}}^2F$. The first order deformations for which ${{\mathscr B}}$ still maps to the deformed object can be identified with the dual of the ${{\mathscr B}}$-coinvariants of ${\operatorname{Sym}}^2F$, i.e., the dual of $({\operatorname{Sym}}^2F)_{L({{\mathscr B}})}\cong ({\operatorname{Sym}}^2F)^{L({{\mathscr B}})}$. The more precise statement is that we have locally a universal deformation in the sense of Schlessinger with smooth base and whose cotangent space at the closed point can be identified with $({\operatorname{Sym}}^2F)^{L({{\mathscr B}})}$.\
We now turn to the proof of Theorem \[endo\]. We first reduce it to Lemma \[lemma:endo\] below. We write ${{\mathscr B}}$ for ${\operatorname{End}}_{{\mathbb R}}(J(C))$ and $F$ for $H^0(C, \Omega_C)$. The property that ${{\mathbb Q}}G\to {\operatorname{End}}_{{\mathbb Q}}(J(C))$ be surjective is equivalent to ${{\mathbb R}}G\to {{\mathscr B}}$ being surjective and since $L({{\mathscr B}})$ generates ${{\mathscr B}}$, the latter is equivalent to $L({{\mathscr B}})$ being contained in the image of ${{\mathbb R}}G$. This is what we will show.

Since $L({{\mathscr B}})$ is compact, $({\operatorname{Sym}}^2 F)^{L({{\mathscr B}})}$ has a ${L({{\mathscr B}})}$-invariant supplement $({\operatorname{Sym}}^2F)^{{L({{\mathscr B}})}\not=1}$ in ${\operatorname{Sym}}^2 F$. This is also $G$-invariant and so $$({\operatorname{Sym}}^2F)^{G,{L({{\mathscr B}})}\not=1}:=({\operatorname{Sym}}^2F)^G\cap  ({\operatorname{Sym}}^2F)^{{L({{\mathscr B}})}\not=1}$$ is a $G$-invariant supplement of $({\operatorname{Sym}}^2 F)^{L({{\mathscr B}})}$ in $({\operatorname{Sym}}^2 F)^G$. In view of our hypotheses and Lemma \[lemma:quadratic\], Theorem \[endo\] will follow from:

\[lemma:endo\] We have $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}=0$.

The proof of Lemma \[lemma:endo\] involves a Hilbert scheme. We write ${{\mathbb P}}$ for the projective space $\check{{\mathbb P}}(F)$ of hyperplanes of $F$. The very general $G$-curve yields a canonical embedding $C\subset{{\mathbb P}}$. Let us denote by ${{\mathscr H}}$ the open subscheme of the irreducible component of ${\operatorname{Hilb}}({{\mathbb P}})$ which parameterizes canonically embedded smooth projective curves. The $G$-equivariant canonical embeddings of $G$-curves in ${{\mathbb P}}$ of our given topological type define an irreducible closed subscheme ${{\mathscr H}}^G$ of ${{\mathscr H}}$. The action of $G$ on $F$ induces one on ${{\mathscr H}}$ and then ${{\mathscr H}}^G$ is an irreducible component of the fixed point locus for this action.

\[quadric\]Let $X$ be a $G$-invariant quadric hypersurface of ${{\mathbb P}}$ containing the $G$-invariant canonical curve $C$. Then the $G$-invariant Hilbert scheme ${{\mathscr H}}_X^G$ (of $G$-curves contained in $X$) is a proper subscheme of the $G$-invariant Hilbert scheme ${{\mathscr H}}^G$.

In what follows we use the following notational convention. Given an immersion $Y\hookrightarrow Z$ of schemes, we write ${{\mathcal C}}_{Y/Z}$ for its conormal sheaf on $Y$. This is an ${{\mathscr O}}_Y$-module. The normal sheaf ${{\mathcal N}}_{Y/Z}$ of this immersion is by definition the ${{\mathscr O}}_Y$-dual of ${{\mathcal C}}_{Y/Z}$.

In the proof of Lemma 2.2 in [@bl], we observed that to the nested embeddings $C\subset X\subset{{\mathbb P}}$ is associated a short exact sequence $$0\to(\Omega_C^{\otimes 2})^\vee\to{\mathcal C}_{C/{{\mathbb P}}}\to{\mathcal C}_{C/X}\to 0,$$ which, by Lemma 2.2 in [@bl], determines an exact sequence $$0\to H^0(C,{{\mathcal N}}_{C/X})\to H^0(C,{{\mathcal N}}_{C/{{\mathbb P}}})\to H^0(C, \Omega^{\otimes 2}_C)\to {\operatorname{Ext}}_C^1({{\mathcal C}}_{C/X}, {{\mathscr O}}_C)\to 0.$$ Passing to $G$-invariants is an exact functor, and so we get the exact sequence $$\label{exact}
0\to H^0(C,{{\mathcal N}}_{C/X})^G\to H^0(C,{{\mathcal N}}_{C/{{\mathbb P}}})^G\to H^0(C, \Omega^{\otimes 2}_C)^G\to {\operatorname{Ext}}_C^1({{\mathcal C}}_{C/X}, {{\mathscr O}}_C)^G\to 0,$$ where $H^0(C,{{\mathcal N}}_{C/X})^G$ and $H^0(C,{{\mathcal N}}_{C/{{\mathbb P}}})^G$ identify with the tangent space at $[C\subset X]$ to the scheme ${{\mathscr H}}_X^G$ resp.  the tangent space at $[C\subset{{\mathbb P}}]$ to the scheme ${{\mathscr H}}^G$, while $H^0(C, \Omega^{\otimes 2}_C)^G$ identifies with the cotangent space to ${{\mathscr M}}^\phi$ at $s$. By assumption $H^0(C, \Omega^{\otimes 2}_C)^G\neq\{ 0\}$. It is well-known (see [@bl], Lemma 2.1) that ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/{{\mathbb P}}},{{\mathscr O}}_C)^G= H^1(C,{{\mathcal N}}_{C/{{\mathbb P}}})^G=0$, so that the $G$-invariant Hilbert scheme ${{\mathscr H}}^G$ is smooth at $[C\subset{{\mathbb P}}]$. In case ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G=0$, then the exact sequence (\[exact\]) shows that the inclusion ${{\mathscr H}}_X^G\subseteq{{\mathscr H}}^G$ is strict.

When ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G\not=0$, we need the interpretation of this space as an obstruction space, as provided by Lemma \[obstruction3\] below. It then tells us that the $G$-equivariant deformation theory of $C$ in the quadric $X$ is obstructed. So the $G$-invariant Hilbert scheme ${{\mathscr H}}_X^G$ is singular at $[C\subset X]$ and then the inclusion ${{\mathscr H}}_X^G\subseteq{{\mathscr H}}^G$ is also strict.

\[obstruction3\]A minimal obstruction space for the $G$-equivariant deformation theory of $C$ in the quadric $X$ is ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G$.

We proved in Theorem 1.1 in [@bl] that ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)$ is a minimal obstruction space for the deformation theory of $C$ in the quadric $X$ and so we only need to check that this retains this interpretation in a $G$-equivariant setting. This is a matter of carefully going through the definitions.

Let $\omega$ be the map which assigns to every small extension $0\to I\to A'\to A\to 0$ of Artin ${{\mathbb C}}$-algebras and every deformation $\xi\co{{\mathscr C}}\subset X_A$ over $A$ of the embedding $C\subset X$ (where $X_A=X\times {\operatorname{spec}}(A)$) an element $\omega(\xi,A')\in{\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)\otimes I$ which is the obstruction to lifting $\xi$ to $A'$ (cf. Definition 5.1 in [@TV]).

The group $G$ acts on deformations of the embedding $C\subset X$ by changing the identification of the central fiber. Let us assume that $0\to I\to A'\to A\to 0$ is a tiny extension, i.e. that $I\cong{{\mathbb C}}$. Then, from the canonicity of minimal obstruction spaces (cf. Exercise 5.8 in [@TV]), it follows that $\omega(\alpha\cdot\xi,A')=\alpha\cdot\omega(\xi,A')$, for all $\alpha\in G$. Therefore, obstructions arising from $G$-equivariant lifting problems are contained in ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G$.

We have to show that all elements of ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G$ are obtained as obstructions to $G$-equivariant lifting problems. For this, it is enough to observe the following. Every element of ${\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)$ is of the form $\omega(\xi,A')$, for some tiny extension $0\to I\to A'\to A\to 0$ and some deformation $\xi\co{{\mathscr C}}\subset X_A$ over $A$ of the embedding $C\subset X$. By the construction appearing in the proof of Proposition 5.6 in [@TV], the sum of the elements appearing in the $G$-orbit of $\omega(\xi,A')$: $$\textstyle \sum_{\alpha\in G}\alpha\cdot\omega(\xi,A')=\sum_{\alpha\in G}\omega(\alpha\cdot\xi,A')\in{\operatorname{Ext}}^1_C({{\mathcal C}}_{C/X},{{\mathscr O}}_C)^G$$ is realized as the obstruction to lifting the $G$-equivariant deformation $\times_{\alpha\in G}\alpha\cdot\xi$ over $\otimes^{|G|}A$ of the embedding $C\subset X$ in a $G$-equivariant fashion.

For this we use an interpretation of $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}$ as a conormal space of moduli spaces. We first note that the restriction map $$\Phi: ({\operatorname{Sym}}^2F)^G=({\operatorname{Sym}}^2 H^0(C, \Omega_C))^G \to H^0(C, \Omega^{\otimes 2}_C)^G$$ can be understood as follows: the dual of $H^0(C, \Omega^{\otimes 2}_C)^G$ is the space of first order deformations of $C$ as a $G$-curve and the dual of $({\operatorname{Sym}}^2F)^G$ is the space of first order deformations of deformations of the Jacobian $J(C)$ as a ppav with $G$-action, and $\Phi$ is then the obvious map. (By Max Noether’s theorem, the natural map ${\operatorname{Sym}}^2F={\operatorname{Sym}}^2H^0(C, \Omega_C)\to H^0(C, \Omega^{\otimes 2}_C)$ is onto and hence the same is true for $\Phi$.) As mentioned above, the dual of $({\operatorname{Sym}}^2F)^{L({{\mathscr B}})}$ has the interpretation of the space of first order deformations of $J(C)$ as a ppav endowed with an involution preserving homomorphism of ${{\mathscr B}}$ to the endomorphism algebra tensored with ${{\mathbb R}}$. In other words, $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}$ vanishes on the first order deformations of $J(C)$ as $G$-ppav which retain ${{\mathscr B}}$ in their endomorphism algebra tensored with ${{\mathbb R}}$. By construction, the first order deformations of $J(C)$ as a $G$-Jacobian are of this type. It follows that the kernel of $\Phi$ contains $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}$. As $C$ is very general, it is also very general as a member of ${{\mathscr H}}^G$ and so this will then be true for all members parameterized by ${{\mathscr H}}^G$ (we here use that the associated morphism ${{\mathscr H}}^G\to {{\mathscr M}}(G)$ is dominant, see Remark \[rem:verygeneral=dominant\]). It follows that any nonzero element of $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}$ gives us a $G$-invariant quadric $X$ for which ${{\mathscr H}}_X^G={{\mathscr H}}^G$. As this would contradict Lemma \[quadric\], we must have $({\operatorname{Sym}}^2 F)^{G,{L({{\mathscr B}})}\not= 1}=0$.

The hypotheses of Theorem \[endo\] are satisfied in most of the situations of interest. For instance, if the genus of the quotient orbifold curve $C_G$ is positive, then so is the dimension of the corresponding moduli stack ${{\mathscr M}}^\phi$ and the center of $G$ cannot contain a hyperelliptic involution. In order to establish when the condition on the structure of the $G$-module of abelian differentials $H^0(C,\Omega_C)$ is also satisfied, we use the classical Chevalley-Weil formula, which we now recall.

Let $C\to C_G$ be a Galois covering defined by a smooth projective $G$-curve and let $\chi\in X({{\mathbb C}}G)$ be *nontrivial* (that is, not the character of the trivial irreducible ${{\mathbb C}}G$-module ${{\mathbb C}}$, which is constant equal to $1$). Denote by $B\subset C_G$ the inertia locus (a finite set). For every $Q\in B$, we choose a representative $\tilde Q\in C$ and let ${\gamma}_{\tilde Q}$ be the positive generator of the (cyclic) stabilizer $G_{\tilde Q}\subset G$. Since the conjugacy class of ${\gamma}_{\tilde Q}$ only depends on $Q$, the same is true for the characteristic polynomial of ${\gamma}_{\tilde Q}$ in $V_\chi$. For $a\in {{\mathbb Q}}\cap (0,1]$, let $N_{\chi, Q}(a)$ be the multiplicity of $\exp(2\pi\sqrt{-1}a)$ as a root of this polynomial and put $N_\chi(a):=\sum_{Q\in B} N_{\chi, Q}(a)$. Then the Chevalley-Weil formula [@cw] asserts that $$\label{eqn:CW1}
\textstyle {\operatorname{dim}}_{{\mathbb C}}H^0(X,\Omega_X)[\chi] =(g(C_G)-1) \chi (1)+ \sum_{a\in {{\mathbb Q}}\cap (0,1]} N_{\chi}(a)(1-a).$$ If $\chi$ is the trivial character, the left hand side is of course equal to $g(C_G)$. This shows that when $g(C_G)\ge 2$, all irreducible characters of $G$ are afforded by the ${{\mathbb C}}G$-module $H^0(C,\Omega_C)$, so that the natural homomorphism ${{\mathbb Q}}G\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is injective.

\[endo2\] For a very general $G$-curve $C$ for which the genus of the orbit curve $C_G$ is at least $3$, the natural homomorphism ${{\mathbb Q}}G\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is an isomorphism.

The Chevalley-Weil formula shows that every $\chi\in X({{\mathbb C}}G)$ appears in $H^0(C,\Omega_C)$ with multiplicity $\ge 2$ and even with multiplicity $\ge 4$, unless $\chi$ is the character of the trivial representation. Now apply Theorem \[endo\].

We shall see (Remark \[g=2\]) that this corollary can fail if $g(C_G)=2$.

Since the dual of $H^0(C,\Omega_C)$ can be identified with its Hodge supplement in $H^1(C; {{\mathbb C}})$, the character of $H^1(C; {{\mathbb C}})$ must be equal to $\chi^\phi +\overline{\chi^\phi}$. The character of $H^1(X,{{\mathbb C}})$ can also be computed by means of a $G$-invariant cellular decomposition of the underlying oriented surface $S$: choose one such that the induced decomposition $S_G$ has as zero skeleton the union of $B$ and a base point of $S_G{\smallsetminus}B$, has $2g(S_G)+|B|$ 1-cells and has one $2$-cell. The associated cellular complex is then $$\label{display:complex}
0\to {{\mathbb Z}}G \to ({{\mathbb Z}}G)^{2g(C_G)}\oplus ({{\mathbb Z}}G)^B\to  {{\mathbb Z}}(G)\oplus\underset{Q\in B}\oplus{{\mathbb Z}}(G/G_{\tilde Q}) \to 0.$$ The complex cohomology of $S$ as a ${{\mathbb C}}G$-module is that of the complex obtained by applying ${\operatorname{Hom}}(\; , {{\mathbb C}})$ and from this we see that for a nontrivial irreducible ${{\mathbb C}}G$-module $V_\chi$, $$\label{formula2}
\textstyle {\operatorname{dim}}_{{\mathbb C}}H^1(X,{{\mathbb C}})[\chi] =2( g(S_G)-1)\chi (1)+\sum_{Q\in B} {\operatorname{dim}}(V_\chi/V_\chi^{G_{\tilde Q}}).$$ So if $\chi^\phi$ is self-dual, then the character $H^0(C,\Omega_C)$ will then be half that of $H^1(C,{{\mathbb C}})$. Conversely, when $G$ acts freely on $S$, then a comparison with the Chevalley-Weil formula (\[eqn:CW1\]) shows that $\chi^\phi$ is half the character of $H^1(C,{{\mathbb C}})$, so that $\chi^\phi$ must be self-dual.

\[rem:\] It is not hard to see that complex conjugation sends ${{\mathscr M}}^\phi$ to a ${{\mathscr M}}^{\bar\phi}$, where $\bar\phi$ is represented by retaining our $\phi : G\hookrightarrow {\operatorname{Mod}}(S)$ and changing the orientation of $S$. We then have $\chi^{\bar\phi}=\overline{\chi^\phi}$. In particular, if ${{\mathscr M}}^\phi$ is real, then so is $\chi^\phi$. Whether this is a useful result is another matter. Let us just note that it fits in a more general Galois setting: if ${{\mathscr M}}^{(G)}$ is defined over the number field $K$ (Collas and Maugeais have shown in [@cm] and [@cm2014] that we can take $K={{\mathbb Q}}$ for $G$ cyclic), then the individual ${{\mathscr M}}^\phi$ are defined over a (common) finite extension $L/K$ so that the Galois group of $\overline K/K$ permutes them through ${\operatorname{Gal}}(L/K)$. The induced action of ${\operatorname{Gal}}(L/K)$ on the characters $\chi^\phi$ will then be the obvious one (via a cyclotomic character, since all the characters of $G$ take their values in the extension of ${{\mathbb Q}}$ generated by the $|G|$-th roots of unity).

Even though we cannot expect that the hypotheses of Theorem \[endo\] are always satisfied by $G$-curves $C$ whose orbit curve has genus zero, there is a particular case for which this happens:

\[theorem:GN\] Let $C$ be a smooth projective $G$-curve and $N\subset G$ a subgroup of index 2 which acts freely on $C$ and is such that $C_N$ is of genus $\geq 2$ with $G/N$ acting on $C_N$ as a hyperelliptic involution (so that $C_G$ is of genus zero). Then

1.  $G$ is a semi-direct product of $N$ and a group of order $2$ and the $G$-invariant subspace $H^0(C_N,\Omega_{C_N})$ of $H^0(C,\Omega_C)$ is acted on via $G/N$ with the nontrivial element acting as scalar multiplication by $-1$.

2.  Every irreducible ${{\mathbb C}}G$-submodule $V$ of $H^0(C,\Omega_C)$ not contained in $H^0(C_N,\Omega_{C_N})$ decomposes into two non-isomorphic irreducible ${{\mathbb C}}N$-modules, which are characterized by the property that if we induce them up to $G$, we get a ${{\mathbb C}}G$-module equivalent to $V$. The two are exchanged by the outer action of $G/N$ and $V$ appears in $H^0(C,\Omega_C)$ with multiplicity $\tfrac{1}{2}(g(C_{N})-1) {\operatorname{dim}}V$.

3.  If $N$ is nontrivial, then $C$ is nonhyperelliptic.

Moreover, if $C_N$ is of genus $\ge 4$ and $(C, G)$ is very general as a $G$-curve, then the natural homomorphism ${{\mathbb Q}}G\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is surjective.

In case $N$ is trivial (so that $C$ is hyperelliptic), the three assertions are trivial or empty, and the last clause follows from Theorem 1.2 in [@CGT]. We will therefore assume in the proof below that $N$ is nontrivial.

We first show that the involution $\iota$ of $C_{N}$ induced by the nontrivial element of $G/N$ lifts to an involution $\tilde\iota$ in $G$. To see this, let $x\in C$ be such that its image $x_{N}$ in $C_{N}$ is fixed under $\iota$. Since $N$ acts freely on $C$, the $G$-stabilizer $G_x$ of $x$ meets $N$ trivially. So $G_x$ is of order two and its nontrivial element $\tilde\iota$ is a lift of $\iota$. In particular, $G$ is a semi-direct product $\langle \tilde\iota\rangle\ltimes N$. Part (i) now follows, because the hyperelliptic involution $\iota$ acts on $H^0(C_N,\Omega_{C_N})$ as multiplication by $-1$.

Since $N$ is of index 2 in $G$, we have for every irreducible $G$-submodule $V$ of $H^0(C,\Omega_C)$ two possibilities, according to whether the outer action of $\iota\in G/N$ fixes the character of $V$ or not (cf. Proposition 5.1 in [@FH]):

(irr)

:   $V$ is also irreducible as a ${{\mathbb C}}N$-module (with ${\tilde}{\iota}$ acting on it trivially or by multiplication by $-1$) or

(red)

:   $V$ splits into two non-isomorphic irreducible ${{\mathbb C}}N$-modules $V'\oplus V''$, where $V''$ is obtained from $V'$ by precomposition of $N$ with conjugation by $\tilde\iota$, and ${\operatorname{Ind}}^G_N V'$ and ${\operatorname{Ind}}^G_N V''$ are equivalent to $V$.

We now invoke the Chevalley-Weil formula (\[eqn:CW1\]). The group $N$ acts freely on $C$, while the involution $\iota$ in $C_N$ is hyperelliptic, and so $G$ has $2g(C_{N})+2$ irregular orbits in $C$, each of which contains a fixed point of $\tilde\iota$. So the sum in the right hand side of (\[eqn:CW1\]) is in fact a single term $N_V(a)(1-a)$ with $N_V(a)=(2g(C_{N})+2){\operatorname{dim}}V$ and either $a=1$ or $a=\tfrac{1}{2}$, according to whether $\tilde\iota$ acts in $V$ as $1$ or as $-1$. We find that in the first case, the multiplicity of $V$ in $H^0(C,\Omega_C)$ is $-{\operatorname{dim}}V$ (which is of course absurd) and so $\tilde\iota$ must act in $V$ as $-1$ and $V$ appears with multiplicity $-{\operatorname{dim}}V+ (2g(C_{N})+2){\operatorname{dim}}V.\frac{1}{2}=g(C_{N}){\operatorname{dim}}V$. But $N$ acts freely on $C$ and hence the Chevalley-Weil formula (\[eqn:CW1\]) applied to the $N$-action implies that the only irreducible representation of $N$ which appears in $H^0(C, \Omega_C)$ with multiplicity $g(C_{N})$ is the trivial one, in other words, $V$ is then contained in $H^0(C_N, \Omega_{C_N})$.

The above dichotomy also shows that if the irreducible ${{\mathbb C}}G$-module $V$ is not contained in $H^0(C_N, \Omega_{C_N})$, then $V$ decomposes as a ${{\mathbb C}}N$-module as in (ii): $V=V'\oplus V''$. Since $V'$ appears in $H^0(C, \Omega_C)$ with multiplicity $(g(C_{N})-1){\operatorname{dim}}V'$, the ${{\mathbb C}}G$-module $V\cong {\operatorname{Ind}}^G_N V'$ appears in $H^0(C, \Omega_C)$ with the same multiplicity, i.e., $\frac{1}{2}(g(C_{N})-1){\operatorname{dim}}V$. This proves (ii). Since $\tilde\iota$ has the effect of exchanging $V'$ and $V''$, this also shows that $\tilde\iota$ is not hyperelliptic.

Suppose now $g(C_N)\ge 4$ and $(C, G)$ is very general. The one-dimensional nontrivial representation of $G/N$ appears in $H^0(C, \Omega_C)$ and its isotypical subspace is $H^0(C_N, \Omega_{C_N})$. So it has multiplicity $\ge 4$ in $H^0(C, \Omega_C)$. According to (ii), any other nontrivial irreducible representation $V$ which appears in $H^0(C, \Omega_C)$ is of the form $V'\oplus V''$ and hence appears with multiplicity $(g(C_{N})-1){\operatorname{dim}}V'\ge 3$. It then remains to apply Theorem \[endo\].

\[g=2\] Theorem \[theorem:GN\] implies that the image ${{\mathbb Q}}N\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ cannot contain the lift $\tilde\iota$ of the hyperelliptic involution (for $\tilde\iota$ permutes non-isomorphic irreducible ${{\mathbb Q}}N$-modules), so that in particular, ${{\mathbb Q}}N\to{\operatorname{End}}_{{\mathbb Q}}(J(C))$ is not surjective. Since it is easy to find nontrivial unramified covers of a closed genus 2 surface such that a hyperelliptic involution lifts, we see that Corollary \[endo2\] can fail when the genus of the orbit curve is $2$.

Symplectic $G$-modules and associated unitary groups
====================================================

We here collect (or derive) some facts regarding symplectic $G$-modules and the algebraic groups that they give rise to, insofar they are needed in this paper.

Irreducible $G$-modules {#irreducible-g-modules .unnumbered}
-----------------------

We review the basics of the theory, as can be found for instance in [@isaacs] and use this opportunity to establish notation. Let $k$ be a number totally real field or be equal to ${{\mathbb R}}$ (but for us only the cases $k={{\mathbb Q}}$ and $k={{\mathbb R}}$ will be relevant). The set $X(k G)$ of irreducible characters of $k G$ is a set of $k$-valued class functions on $G$, which is invariant under the Galois group of $k/{{\mathbb Q}}$. For every $\chi\in X({{\mathbb Q}}G)$, there is always a positive integer $m$ and an orbit in the Galois group of ${{\mathbb R}}/{{\mathbb Q}}$ in $X({{\mathbb R}}G)$ such that $\chi$ is $m$ times the sum over this Galois orbit (see e.g., Theorem (9.21) in [@isaacs]). We shall express this shortly in terms of the representations themselves.

For a given $\chi\in X(k G)$, Schur’s lemma implies that ${\operatorname{End}}_{k G}(V_\chi)$ is a division algebra. We denote the opposite algebra by $D_\chi$ and regard $V_\chi$ as a left $k G$-module and as a right $D_\chi$-module. We also fix a positive definite $G$-invariant inner product $s_\chi :V_\chi\times V_\chi\to k$. For any bilinear form $\phi: V_\chi\times V_\chi\to k$, there exists a unique $\sigma \in {\operatorname{End}}_k(V_\chi)$ such that $\phi(v, v')= s_\chi(\sigma(v), v')$. The form $\phi$ is $G$-invariant if and only if $\sigma$ is $G$-equivariant, so that the latter will be given by right multiplication with some $\lambda\in D_\chi$. For the same reason there exists a $\lambda^\dagger\in D_\chi$ such that $\phi (v, v')= s_\chi(v, v'\lambda^\dagger)$. Note that thus is defined an anti-involution $\lambda \mapsto \lambda^\dagger$ on $D_\chi$ characterized by the property that $s_\chi(v\lambda , v')=s_\chi(v, v'\lambda^\dagger)$. So if $\lambda\in D_\chi$ is symmetric in the sense that $\lambda=\lambda^\dagger$, then $(v, v')\mapsto s_\chi(v\lambda , v')=s_\chi(v, v'\lambda)$ is also a $G$-invariant symmetric bilinear form with values in $k$. In particular, $s_\chi$ need not be unique up to scalar. The identity $s(v\lambda, v'\lambda)=s(v\lambda\lambda^\dagger, v')$, shows that $\dagger$ is positive in the following sense. Denote by $L_\chi$ the center of $D_\chi$ and by $K_\chi\subset L_\chi$ the subfield fixed under the anti-involution $\dagger$. Then $K_\chi$ is finite extension of $k$ that is totally real and for every nonzero $\lambda\in D_\chi$, the trace ${\operatorname{tr}}_{D_\chi/K_\chi}(\lambda\lambda^\dagger)\in K_\chi$ has positive image under every field embedding $K_\chi\hookrightarrow {{\mathbb R}}$.

Recall that for a finitely generated $k G$-module $H$ and $\chi\in X(kG)$, we write $H[\chi]$ for ${\operatorname{Hom}}_{k G}(V_\chi, H)$. The right $D_\chi$-module structure on $V_\chi$ determines a left $D_\chi$-module structure on $H[\chi]$ by the rule $(du)(v):=u(vd)$ and then the natural map $$\oplus_{\chi\in X(k G)} V_\chi\otimes_{D_\chi} H[\chi]\to H, \quad v\otimes_{D_\chi} u\in V_\chi\otimes_{D_\chi} H[\chi]\mapsto u(v)$$ is an isomorphism of $k G$-modules. This is the $kG$-isotypical decomposition of $H$ with the image $H_\chi$ of $V_\chi\otimes_{D_\chi} H[\chi]$ being its $\chi$-isotypical summand. Any $k G$-linear automorphism of $H$ preserves its $G$-isotypical decomposition and acts in the summand $H_\chi\cong V_\chi\otimes_{D_\chi}  H[\chi]$ through a $D_\chi$-linear transformation of $H[\chi]$. Thus, the $G$-centralizer ${\operatorname{End}}_k(H)^G\subset {\operatorname{End}}_k(H)$ is identified with $\prod_{\chi\in X(k G)} {\operatorname{End}}_{D_\chi}(H[\chi])$. The natural map $$\textstyle  kG\to \prod_{\chi\in X(k G)} {\operatorname{End}}^{D_\chi}(V_\chi),$$ where ${\operatorname{End}}^{D_\chi}(V_\chi)$ stands for the algebra of endomorphisms of $V_\chi$ as a right $D^\chi$-module, is an isomorphism of $k$-algebras (this can be regarded as an instance of one of Wedderburn’s structure theorems).

We discuss the passage from $k={{\mathbb Q}}$ to $k={{\mathbb R}}$. For this, let us for now fix $\chi\in X( kG)$ and temporarily omit $\chi$ in the notation. So we write $D=D_\chi$, $L=L_\chi$, $V=V_\chi$ etc.

When $k={{\mathbb R}}$, we have the three classical cases: $D={{\mathbb R}}$ (so $\dagger$ is then the identity) or $D$ is isomorphic to ${{\mathbb K}}$ or ${{\mathbb C}}$, with $\dagger$ in both cases corresponding to the usual conjugation. (These three cases correspond to $\chi\in X({{\mathbb R}}G)$ being resp. an element of $X({{\mathbb C}}G)$, twice an element of $X({{\mathbb C}}G)$, the sum of a complex conjugate distinct pair in $X({{\mathbb C}}G)$.)

When $k={{\mathbb Q}}$ there are in some sense also three cases. Given a field embedding $\sigma :K\hookrightarrow {{\mathbb R}}$, let us use write $\sigma^*$ for the associated base change: if $F$ is a $K$-vector space, then $\sigma^*F={{\mathbb R}}\otimes_\sigma F$. For the irreducible ${{\mathbb Q}}G$-module $V=V_\chi$, the ${{\mathbb R}}G$-module $\sigma^*V$ is in general no longer irreducible, but it will be isotypical for an irreducible ${{\mathbb R}}G$-module $V^\sigma$ with character $\chi^\sigma\in X({{\mathbb R}}G)$, say with multiplicity $m^\sigma$. The characters $\{\chi^\sigma\}_\sigma$ are pairwise distinct and make up an orbit of the Galois group of ${{\mathbb R}}/{{\mathbb Q}}$ and $m^\sigma$ is independent of $\sigma$: $m^\sigma=m$. Denote by $D^\sigma$ the opposite of ${\operatorname{End}}_{{{\mathbb R}}G} (V^\sigma)$. It is a real division algebra, which will be isomorphic to ${{\mathbb R}}$, ${{\mathbb C}}$ or ${{\mathbb K}}$ and whose isomorphism type is independent of $\sigma$. We put $I^\sigma={\operatorname{Hom}}_{{{\mathbb R}}G} (V^\sigma, \sigma^*V)$ (so we could also write this as $ \sigma^*V[\chi^\sigma]$). Then $I^\sigma$ is a left $D^\sigma$-module (of rank $m$) and a right $\sigma^*D$-module and we have the isotypical decomposition as ${{\mathbb R}}G$-modules $$V_{{\mathbb R}}(:={{\mathbb R}}\otimes_{{\mathbb Q}}V)=\oplus_\sigma  \,  \sigma^*V \cong \oplus_\sigma  \, V^\sigma\otimes_{D^\sigma} I^\sigma.$$ This also identifies $\sigma^*D$ with the opposite of ${\operatorname{End}}_{{{\mathbb R}}G}(\sigma^*V)={\operatorname{End}}_{D^\sigma}(I^\sigma)$ and gives rise to the isomorphism $$\begin{gathered}
\sigma^*(H[\chi])=\sigma^*({\operatorname{Hom}}_{{{\mathbb Q}}G}(V, H))\cong{\operatorname{Hom}}_{{{\mathbb R}}G}(\sigma^*V, H_{{\mathbb R}})\cong  {\operatorname{Hom}}_{{{\mathbb R}}G}(V^\sigma\otimes_{D^\sigma} I^\sigma, H_{{\mathbb R}})\cong\\
\cong {\operatorname{Hom}}_{D^\sigma} (I^\sigma,  {\operatorname{Hom}}_{{{\mathbb R}}G}(V^\sigma, H_{{\mathbb R}}))= {\operatorname{Hom}}_{D^\sigma} (I^\sigma, H_{{\mathbb R}}[\chi^\sigma])
 \end{gathered}$$ as left $\sigma^*D$-modules. The anti-involution $\dagger$ on $D$ yields one on $\sigma^*D$ and via the identification of the latter with ${\operatorname{End}}_{D^\sigma}(I^\sigma)^{\mathrm{opp}}$ becomes ‘taking the adjoint’ with respect to a hermitian form $h^\sigma$ on $I^\sigma$ obtained as follows. There is up to scalar exactly one $G$-invariant inner product on the ${{\mathbb R}}G$-module $V^\sigma$. Let us fix one and denote it $s^\sigma$. Then the $G$-invariant symmetric bilinear form $s$ on $V$ determines a positive definite hermitian form $h^\sigma: I^\sigma\times I^\sigma\to D^\sigma$ (relative to the standard conjugation in $D^\sigma$) characterized by the property that for all $v, v'\in V^\sigma$ and $w, w'\in I^\sigma$, $s^\sigma( w(v), w'(v'))=s^\sigma(vh^\sigma (w, w'),v')$.

Symplectic $G$-modules {#symplectic-g-modules .unnumbered}
----------------------

Suppose that our $kG$-module $H$ comes with a nondegenerate $G$-invariant symplectic form $(a,b)\in H\times H\mapsto {\langle}a, b{\rangle}\in k$ (for example $H=H^1(S; k)$, and the symplectic form being the intersection product). We then first observe that we have a decomposition $$\textstyle {\operatorname{Sp}}(H)^G=\prod_{\chi\in X(k G)} {\operatorname{Sp}}(H_\chi)^G.$$ Let us now fix some $\chi\in X(k G)$. Given $u,u'\in H[\chi]$, then $(v, v')\mapsto {\langle}u(v), u'(v'){\rangle}$ is a $G$-invariant bilinear form on $V_\chi$. Since $s_\chi$ is nondegenerate, there is an $h\in {\operatorname{End}}(V_\chi)$ such that ${\langle}u(v), u'(v'){\rangle}=s_\chi (h(v), v')$ for all $v, v'\in V_\chi$. The $G$-invariance of the form implies that $h$ is right multiplication with some $h_\chi (u, u')\in D_\chi$: $${\langle}u(v), u'(v'){\rangle}=s_\chi(vh_\chi (u, u'),v').$$ It is then straightforward to verify that $h_\chi$ is $D_\chi$-linear in the first variable and satisfies $h_\chi(u, u')=-h_\chi(u',u)^\dagger$ (so that $h_\chi(u, du')=h_\chi(u',u)d^\dagger$). In other words, it is a skew-hermitian form on the $D_\chi$-module $H[\chi]$. We shall therefore denote this form by ${\langle}\; ,\; {\rangle}_\chi$. It is clearly nondegenerate. We thus find an identification $${\operatorname{Sp}}(H_\chi)^G\cong  {\operatorname{U}}_{D_\chi}(H[\chi]),$$ where ${\operatorname{Sp}}(H_\chi)^G$ stands for the $G$-centralizer in ${\operatorname{Sp}}(H_\chi)$ and ${\operatorname{U}}_{D_\chi} (H[\chi])$ denotes the group of automorphisms of $H[\chi]$ as a skew-hermitian $D_\chi$-module (but we write ${\operatorname{U}}(H[\chi])$ when $D_\chi$ is clear from the context). The group on the left is in a natural manner the group of $k$-points of a reductive $k$-algebraic group ${\operatorname{{\mathscr Sp}}}(H_\chi)^G$ and the group on the right can be regarded as the group of $K_\chi$-points of an algebraic group ${{\mathscr U}}(H[\chi])$ defined over $K_\chi$. In other words, ${\operatorname{{\mathscr Sp}}}(H_\chi)^G$ is obtained from ${{\mathscr U}}(H[\chi])$ by restriction of scalars (from $K_\chi$ to $k$).

This enables us to understand what the group of real points of ${\operatorname{{\mathscr Sp}}}(H_\chi)^G$ is like when $k={{\mathbb Q}}$. Given a field embedding $\sigma :K_\chi\to {{\mathbb R}}$, then, as we noted, the ${{\mathbb R}}$-algebra $\sigma^*D_\chi$ is isomorphic to a matrix algebra over ${{\mathbb R}}$, ${{\mathbb C}}$ or ${{\mathbb K}}$. Clearly, $\sigma^*H[\chi]$ is a left $\sigma^*D_\chi$-module and ${\langle}\; , \: {\rangle}$ determines a skew-hermitian form $\sigma^*{\langle}\; , \: {\rangle}$ on it. The above identification then yields an isomorphism $$\textstyle {\operatorname{{\mathscr Sp}}}(H_\chi)^G({{\mathbb R}})\to \prod_\sigma {\operatorname{U}}_{\sigma^*D_\chi} (\sigma^*H[\chi]),$$ where the product is over all field embeddings $\sigma: K_\chi\hookrightarrow {{\mathbb R}}$.

The factor ${\operatorname{U}}_{\sigma^*D_\chi} (\sigma^*H[\chi])$ can be identified with a unitary group over the classical division algebra $D^\sigma=D_{\chi^\sigma}$ as follows. Let us recall that we write $V^\sigma$ for $V_{\chi^\sigma}$ and $I^\sigma$ for ${\operatorname{Hom}}_{{{\mathbb R}}G} (V^\sigma, \sigma^*V)$ and that we have isomorphisms $$\sigma^*D_\chi \cong {\operatorname{End}}_{D^\sigma}(I^\sigma)^{\mathrm{opp}}\text{ (of algebras)}, \quad \sigma^*H[\chi]\cong  {\operatorname{Hom}}_{D^\sigma} (I^\sigma, H_{{\mathbb R}}[\chi^\sigma]) \text{ (of  $\sigma^*D_\chi$-modules)}.$$ If we apply the preceding to the case $k={{\mathbb R}}$, then we see that $H_{{{\mathbb R}}, \chi^\sigma}$ is a nondegenerate subspace of $H_{{\mathbb R}}$ for the symplectic form and that the $D^\sigma$-module $H_{{\mathbb R}}[\chi^\sigma]$ comes with a nondegenerate $D^\sigma$-valued skew-hermitian form. Let us denote the latter by ${\langle}\; ,\; {\rangle}^\sigma$. So if $\phi, \phi'\in \sigma^*H[\chi]\cong  {\operatorname{Hom}}_{D^\sigma} (I^\sigma, H_{{\mathbb R}}[\chi^\sigma]) $, then a skew-hermitian form on $I^\sigma$ is given by $$(u,u')\mapsto {\langle}\phi (u) ,\phi' (u') {\rangle}^\sigma$$ We had already defined a positive definite hermitian form $h^\sigma: I^\sigma\times  I^\sigma\to D^\sigma$. Hence there is a unique $g(\phi, \phi')\in {\operatorname{End}}_{D^\sigma}(I^\sigma)$ such that $${\langle}\phi (u) ,\phi' (u') {\rangle}^\sigma= h^\sigma(g(\phi, \phi')(u), u')$$ for all $u, u'\in I^\sigma$. It is straightforward to verify that $g$ defines a ${\operatorname{End}}_{D^\sigma}(I^\sigma)^{\mathrm{opp}}$-valued skew-hermitian form on ${\operatorname{Hom}}_{D^\sigma} (I^\sigma, H_{{\mathbb R}}[\chi^\sigma])$. Via the isomorphisms above this is up to a scalar the $\sigma^*D_\chi$-valued skew-hermitian form on $\sigma^*H[\chi]$. We thus find:

\[cor:\] We have a natural identification ${\operatorname{U}}_{\sigma^*D_\chi} (\sigma^*H[\chi])\cong {\operatorname{U}}_{D_{\chi^\sigma}}(H_{{\mathbb R}}[\chi^\sigma])$ and hence an identification ${\operatorname{{\mathscr Sp}}}(H_\chi)^G({{\mathbb R}})\cong \prod_\sigma  {\operatorname{U}}_{D_{\chi^\sigma}}(H_{{\mathbb R}}[\chi^\sigma])$.

So if $r$ denotes the rank of $H_{{\mathbb R}}[\chi^\sigma]$ as a $D_{\chi^\sigma}$-module, then

real

:   $H_{{\mathbb R}}[\chi^\sigma]$ is a real symplectic vector space so that ${\operatorname{U}}(\sigma^*H[\chi])$ is isomorphic to a symplectic group in $r$ real variables.

quaternion

:   $H_{{\mathbb R}}[\chi^\sigma]$ is a skew-hermitian module over $D_{\chi^\sigma}\cong {{\mathbb K}}$ and ${\operatorname{U}}(\sigma^*H[\chi])$ is isomorphic to a unitary group in $r$ quaternionic variables.

complex

:   $H_{{\mathbb R}}[\chi^\sigma]$ is a skew-hermitian module $D_{\chi^\sigma}=L_{\chi^\sigma}\cong {{\mathbb C}}$ and ${\operatorname{U}}(\sigma^*H[\chi])$ is isomorphic to a unitary group in $r$ complex variables.

Virtual linear representations of the mapping class group {#virtual}
=========================================================

Hodge group and connected monodromy group {#hodge-group-and-connected-monodromy-group .unnumbered}
-----------------------------------------

We recall that a weight $m$ Hodge structure on a ${{\mathbb Q}}$-vector space $W$ amounts to giving an action of the circle group ${\operatorname{U}}(1)$ on $W_{{\mathbb R}}$ so that $W_{{\mathbb C}}=\oplus_{p+q=m}W^{p,q}$ can be regarded as the eigenspace decomposition of an ${\operatorname{U}}(1)$-action, where $\lambda\in {\operatorname{U}}(1)$ acts as multiplication by $\lambda^{q-p}$. The *Hodge group* of this Hodge structure is defined to be the smallest ${{\mathbb Q}}$-subgroup ${\mathscr{H}\! g}(W)$ of ${\operatorname{GL}}(W)$ whose group of real points contains the image of this action. It is a connected reductive group. Note that $W$ is indecomposable as a Hodge structure, if and only if ${\mathscr{H}\! g}(W)$ acts ${{\mathbb Q}}$-irreducibly on $W$.

Now let ${{\mathbb W}}$ be a *variation of polarizable Hodge structure* ${{\mathbb W}}$ over a connected complex manifold $Y$ with Hodge numbers $(1,0)$ and $(0,1)$ (our definition assumes that there exists a sublattice ${{\mathbb W}}_{{\mathbb Z}}\subset{{\mathbb W}}$, but this is a property and not part of the data). Since the Hodge group gets smaller as the Hodge structure becomes more special and is locally constant on a dense subset, we have a locally constant family of subgroups $\{{\mathscr{H}\! g}({{\mathbb W}})_y\subseteq {\operatorname{GL}}({{\mathbb W}}_y)\}_{y\in Y}$ characterized by the property that ${\mathscr{H}\! g}({{\mathbb W}}_y)\subseteq {\mathscr{H}\! g}({{\mathbb W}})_y$ for all $y\in Y$ and with equality for some $y$. This is what is called the *generic Hodge group* (of ${{\mathbb W}}$ at $y$). Deligne’s semi-simplicity theorem ([@Deligne], Thm. 4.2.6, Cor. 4.2.8 and Cor. 4.2.9) tells us that ${{\mathbb W}}$ is semi-simple as a variation of Hodge structure and that its complexification ${{\mathbb W}}_{{\mathbb C}}$ is semisimple as a local system. The last property implies that the identity component of the Zariski closure of the monodromy group of ${{\mathbb W}}$ in ${\operatorname{GL}}({{\mathbb W}}_y)$—which we shall denote by ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ and refer to as the *connected monodromy group*—is semisimple as well. Note that ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ does not change under base change by an unramified finite cover of $Y$.

The endomorphism algebra of the local system underlying ${{\mathbb W}}$, ${\operatorname{End}}({{\mathbb W}})$, has a natural Hodge structure with Hodge numbers $(1,-1)$, $(0,0)$ and $(-1, 0)$. Note that the subalgebra ${\operatorname{End}}_{HS}({{\mathbb W}})$ of endomorphisms preserving the Hodge structure is ${\operatorname{End}}({{\mathbb W}})^{(0,0)}({{\mathbb Q}})=
{\operatorname{End}}({{\mathbb W}}_{{\mathbb C}})^{(0,0)}\cap {\operatorname{End}}({{\mathbb W}})$. If ${{\mathbb W}}$ is indecomposable as a VHS, this will be a division algebra.

Taking into account the results quoted above, then the essential part of the following lemma is second part, which is due to Masa-Hiko Saito [@mhsaito] (see also the discussion in section (1.5) of [@MZ]).

\[lemma:rigid\] Let ${{\mathbb W}}$ be a polarized variation of Hodge structure over a connected complex manifold $Y$ with Hodge numbers $(1,0)$ and $(0,1)$. Suppose ${{\mathbb W}}$ is indecomposable and has infinite monodromy (or equivalently, is not isotrivial). Then:

1.  ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ is a product of ${{\mathbb Q}}$-simple factors of the derived Hodge group ${{\mathscr {DH}}\!g}({{\mathbb W}})_y$, and its representation on ${{\mathbb W}}_y$ is isotypical. The group ${\mathscr{H}\! g}({{\mathbb W}})_y$ acts on ${\operatorname{End}}({{\mathbb W}})$ via ${\mathscr{H}\! g}({{\mathbb W}})_y/{{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ and the latter’s action on ${\operatorname{End}}({{\mathbb W}})$ has as its kernel a central subgroup.

2.  The variation of Hodge structure ${{\mathbb W}}$ is *rigid* in the sense that ${\operatorname{End}}({{\mathbb W}})$ is of type $(0,0)$ if and only if ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ acts ${{\mathbb Q}}$-irreducibly on ${{\mathbb W}}_y$; this is also equivalent to ${{\mathscr {DH}}\!g}({{\mathbb W}})_y={{\mathscr M}{on}}^\circ({{\mathbb W}})_y$.

3.  When ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ has positive ${{\mathbb Q}}$-rank, then ${{\mathscr {DH}}\!g}({{\mathbb W}})_y={{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ and this group is ${{\mathbb Q}}$-almost simple. In particular, ${{\mathbb W}}$ is rigid.

We merely show how this follows from the literature. We may assume that ${{\mathbb W}}_y$ is indecomposable as Hodge structure and that ${\mathscr{H}\! g}({{\mathbb W}}_y)={\mathscr{H}\! g}({{\mathbb W}})_y$. It is a general result of André [@Andre] that ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ is normal in ${{\mathscr {DH}}\!g}({{\mathbb W}})_y$. Since ${{\mathscr {DH}}\!g}({{\mathbb W}})_y$ is a semisimple ${{\mathbb Q}}$-algebraic group, ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ will be a product of ${{\mathbb Q}}$-simple factors of it. The isotypical decomposition of ${{\mathbb W}}_y$ as a representation of ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ will respect the Hodge decomposition on ${{\mathbb W}}_y$ and hence must be trivial in the sense that ${{\mathbb W}}_y$ will be an isotypical ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$-module. We can state this as follows. The isogeny type of ${{\mathbb W}}$ is represented by an irreducible local system ${{\mathbb V}}$ of ${{\mathbb Q}}$-vector spaces over $Y$ such that ${{\mathbb W}}$ is as local system isomorphic a sum of copies of ${{\mathbb V}}$. More precisely, $D:={\operatorname{End}}({{\mathbb V}})$ is a division algebra and if we put $U:={\operatorname{Hom}}({{\mathbb V}}, {{\mathbb W}})$, then $U$ is in a natural manner a right $D$-module such that the evident map $U\otimes_D {{\mathbb V}}\cong {{\mathbb W}}$ is an isomorphism of local systems. This isomorphism also identifies ${\operatorname{End}}({{\mathbb W}})$ with the matrix algebra ${\operatorname{End}}^D(U)$ (the endomorphisms of $U$ as a right $D$-module). But note that the tensor decomposition ${{\mathbb W}}\cong U\otimes_D {{\mathbb V}}$ cannot respect the Hodge structure, unless this decomposition is trivial. In particular, $U$ need not have a Hodge structure. On the other hand, ${\operatorname{End}}^D(U)\cong {\operatorname{End}}({{\mathbb W}})$ has one. Since this Hodge structure is invariant under the monodromy, ${\mathscr{H}\! g}({{\mathbb W}})_y$ acts on it through ${\mathscr{H}\! g}({{\mathbb W}})_y/{{\mathscr M}{on}}^\circ({{\mathbb W}})_y$. It is clear that the kernel of this action is a central subgroup of ${\mathscr{H}\! g}({{\mathbb W}})_y/{{\mathscr M}{on}}^\circ({{\mathbb W}})_y$.

We now turn to the second assertion. Since Saito does not express his results in terms of the Hodge group, we must make the translation. We address the nontrivial direction: if ${{\mathbb W}}$ is rigid, then we must show that every ${{\mathbb Q}}$-almost simple factor ${{\mathscr G}}$ of ${\mathscr{H}\! g}({{\mathbb W}})_y$ is contained in ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$. Let ${{\mathscr G}}'$ be a complementary normal ${{\mathbb Q}}$-subgroup of ${\mathscr{H}\! g}({{\mathbb W}})_y$ so that ${{\mathscr G}}$ and ${{\mathscr G}}'$ commute with each other and have finite intersection. The Hodge structure on ${{\mathbb W}}_y$ is given by a morphism ${\operatorname{U}}(1)\to{\mathscr{H}\! g}({{\mathbb W}})_y$. Denote its composite with the projection ${\mathscr{H}\! g}({{\mathbb W}})_y\to {\mathscr{H}\! g}({{\mathbb W}})_y/{{\mathscr G}}'$ by $\rho_y$. This composite is independent of $y$ if and only if ${{\mathscr G}}$ is not an irreducible constituent of ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ (we could paraphrase this by saying that variations of Hodge structures manufactured out of ${{\mathbb W}}$ which do not involve ${{\mathscr G}}'$ will be constant). But as Saito makes clear, this just means that $\rho_y$ describes a (necessarily) nontrivial Hodge structure on ${\operatorname{End}}({{\mathbb W}})$. As this precludes rigidity, this shows that ${{\mathscr G}}\subseteq {{\mathscr M}{on}}^\circ({{\mathbb W}})_y$.

His argument also shows that when every factor of ${{\mathscr G}}({{\mathbb R}})$ is noncompact, we have in fact equality: ${{\mathscr G}}={\mathscr{H}\! g}({{\mathbb W}})_y$. (Apply Cor. 6.27 to the family of abelian varieties which deforms the Hodge structures in the ${{\mathscr G}}$-direction only: it implies that the group denoted there by $G'_{{\mathbb R}}$—in our situation $G'$ is essentially the centralizer of ${{\mathscr G}}$ in the the symplectic group of ${{\mathbb W}}_y$—has to be compact, so that the defining homomorphism ${\operatorname{U}}(1)\to  {\operatorname{Sp}}({{\mathbb W}}_y)$ actually lands in ${{\mathscr G}}({{\mathbb R}})$.) If ${{\mathscr M}{on}}^\circ({{\mathbb W}})_y$ has positive ${{\mathbb Q}}$-rank, then it has a ${{\mathbb Q}}$-simple factor ${{\mathscr G}}$ of positive ${{\mathbb Q}}$-rank. Since every factor of such a ${{\mathscr G}}({{\mathbb R}})$ is noncompact, (iii) also follows.

Application to $G$-curves {#application-to-g-curves .unnumbered}
-------------------------

We are going to apply the preceding to $H^1(S; {{\mathbb Q}})$, where $S$ is as before: a closed connected oriented surface with a faithful $G$-action. So then for every $\chi\in X({{\mathbb Q}}G)$, we have a skew-hermitian $D_\chi$-module $H^1(S; {{\mathbb Q}})[\chi]$.

We equip $V_\chi$ with the trivial Hodge structure of bidegree $(0,0)$; it is then polarized by $s_\chi$ (but this can be one of many polarizations). When we endow $S$ with a $G$-invariant complex structure, resulting in a $G$-curve $C$, then $H^1(C; {{\mathbb Q}})[\chi]$ comes with a polarizable $D_\chi$-invariant Hodge structure with Hodge numbers $(1,0)$ and $(0,1)$ and $D_\chi$ acts by endomorphisms of type $(0,0)$. If the natural algebra homomorphism ${{\mathbb Q}}G\to {\operatorname{End}}_{{\mathbb Q}}(J(C))\cong{\operatorname{End}}_{HS} (H^1(C; {{\mathbb Q}}))$ is onto, then the $G$-isotypical decomposition cannot be refined any further in a $G$-equivariant manner such that it also respects the Hodge structure: otherwise we would get a projector in ${\operatorname{End}}_{{\mathbb Q}}(J(C))$ that is not in the image of ${{\mathbb Q}}G$. In other words, the endomorphism algebra of the Hodge structure on $H^1(C; {{\mathbb Q}})[\chi]$ is then $D_\chi$.

This becomes more interesting in a relative setting. We aim to formulate the consequences in topological language.

Let $(S, G)$ be a closed connected oriented $G$-surface. Let us denote by ${{\mathscr M}{on}}^\circ(S)^G$ the identity component of the Zariski closure of the image of the action of ${\operatorname{Mod}}(S)^G$ on $H^1(S;{{\mathbb Q}})$. It respects the isotypical decomposition $H^1(S; {{\mathbb Q}})\cong \oplus_\chi V_\chi\otimes_{D_\chi} H^1(S; {{\mathbb Q}})[\chi]$ and ${{\mathscr M}{on}}^\circ(S)^G$ acts on the $\chi$-summand via $H^1(S; {{\mathbb Q}})[\chi]$. We denote the image of this action by ${{\mathscr M}{on}}^\circ(S)[\chi]$. A $G$-invariant conformal structure on $S$ yields a $G$-curve $C$. The Hodge group of $H^1(C; {{\mathbb Q}})$ is a reductive ${{\mathbb Q}}$-subgroup of ${\operatorname{Sp}}(H^1(S; {{\mathbb Q}}))^G$. Since these conformal structures are (up to $G$-equivariant isotopy) parameterized by the (connected) complex manifold ${\operatorname{Teich}}(S)^G={\operatorname{Teich}}(S_G)$, the preceding discussion shows that for a very general choice of a $G$-invariant conformal structure, this Hodge group is in fact independent of that choice. We therefore denote it by ${\mathscr{H}\! g}(G,S)$. This group respects the isotypical decomposition and like ${{\mathscr M}{on}}^\circ(S)^G$, it acts on the $\chi$-summand via $H^1(S; {{\mathbb Q}})[\chi]$. We denote the image of this action by ${\mathscr{H}\! g}(S)[\chi]$.

Our main application is Theorem \[topology1\] below. We have already shown that its hypotheses are fulfilled when $S_G$ has genus at least $3$ or when we are in the situation of Theorem \[theorem:GN\] (where $G$ contains a subgroup $N$ of index $2$ which acts freely on $S$ such that $S_N$ is of genus $\ge 4$ and $G/N$ acts on $S_N$ as a hyperelliptic involution).

\[topology1\] Let $(S, G)$ be a closed connected oriented $G$-surface. Assume that for a very general $G$-invariant conformal structure on $S$, the resulting $G$-curve $C$ has the property that ${{\mathbb Q}}G$ maps onto ${\operatorname{End}}_{HS}(H^1(C; {{\mathbb Q}}))$. Then:

1.  The centralizer of ${\mathscr{H}\! g}(S)[\chi]$ in ${\operatorname{End}}(H^1(S; {{\mathbb Q}})[\chi])$ is $D_\chi$.

2.  ${\operatorname{Mod}}(S)[\chi]$ is a normal subgroup of the derived group ${{\mathscr {DH}}\!g}(S)[\chi]$ of ${\mathscr{H}\! g}(S)[\chi]$ and the quotient ${\mathscr{H}\! g}(S)[\chi]/{{\mathscr M}{on}}^\circ(S)[\chi]$ acts on ${\operatorname{End}}_{{{\mathscr M}{on}}^\circ(S)[\chi]}(H^1(S; {{\mathbb Q}})[\chi])$ with finite central kernel.

3.  The group ${\mathscr{H}\! g}(S)[\chi]$ (resp. ${{\mathscr M}{on}}^\circ(S)[\chi]$) acts ${{\mathbb Q}}$-irreducibly (resp. ${{\mathbb Q}}$-isotypically) on the $D_\chi$-module $H^1(S;{{\mathbb Q}})[\chi]$.

4.  If ${{\mathscr M}{on}}^\circ(S)[\chi]$ has positive ${{\mathbb Q}}$-rank, then ${{\mathscr M}{on}}^\circ(S)[\chi]={{\mathscr {DH}}\!g}(S)[\chi]$ and this group is ${{\mathbb Q}}$-almost simple.

5.  If ${{\mathscr M}{on}}^\circ(S)[\chi]$ is trivial, then ${\operatorname{Mod}}(S)^G$ acts through a finite group on $H^1(S; {{\mathbb Q}})_\chi$ and all $G$-invariant conformal structures on $S$ define the same indecomposable Hodge structure on $H^1(S;{{\mathbb Q}})[\chi]$.

\[rem:\] It is not known whether the case (v) of finite monodromy occurs at all. Putman and Wieland conjecture (1.2 of [@P-W]) that it does not (even when the genus of $S_G$ is $\ge 2$), and prove that this non-occurrence is essentially equivalent to a conjecture of Ivanov, which states that the first Betti number of a finite index subgroup of a mapping class group of genus $\ge 3$ is trivial.

In order to prove the theorem for a cofinite subgroup ${\Gamma}\subset {\operatorname{Mod}}(S)^G$, there is no loss of generality in passing to a smaller one and so we may assume that ${\Gamma}\subset {\operatorname{Mod}}(S)^G$ is a normal subgroup which acts trivially on $H^1(S, {{\mathbb Z}}/3)$. By a classical observation of Serre, ${\Gamma}$ is then torsion free, so if we divide out ${{\mathscr C}}_{{\operatorname{Teich}}^G}/{\operatorname{Teich}}^G$ by the freely acting ${\Gamma}$ we obtain a $G$-curve $f: {{\mathscr C}}\to Y$, where $Y$ has the structure of a smooth quasi-projective variety étale over ${{\mathscr M}}^\phi$. For $y\in Y$, we have an isomorphism $\pi_1(Y,y)\cong {\Gamma}$, unique up to inner automorphism and via such an isomorphism, the $\pi_1(Y,y)\times G$-action on $H^1(C_y)$ is equivalent to the ${\Gamma}\times G$-action on $H^1(S)$. We are now in a situation in which we can apply Lemma \[lemma:rigid\] to ${{\mathbb W}}=R^1f_*{{\mathbb Q}}[\chi]$ and all the assertions then follow.

\[quest:opology1\] We do not know whether in the case of infinite monodromy (under the hypotheses of Theorem \[topology1\]) which of the inclusions $${{\mathscr M}{on}}^\circ(S)[\chi]\subseteq {{\mathscr {DH}}\!g}(C)[\chi]\subseteq {\mathscr {D\!U}}(H^1(S, {{\mathbb Q}})[\chi])$$ can be strict. (According to Theorem \[topology1\]-iv, the first inclusion is an isomorphism when ${{\mathscr M}{on}}^\circ(S)[\chi]$ has positive ${{\mathbb Q}}$-rank, but we do not know whether that is always the case; see also Corollary \[cor:primmon\] below.) Nor do we know whether the image of the ${\operatorname{Mod}}(S)^G$-action on $H^1(S;{{\mathbb Q}})[\chi])$ is an arithmetic subgroup. When $G$ is abelian and acts freely on $S$ and is such that $g(S_G)\ge 2$, then it was verified in [@L] that the answer is yes; the essential case is then $G$ cyclic, which means that $S\to S_G$ is trivial over a connected subsurface of genus $g(S_G)-1$. Grünewald, Larsen, Lubotzky and Malestein [@GLLM] generalized this to the case when $S\to S_G$ is trivial over a connected subsurface of genus one that is the complement of $g(S_G)-1$ pairwise disjoint loops in $S_G$ (with $G$ still acting acts freely and $g(S_G)\ge 2$). This implies that $G$ has $\le g(S_G)-1$ generators, but can be very much non-abelian. Venkataramana [@venky] proved that the answer is also yes when $g(S_G)=0$, $G$ abelian and such that the number of irregular orbits does not exceed half the order of $G$. In none of these cases a nontrivial $H^1(S, {{\mathbb Q}})[\chi]$ occurs with finite monodromy group.

Another question is whether these representations are independent, in the sense that the natural map ${{\mathscr M}{on}}^\circ(S)^G\to  \prod_\chi {{\mathscr M}{on}}^\circ(S)[\chi]$ is an isomorphism (again assuming that $g(S_G)\ge 3$). The results in [@L] imply that this is so when $B=\emptyset$ and $G$ is abelian.

Virtual representations {#virtual-representations .unnumbered}
-----------------------

Let us now explain the title of this section. For this we need to elaborate a bit more on the structure of the D-M stack ${{\mathscr M}}^\phi$. Choose a base point $y\in S_G$ away from the set of inertia points. The data of $S_G$ as the $G$-orbifold quotient of $S$, determines and is determined by an epimorphism $p:\pi_1(S_G, y)\to G$ up to an inner automorphism of $G$. Here $\pi_1(S_G, y)$ is to be understood as the fundamental group of the orbifold $S_G$. The kernel $K$ of $p$ is of course intrinsic to this situation, but it does not quite determine the topological type $\phi$ up to a $G$-isomorphism, because it is also necessary to specify an ${\operatorname{In}}(G)$-orbit of isomorphisms $\pi_1(S_G, y)/K\cong  G$. However, the relevant stabilizer subgroup in ${\operatorname{Mod}}(S_G)$, that is, the group of mapping classes of ${\operatorname{Mod}}(S_G)$ which, when viewed as a group of outer automorphisms of $\pi_1(S_G, y)$, preserve $K$ and change $p$ by an inner automorphism of $G$, only depends on $K$, and can therefore be denoted by ${\operatorname{Mod}}(S_G)[K]$. It consists of the mapping classes which lift to $S$ and commute with the $G$-action. It is of finite index in ${\operatorname{Mod}}(S_G)$. We thus obtain an étale covering ${{\mathscr M}}(S_G)[K]\to {{\mathscr M}}(S_G)$ and a short exact sequence: $$\label{shortexact}
1\to Z(G)\to {\operatorname{Mod}}(S)^G\to {\operatorname{Mod}}(S_G)[K]\to 1.$$ (This exact sequence expresses the fact that the smooth D–M stack ${{\mathscr M}}^\phi$ is a $Z(G)$-gerbe over ${{\mathscr M}}(S)[K]$.) If $\overline g$ is the genus of $S_G$ and $\overline n$ the cardinality of the inertia set, then ${\operatorname{Mod}}(S_G)$ contains ${\operatorname{Mod}}_{\overline g, \overline n}$ as a subgroup of finite index. Since $Z(G)$ is finite and the group ${\operatorname{Mod}}(S_G)[K]$ is of finite index in ${\operatorname{Mod}}(S_G)$, the group ${\operatorname{Mod}}(S)^G$ has a subgroup of finite index which is also of finite index in ${\operatorname{Mod}}_{\overline g, \overline n}$. So in that sense, the monodromy representation of ${\operatorname{Mod}}(S)^G$ appearing in Theorem \[topology1\] can be regarded as a virtual representation of ${\operatorname{Mod}}_{\overline g, \overline n}$.

The Dehn groups {#the-dehn-groups .unnumbered}
---------------

Given an oriented surface $\Sigma$, we shall denote by ${\operatorname{\mathscr{X}}}(\Sigma)$ the vertices of the curve complex of $\Sigma$, that is, the set of isotopy classes of embedded circles in $\Sigma$ which do not bound a disk or a once punctured disk. We recall that every $\alpha\in {\operatorname{\mathscr{X}}}(\Sigma)$ defines a Dehn twist $\tau_\alpha\in {\operatorname{Mod}}(\Sigma)$ and carries, after an orientation, a homology class $[\alpha]\in H_1(\Sigma)$ (which is therefore only given up to sign). The action of $\tau_\alpha$ on $H_1(\Sigma)$ is given by the transvection $T_{[\alpha]}: c\in H_1(\Sigma)\mapsto c+ {\langle}c ,[\alpha]{\rangle}[\alpha]$, where ${\langle}\; , \; {\rangle}$ is the intersection form on $H_1(\Sigma)$. Here the choice of orientation of $\alpha$ clearly doesn’t matter.

Let us write $S_G^\circ$ for $S_G{\smallsetminus}B$, where $B\subset S_G$ denotes the set of inertia points. We observe that every $\bar\alpha\in {\operatorname{\mathscr{X}}}(S_G^\circ)$ determines a $G$-orbit $A=A_{\bar\alpha}\subset {\operatorname{\mathscr{X}}}(S)$, namely the set of connected components of its preimage of a representative of $\bar\alpha$. We denote by ${\operatorname{\mathscr{X}}}(S, G)\subset {\mathscr {X}}(S)$ the union of the $G$-orbits $A$ so obtained. It is clear that a $G$-orbit $A$ as above spans an isotropic subspace for the intersection pairing. The Dehn twists $\{\tau_{\alpha}\}_{\alpha\in A}$ commute pairwise and their product $\tau_{A}$ is an element of ${\operatorname{Mod}}(S)^G$ which canonically lifts $\tau_{\bar \alpha}^{m_{\bar\alpha}}\in {\operatorname{Mod}}(S_G)$, where $m_{\bar\alpha}$ denotes the common degree of the members of $A$ over $\bar\alpha$. Then $T_{A}:=\tau_{A*}\in {\operatorname{Sp}}(H_1(S))^G$ is given by $$\textstyle T_{A}(x)=x+\sum_{\alpha\in A} {\langle}x,[\alpha]{\rangle}[\alpha]$$ and hence generates the one-parameter subgroup $$\textstyle  {\mathscr {T}}_{A}:{\operatorname{\mathbb{G}}}_a\to {\operatorname{{\mathscr Sp}}}(H)^G, \quad {\mathscr {T}}_{A}(t) (x)= x+ \sum_{\alpha\in A} {\langle}x,[\alpha]{\rangle}t[\alpha].$$ It is clearly contained in the connected monodromy group ${{\mathscr M}{on}}^\circ(S)^G$.

\[cor:primmon\] Assume that we are in the situation of Theorem \[topology1\] and that $\chi\in X({{\mathbb Q}}G)$ is such that ${\operatorname{\mathscr{X}}}(S,G)$ has a nonzero image in $H_1(S;{{\mathbb Q}})_\chi$. Then this image spans $H_1(S;{{\mathbb Q}})_\chi$ over ${{\mathbb Q}}$. Moreover, ${\operatorname{Mod}}(S)^G$ acts irreducibly on $H_1(S;{{\mathbb Q}})[\chi]$ and the identity component of the Zariski closure of this representation, ${{\mathscr M}{on}}^\circ(S)[\chi]$, is ${{\mathbb Q}}$-almost simple.

Let $A\subset {\mathscr {X}}(S,G)$ be a $G$-orbit whose image in $H_1(S;{{\mathbb Q}})_\chi$ is nontrivial. Then the one-parameter subgroup ${\mathscr {T}}_{A}$ of ${{\mathscr M}{on}}^\circ(S)^G$ has a nontrivial image in ${{\mathscr M}{on}}^\circ(S)[\chi]$ and hence ${{\mathscr M}{on}}^\circ(S)[\chi]$ has positive ${{\mathbb Q}}$-rank. It then follows from Theorem \[topology1\]-iv that ${{\mathscr M}{on}}^\circ(S)[\chi]$ is ${{\mathbb Q}}$-almost simple and acts irreducibly on $H_1(S;{{\mathbb Q}})[\chi]$. It then also follows that $G\times {\operatorname{Mod}}(S)^G$ acts irreducibly on $H_1(S;{{\mathbb Q}})_\chi$. Since the ${{\mathbb Q}}G$-submodule of $H_1(S;{{\mathbb Q}})_\chi$ spanned by the image of ${\operatorname{\mathscr{X}}}(S,G)$ is $G\times {\operatorname{Mod}}(S)^G$-invariant, it must be all of $H_1(S;{{\mathbb Q}})_\chi$.

\[rem:\] Suppose that $S_G{\smallsetminus}B$ contains an embedded circle $\gamma$ over which the covering is trivial and is such that its preimage $\tilde\gamma$ in $S$ has the property that $S{\smallsetminus}\tilde\gamma$ is still connected. This implies that that ${{\mathbb Z}}G\cong H_1(\tilde\gamma)\to  H_1(S)$ is an injection of ${{\mathbb Z}}G$-modules. Hence, if in addition $g(S_G)\ge 3$, then the hypotheses of Corollary \[cor:primmon\] are satisfied for all $\chi$, so that for *every* $\chi\in X({{\mathbb Q}}G)$, ${\operatorname{Mod}}(S)^G$ acts ${{\mathbb Q}}$-irreducibly on $H_1(S;{{\mathbb Q}})[\chi]$ and ${{\mathscr M}{on}}^\circ(S)[\chi]$ is ${{\mathbb Q}}$-almost simple.

We close the paper with asking a question and proposing a conjecture.

\[conj:liegeneration\] In the situation of Corollary \[cor:primmon\], the group generated isotropic transvections $\{T_{A}[\chi]\}_{A\in G{\backslash}{\operatorname{\mathscr{X}}}(S,G)}$ maps onto an arithmetic subgroup of ${{\mathscr M}{on}}^\circ (S)[\chi]$.

\[quest:arithmetic2\] Malestein and Putman [@mp] have recently shown that it can happen that ${\mathscr {X}}(S, G)$ does not span $H_1(S;{{\mathbb Q}})$. It then follows from Corollary \[cor:primmon\] that for some $\chi\in X({{\mathbb Q}}G)$, the image of ${\mathscr {X}}(S, G)$ in $H_1(S;{{\mathbb Q}})[\chi]$ is trivial. So then any Dehn twist in $S_G{\smallsetminus}B$ admits a power which lifts to a multi-Dehn twist in ${\operatorname{Mod}}(S)^G$ that acts trivially on $H_1(S;{{\mathbb Q}})_\chi$. Since a Dehn twist in $S_G{\smallsetminus}B$ represents a loop in the moduli space of curves of orbifold type $S_G$, there exists then an étale cover $Y\to {{\mathscr M}}^\phi$ of our moduli space ${{\mathscr M}}^\phi$ with the property that for the resulting pull-back $f:{{\mathscr C}}_Y\to Y$ (a family of $G$-curves), the local variation of Hodge structure $R^1f_*{{\mathbb Q}}[\chi]$ extends across the Deligne-Mumford compactification of $Y$. Since the kind of representations of ${\operatorname{Mod}}(S_G)$ which come from the theory of conformal blocks (the so-called quantum representations) also have this property, one of us [@Looij:OW] has raised the question whether in such a case the (virtual) representation of ${\operatorname{Mod}}(S_G)$ on $H_1(S; {{\mathbb Q}})[\chi]$ is of this type. (See also [@ks].) It is likely that the Zariski closure of such a representation is anisotropic (i.e., has ${{\mathbb Q}}$-rank zero).

[PP]{} Y. André: *Mumford-Tate groups of mixed Hodge structures and the theorem of the fixed part*. Compositio Mathematica [**82**]{} (1992), 1–24.

M. Boggi, E. Looijenga: *Deforming a canonical curve inside a quadric*. Int. Math. Res. Not. IMRN,\
<https://doi.org/10.1093/imrn/rny027> (2018).

C. Chevalley, A. Weil, E.  Hecke: *Über das Verhalten der Integrale 1. Gattung bei Automorphismen des Funktionenkörpers.* Abh. Math. Sem. Univ. Hamburg [**10**]{} (1934), 358–361.

C. Ciliberto: *Endomorfismi di Jacobiane*. Rend. Sem. Mat. Fis. Milano [**59**]{} (1989), 213–242.

C. Ciliberto, G. van der Geer, M. Teixidor i Bigas: *On the number of parameters of curves whose Jacobians possess nontrivial endomorphisms*. J. Alg. Geom. [**1**]{} (1992), 215–229.

B. Collas, S. Maugeais: *Composantes irréductibles de lieux spéciaux d’espaces de modules de courbes, action galoisienne en genre quelconque.* Ann. Inst. Fourier (Grenoble) **65** (2015), 245–276, <https://arxiv.org/pdf/1403.2517.pdf>

B. Collas, S. Maugeais: *On Galois action on stack inertia of moduli space of curves.*\
<https://arxiv.org/pdf/1412.4644v1.pdf> (2014).

P. Deligne: *Th[é]{}orie de Hodge, II*. Publ. Math. IHÉS [**40**]{} (1971), 5–57.

W. Fulton, J. Harris. *Representation Theory: A First Course*. Graduate Texts in Mathematics [**129**]{}, Springer-Verlag (1991).

F. Grünewald, M. Larsen, A. Lubotzky, J. Malestein: *Arithmetic quotients of the mapping class group.* Geom. Funct.  Anal. **25** (2015) 1493–1542, <https://arxiv.org/pdf/1307.2593v2>

I.M. Isaacs: *Character theory of finite groups.* Academic Press (1976).

T. Koberda, R. Santharoubane: *Quotients of surface groups and homology of finite covers via quantum representations.* Invent. Math. **206** (2016), 269–292, <https://arxiv.org/pdf/1510.00677>

S. Lefschetz: *A Theorem on Correspondences on Algebraic Curves.* Am. J. of Math., [**50**]{} (1928), 159–166.

E. Looijenga: *Prym Representations of Mapping Class Groups*. Geometriae Dedicata [**64**]{} (1997), 69–83.

E. Looijenga: *Some algebraic geometry related to the mapping class group,* abstract of a talk at the Oberwolfach workshop *New perspectives on the the Interplay between Discrete Groups in Low-Dimensional Topology and Arithmetic Lattices* (June 2015), available at <http://www.staff.science.uu.nl/~looij101/eduardlooijengaOWjuni2015.pdf>

A. Putman, J. Malestein: *Simple closed curves, finite covers of surfaces, and power subgroups of ${\operatorname{Out}}(F_n)$,* <https://arxiv.org/pdf/1708.06486v3.pdf>

B.J.J. Moonen, Yu.G. Zarhin: *Hodge classes on abelian varieties of low dimension*. Math. Annalen [**315**]{} (1999), 711–733.

A. Putman, B. Wieland: *Abelian quotients of the mapping class group and higher Prym representations*, J. Lond. Math. Soc.  (2) [**88**]{} (2013), 79–96, <https://arxiv.org/pdf/1106.2747.pdf>

Masa-Hiko Saito: *Classification of nonrigid families of abelian varieties,* Tohoku Math. J.  [**45**]{} (1993), 159–189.

J-P. Serre: *Linear Representations of Finite Groups*. Graduate Texts in Mathematics [**42**]{}, Springer-Verlag (1977).

T.A. Springer: *Linear algebraic groups,* 2nd ed., Progress in Mathematics, **9** Birkhäuser Boston, Inc., Boston, MA (1998).

M. Talpo, A. Vistoli: *Deformation theory from the point of view of fibered categories.* The Handbook of Moduli (G. Farkas, I. Morrison, editors), Vol. III, pp. 281–397, Adv. Lect. in Math. [**25**]{}, Int. Press (2013), <https://arxiv.org/pdf/1006.0497.pdf>

T. N. Venkataramana: *Monodromy of cyclic coverings of the projective line.* Invent. Math. 197 (2014), 1–45. <https://arxiv.org/abs/1204.4778>

Y. Zarhin: *Hodge classes on certain hyperelliptic prymians,* in *Arithmetic, geometry, cryptography and coding theory.* 171–183, Contemp. Math. [**574**]{}, Amer. Math. Soc. Providence, RI (2012), <https://arxiv.org/pdf/1012.3731.pdf>

Y. Zarhin: *The endomorphism rings of Jacobians of cyclic covers of the projective line.* Math. Proc. Cambridge Philos. Soc.  [**136**]{} (2004), 257–267, <https://arxiv.org/pdf/math/0103203.pdf>

[^1]: E.L. was supported by the Chinese National Science Foundation.

[^2]: When a notion only depends on the image of $\phi$ in ${\operatorname{Mod}}(S)$, we usually suppress $\phi$ in the notation and write $G$ instead.
