Rows vs Columns for Linear Systems of Equations - Randomized Kaczmarz or Coordinate Descent?

Abstract: This paper is about randomized iterative algorithms for solving a linear system of equations $X \beta = y$ in different settings. Recent interest in the topic was reignited when Strohmer and Vershynin (2009) proved the linear convergence rate of a Randomized Kaczmarz (RK) algorithm that works on the rows of $X$ (data points). Following that, Leventhal and Lewis (2010) proved the linear convergence of a Randomized Coordinate Descent (RCD) algorithm that works on the columns of $X$ (features). The aim of this paper is to simplify our understanding of these two algorithms, establish the direct relationships between them (though RK is often compared to Stochastic Gradient Descent), and examine the algorithmic commonalities or tradeoffs involved with working on rows or columns. We also discuss Kernel Ridge Regression and present a Kaczmarz-style algorithm that works on data points and having the advantage of solving the problem without ever storing or forming the Gram matrix, one of the recognized problems encountered when scaling kernelized methods.

\section{Introduction}
Solving linear systems of equations is a classical topic and our interest in this problem is fairly limited. While we do compare two algorithms - Randomized Kaczmarz (RK) (see \citet{StrVer09}) and Randomized Coordinate Descent (RCD) (see \citet{LevLew10}) - to each other, we will not be presently concerned with comparing them to the host of other classical algorithms in the literature, like Conjugate Gradient and  Gradient Descent methods.
Our primary aim will be to understand the algorithmic similarities and differences involved in working with rows and columns (RK and RCD) of a given input.

Assume we have a known $n \times p$ matrix $X$ (representing $n$ data points with $p$ features each), an unknown $p$-dimensional vector $\beta$ (regression coefficients), and a known $n$-dimensional vector $y$ (observations). The system of equations that one wants to solve is
\begin{equation}\label{eq:syseq}
X \beta = y
\end{equation}

When there exists at least one solution to the above system, we say that it is consistent.
When there is a unique consistent solution, solving \eqref{eq:syseq} is a special case of the more general problem of minimizing the residual norm (makes sense when there are no consistent solutions)
\begin{equation}\label{eq:LS}
\min_{\beta \in \mathbf{R}^p} \tfrac1{2} \|y-X\beta\|^2 =: L(\beta)
\end{equation}

When there is a unique consistent solution, it is also a special case of the more general problem of finding the minimum norm consistent solution (makes sense when there are infinite solutions)
\begin{equation}\label{eq:minnorm}
\min_{\beta \in \mathbf{R}^p} \|\beta\|^2 \mbox{ \ s.t. \ } y = X\beta
\end{equation}

Sparse versions of the above problems, while interesting, and not in the scope of this work. We will later consider the Ridge Regression extension to Eq.\eqref{eq:LS}.

We represent the $i$-th row ($i=1,...,n$) of $X$ by $X^i$, and the $j$-th column ($j=1,...,p$) by $X_j$. Similarly, the $i$-th observation is $y^i$, and the $j$-th regression coefficient is $\beta_j$. The reason for the linear regression setup using statistical notation of $n,p$ and $X,\beta,y$, is simply author comfort, and the literature sometimes uses $Ax=b$ instead, with an $m \times n$ matrix A.

For the rest of this paper, we refer to a method as being Kaczmarz-like when its updates depend on rows (data points) of $X$, like
$$
\beta_{t+1} := \beta_t + \delta_i X^i
$$

where stepsize $\delta_i$ could also depend on $X^i$.
and we refer to a method as being Coordinate Descent style when its updates are on coordinates of $\beta$ and depend on columns (features) of $X$, like
$$
\beta_{t+1} := \beta_t + \delta_j e_j
$$

where stepsize $\delta_j$ could depend on $X_j$.

Randomized Kaczmarz (and its variations) have been likened to Stochastic Gradient Descent (see \citet{sgdkacz}). Indeed, even before having mentioned the form of $\delta_i$, the update already looks a lot like that of the Perceptron algorithm by \citet{perceptron}, a well known stochastic gradient descent algorithm. However, even though we will later describe some differences from RCD, we argue that RK-style methods are still much more like randomized coordinate descent than stochastic gradient descent algorithms - this is useful not only for interpretation but also for new derivations. We will bring out the intuitive similarity between RK and RCD by establishing striking parallels in proofs of convergence (these proofs are traditionally presented in a different manner), and exploit this in designing a simple RK algorithm for Kernel Ridge Regression.

There has been a lot of interest and extensions on both these interesting algorithms, and perhaps connections between the two have been floating around in a subtle manner. One of our aims will be to make these connections explicit, intuitively clear, and enable the reader to build an understanding of ideas and proof techniques by the end.  We first need to introduce the aforementioned algorithms before we summarize our contributions. In doing so, we will only refer to papers that are directly relevant to our work.

\subsection{Randomized Kaczmarz (RK)}
Taking  $X, y$ as input and starting from an arbitrary  $\beta_0$, it repeats the following in each iteration. First, pick a random row $r \in \{1...n\}$ with probability proportional to its  Euclidean norm, i.e.
$$
\mathbb{P}(r = i) = \frac{\|X^i\|^2}{\|X\|_F^2}
$$
Then, project the current iterate onto that row, i.e.
\begin{equation}
\beta_{t+1} := \beta_t + \frac{(y^r - X^{rT}\beta_t)}{\|X^r\|^2} X^r
\end{equation}

Intuitively, this update can be seen as greedily satisfying the $r$th equation in the linear system, because it is easy to see that after the update,
\begin{equation}
X^{rT}\beta_{t+1} = y^r
\end{equation}

Alternatively, referring to Eq.\eqref{eq:LS}, since
$
L(\beta) = \tfrac1{2} \|y-X\beta\|^2 = \tfrac1{2} \sum_{i=1}^n (y^i - X^{iT} \beta)^2,
$ we can interpret this update as stochastic gradient descent (we pick a random data-point on which to update), where the stepsize is the inverse Lipschitz constant of the stochastic gradient
$$
\nabla^2 \tfrac1{2} (y^i - X^{iT} \beta)^2 = \|X^i\|^2.
$$

\citet{StrVer09} showed that the above algorithm
has an expected linear convergence. We will formally discuss the convergence properties of this algorithm in future sections.

\subsection{Randomized Coordinate Descent (RCD)}
Takeing $X,y$ as input, starting from an arbitrary $\beta_0$, it repeats the following in each iteration. First, pick a random column $c \in \{1...p\}$ with probability proportional to its  Euclidean norm, i.e.
$$
\mathbb{P}(c = j) = \frac{\|X_j\|^2}{\|X\|_F^2}
$$

We then minimize the objective $L(\beta) = \tfrac1{2} \|y-X\beta\|^2$ with respect to this coordinate to get
\begin{equation}
\beta_{t+1} := \beta_t + \frac{X_c^T(y-X\beta_t)}{\|X_c\|^2} e_c
\end{equation}

where $e_{c}$ is the $c$th coordinate axis. It can be seen as greedily minimizing the objective with respect to the $c$-th coordinate. Indeed, letting $X_{-c},\beta_{-c}$ represent $X$ without its $c$-th column and $\beta$ without its $c$-th coordinate,
\begin{equation}
\frac{\partial L}{\partial \beta_c} = -X_c^T(y-X\beta) = -X_c^T(y-X_{-c}\beta_{-c} - X_c\beta_c)
\end{equation}

Setting this equal to zero for the coordinatewise minimization, we get the aforementioned update for $\beta_c$. Alternately, since $[\nabla L(\beta)]_c = -X_c^T(y-X\beta)$, the above update can intuitively be seen as a univariate descent step where the stepsize is the inverse Lipschitz constant of the gradient along the $c$-th coordinate, since $$[\nabla^2 L(\beta)]_{c,c} = (X^TX)_{c,c} = \|X_c\|^2.$$

\citet{LevLew10} showed that this algorithm has an expected linear convergence. We will discuss the convergence properties of this algorithm in detail in future sections.

\section{Main Results}

We first examine the differences in behavior of the two algorithms in three distinct but related settings. This will bring out the opposite behaviors of the two similar algorithms.

When the system of equations \eqref{eq:syseq} has a unique solution, we represent this by $\beta^*$. This happens when $n \geq p$, and the system is (luckily) consistent. Assuming  that $X$ has full column rank,
\begin{equation}
\beta^* = (X^TX)^{-1}X^T y
\end{equation}

When \eqref{eq:syseq} does not have any consistent solution, we refer to the least-squares solution of Eq. \eqref{eq:LS} as $\beta_{LS}$. This could happen in the overconstrained case, when $n > p$. Again, assuming  that $X$ has full column rank, we have
\begin{equation}
\beta_{LS} = (X^TX)^{-1}X^T y
\end{equation}

When \eqref{eq:syseq} has infinite solutions, we call the minimum norm solution to \eqref{eq:minnorm} as $\beta^*_{MN}$. This could happen in the underconstrained case, when $n < p$. Assuming  that $X$ has full row rank, we have
\begin{equation}
\beta_{MN}^* = X^T (XX^T)^{-1} y
\end{equation}

In the above notation, the $*$ is used to denote the fact that it is consistent, i.e. it solves the system of equations, the $LS$ stands for Least Squares and $MN$ for minimum norm. We shall return to each of these three situations in that order in future sections of this paper.

One of our main contributions is to achieve a unified understanding of the behaviour of RK and RCD in  these different situations.
The literature for RK deals with only the first two settings (see  \citet{StrVer09}, \citet{Nee10}, \citet{ZouFre13}), but avoids the third.
The literature for RCD typically focuses on more general setups than our specific quadratic least squares loss function $L(\beta)$ (see \citet{Nesterov12} or \citet{RicTak12}). However, for both the purposes of completeness, and for a more thorough understanding the relationship between RK and RCD, it turns out to be crucial to analyse all three settings (for equations \eqref{eq:syseq}-\eqref{eq:minnorm}).

\begin{enumerate}
\item When $\beta^*$ is a unique consistent solution, we present proofs of the linear convergence of both algorithms - the results are known from papers by \citet{StrVer09} and \citet{LevLew10} but are presented in  a novel manner so that their relationship becomes clearer and direct comparison is easily possible.

\item When $\beta_{LS}$ is the (inconsistent) least squares solution, we show why RCD iterates converge linearly to $\beta_{LS}$, but RK iterates do not - making RCD preferable. These facts are not hard to see, but we make it more intuitively and mathematically clear why this should be the case.
\item When $\beta^*_{MN}$ is the minimum norm consistent solution, we explain why RK converges linearly to it, but RCD iterates do not (both so far undocumented observations) - making RK preferable.
\end{enumerate}

Together, the above three points complete the picture (with solid accompanying intuition) of the opposing behavior of RK and RCD. We then use the insights thus gained to develop a Kaczmarz style algorithm for Ridge Regression and its kernelized version. It is well known that the solution to
\begin{equation}
\min_{\beta \in \mathbb{R}^p} \|y-X\beta\|^2 + \lambda \|\beta\|^2
\end{equation}

can be given in two equivalent forms (using the covariance and gram matrices) as
\begin{equation}
\beta_{RR} ~=~ (X^T X + \lambda I)^{-1}X^T y ~=~ X^T(XX^T + \lambda I)^{-1}y
\end{equation}

The presented algorithms completely avoid inverting, storing or even forming  $XX^T$ and $X^TX$.
Later, we will show that the following updates take only $O(p)$ computation per iteration like RK (starting with $\delta=0, \alpha=\mathbf{0}_n, \beta = \mathbf{0}_p, r = y$) and have expected linear convergence
\begin{eqnarray}
\delta_t &=& \frac{y^i - \beta_t^TX^i - \lambda \alpha^i_t}{\|X^i\|^2 + \lambda}\label{eq:RR1}\\
\alpha^i_{t+1} &=& \alpha^i_t + \delta_t\label{eq:RR2} \\
\beta_{t+1} &=& \beta_t + \delta_t X^i\label{eq:RR3}
\end{eqnarray}

where the $i$-th row is picked with probability proportional to $\|X^i\|^2 + \lambda$.
If $\mathcal{H}_k$ is a Reproducing Kernel Hilbert Space (RKHS, see \citet{learningkernels} for an introduction) associated to positive definite  kernel $k$ and feature map $\phi_x$, it is well known that the solution to the corresponding Kernel Ridge Regression (see \citet{krr}) problem  is
\begin{eqnarray}
\label{eq:KRRf}
f_{KRR} &=& \arg\min_{f \in \mathcal{H}_k} \sum\limits_{i=1}^n (y_i - f(x_i))^2 + \lambda \|f\|^2_{\mathcal{H}_k} \\
&=& \Phi^T (K+\lambda I)^{-1}y
\end{eqnarray}

where $\Phi = (\phi_{x_1},...,\phi_{x_n})^T$  and $K$ is the gram matrix with $K_{ij}=k(x_i,x_j)$.

One of the main problems with kernel methods is as data size grows, the gram matrix becomes too large to store. This has motivated the study of approximation techniques for such kernel matrices, but we have an alternate suggestion. The aim of a Kaczmarz style algorithm would be to solve the problem by never forming $K$ as exmplified in updates for $\beta_{RR}$.
For KRR, the update is
\begin{eqnarray}
\alpha^i_{t+1} &=&  \frac{y - \sum_{j\neq i} K(x_i,x_j) \alpha_t^j}{K(x_i,x_i) + \lambda}
\end{eqnarray}

and costs $O(n)$ per iteration, and results in linear convergence as described later. Note that here RK for Kernel Ridge Regression costs $O(n)$ per iteration and RK for Ridge Regression cost $O(p)$ per iteration due to different parameterization. In the latter, we can keep track of $\beta_t$ as well as $\alpha_t$ easily, see Eq.\eqref{eq:RR2},\eqref{eq:RR3}, but for KRR, calculations can only be performed via evaluations of the kernel only ($\beta_t$ corresponds to a function and cannot be stored), and hence have a different cost.

The aforementioned updates and their convergence can  be easily derived after we develop a clear understanding of how RK and RCD methods relate to each other and jointly to positive semi-definite systems of equations. We shall see more of this in Sec.\ref{sec:RR}.

\newpage

\section{Overconstrained System, Consistent}

To be clear, here we will assume that $n > p$, $X$ has full column rank, and that the system is consistent, so
 $y = X\beta^*$.
First, let us write the updates used by both algorithms in a revealing fashion. If RK and RCD picked row $i$ and column $j$ at step $t+1$, and $e^i$ is $1$ in the $i$-th position and $0$ elsewhere, then the updates can be rewritten as below:
\begin{align}\label{eq:RKupdate}
&\mbox{(RK)}& \beta_{t+1} &:= \beta_t + \frac{e^{iT} r_t}{\|X^i\|^2} X^i&\\
&\mbox{(RCD)}& \beta_{t+1} &:= \beta_t + \frac{X_j^T r_t}{\|X_j\|^2}e_j &
\end{align}
where $r_t = y - X\beta_t = X\beta^*-X\beta_t$ is the residual vector. Then multiplying both equations by $X$ gives
\begin{align}
&\mbox{(RK)}&X\beta_{t+1} &:= X\beta_t + \frac{X^{iT}(\beta^*-\beta_t)}{\|X^i\|^2} X X^i&\\
&\mbox{(RCD)}& X\beta_{t+1} &:= X\beta_t + \frac{X_j^T X(\beta^*-\beta_t)}{\|X_j\|^2}X_j& \label{eq:RCDupdate}
\end{align}
We now come to an important difference, which is the key update equation for RK and RCD.

Firstly, from the update Eq.(\ref{eq:RKupdate}) for RK, we have $\beta_{t+1}-\beta_t$ is parallel to $X^i$. Also, $\beta_{t+1}-\beta^*$ is orthogonal to $X^i$ (why? because $X^{iT}(\beta_{t+1}-\beta^*)=y^i - y^i = 0$). By Pythagoras,
\begin{equation}\label{eq:RKrecursion}
\|\beta_{t+1} - \beta^*\|^2 = \|\beta_t - \beta^*\|^2 - \|\beta_{t+1} - \beta_t\|^2
\end{equation}
Note that from the update Eq.(\ref{eq:RCDupdate}), we have $X\beta_{t+1} - X\beta_t$ is parallel to $X_j$. Also, $X\beta_{t+1} - X\beta^*$ is orthogonal to $X_j$ (why? because $X_j^T(X\beta_{t+1} - X\beta^*) = X_j^T(X\beta_{t+1} - y) = 0$ by the optimality condition $\partial L/\partial \beta_{j} = 0$). By Pythagoras,
\begin{equation}\label{eq:RCDrecursion}
\|X\beta_{t+1} - X\beta^*\|^2 = \|X\beta_t - X\beta^*\|^2 - \|X\beta_{t+1} - X\beta_t\|^2
\end{equation}
The rest of the proof follows by simply substituting for the last term in the above two equations, and is presented in the following table for easy comparison. Note $\Sigma=X^TX$ is the full-rank covariance matrix and we first
 take expectations with respect to the randomness at the $t+1$-st step, conditioning on all randomness up to the $t$-th step. We later iterate this expectation.
\begin{table}[h]
\begin{tabular}{|l|l|}
\hline
Randomized Kaczmarz $\mathbb{E}\|\beta_{t+1} - \beta^*\|^2$ & Randomized Coordinate Descent $\mathbb{E}\|X\beta_{t+1} - X\beta^*\|^2$ \\
\hline
$= \|\beta_t - \beta^*\|^2 - \mathbb{E}\|\beta_{t+1} - \beta_t\|^2$ & $= \|X\beta_t - X\beta^*\|^2 - \mathbb{E}\|X\beta_{t+1} - X\beta_t\|^2$\\
$= \| \beta_{t} - \beta^*\|^2 -  \sum_i \frac{\|X^i\|^2}{\|X\|_F^2} \frac{(X^i(\beta_t - \beta^*))^2}{(\|X^i\|^2)^2} \|X^i\|^2$ & $= \|X \beta_{t} - X\beta^*\|^2  - \sum_j \frac{\|X_j\|^2}{\|X\|_F^2} \frac{(X_j^T X(\beta_t - \beta^*))^2}{(\|X_j\|^2)^2}\|X_j\|^2$ \\
$= \|\beta_{t} - \beta^*\|^2 \left(1 - \frac1{\|X\|_F^2} \frac{\| X(\beta_t - \beta^*)\|^2}{\| \beta_{t} - \beta^*\|^2} \right)$ & $= \|X \beta_{t} - X\beta^*\|^2 \left(1 - \frac1{\|X\|_F^2}  \frac{\|X^T X(\beta_t - \beta^*)\|^2}{\|X \beta_{t} - X\beta^*\|^2} \right)$ \\
$\leq \|\beta_t-\beta^*\|^2 (1 - \frac{\sigma_{\min}(\Sigma)}{Tr(\Sigma)})$ & $\leq \|X\beta_t-X\beta^*\|^2 (1 - \frac{\sigma_{\min}(\Sigma)}{Tr(\Sigma)})$ \\
\hline
\end{tabular}
\end{table}

Here, $\sigma_{\min}(\Sigma)\| \beta_t - \beta^*\|^2 \leq \|X(\beta_t - \beta^*)\|^2 $  i.e. $\sigma_{\min}(\Sigma)$ is the smallest eigenvalue.
It follows tha
\begin{align}
&\mbox{(RK)}& \mathbb{E}\|\beta_t - \beta^*\|^2 &\leq \left( 1 - \frac{\sigma_{\min}(\Sigma)}{Tr(\Sigma)} \right)^{t}\|\beta_0-\beta^*\|^2&\\ \label{eq:RCDlin}
&\mbox{(RCD)}& \mathbb{E}\|\beta_t - \beta^*\|_\Sigma^2 &\leq \left( 1 - \frac{\sigma_{\min}(\Sigma)}{Tr(\Sigma)} \right)^{t}\|\beta_0-\beta^*\|_\Sigma^2
\end{align}

Since $\Sigma$ is invertible when $n>p$ and $X$ has full column rank, the last equation also implies linear convergence of $\mathbb{E}\|\beta_t-\beta^*\|^2$.
The final results do exist in \citet{StrVer09,LevLew10} but there is utility in seeing the two proofs in a form that differs from their original presentation, side by side. In this setting, both RK and RCD are essentially equivalent (without computational considerations).

\newpage

\section{Overconstrained System, Inconsistent}

Here, we will assume that $n > p$, $X$ is full column rank, and the system is (expectedly) inconsistent, so $y = X\beta_{LS} + z$, where $z$ is such that $X^T z = 0$. It is easy to see this condition, because as mentioned earlier,
\[
\beta_{LS} = (X^T X)^{-1}X^T y
\]
implying that $X^T X \beta_{LS} = X^T y$. Substituting $y=X\beta_{LS}+z$ gives that $X^Tz=0$.

In this setting, RK is known to not converge to the least squares solution, as is easily verified experimentally. The tightest convergence upper bounds known are by \citet{Nee10} and \citet{ZouFre13} who show that
\[
\mathbb{E}\|\beta_t - \beta_{LS}\|^2 \leq \left( 1 - \frac{\sigma_{\min}^+(\Sigma)}{Tr(\Sigma)} \right)^{t}\|\beta_0-\beta_{LS}\|^2 + \frac{\|w\|^2}{\sigma_{\min}^+(\Sigma)^2}
\]
If you tried to follow the previous proof, Eq.\eqref{eq:RKrecursion} does not go through - Pythagoras fails because $\beta_{t+1}-\beta_{LS}\not\perp X^i$, since $X^{iT}(\beta_{t+1}-\beta_{LS}) = y^i-X^{iT}\beta_{LS} \neq 0 $.
Intuitively, the reason RK does not converge  is that every update of RK (say of row $i$) is a projection onto the ``wrong'' hyperplane that has constant $y^i$ (where the ``right'' hyperplane would involve projecting onto a parallel hyperplane with constant $y^i-z^i$ where $z$ was defined above). An alternate intuition is that all RK updates are in the span of the rows, but $\beta_{LS}$ is not in the row span. These intuitive explanations are easily confirmed by experiments seen in \citet{Nee10,ZouFre13}.

The same doesn't hold for RCD. Almost magically, in the previous proof, Pythagoras works because
$$X_j^T(X\beta_{t+1} - X\beta_{LS}) = X_j^T(X\beta_{t+1} - y) + X_j^T(y-X\beta_{LS}) = 0$$
The first term is 0 by the  optimality condition for $\beta_{t+1}$ i.e. $X_j^T(X\beta_{t+1} - y) = \partial L/\partial \beta_j = 0$. The second term is zero by the global optimality of $\beta_{LS}$ i.e. $X^T(y - X\beta_{LS}) = \nabla L = 0$. Also, $\Sigma$ is full rank as before.
Hence, since RCD works in the space of fitted values $X\beta$ and not the iterates $\beta$.

In summary, RK does not converge to the LS solution, but RCD does at the same linear rate.
The Randomized Extended Kaczmarz by \citet{ZouFre13} is a modification of RK designed to converge by randomly projecting out $z$, but its discussion is beyond our  scope. We were also alerted to a recent, independent Arxiv paper by \citet{frek}.

\section{Underconstrained System, Infinite Solutions}

Here, we will assume that $p > n$, $X$ is full row rank and the system is (expectedly) consistent with infinite solutions. As mentioned earlier, it is easy to show that
\[
\beta^*_{MN} = X^T(XX^T)^{-1}y
\]
(which clearly satisfies $X\beta^*_{MN} = y$). Every other consistent solution can be expressed as
$$
\beta^* = \beta^*_{MN} + z ~\mbox{~ where ~}~ Xz=0
$$
Clearly any such $\beta^*$ would also satisfy $X\beta^* = X\beta^*_{MN}=0$. Since $Xz=0$, $z \perp \beta^*_{MN}$ implying $\|\beta^*\|^2 = \|\beta^*_{MN}\|^2 + \|z\|^2$, showing that $\beta^*_{MN}$ is indeed the minimum norm solution as claimed.

In this case, RK has good behaviour, and starting from $\beta_0=0$, it does converge linearly to $\beta^*_{MN}$. Intuitvely, $\beta^*_{MN} = X^T \alpha$ (for $\alpha = (XX^T)^{-1}y$) and hence is in the row span of $X$. Starting from $\beta_0=0$, RK only adds multiples of rows to its iterates, and hence will never have any component orthogonal to the row span of $X$ (i.e. will never add any $z$ such that $Xz=0$). There is exactly one solution with no component orthogonal to the row span of $X$, and that is $\beta^*_{MN}$, and hence RK converges linearly to the required point. It is important not to start from an arbitrary $\beta_0$ since the RK updates can never wipe out any component of $\beta_0$ that is perpendicular to the row span of $X$.

Mathematically, the previous earlier proof works because Pythagoras goes through since it is a consistent system.  However, $\Sigma$ is not full rank but note that since both $\beta^*_{MN}$ and $\beta_t$ are in the row span, $\beta_t - \beta^*_{MN}$ has no component orthogonal to $X$ (unless it equals zero and we're done). Hence $\sigma_{\min}^+(\Sigma)\| \beta_t - \beta^*\|^2 \leq \|X(\beta_t - \beta^*)\|^2$ does hold $\sigma_{\min}^+$ being the smallest positive eigenvalue of $\Sigma$.

RCD unfortunately suffers the opposite fate. The iterates do not converge to $\beta_{LS}^*$, even though $X\beta_t$ does converge to $X\beta^*$. Mathematically, the convergence proof still carries forward as before till Eq.\eqref{eq:RCDlin}, but in the last step where $X^T X$ cannot be inverted because it is not full rank. Hence we get convergence of the residual to zero, without getting convergence of the iterates to the least squares solution.

Unfortunately, when each update is cheaper for RK than RCD (due to matrix size), RCD is preferred for  reasons of convergence and when it is cheaper for RCD than RK, RK is preferred.

\section{Randomized Kaczmarz for Ridge Regression}\label{sec:RR}

Both RK and RCD can be viewed in the following fashion. Suppose we have a positive definite matrix $A$, and we want to solve $Ax=b$. Instead of casting it as $\min_x \|Ax-b\|^2$, we can alternatively pose the different problem $\min_x \tfrac1{2} x^TAx - b^Tx$. Then one could use the update
\[
x_{t+1} = x_t + \frac{b_i-A_i^Tx_t}{A_{ii}} e_i
\]
where the $b_i-A_i^T x_t$ is basically the $i$-th coordinate of the gradient, and $A_{ii}$ is the Lipschitz constant of the $i$-th coordinate of the gradient (related works include \citet{LevLew10}, \citet{Nesterov12},\citet{RicTak12},\citet{SidLee13}).

In this light, the RK update can be seen as the randomized coordinate descent rule for the psd system $XX^T \alpha = y$ (substituting $\beta = X^T \alpha$) and treating $A = XX^T$ and $b=y$.

Similarly, the RCD update can be seen as the randomized coordinate descent rule for  the psd system $X^T X \beta = X^T y$ and treating $A = X^T X$ and $b=X^T y$.

Using this connection, we propose the following update rule:
\begin{eqnarray}
\delta_t &=& \frac{y^i - \beta_t^TX^i - \lambda \alpha^i_t}{\|X^i\|^2 + \lambda} \label{eq:kacz1}\\
\alpha^i_{t+1} &=& \alpha^i_t + \delta_t  \label{eq:kacz2}\\
\beta_{t+1} &=& \beta_t + \delta_t X^i \label{eq:kacz3}
\end{eqnarray}
where the $i$-th row was picked with probability proportional to $\|X^i\|^2 + \lambda$. If all rows are normalized, then this is still a uniform distribution. However, it is more typical to normalize the columns in statistics, and hence one pass over the data must be made to calculate row norms.

We argue that this update rule exhibits linear convergence for $\min_\beta \|y - X\beta\|^2 + \lambda \|\beta\|^2$. Similarly, for Kernel Ridge Regression as mentioned in Eq.\eqref{eq:KRRf}, since one hopes to calculate $f_{KRR} = \Phi^T(K + \lambda I_n)^{-1} y$, the RK-style update can be derived from the randomized coordinate descent update rule for the psd system
\[
(K + \lambda I) \alpha = y
\]
by setting $f_{KRR}=\Phi^T \alpha$. The update for $\alpha$ looks like (where $S_a(z) = \frac{z}{1+a}$)
\begin{eqnarray}\label{eq:kerkacz1}
\alpha^i_{t+1} &=& \frac{K(x_i,x_i)}{K(x_i,x_i)+\lambda}\alpha^i_t + \frac{y_i - \sum_j K(x_i,x_j) \alpha_t^j}{K(x_i,x_i) + \lambda}\\
&=& S_{\frac{\lambda}{K(x_i,x_i)}} \left( \alpha_t^i + \frac{r_i}{K(x_i,x_i)} \right)\label{eq:kerkacz3}
\end{eqnarray}
where row $i$ is picked proportional to $K(x_i,x_i)+\lambda$ (uniform for translation invariant kernels).

Let us contrast this with the randomized coordinate descent update rule   for the loss function $\min_x \tfrac1{2} \beta^T (X^TX + \lambda I_p) \beta - y^T X\beta$ i.e. the system $(X^T X + \lambda I_p)\beta = X^T y$.
\begin{eqnarray}
\beta^i_{t+1} &=&\beta^i_t + \frac{X_i^T y - X_i^T X \beta -\lambda \beta_i}{\|X_i\|^2 + \lambda}\label{eq:rcdridge1}\\
&=& \frac{\|X_i\|^2}{\|X_i\|^2 + \lambda}\beta^i_t + \frac{X_i^T r_t}{\|X_i\|^2 + \lambda} \\
&=& S_{\frac{\lambda}{\|X_i\|^2}} \left(\beta^i_t + \frac{X_i^T r_t}{\|X_i\|^2} \right)\label{eq:rcdridge3}
\end{eqnarray}

\subsection{Computation and Convergence}
The RCD updates in \eqref{eq:rcdridge1}-\eqref{eq:rcdridge3} take $O(n)$ time, since each column (feature) is of size $n$. In contrast, the proposed RK updates in \eqref{eq:kacz1}-\eqref{eq:kacz3} takes $O(p)$ time since that is the length of a data point.

Lastly the RK updates in \eqref{eq:kerkacz1}-\eqref{eq:kerkacz3} take $O(n)$ time (to update $r$) not counting time for kernel evaluations. The difference between the two RK updates for Ridge Regression and Kernel Ridge Regression is that for KRR, we cannot maintain $\alpha$ and $\beta$ since the $\beta$ is a function in the RKHS. This different parameterization makes the updates to $\alpha$ cost $O(n)$ instead of $O(p)$.

While the RK and RCD algorithms are similar and related, one should not be tempted into thinking their convergence rates are the same. Indeed, with no normalization assumption, using a similar style proof as presented earlier, one can show that the convergence rate of the RK for Kernel Ridge Regression is
\begin{eqnarray}
\mathbb{E}\|\alpha_t - \alpha^*\|_{K+\lambda I_n}^2 &\leq& \left( 1 - \frac{\sigma_{\min}(K + \lambda I_n)}{Tr(K+\lambda I_n)}\right)^{t} \|\alpha_0 - \alpha^*\|_{K+\lambda I_n}^2\\
&=& \begin{cases} \left( 1 - \frac{\lambda}{\sum_i \sigma_i^2 + n\lambda}\right)^{t} \|\alpha_0 - \alpha^*\|_{K+\lambda I_n}^2 ~\mbox{~if $n>p$ }\\
\left( 1 - \frac{\sigma_1^2 + \lambda}{\sum_i \sigma_i^2 + n\lambda}\right)^{t} \|\alpha_0 - \alpha^*\|_{K+\lambda I_n}^2 ~\mbox{~if $p>n$}
\end{cases}
\end{eqnarray}
and the rate of convergence of the RCD for Ridge Regression is subtly different:
\begin{eqnarray}
\mathbb{E}\|\beta_t - \beta^*\|_{\Sigma+\lambda I_p}^2 &\leq& \left( 1 - \frac{\sigma_{\min}(\Sigma + \lambda I_p)}{Tr(\Sigma + \lambda I_p)}\right)^{t} \|\beta_0 - \beta^*\|_{\Sigma+\lambda I_p}^2 \\
&=& \begin{cases} \left( 1 - \frac{\sigma_1^2 + \lambda}{\sum_i \sigma_i^2 + p\lambda}\right)^{t} \|\beta_0 - \beta^*\|_{\Sigma+\lambda I_p}^2 ~\mbox{~if $n>p$} \\
\left( 1 - \frac{ \lambda}{\sum_i \sigma_i^2 + p\lambda}\right)^{t} \|\beta_0 - \beta^*\|_{\Sigma+\lambda I_p}^2 ~\mbox{~if $p>n$}
\end{cases}
\end{eqnarray}

Kernel Ridge Regression is used as a subroutine in many kernelized machine learning problems (for example, see section 4.3 of \citet{FukSonGre14} for a recent application to a nonparametric state space models). One of the major issues involved with scaling kernel methods is the formation of the gram matrix, which could be prohibitively large to form, store, invert, etc. Our RK-style algorithm gets around this issue completely by never forming the kernel matrix, just as RCD avoids forming $\Sigma$, making it a great choice for scaling kernel methods.

\section{Conclusion}

In this paper, we studied the close connections between the RK and RCD algorithms that have both received a lot of recent attention in the literature, and which we show are in some sense instances of each other when appropriately viewed. While RK is often viewed as a stochastic gradient algorithm, we saw that its ties to RCD are much stronger and that it is easier to understand its convergence through the RCD perspective than the SGD viewpoint.

We first analysed their opposite behavior with linear systems. If the system was consistent and unique then we showed that both algorithms approached the solution at (approximately) the same linear rate, with extremely similar proofs presented for direct comparison. However, if the system was consistent with infinite solutions then we saw that RK converged to the minimum norm solution but RCD didn't, making RK preferable. In contrast, if the system was inconsistent, RCD converged to the least squares solution but RK did not, making RCD the preferred choice. Unfortunately, in both cases, the preferred choices have costlier updates.

We then exploited the connection between RK and RCD to design an RK-style algorithm for Kernel Ridge Regression (KRR) which avoided explicitly forming and inverting the potentially large kernel matrix. We anticipate this and other randomization techniques will help the scalability of kernel methods.

\subsection*{Acknowlegements}
The authors would like to thank
Deanna Needell and Anna Ma
for some discussions,
Peter Richtarik   for pointing out \citet{frek}, and
Ryan Tibshirani, for memorable class notes on coordinate descent and for corrections on an earlier version of this manuscript

\bibliographystyle{agsm}

Optimization and Control (math.OC); Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)


Counting elliptic curves with prescribed torsion

Abstract: Mazur's theorem states that there are exactly 15 possibilities for the torsion subgroup of an elliptic curve over the rational numbers. We determine how often each of these groups actually occurs. Precisely, if $G$ is one of these 15 groups, we show that the number of elliptic curves up to height $X$ whose torsion subgroup is isomorphic to $G$ is on the order of $X^{1/d}$, for some number $d=d(G)$ which we compute.

\section{Introduction}

The arithmetic of elliptic curves has long been a central area of study in number theory. One of the first general results, due to Mordell \cite{mordell} (see also \cite[Ch.~VIII]{silverman}), states that the group of rational points of an elliptic curve is finitely generated. In other words, if $E/\mathbf{Q}$ is an elliptic curve then we can write $E(\mathbf{Q})=\mathbf{Z}^r \times E(\mathbf{Q})_{\mathrm{tors}}$, where $r \ge 0$ is an integer (the \emph{rank} of $E$), and $E(\mathbf{Q})_{\mathrm{tors}}$ is a finite group (the \emph{torsion subgroup} of $E$). Given this result, one would like to understand what the possibilities are for the rank and torsion subgroup. Little is known regarding the rank. On the other hand, work of several mathematicians (most notably Mazur \cite{mazur}) established that there are only 15 possibilities for the torsion subgroup, namely:
\begin{equation}
\label{mazur}
\begin{aligned}
\mathbf{Z}/N\mathbf{Z} & \quad \textrm{with $1 \le N \le 10$ or $N=12$} \\
\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/N\mathbf{Z} & \quad \textrm{with $N=2,4,6,8$.}
\end{aligned}
\end{equation}
With this classification in hand, it is natural to ask a more refined question: how often does each of these groups occur? Our purpose here is to provide an answer.

\subsection{The main result}

To make the question precise, we formulate it as a counting problem. Every elliptic curve $E$ over $\mathbf{Q}$ admits a unique equation of the form
\begin{displaymath}
y^2=x^3+Ax+B
\end{displaymath}
where $A$ and $B$ are integers and $\gcd(A^3,B^2)$ is not divisible by any twelfth powers. We call such an equation \emph{minimal}, and define the (\emph{na\"{i}ve}) \emph{height} of $E$ to be $\max(|A|^3, |B|^2)$. Clearly, there are only finitely many elliptic curves (up to isomorphism) of height $<X$, for any real number $X$. We can therefore introduce a meaningful counting function: if $G$ is one of the groups in \eqref{mazur}, we let $N_G(X)$ be the number of (isomorphism classes of) elliptic curves $E/\mathbf{Q}$ of height at most $X$ for which $E(\mathbf{Q})_{\mathrm{tors}}$ is isomorphic to $G$. The following theorem is our main result.

\begin{theorem}
\label{thm:main}
For any group $G$ in \eqref{mazur}, the limit
\begin{displaymath}
\frac{1}{d(G)}=\lim_{X \to \infty} \frac{\log{N_G(X)}}{\log{X}}
\end{displaymath}
exists. The value of $d(G)$ is as indicated in Table~\ref{f:main}.
\end{theorem}

\begin{table}[!h]
\caption{The values of $d(G)$.}
{\tabulinesep=1.2mm
\begin{tabu}{|c|c||c|c||c|c|}
\hline
$G$ & $d$ & $G$ & $d$ & $G$ & $d$ \\
\hline
\hline
0 & 6/5 & $\mathbf{Z}/6\mathbf{Z}$ & 6 & $\mathbf{Z}/12\mathbf{Z}$ & 24 \\
\hline
$\mathbf{Z}/2\mathbf{Z}$ & 2 & $\mathbf{Z}/7\mathbf{Z}$ & 12 & $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z}$ & 3 \\
\hline
$\mathbf{Z}/3\mathbf{Z}$ & 3 & $\mathbf{Z}/8\mathbf{Z}$ & 12 & $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/4\mathbf{Z}$ & 6 \\
\hline
$\mathbf{Z}/4\mathbf{Z}$ & 4 & $\mathbf{Z}/9\mathbf{Z}$ & 18 & $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/6\mathbf{Z}$ & 12 \\
\hline
$\mathbf{Z}/5\mathbf{Z}$ & 6 & $\mathbf{Z}/10\mathbf{Z}$ & 18 & $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/8\mathbf{Z}$ & 24 \\
\hline
\end{tabu}}
\label{f:main}
\end{table}

Since $d(0)<d(G)$ for all non-trivial $G$, we recover the following well-known result due to Duke \cite{duke}.

\begin{corollary}
Almost all elliptic curves over $\mathbf{Q}$ have trivial torsion.
\end{corollary}

\subsection{Refined results}
\label{refined}

We actually prove a stronger result than Theorem~\ref{thm:main}: given $G$ as in \eqref{mazur}, there exist positive constants $K_1$ and $K_2$ such that
\begin{displaymath}
K_1 X^{1/d(G)} \le N_G(X) \le K_2 X^{1/d(G)}
\end{displaymath}
holds for all $X \ge 1$. This suggests that the limit
\begin{displaymath}
c(G)=\lim_{X \to \infty} \frac{N_G(X)}{X^{1/d(G)}}
\end{displaymath}
might exist. We prove this is the case for $\#G\leq3$.

\begin{theorem}
\label{thm:asymptotics}
Write $c_N$ for $c(\mathbf{Z}/N\mathbf{Z})$. Then the limits defining $c_1$, $c_2$, and $c_3$ exist, and
\begin{align*}
c_1 &= \frac{4}{\zeta(10)} \approx 3.9960, \\
c_2 &= \frac{1}{\zeta(6)} \left( 2\log(\alpha_-/\alpha_+)+\tfrac{4}{3} (\alpha_++\alpha_-) \right) \approx 3.1969, \\
c_3 &= \frac{1}{\zeta(4)} \left( 2I_+-2I_-+\frac{1}{3}\log\left(\displaystyle\frac{\beta_0\beta_1\beta_5}{\beta_2\beta_3\beta_4}\right)+\frac{9}{4}\left(\beta_0^4+\beta_2^4+\beta_4^4-\beta_1^4-\beta_3^4-\beta_5^4\right) \right) \approx 1.5221,
\end{align*}
where the $\alpha$'s and $\beta$'s are algebraic numbers and the $I$'s are hyperelliptic integrals (see \S\ref{sec:asymptotics}).
\end{theorem}

The case $N=1$ is straightforward; see \cite[Lemma~4.3]{brumer}. The case $N=2$ was previously carried out in \cite[\S2]{grant}. The case $N=3$ does not seem to occur in the literature. We, in fact, obtain power-saving error terms (see Theorem~\ref{lem:main_asymp_lemma} for this more precise version).

\subsection{Interpretation of $d(G)$}
\label{ss:interp}

If $f \colon \mathbf{P}^1 \to \mathbf{P}^1$ is a degree $d$ map, then the number of points in $f(\mathbf{P}^1(\mathbf{Q}))$ up to height $X$ is approximately $X^{2/d}$. This suggests that $d(G)$ should be related to the degree of the map $f \colon \mathcal{X}(G) \to \mathcal{X}(1)$, where here $\mathcal{X}(-)$ denotes the appropriate moduli stack. When $\mathcal{X}(G)$ is a scheme, Table~\ref{f:main} shows that
\begin{displaymath}
d(G)=\tfrac{1}{4} \deg(f)=\tfrac{1}{2} \deg(|f|),
\end{displaymath}
where $|f|$ denotes the map of coarse spaces. More generally, as long as $G \ne 0$, we have
\begin{displaymath}
d(G) = \tfrac{1}{2} \deg(|f|)+\tfrac{1}{2} \nu_2 + \nu_3 + \nu_\infty^\prime,
\end{displaymath}
where $\nu_2$ and $\nu_3$ are the number of elliptic points of orders 2 and 3 on $\mathcal{X}(G)$ and $\nu_\infty^\prime$ is the number of irregular cusps (in the sense of \cite[\S\S 1.2 \& 2.1]{shimura}). We do not have a conceptual explanation of this formula, so it could simply be a numerical coincidence.

\begin{remark}
If the moduli stacks $\mathcal{X}(G)$ and $\mathcal{X}(1)$ were actually schemes then Theorem~\ref{thm:main} would follow immediately from the aforementioned fact about maps of $\mathbf{P}^1$, and $d(G)$ would be half the degree of $f$. It would be desirable to have a general result for counting points in the image of a map of stacks, from which Theorem~\ref{thm:main} could be deduced; see \S \ref{s:future} for a more precise discussion.
\end{remark}

\subsection{Overview of the proof}

We now go over the main ideas in the proof of Theorem~\ref{thm:main} and make some preliminary reductions.

\subsubsection{The function $N'_G(X)$}

Let $N'_G(X)$ be the number of (isomorphism classes of) elliptic curves $E$ of height at most $X$ such that $E(\mathbf{Q})$ contains a subgroup isomorphic to $G$. We prove the following theorem.

\begin{theorem}
\label{thm:main2}
For any group $G$ in \eqref{mazur}, there exist positive constants $K_1$ and $K_2$ such that $K_1 X^{1/d(G)} \le N'_G(X) \le K_2 X^{1/d(G)}$ for all $X \ge 1$.
\end{theorem}

Theorem~\ref{thm:main} follows easily from this. Indeed, we have obvious bounds
\begin{displaymath}
N'_G(X)-\sum_{G \subsetneqq H} N'_H(X) \le N_G(X) \le N'_G(X).
\end{displaymath}
As $d(G)<d(H)$, Theorem~\ref{thm:main2} gives $N'_H(X)/N'_G(X) \to 0$ as $X \to \infty$. Thus $N_G(X)/N'_G(X) \to 1$, and so Theorem~\ref{thm:main} (and the stronger form stated in \S \ref{refined}) follows from Theorem~\ref{thm:main2}.

\subsubsection{Proof of Theorem~\ref{thm:main2} when $2G \ne 0$}
\label{sss:oddg}

Suppose that $G$ is one of the 12 groups in \eqref{mazur} for which $2G \ne 0$. Then there is a \emph{universal elliptic curve} $\mathcal{E}$ over an open subset of $\mathbf{A}^1$ equipped with a subgroup isomorphic to $G$. (This is not exactly true for $G=\mathbf{Z}/3\mathbf{Z}$, see \S \ref{ss:z3} for details.) The universal property implies that an arbitrary curve $E/\mathbf{Q}$ admits a copy of $G$ in its rational points if and only if $E$ is isomorphic to $\mathcal{E}_t$ for some $t \in \mathbf{Q}$. We can describe $\mathcal{E}$ by an equation of the form
\begin{displaymath}
y^2=x^3+f(t)x+g(t)
\end{displaymath}
where $f$ and $g$ are polynomials. In \S \ref{s:general}, we prove the following general theorem for counting elliptic curves that appear in such families.

\begin{theorem}
\label{general}
Let $f,g \in \mathbf{Q}[t]$ be non-zero coprime polynomials of degrees $r$ and $s$, with at least one of $r$ or $s$ positive, and write
\begin{displaymath}
\max \left( \frac{r}{4}, \frac{s}{6} \right)=\frac{n}{m},
\end{displaymath}
with $n$ and $m$ coprime. Assume $n=1$ or $m=1$. Let $\mathcal{E}$ be the family of elliptic curves defined by
\begin{displaymath}
y^2=x^3+f(t)x+g(t).
\end{displaymath}
Let $N(X)$ be the number of (isomorphism classes of) elliptic curves $E/\mathbf{Q}$ of height at most $X$ for which $E \cong \mathcal{E}_t$ for some $t \in \mathbf{Q}$. Then there exist positive constants $K_1$ and $K_2$ such that
\begin{displaymath}
K_1 X^{(m+1)/12n} \le N(X) \le K_2 X^{(m+1)/12n}
\end{displaymath}
for all $X \ge 1$.
\end{theorem}

In each case of interest, the hypotheses of the theorem are satisfied. Table~\ref{f:params} lists the invariants for the various groups (see \S \ref{explan} for proofs). This establishes Theorem~\ref{thm:main2} in the cases $2G \ne 0$.

\begin{table}
\caption{Data for the universal elliptic curve with $G$-structure.}
{\small\tabulinesep=1.2mm
\begin{tabu}{|c|c|c|c|c|c|}
\hline
$G$ & $r$ & $s$ & $n$ & $m$ & $12n/(m+1)$ \\
\hline
\hline
 3 & 1 & 2 & 1 & 3 & 3 \\
\hline
 4 & 2 & 3 & 1 & 2 & 4 \\
\hline
 5 & 4 & 6 & 1 & 1 & 6 \\
\hline
 6 & 4 & 6 & 1 & 1 & 6 \\
\hline
 7 & 8 & 12 & 2 & 1 & 12 \\
\hline
 8 & 8 & 12 & 2 & 1 & 12 \\
\hline
 9 & 12 & 18 & 3 & 1 & 18 \\
\hline
10 & 12 & 18 & 3 & 1 & 18 \\
\hline
12 & 16 & 24 & 4 & 1 & 24 \\
\hline
$(2,4)$ & 4 & 6 & 1 & 1 & 6 \\
\hline
$(2,6)$ & 8 & 12 & 2 & 1 & 12 \\
\hline
$(2,8)$ & 16 & 24 & 4 & 1 & 24\\
\hline
\end{tabu}}
\label{f:params}
\end{table}

\subsubsection{Proof of Theorem~\ref{thm:main2} when $2G=0$}

If $2G=0$ then there is not a universal curve (over a scheme), and so the above method does not directly apply. When $G$ is trivial or $\mathbf{Z}/2\mathbf{Z}$, the situation is simple enough to understand directly (see \S \ref{sec:asymptotics}). When $G=\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z}$, we use a variant of Theorem~\ref{general} to understand $N'_G$ (see \S \ref{sec:Zmod2timesZmod2}). In this way, the remaining cases of Theorem~\ref{thm:main2} are established.

\subsubsection{Proof of Theorem~\ref{general}}
\label{sss:general}

We now comment on the proof of Theorem~\ref{general}, which is the heart of the paper. Recall that if $E_i$ (for $i=1,2$) are elliptic curves over $\mathbf{Q}$ defined by equations
\begin{displaymath}
y^2=x^3+a_ix+b_i,
\end{displaymath}
then $E_1$ and $E_2$ are isomorphic if and only if there exists a rational number $u \in \mathbf{Q}$ for which $a_1=u^4 a_2$ and $b_1=u^6 b_2$ \cite[Ch.~III \S 1]{silverman}. We therefore see that an elliptic curve $E/\mathbf{Q}$ given by an equation
\begin{equation}
\label{eq3}
y^2=x^3+Ax+B
\end{equation}
belongs to a family $\mathcal{E}$ as above if and only if there exist $u, t \in \mathbf{Q}$ such that $A=u^4f(t)$ and $B=u^6g(t)$. It follows that $N(X)$ (as in Theorem~\ref{general}) counts the number of pairs $(A,B) \in \mathbf{Z}^2$ satisfying the following conditions:
\begin{itemize}
\item $4A^3+27B^2 \ne 0$.
\item $\gcd(A^3, B^2)$ is not divisible by any 12th power.
\item $|A|<X^{1/3}$ and $|B|<X^{1/2}$.
\item There exist $u,t \in \mathbf{Q}$ for which $A=u^4f(t)$ and $B=u^6g(t)$.
\end{itemize}
The first condition ensures that \eqref{eq3} defines an elliptic curve; the second that the equation is minimal; the third that it has height $<X$; and the fourth that it belongs to $\mathcal{E}$. This reduces the analysis of $N(X)$ to an elementary number theory problem, which we solve directly.

\subsection{Future directions}
\label{s:future}

There are several generalizations of our results that would be of interest. The most immediate, perhaps, is to extend our results to rationally defined subgroups. Precisely, let $G$ be a product of two finite cyclic groups, and let $N^0_G(X)$ be the number of elliptic curves $E/\mathbf{Q}$ up to height $X$ admitting a subgroup defined over $\mathbf{Q}$ which (over $\mathbf{C}$) is isomorphic to $G$. One would then like to compute the limit of $\log{N^0_G(X)}/\log{X}$. Again, there are only finitely many $G$ for which one gets a non-zero answer, namely, those for which the moduli space has genus 0.

It would also be interesting to generalize Theorem~\ref{general} to allow $f$ and $g$ to have a common factor and to remove the restriction on $n$ and $m$. How does this affect the count?

Perhaps the ultimate generalization in this direction is the following. Let $\mathcal{X}$ and $\mathcal{Y}$ be proper smooth Deligne--Mumford stacks over $\mathbf{Q}$ with coarse space $\mathbf{P}^1$, and let $f \colon \mathcal{Y} \to \mathcal{X}$ be a map. Suppose that there is a good notion of height $h_{\mathcal{X}}$ on the set $|\mathcal{X}(\mathbf{Q})|$, where $|\cdot|$ denotes isomorphism classes. Then one would like a formula for
\begin{displaymath}
\lim_{T \to \infty} \frac{\# \{ x \in f(|\mathcal{Y}(\mathbf{Q})|) \mid h_{\mathcal{X}}(x) \le T \}}{\log{T}}
\end{displaymath}
in terms of invariants of $\mathcal{X}$, $\mathcal{Y}$, and $f$ (in the style of \S \ref{ss:interp}).

More generally, one may ask these questions over general global fields. What kind of dependence is there on the base field?

In addition to these generalizations, it would be interesting to complete the results of \S \ref{refined}, and compute the value of $c(G)$ for other $G$'s.

\subsection{Notation}

If $f$ and $g$ are two functions of a real variable $X$, we write $f \lesssim g$ to mean ``there exists a positive constant $c$ such that $f(X) \le c g(X)$ holds for all $X \ge 1$.'' For a prime number $p$, we let $\val_p$ be the usual $p$-adic valuation on $\mathbf{Q}$ (so $\val_p(p)=1$). For a place $p \le \infty$ of $\mathbf{Q}$, we let $\vert \cdot \vert_p$ be the usual absolute value on $\mathbf{Q}$ (so $\vert p \vert_p=p^{-1}$ if $p<\infty$). We often write $\vert \cdot \vert$ in place of $\vert \cdot \vert_{\infty}$. We write $\lfloor x \rfloor$ and $\lceil x \rceil$ for the floor and ceiling of $x \in \mathbf{Q}$.

\subsection*{Acknowledgements}

We would like to thank Jordan Ellenberg, Wei Ho, and Melanie Matchett Wood for useful conversations, and Bjorn Poonen for pointing out an innaccuracy in an earlier version. We would also like to acknowledge the Sagemath Cloud and Sage \cite{sage} in which we carried out several helpful computations. The first author was supported by NSA Young Investigator Grant \#H98230-13-1-0223 and NSF RTG Grant ``Number
Theory and Algebraic Geometry at the University of Wisconsin''. The second author was supported by NSF Grant DMS-1303082.

\section{Counting elliptic curves in families}
\label{s:general}

The purpose of this section is to prove Theorem~\ref{general}. As explained in \S \ref{sss:general}, it is equivalent to the following proposition.

\begin{proposition}
\label{bd}
Let $f,g \in \mathbf{Q}[t]$ be coprime polynomials of degrees $r$ and $s$. Assume at least one of $r$ or $s$ is positive. Write
\begin{displaymath}
\max \left( \frac{r}{4}, \frac{s}{6} \right)=\frac{n}{m}
\end{displaymath}
with $n$ and $m$ coprime. Assume $n=1$ or $m=1$. Let $S(X)$ be the set of pairs $(A,B) \in \mathbf{Z}^2$ satisfying the following conditions:
\begin{itemize}
\item $4A^3+27B^2 \ne 0$.
\item $\gcd(A^3,B^2)$ is not divisible by any 12th power.
\item $|A|<X^{1/3}$ and $|B|<X^{1/2}$.
\item There exist $u,t \in \mathbf{Q}$ such that $A=u^4 f(t)$ and $B=u^6 g(t)$.
\end{itemize}
Then $X^{(m+1)/12n} \lesssim \# S(X) \lesssim X^{(m+1)/12n}$.
\end{proposition}

\subsection{The upper bound}

We begin by establishing the upper bound. Let $S_1(X)$ be the set of pairs $(u,t) \in \mathbf{Q}^2$ such that $(A,B)=(u^4f(t), u^6g(t))$ belongs to $S(X)$.

\begin{lemma}
\label{bd1}
For each place $p \le \infty$ of $\mathbf{Q}$ there exists a constant $c_p>0$ such that for all $t \in \mathbf{Q}$
\begin{displaymath}
\max(|f(t)|_p, |g(t)|_p) \ge c_p.
\end{displaymath}
If $p$ is a sufficiently large finite prime, one can take $c_p=1$.
\end{lemma}

\begin{proof}
Let $\ol{\mathbf{Q}}$ be a fixed algebraic closure of $\mathbf{Q}$, and extend $\vert \cdot \vert_p$ to $\ol{\mathbf{Q}}$ in any way. Let $\{\alpha_i\}$ be the roots of $f$ and $\{\beta_j\}$ the roots of $g$ in $\ol{\mathbf{Q}}$. Note that $\alpha_i \ne \beta_j$ for all $i$ and $j$, since $f$ and $g$ are coprime. Let $\delta=\min(|\alpha_i-\beta_j|_p)$. Let $\epsilon>0$ be such that $|f(t)|_p<\epsilon$ implies $|t-\alpha_i|<\delta/2$ for some $i$, and similarly for $g$. If $|f(t)|_p<\epsilon$ and $|g(t)|_p<\epsilon$ then $|t-\alpha_i|_p<\delta/2$ for some $i$ and $|t-\beta_j|_p<\delta/2$ for some $j$, which implies $|\alpha_i-\beta_j|_p<\delta$, a contradiction. We must therefore have $|f(t)|_p \ge \epsilon$ or $|g(t)|_p \ge \epsilon$ for all $t$, and so can take $c_p=\epsilon$.

Now let $p$ be a prime large enough so that: (1) the coefficients of $f$ and $g$ are $p$-integral; (2) the leading coefficients of $f$ and $g$ are $p$-units; and (3) $\alpha_i-\beta_j$ are $p$-units, for all $i$ and $j$. If $|f(t)|_p<1$ then $|t-\alpha_i|<1$ for some $i$. Similarly for $g$. Thus if $|f(t)|_p<1$ and $|g(t)|_p<1$ then $|t-\alpha_i|<1$ and $|t-\beta_j|<1$, which implies $|\alpha_i-\beta_j|<1$, a contradicition. We conclude that we can take $c_p=1$ for such $p$.
\end{proof}

\begin{lemma}
\label{bd2}
For each prime number $p$ there exists a constant $C_p$ with the following property. Suppose $(u,t) \in S_1(X)$. Then
\begin{displaymath}
\val_p(u)=\epsilon+\begin{cases} \lceil -\tfrac{n}{m} \val_p(t) \rceil & \textrm{if $\val_p(t)<0$} \\
0 & \textrm{if $\val_p(t) \ge 0$,} \end{cases}
\end{displaymath}
where $|\epsilon| \le C_p$. Furthermore, one can take $C_p=0$ for $p \gg 0$.
\end{lemma}

\begin{proof}
Suppose $(u,t) \in S_1(X)$, and let $p$ be a prime. Since $A$ and $B$ are integral, we have $4\val_p(u)+\val_p(f(t)) \ge 0$ and $6\val_p(u)+\val_p(g(t)) \ge 0$. Furthermore, $\val_p(u)$ must be minimal subject to these inequalities, or else $p^{12}$ would divide $\gcd(A^3, B^2)$. We can thus write
\begin{displaymath}
\val_p(u)=\max(\lceil -\tfrac{1}{4} \val_p(f(t)) \rceil, \lceil -\tfrac{1}{6} \val_p(g(t)) \rceil).
\end{displaymath}
Equivalently,
\begin{equation}
\label{eq1}
-\val_p(u)=\min(\lfloor \tfrac{1}{4} \val_p(f(t)) \rfloor, \lfloor \tfrac{1}{6} \val_p(g(t)) \rfloor).
\end{equation}

Suppose $\val_p(t)<0$. Let $K_1$ be a constant such that $|\val_p(f(t))-r\val_p(t)|<K_1$ and $|\val_p(g(t))-s\val_p(t)|<K_1$ for all such $t$. (This exists since the functions $\val_p(f(t)/t^r)$ and $\val_p(g(t)/t^s)$ are bounded on $\mathbf{Q}_p \setminus \mathbf{Z}_p$.) Then for $\val_p(t)<0$ we have
\begin{displaymath}
\val_p(u)=\epsilon+\max(\lceil -\tfrac{r}{4} \val_p(t) \rceil, \lceil -\tfrac{s}{6} \val_p(t) \rceil)
\end{displaymath}
where $|\epsilon|<K_2$, for some $K_2$. (One can take $K_2=1+\tfrac{n}{m} K_1$.) Since $\val_p(t)<0$, we have
\begin{displaymath}
\max(\lceil -\tfrac{r}{4} \val_p(t) \rceil, \lceil -\tfrac{s}{6} \val_p(t) \rceil)
=\lceil -\tfrac{n}{m} \val_p(t) \rceil,
\end{displaymath}
and so
\begin{displaymath}
\val_p(u)=\epsilon+\lceil -\tfrac{n}{m} \val_p(t) \rceil,
\end{displaymath}
with $|\epsilon|<K_2$.

Now, consider $\val_p(t) \ge 0$. Let $K_3$ be a constant such that $\min(\val_p(f(t)), \val_p(g(t))) \le K_3$ for all such $t$, which exists by Lemma~\ref{bd1}. Appealing to \eqref{eq1}, we find $-\val_p(u) \le K_4$, for an appropirate $K_4$. Let $K_5$ be so that $\val_p(f(t)) \ge K_5$ and $\val_p(g(t)) \ge K_5$ for all $t$ with $\val_p(t) \ge 0$. Then appealing to \eqref{eq1} again, we find $-\val_p(u) \ge K_6$, for an appropriate $K_6$. We thus see that $|\val_p(u)|<K_7$ whenever $\val_p(t) \ge 0$, where $K_7=\max(K_4,K_6)$.

Combining the above two paragraphs, we see that the formula in the statement of the lemma holds with $C_p=\max(K_2, K_7)$.

Now suppose $p$ is large enough so that: (1) the coefficients of $f$ and $g$ are $p$-integral; (2) the leading coefficients of $f$ and $g$ are $p$-unit; and (3) the constant $c_p$ from the previous lemma can be taken to be 1. For $\val_p(t)<0$ we have equalities $\val_p(f(t))=r\val_p(t)$ and $\val_p(g(t))=s\val_p(t)$, which shows that
\begin{displaymath}
\val_p(u)=\max(\lceil -\tfrac{r}{4} \val_p(t) \rceil, \lceil -\tfrac{s}{6} \val_p(t) \rceil)
=\lceil -\tfrac{n}{m} \val_p(t) \rceil.
\end{displaymath}
For $\val_p(t) \ge 0$, we have $\val_p(f(t)) \ge 0$ and $\val_p(g(t)) \ge 0$, with at least one inequality being an equality. Thus $\val_p(u)=0$.  This proves that we can take $C_p=0$ in this case.
\end{proof}

\begin{lemma}
\label{bd3}
There exists a finite set $Q$ of non-zero rational numbers with the following property. Suppose $(u,t) \in S_1(X)$. Then we can write $t=a/b^m$, where $a$ and $b$ are integers such that $b>0$ and $\gcd(a,b^m)$ is not divisible by any $m$th power, and $u=qb^n$, with $q \in Q$.
\end{lemma}

\begin{proof}
Suppose $(u,t) \in S_1(X)$. We then have a unique expression $t=a/b^m$, where $a$ and $b>0$ are integers with $\gcd(a,b^m)$ not divisible by any $m$th power. If $p\mid b$ then $-\val_p(t)=m\val_p(b)-k$, where $0 \le k<m$. If $m=1$ then $k=0$, while if $n=1$ then $0 \le \tfrac{n}{m} k<1$; in either case, $\lceil-\tfrac{n}{m} k \rceil=0$. Thus $\val_p(u)=\epsilon+n\val_p(b)$ by Lemma~\ref{bd2}, with $|\epsilon| \le C_p$. If $p \nmid b$ then $\val_p(t) \ge 0$, and so $|\val_p(u)| \le C_p$. We thus see that $|\val_p(u/b^n)| \le C_p$ for all $p$. Since we can take $C_p=0$ for $p \gg 0$, this means that there are only finitely many possibilities for the rational number $u/b^n$.
\end{proof}

\begin{lemma}
\label{bd4}
Suppose $(u,t) \in S_1(X)$ and write $t=a/b^m$ and $u=qb^n$ per Lemma~\ref{bd3}. Then $|a| \lesssim X^{m/12n}$ and $|b| \lesssim X^{1/12n}$.
\end{lemma}

\begin{proof}
Suppose $(u,t) \in S_1(X)$ and write $t=a/b^m$ and $u=qb^n$ as above. The inequality $\max(|A|^3,|B|^2)<X$ translates to
\begin{displaymath}
|u| \max(|f(t)|^{1/4}, |g(t)|^{1/6})<X^{1/12}.
\end{displaymath}
Let $K_1>0$ be a constant such that $\max(|f(t)|^{1/4}, |g(t)|^{1/6}) \ge K_1$ for all $t$, which exists by Lemma~\ref{bd1}. Then $|u| \le K_1^{-1} X^{1/12}$, and so
\begin{displaymath}
|b| \le K_2 X^{1/12n},
\end{displaymath}
with $K_2=K_1^{-1/n} \max_{q \in Q}(q^{-1/n})$. Suppose for the moment that $|t| \ge 1$. Let $K_3>0$ be a constant so that $K_3^4 |t|^r \le |f(t)|$ and $K_3^6 |t|^s \le |g(t)|$ holds for all such $t$. We thus have
\begin{displaymath}
X^{1/12}>|u| \max(|f(t)|^{1/4}, |g(t)|^{1/6}) \ge K_3 |u| \max(|t|^{r/4}, |t|^{s/6}) = K_3 |u| |t|^{n/m}
=K_4(q) |a|^{n/m},
\end{displaymath}
where $K_4(q)=K_3 q$. We therefore find $|a|<K_5 X^{m/12n}$ with $K_5=\max_{q \in Q}K_4(q)^{-m/n}$. Now suppose $|t|<1$. Then $|a|<|b^m|\leq K_2^m X^{m/12n}$. Thus, in all cases, we have
\begin{displaymath}
|a|<K_6 X^{m/12n}
\end{displaymath}
with $K_6=\max(K_5,K_2^m)$.
\end{proof}

The lemma shows that $\# S_1(X) \lesssim X^{(m+1)/12n}$, which implies the same for $\# S(X)$. We have thus established the upper bound in Proposition~\ref{bd}.

\subsection{The lower bound}

We now turn to the lower bound. Observe that by changing $u$ to $Mu$ for appropriate $M$, it suffices to consider the case where $f$ and $g$ have integer coefficients, which we now assume. For $(a,b) \in \mathbf{Z}^2$ put $u=b^n$ and $t=a/b^m$, and $A=u^4f(t)$ and $B=u^6g(t)$. Note that $A$ and $B$ are integers. Fix a small constant $\kappa>0$ and let $S_2(X)$ denote the set of pairs $(a,b) \in \mathbf{Z}^2$ satisfying the following conditions:
\begin{itemize}
\item $a$ and $b$ are coprime and $b>0$.
\item $|a|<\kappa X^{m/12n}$ and $|b|<\kappa X^{1/12n}$.
\item $4A^3+27B^2 \ne 0$.
\end{itemize}
We note that for appropriately chosen $\kappa$, if $(a,b) \in S_2(X)$ then $|A|<X^{1/3}$ and $|B|<X^{1/2}$.

\begin{lemma}
\label{bd5}
There exists a non-zero integer $D$ with the following property: if $(a,b) \in S_2(X)$ then $\gcd(A^3, B^2)$ divides $D$.
\end{lemma}

\begin{proof}
It suffices to find constants $e_p$ such that $\val_p(\gcd(A^3,B^2)) \le e_p$ and $e_p=0$ for $p \gg 0$, for then we can take $D=\prod_p p^{e_p}$.

Suppose $(a,b) \in S_2(X)$, and let $p$ be a prime. Let $K_1$ be a constant so that $|3\val_p(f(t))-3r\val_p(t)| \le K_1$ and $|2\val_p(g(t))-2s\val_p(t)| \le K_1$ for all $t \in \mathbf{Q}$ with $\val_p(t)<0$. Note that for $p \gg 0$ we can take $K_1=0$. Suppose $\val_p(b)=k>0$, so that $\val_p(t)=-mk$ and $\val_p(u)=nk$. Then
\begin{displaymath}
\val_p(A^3)=\val_p(u^{12} f(t)^3)=12nk-3rmk+\epsilon=12m(\tfrac{n}{m}-\tfrac{r}{4})k+\epsilon
\end{displaymath}
and
\begin{displaymath}
\val_p(B^2)=\val_p(u^{12} g(t)^2)=12nk-2smk+\delta = 12m(\tfrac{n}{m}-\tfrac{s}{6})k+\delta
\end{displaymath}
where $|\epsilon| \le K_1$ and $|\delta| \le K_1$. However, $\tfrac{n}{m}=\max(\tfrac{r}{4}, \tfrac{s}{6})$, and so $\val_p(\gcd(A^3, B^2)) \le K_1$.

Let $K_2$ be a constant so that $\max(3\val_p(f(t)), 2\val_p(g(t))) \le K_2$ for all $t \in \mathbf{Q}$ with $\val_p(t) \ge 0$. This constant exists by Lemma~\ref{bd1}; furthermore, we can take $K_2=0$ for $p$ sufficiently large. Since $p$ does not divide $b$, this gives $\val_p(\gcd(A^3,B^2)) \le K_2$.

We thus see that we can take $e_p=\max(K_1, K_2)$, and this can be taken to be 0 for $p \gg 0$.
\end{proof}

Let $S_3(X)$ denote the set of pairs $(A,B) \in \mathbf{Z}^2$ coming from $S_2(X)$. We have a map $S_3(X) \to S(X)$ taking $(A,B)$ to $(A/d^4,B/d^6)$, where $d^{12}$ is the largest 12th power dividing $\gcd(A^3,B^2)$.

\begin{lemma}\label{lem:S3toS}
There exists a constant $N$ such that every fiber of the map $S_3(X) \to S(X)$ has cardinality at most $N$.
\end{lemma}

\begin{proof}
Let $(A_0,B_0) \in S(X)$ be given. Suppose $(A,B) \in S_3(X)$ maps to $(A_0, B_0)$, i.e., $A=d^4A_0$ and $B=d^6B_0$ for some $d \in \mathbf{Z}$. By Lemma~\ref{bd5}, $d^{12} \mid D$. Therefore, if $N$ is the number of 12th powers dividing $D$, the fibers have cardinality at most $N$.
\end{proof}

\begin{lemma}
There exists a constant $M$ such that every fiber of the map $S_2(X) \to S_3(X)$ has cardinality at most $M$.
\end{lemma}

\begin{proof}
Let $(A,B) \in S_3(X)$ be given. An element $(a,b) \in S_2(X)$ of the fiber gives a solution to the equations $Ax^4=f(y)$ and $Bx^6=g(y)$ by $x=u^{-1}=b^{-n}$ and $y=t=a/b^m$. Furthermore, $(a,b)$ is determined from $(x,y)$. These two equations define plane curves of degree $\max(4,r)$ and $\max(6,s)$, which are easily seen to have no common components. By B\'ezout's theorem, they therefore have at most $M=\max(4,r) \max(6,s)$ points in common, and this bounds the cardinality of the fibers.
\end{proof}

We leave to the reader the standard estimate $X^{(m+1)/12n} \lesssim \# S_2(X)$. This gives the lower bound in Proposition~\ref{bd} since we have a map $S_2(X) \to S(X)$ whose fibers have bounded size. We have thus completed the proof of Proposition~\ref{bd}.

\section{Equations for universal curves}
\label{explan}

Let $G$ be one of the 12 groups in \eqref{mazur} for which $2G \ne 0$. Let $E$ be a family of elliptic curves over a base scheme $S/\mathbf{Q}$. By a \emph{$G$-structure} on $E$ we mean an injection of groups $i \colon G_S \to E$. When $G=\mathbf{Z}/N\mathbf{Z}$, a $G$-structure is just a section of $E$ of order $N$. There is a moduli space $Y/\mathbf{Q}$ of elliptic curves equipped with a $G$-structure, over which there is a universal elliptic curve $\mathcal{E}$ with a $G$-structure. (Although for $G=\mathbf{Z}/3\mathbf{Z}$ we must slightly change the definition of $Y$, see below.) In fact, $Y$ can be identified with an open subvariety of $\mathbf{A}^1$; choose such an identification. Then $\mathcal{E}$ is given by an equation of the form
\begin{equation}
\label{univeq}
y^2=x^3+f(t)x+g(t),
\end{equation}
where $f$ and $g$ are rational functions in $t$ with rational coefficients. Making a change of variables, we can assume that $f$ and $g$ are polynomials and $\gcd(f^3,g^2)$ is not divisible by any 12th powers. Note that neither $f$ nor $g$ is zero, as then every geometric fiber of $\mathcal{E}$ would have CM. We show that $f$ and $g$ are coprime and compute their degree, which allows us to apply Theorem~\ref{general} to prove Theorem~\ref{thm:main2}.

\subsection{The case $\mathbf{Z}/4\mathbf{Z}$}

Let $G=\mathbf{Z}/4\mathbf{Z}$. We have the following result.

\begin{proposition}
For an appropriate choice of embedding $Y \to \mathbf{A}^1$, the polynomials $f$ and $g$ are coprime and satisfy $\deg(f)=2$ and $\deg(g)=3$.
\end{proposition}

\begin{proof}
By \cite[Tab.~3]{kubert}, the curve $\mathcal{E}$ is given by
\begin{displaymath}
y^2+xy-ty=x^3-tx^2,
\end{displaymath}
for some embedding $Y \to \mathbf{A}^1$. The result is obtained from the change of variables:
\[
	x\mapsto x+\frac{t}{3}-\frac{1}{12}\quad\text{and}\quad y\mapsto y-\frac{x}{2}+\frac{t}{3}+\frac{1}{24}.
\]
\end{proof}

This establishes the second row of Table~\ref{f:params}, from which Theorem~\ref{thm:main2} follows for $G$.

\subsection{The cases with $\# G>4$}

We assume now that $G$ is one of the 10 groups in \eqref{mazur} of order greater than 4.

\begin{proposition}
\label{prop1}
The polynomials $f$ and $g$ are coprime, and $\deg(f)=4\ell$ and $\deg(g)=6\ell$ for some integer $\ell$. In fact, if $k$ denotes the number of injective group homomorphisms $G \to (\mathbf{Q}/\mathbf{Z})^2$ then $\ell=k/24$.
\end{proposition}

We first require a lemma.

\begin{lemma}
Let $A$ be a DVR with fraction field $K$ and residue characteristic 0. Let $E/K$ be an elliptic curve admitting a $G$-structure. Then $E$ has semi-stable reduction.
\end{lemma}

We give two proofs.

\begin{proof}[First proof]
Suppose $E$ has additive reduction. Let $\ol{E}/A$ be the N\'eron model of $E$. The $G$-structure on $E$ extends to one on $\ol{E}$ by the N\'eron mapping property. Thus the special fiber $\ol{E}_0$ of $\ol{E}$ contains a subgroup isomorphic to $G$. The map $G \to \pi_0(\ol{E}_0)$ necessarily has a kernel, by the classification of special fibers of N\'eron models. Thus $G$ meets the identity component $\ol{E}_0^{\circ}$ non-trivially. But this is a contradiction, since $\ol{E}_0^{\circ} \cong \mathbf{G}_a$ is torsion-free.
\end{proof}

\begin{proof}[Second proof]
Let $X$ be the moduli space of generalized elliptic curves with $G$-structure, in the sense of Deligne--Rapoport. Then $X$ is a proper scheme. It follows that $E/K$ extends to a generalized elliptic curve $\ol{E}/A$ with $G$-structure. The identity component of the smooth locus of the special fiber of $\ol{E}$ is either an elliptic curve or a torus, and so $E$ has semi-stable reduction.
\end{proof}

\begin{proof}[Proof of Proposition~\ref{prop1}]
By the lemma, the universal curve $\mathcal{E}$ has semi-stable reduction at all places of $\mathbf{P}^1_{\mathbf{Q}}$. Since the equation \eqref{univeq} for $\mathcal{E}$ is minimal at all finite places (in the sense of \cite[Ch.~VII \S 1]{silverman}), it follows that $f$ and $g$ are coprime (see \cite[Ch.~VII, Prop.~5.1]{silverman} and \cite[Exc.~7.1]{silverman}). Now, \eqref{univeq} is not necessarily minimal at infinity. The minimal equation is given by
\begin{displaymath}
y^2=x^3+t^{-4\ell} f(t)+t^{-6\ell} g(t)
\end{displaymath}
where $\ell$ is minimal so that $t^{-4\ell} f(t)$ and $t^{-6\ell} g(t)$ have non-negative valuation at $\infty$. (Note: the valuation at $\infty$ is $-\deg$.) Now, $Y \subset \mathbf{P}^1$ is exactly the locus where $\mathcal{E}$ has good reduction, by universality. (Alternatively, the Deligne--Rapoport theory shows that $\mathcal{E}$ extends to a genuinely generalized elliptic curve over $\infty$). Thus $\mathcal{E}$ has multiplicative reduction at $\infty$, and so both $t^{-4\ell} f(t)$ and $t^{-6\ell} g(t)$ have valuation zero at $\infty$ \cite[Exc.~7.1]{silverman}, which exactly says that $\deg(f)=4\ell$ and $\deg(g)=6\ell$.

Now, the $j$-invariant of the family $\mathcal{E}$ is (up to a scalar)
\begin{displaymath}
\frac{f^3}{4f^3+27g^2}.
\end{displaymath}
We regard this as a self-map of $\mathbf{P}^1$. Since $f$ and $g$ are coprime, it has degree $12\ell$. On the other hand, the degree of the map $j \colon Y \to \mathbf{P}^1$ is the number of points in a typical fiber over a complex point. If $E/\mathbf{C}$ is an elliptic curve, the fiber of $j$ over $E$ is the set of isomorphism classes of pairs $(E, i)$ where $i \colon G \to E$ is a $G$-structure. Generically, $\Aut(E)=\{\pm 1\}$, and so the pairs $(E, i)$ and $(E, -i)$ are isomorphic and there are no other identifications. Thus the fiber of $j$ over $E$ has $k/2$ points, and so $12\ell=k/2$.
\end{proof}

Now, in the notation of Table~\ref{f:params}, we have $r=4\ell$, $s=6\ell$, $n=\ell$, and $m=1$. As $\ell=k/24$, it suffices to compute $k$. When $G=\mathbf{Z}/N\mathbf{Z}$, we have $k=N^2 \prod_{p|N} (1-p^{-2})$; and when $G=\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/N\mathbf{Z}$ we have $k=2N^2 \prod_{p \mid N} (1-p^{-2})$. This establishes the row in Table~\ref{f:params} corresponding to $G$. Theorem~\ref{thm:main2} for $G$ then follows from Theorem~\ref{general}.

\subsection{The case $\mathbf{Z}/3\mathbf{Z}$}
\label{ss:z3}

We now treat the case $G=\mathbf{Z}/3\mathbf{Z}$. There is a minor complication owing to the fact that $\mathcal{Y}_1(3)$ is not a scheme. Since we will treat this group more thoroughly in \S \ref{sec:asymptotics}, we do not include all the details here.

Call a pair $(E, P)$ consisting of an elliptic curve $E$ over a field $k$ of characteristic 0 and a 3-torsion point $P$ \emph{exceptional} if there is a non-trivial automorphism of $E$ over $\ol{k}$ fixing $P$. Let $\rho=e^{2\pi i/3}$, let $E=\mathbf{C}/\mathbf{Z}[\rho]$, and let $P$ be the 3-torsion point on $E$ given by $(1-\rho)/3$. Then $(E, P)$ is exceptional. Furthermore, it is the unique exceptional pair over $\mathbf{C}$, up to isomorphism.

There is a moduli space $Y$ of unexceptional pairs, which can be identified with an open subvariety of $\mathbf{A}^1$. (In fact, $Y$ is the complement of the single exceptional point in the stack $\mathcal{Y}_1(3)$.) Let $\mathcal{E}$ be the universal elliptic curve over $Y$.

\begin{lemma}
\label{lem:z3}
Let $E/\mathbf{Q}$ be an elliptic curve and let $P$ be a rational point of order 3. Then $E$ admits an equation of the form
\begin{displaymath}
y^2+axy+by = x^3
\end{displaymath}
with $a,b \in \mathbf{Q}$ such that $P=(0,0)$. The pair $(E,P)$ is exceptional if and only if $a=0$.
\end{lemma}

\begin{proof}
The first statement is well-known; see \cite[Tab.~3]{kubert}, for instance. The second is left to the reader
\end{proof}

\begin{proposition}
For an appropriate embedding $Y \to \mathbf{A}^1$, the universal family $\mathcal{E}$ is given by $y^2=x^3+f(t)x+g(t)$ with $f(t)=2t-\tfrac{1}{3}$ and $g(t)=t^2+\tfrac{2}{3}t+\tfrac{2}{27}$.
\end{proposition}

\begin{proof}
The previous lemma can, in fact, be applied to $\mathcal{E}$, and shows that it admits an equation of the form $y^2+2axy+2by=x^3$ where $a$ and $b$ are functions on $Y$, with $a$ invertible. Changing $y$ to $y-(ax+b)$ yields the equation $y^2=x^3+(ax+b)^2$. Changing $(x,y)$ to $(a^2x,a^3y)$ now gives $y^2=x^3+(x+t)^2$ with $t=b/a^3$. Finally, changing $x$ to $x-\tfrac{1}{3}$ yields the stated equation.
\end{proof}

Let $N^1(X)$ (resp.\ $N^2(X)$) be the number of (isomorphism classes of) elliptic curves $E/\mathbf{Q}$ of height at most $X$ admitting a 3-torsion point $P$ such that $(E,P)$ is not (resp.\ is) exceptional. Combining the above proposition with Theorem~\ref{general}, we see that $X^{1/3} \lesssim N^1(X) \lesssim X^{1/3}$. By Lemma~\ref{lem:z3}, an exceptional curve admits an equation of the form $y^2=x^3+b^2$ with $b \in \mathbf{Z}$. Clearly then, $N^2(X) \lesssim X^{1/4}$. Finally, we have the obvious bounds $N^1(X) \le N'_G(X) \le N^1(X)+N^2(X)$. This proves $X^{1/3} \lesssim N'_G(X) \lesssim X^{1/3}$, which establishes Theorem~\ref{thm:main2} for $G$.

\section{The group $\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z}$}\label{sec:Zmod2timesZmod2}

We begin with a variant of Proposition~\ref{bd}.

\begin{proposition}
\label{bd-var}
Let $f,g \in \mathbf{Q}[t]$ be coprime polynomials of degrees $r$ and $s$. Assume one of $r$ or $s$ is positive. Write
\begin{displaymath}
\max \left( \frac{r}{2}, \frac{s}{3} \right)=\frac{n}{m}
\end{displaymath}
with $n$ and $m$ coprime. Assume $n=1$ or $m=1$. Let $S(X)$ be the set of pairs $(A,B) \in \mathbf{Z}^2$ satisfying the following conditions:
\begin{itemize}
\item $4A^3+27B^2 \ne 0$.
\item $|A|<X^{1/3}$ and $|B|<X^{1/2}$.
\item $\gcd(A^3,B^2)$ is not divisible by any 12th power.
\item There exist $u,t \in \mathbf{Q}$ such that $A=u^2 f(t)$ and $B=u^3 g(t)$.
\end{itemize}
Define
\begin{displaymath}
h(X) = \begin{cases} X^{(m+1)/6n} & \textrm{if $m+1>n$} \\
X^{1/6} \log(X) & \textrm{if $m+1=n$} \\
X^{1/6} & \textrm{if $m+1<n$} \end{cases}
\end{displaymath}
Then $h(X) \lesssim \# S(X) \lesssim h(X)$.
\end{proposition}

\begin{proof}
We only sketch the proof, as the details are similar to the proof of Proposition~\ref{bd}. We begin with the upper bound. A version of Lemma~\ref{bd2} holds, but now one only has $C_p=1$ for $p \gg 0$ --- the reason for this is that putting an extra $p$ into $u$ only adds two $p$'s to $A$ and three to $B$, and so does not contribute a $p^{12}$ in $\gcd(A^3, B^2)$. The analog of Lemma~\ref{bd3} then says that we can write $t=a/b^m$, with $\gcd(a, b^m)$ not divisible by any $m$th powers, and $u=qcb^n$, where $q$ belongs to a finite set and $c$ is squarefree. The analog of Lemma~\ref{bd4} yields the inequalities $|ca^{n/m}|\lesssim X^{1/6}$ and $|cb^n| \lesssim X^{1/6}$. To count the number of possibilities, note that for any given $c$ there are at most $X^{m/6n}/c^{m/n}$ possibilities for $a$ and $X^{1/6n}/c^{1/n}$ possibilities for $b$, yielding $X^{(1+m)/6n}/c^{(1+m)/n}$ total possibilities. Now integrate over $c$ (up to $X^{1/6}$) to obtain the upper bound.

We now turn to the lower bound. We only treat the case $m+1>n$, as that is the only one we need and the others are a bit more subtle. It turns out that, in this case, we can find enough points with $c=1$ to establish the bound. More precisely, let $\kappa>0$ be a small constant, and consider the set of pairs of coprime integers $(a,b)$ such that $|a^{n/m}|<\kappa X^{1/6}$ and $|b^n|<\kappa X^{1/6}$ and $\Delta \ne 0$. The number of such pairs is $\gtrsim X^{(m+1)/6n}$. The analog of Lemma~\ref{bd5} shows that $\gcd(A^3, B^2)$ is of the form $\alpha^6 \beta$, where $\beta$ divides a fixed integer $D$, and $\alpha$ is square-free. This is sufficient for the application of Lemma~\ref{lem:S3toS}.
\end{proof}

\begin{proposition}
Let $f(t)=-\frac{1}{3} (t^2-t+1)$ and $g(t)=\tfrac{1}{27} (-2t^3+3t^2+3t-2)$. Suppose $E/\mathbf{Q}$ is an elliptic curve given by an equation $y^2=x^3+Ax+B$. Then all 2-torsion points of $E$ are rational if and only if there exist $u,t \in \mathbf{Q}$ such that $A=u^2f(t)$ and $B=u^3g(t)$.
\end{proposition}

\begin{proof}
It is well-known that all of the 2-torsion of $E$ is rational if and only if it admits an equation of the form $y^2=x(x-a)(x-b)$ with $a,b \in \mathbf{Q}$. Changing $x$ to $x+\tfrac{1}{3}(a+b)$, we obtain the equation
\begin{displaymath}
y^2=x^3-\tfrac{1}{3} (a^2-ab+b^2)+\tfrac{1}{27} (-2a^3+3a^2b+3ab^2-2b^3).
\end{displaymath}
We thus see that all of the 2-torsion of $E$ is rational if and only if there exist $a,b,v \in \mathbf{Q}$ such that
\begin{displaymath}
A=-\frac{v^4}{3} (a^2-ab+b^2), \qquad B=\frac{v^6}{27} (-2a^3+3a^2b+3ab^2-2b^3).
\end{displaymath}
Changing $(a,b)$ to $(a/v^2, b/v^2)$ shows that we can take $v=1$. The above equations are therefore equivalent to $A=u^2f(t)$ and $B=u^3g(t)$ with $u=b$ and $t=a/b$. (Note: a solution with $b=0$ is impossible, as that would give $4A^3+27B^2=0$.)
\end{proof}

\begin{proposition}
Let $G=\mathbf{Z}/2\mathbf{Z} \times \mathbf{Z}/2\mathbf{Z}$. Then $X^{1/3} \lesssim N'_G(X) \lesssim X^{1/3}$.
\end{proposition}

\begin{proof}
This follows from the above two propositions. Note that, in the notation of Proposition~\ref{bd-var}, we have $m=n=1$, and so $(m+1)/6n=1/3$.
\end{proof}

\section{Asymptotics for the trivial group, $\mathbf{Z}/2\mathbf{Z}$, and $\mathbf{Z}/3\mathbf{Z}$}\label{sec:asymptotics}

In this section, we derive asymptotics for $N_G(X)$ when $G$ has order at most 3. The result is stated in Theorem~\ref{lem:main_asymp_lemma} below and is a refinement of Theorem~\ref{thm:asymptotics}. We take a unified approach by first counting integer points in explicit semi-algebraic sets using the Principle of Lipschitz \cite{davenport} and then sieving. Note that these asymptotics imply Theorem~\ref{thm:main} for these cases.

\begin{lemma}
Let $E/\mathbf{Q}$ be an elliptic curve given by $y^2=x^3+Ax+B$ with $A$ and $B$ in $\mathbf{Z}$. Then,
\begin{enumerate}
	\item\label{lempart:2tors} $E$ has a rational point of order 2 if and only if there exists $b \in \mathbf{Z}$ such that $B=b^3+Ab$;
	\item\label{lempart:3tors} $E$ has a rational point of order 3 if and only if there exists $a, b \in \mathbf{Z}$ such that $A=6ab+27 a^4$ and $B=b^2-27a^6$.
\end{enumerate}
\end{lemma}

\begin{proof}
As is well-known, $E$ has a rational 2-torsion point if and only if $x^3+Ax+B$ has a rational root. This yields part (\ref{lempart:2tors}). Part (\ref{lempart:3tors}) is proved in \cite[\S2]{GST}
\end{proof}

This lemma suggests we study integer points in the following three semi-algebraic regions:
\begin{align*}
	R_1(X)&=\{(a,b)\in\mathbf{R}^2:|a|<X^{1/3}\text{ and }|b|<X^{1/2}\},\\
	R_2(X)&=\{(a,b)\in\mathbf{R}^2:|a|<X^{1/3}\text{ and }|b^3+ab|<X^{1/2}\}, \textrm{and}\\
	R_3(X)&=\{(a,b)\in\mathbf{R}^2:|6ab+27a^4|<X^{1/3}\text{ and }|b^2-27a^6|<X^{1/2}\}.
\end{align*}

The Principle of Lipschitz states that the number of integer points $r_{i}(X)$ in $R_i(X)$ is given by its area up to an error given by the maximal length of its projections onto the coordinate axes. From this we immediately see that $r_1(X)=4X^{5/6}+O(X^{1/2})$. The next lemma will allow us to compute the main term and the error for $i=2,3$. Before stating it, we must introduce several quantities.

Let $\alpha_\pm$ be the (unique) real root of $x^3\pm x-1$. Let $f_\pm=3x^4\pm6x^2+12x-1$, each of which has one negative and one positive root. We denote these four roots by
\begin{align*}
	&\alpha_4\approx0.08011\quad\text{and}\quad\alpha_1\approx-1.22259\quad\text{(roots of }f_+);\\
	&\alpha_3\approx0.08711\quad\text{and}\quad\alpha_0\approx-2.01637\quad\text{(roots of }f_-).
\end{align*}
Also, define $\alpha_2=-\sqrt{3}$ and $\alpha_5=\sqrt{3}$. Let $\displaystyle \beta_i=\sgn(\alpha_i)\sqrt{|\alpha_i|/3}$ except take $\beta_3$ to be negative. Then, $\beta_i<\beta_j$ for $i<j$. Define the hyperelliptic integrals
\[
	I_+=\int_{\beta_3}^{\beta_4}\sqrt{1+27a^6}da\approx0.33383
\]
and
\[
	I_-=\int_{\beta_0}^{\beta_1}\sqrt{-1+27a^6}da\approx0.32030.
\]
Finally, let
\[
	A_\pm(a)=\frac{\pm X^{1/3}-27a^4}{6a}\quad\text{ and }\quad B_\pm(a)=\sqrt{\pm X^{1/2}+27a^6}.
\]
We leave the verification of the following lemma to the reader.
\begin{lemma}\mbox{}
	\begin{enumerate}
		\item We have $(a,b)\in R_2(X)$ if and only if
\begin{align*}
& |b|<\alpha_+X^{1/6}\text{ and }|a|^3<X\text{, or} \\
& \alpha_+X^{1/6}\leq|b|<\alpha_-X^{1/6}\text{ and }-X^{1/3}<a<\frac{X^{1/2}}{|b|}-b^2.
\end{align*}
Thus,
\[
	\operatorname{Area}(R_2(1))=2\log(\alpha_-/\alpha_+)+\tfrac{4}{3} (\alpha_++\alpha_-).
\]
		\item Let $R^+_3(X)=R_3(X)\cap\{b\geq0\}$. Then, $(a,b)\in R^+_3(X)$ if and only if
\begin{displaymath}
\beta_iX^{1/12}<a<\beta_{i+1}X^{1/12} \qquad \textrm{and} \qquad g_i(a)<b<f_i(a),
\end{displaymath}
for some $0\leq i\leq 4$, where $f_i(a)$ and $g_i(a)$ are as in the following table.
\begin{center}
{\tabulinesep=1.2mm
\begin{tabu}{|c||c|c|c|c|c|} \hline
$i$ & 0 & 1 & 2 & 3 & 4 \\
\hline
$f_i$ & $A_-(a)$ & $A_-(a)$ & $A_-(a)$ & $B_+(a)$ & $A_+(a)$ \\
\hline
$g_i$ & $B_-(a)$ & $A_+(a)$ & 0 & 0 & 0 \\
\hline
\end{tabu}
}
\end{center}
Thus,
\[
	\operatorname{Area}(R_3^+(1))=I_+-I_-+\frac{1}{6}\log\left(\displaystyle\frac{\beta_0\beta_1\beta_5}{\beta_2\beta_3\beta_4}\right)+\frac{9}{8}\left(\beta_0^4+\beta_2^4+\beta_4^4-\beta_1^4-\beta_3^4-\beta_5^4\right).
\]
Furthermore, $(a,b)\in R_3(X)$ if and only if $(-a,-b)\in R_3(X)$. In particular, $|b|<2\sqrt{7}X^{1/4}$ for all $(a,b)\in R_3(X)$.
	\end{enumerate}
\end{lemma}

Let $(d_1,d_2,d_3)=(6/5,2,3)$ and $(e_1,e_2,e_3)=(2,3,4)$. Note that for each $i$, the region $R_i(X)$ is homogeneous in $X$ in the sense that
\[
	\operatorname{Area}(R_i(X))=X^{1/d_i}\operatorname{Area}(R_i(1)).
\]
Therefore, combining the above lemma with the Principal of Lipschitz yields
\begin{equation}\label{eqn:Ri_count}
	r_i(X)=\operatorname{Area}(R_i(1))X^{1/d_i}+O(X^{1/e_i}).
\end{equation}
All that remains is to address the overcounting that occurs in $R_i(X)$. We do this in three lemmas.
\begin{lemma}
	Define $(A,B)=T_i(a,b)$ by
	\begin{enumerate}
		\item $T_1(a,b)=(a,b)$, if $i=1$;
		\item $T_2(a,b)=(a,b^3+ab)$, if $i=2$;
		\item $T_3(a,b)=(6ab+27a^4,b^2-27a^6)$, if $i=3$.
	\end{enumerate}
	Then, the number of $(a,b)\in R_i(X)\cap\mathbf{Z}^2$ such that $4A^3+27B^2=0$ is $O(X^{1/6})$.
\end{lemma}
\begin{proof}
	Suppose $4A^3+27B^2=0$. First, consider $i=1$. Then, $4a^3=-27b^2$, which implies $a^\prime=-a/3\in\mathbf{Z}$ and $b^\prime=b/2\in\mathbf{Z}$. Since ${a^\prime}^3={b^\prime}^2<X$, the number of $(a,b)$ with discriminant zero is on the order of the number of sixth powers less than $X$, namely it is $O(X^{1/6})$. For $i=2$, B\'{e}zout's theorem says there are at most 3 pairs $(a,b)$ giving $(A,B)$; for $i=3$, the number is 24. Appealing to the argument for $i=1$ again yields the bound $O(X^{1/6})$.
\end{proof}
In view of this lemma, we may (and do) redefine $R_i(X)$ by removing from it the discriminant zero locus, while preserving the truth of \eqref{eqn:Ri_count}.
\begin{lemma}
	The number of $(A,B)$ for which there is more than one $(a,b)\in R_i(X)\cap\mathbf{Z}^2$ with $(A,B)=T_i(a,b)$ is $O(X^{1/e_i})$.
\end{lemma}
\begin{proof}
	This is non-trivial only for $i=2,3$. Consider first $i=2$. Given an $E/\mathbf{Q}$ with equation $y^2=x^3+Ax+B$ and an integer $b$ such that $B=b^3+Ab$, one obtains a non-trivial 2-torsion point $(-b,0)\in E(\mathbf{Q})$. Therefore, if there is another $b^\prime\in\mathbf{Z}$ such that $B=b^{\prime3}+Ab^\prime$, then the torsion subgroup contains $\mathbf{Z}/2\mathbf{Z}\times\mathbf{Z}/2\mathbf{Z}$. But we proved in \S\ref{sec:Zmod2timesZmod2} that the number of such elliptic curves is $O(X^{1/3})$. The case $i=3$ is similar. Indeed, each pair $(a,b)\in\mathbf{Z}^2$ such that $(A,B)=T_3(a,b)$ yields a pair of non-trivial 3-torsion points $(3a^2, \pm(9a^3+b))\in E(\mathbf{Q})$, which are negatives of each other. Ignoring the cases where $ab=0$ (which is $O(X^{1/4})$ many), a second pair $(a^\prime,b^\prime)$ that gives the same $(A,B)$ yields two \emph{new} non-trivial rational 3-torsion points. But this is impossible: an elliptic curve over $\mathbf{Q}$ cannot contain two copies of $\mathbf{Z}/3\mathbf{Z}$ in its Mordell--Weil group.
\end{proof}
Let $\mathfrak{E}_i(X)=\{\text{equations }E_{A,B}:y^2=x^3+Ax+B:(A,B)=T_i(a,b),(a,b)\in R_i(X)\cap\mathbf{Z}^2\}$. The above lemma implies that $\#\mathfrak{E}_i(X)=r_i(X)+O(X^{1/e_i})$. We now sieve out non-minimal equations to make our way from $\mathfrak{E}_i(X)$ to $N_{\mathbf{Z}/i\mathbf{Z}}(X)$.
\begin{theorem}\label{lem:main_asymp_lemma}
	For $i=1,2,$ and $3$,
	\[
		N_{\mathbf{Z}/i\mathbf{Z}}=\frac{\operatorname{Area}(R_i(1))}{\zeta(12/d_i)}X^{1/d_i}+O(X^{1/e_i}).
	\]
\end{theorem}
\begin{proof}
	Given an equation $E_{A,B}\in\mathfrak{E}_i(X)$, let $d$ be the largest twelfth power dividing $\gcd(A^3,B^2)$. Then, the curve defined by $E_{A,B}$ is also given (uniquely) by the minimal equation $E_{d^{-4}A,d^{-3}B}$. Sieving yields
	\begin{align*}
		N_{\mathbf{Z}/i\mathbf{Z}}^\prime(X)	&=\sum_{d=1}^{X^{1/12}}\mu(d)\#\mathfrak{E}_i(d^{-12}X)\\
							&=\sum_{d=1}^{X^{1/12}}\mu(d)\left(\operatorname{Area}(R_i(1))\frac{X^{1/d_i}}{d^{12/d_i}}+O(d^{-12/d_i}X^{1/e_i})\right)\\
							&=\operatorname{Area}(R_i(1))X^{1/d_i}\sum_{d=1}^{X^{1/12}}\frac{\mu(d)}{d^{12/d_i}}+O(X^{1/e_i})\\
							&=\frac{\operatorname{Area}(R_i(1))}{\zeta(12/d_i)}X^{1/d_i}+O(X^{1/e_i}).
	\end{align*}
	Since $N_{\mathbf{Z}/i\mathbf{Z}}(X)-N_{\mathbf{Z}/i\mathbf{Z}}^\prime(X)$ is within the error, the theorem is proved.
\end{proof}
	As remarked in the introduction, the case $i=1$ already appears in \cite[Lemma~4.3]{brumer}. In \cite[\S2]{grant}, the case $i=2$ is shown with the error bound $O(X^{\frac{1}{3}+\epsilon})$, for all $\epsilon>0$. The case $i=3$ appears to be new, and we have found no further cases in the literature either.

Number Theory (math.NT); Algebraic Geometry (math.AG)