On a Balanced Property of Compositions

Abstract: Let $S$ be a finite set of positive integers with largest element $m$. Let us randomly select a composition $a$ of the integer $n$ with parts in $S$, and let $m(a)$ be the multiplicity of $m$ as a part of $a$. Let $0\leq r<q$ be integers, with $q\geq 2$, and let $p_{n,r}$ be the probability that $m(a)$ is congruent to $r$ modulo $q$. We show that if $S$ satisfies a certain simple condition, then $\lim_{n\to \infty} p_{n,r} =1/q$. In fact, we show that an obvious necessary condition on $S$ turns out to be sufficient.

\section{Introduction} A {\em composition} of the positive integer $n$
is a sequence $(a_1,a_2,\cdots ,a_k)$ of positive integers so that
$\sum_{i=1}^k a_i=n$. The $a_i$ are called the {\em parts} of the composition.
It is well-known \cite{bonaw}
 that the number of compositions of $n$ into $k$ parts
is ${n-1\choose k-1}$.
From this fact, it is possible to prove the following.
Let $0\leq r<q$ be integers, with
$q\geq 2$, and let $P_{n,r}$ be the probability of the event that
the number of parts of a randomly selected composition of $n$ is congruent to
$r$ modulo $q$. Then  $\lim_{n\rightarrow \infty} P_{n,r}
=1/q$. In other words,
\[\lim_{n\rightarrow \infty}
 \frac{\sum_{i=0}^{\lfloor (n-1)/q \rfloor} {n-1\choose iq+r}}{2^{n-1}}=
\frac{1}{q}.\]
For $q=2$, this follows from the well-known fact that the number of
even-sized subsets of a non-empty
finite set is equal to the number of odd-sized
subsets of that set. For $q>2$, the statement can be proved, for example,
by the method we will use in this paper. The special cases of $q=3$ and $q=4$
appear as Exercises 4.41 and 4.42 in \cite{bonaw}.
In other words, all residue classes
are equally likely to occur. We will refer to this phenomenon by saying
that the number of part sizes of a randomly selected composition of $n$ is
{\em balanced}.

Now let us impose a restriction on the {\em part sizes} of the compositions
of $n$ that form our sample space by requiring that all part sizes come
from a finite set $S$. Is it still true that the number of part
 sizes of a randomly selected composition of $n$ is balanced? That will
certainly depend on the restriction we impose on the part sizes.
For instance, if the set $S$ of allowed parts consists of odd numbers only,
then the number of part sizes will not be balanced. Indeed, if $q=2$,
then $P_{n,0}=1$ if $n$ is even, and $P_{n,0}=0$ if $n$ is odd.
Let $m$ be the largest element of the set $S$ of allowed parts. It turns
out that it is easier to (directly)
work with the multiplicity $m(a)$ of $m$ as a part of the randomly
selected composition $a$ than with its number of parts.
For the special case when $S$ has only two elements, the
 results obtained for $m(a)$ can then  be translated back to results on
the number of parts of $a$.

In this paper, we prove that if $S$ satisfies a certain obviously necessary
condition, then the parameter $m(a)$ is balanced, as described in the
 abstract.
That is, the remainder of $m(a)$ modulo $q$ is equally likely to take all
possible values.

\section{The Strategy}

Let $S=\{s_1,s_2,\cdots ,s_k=m\}$ be a finite set of positive integers
with at least two elements. Let us assume without loss of generality
 that no integer larger than 1
divides all $k$ elements of $S$.  Clearly, if $s_1,s_2, \cdots ,s_{k-1}$ are
all divisible by a certain prime $h>1$, then the multiplicity
$m(a)$ of $m=s_k$ as a part of a composition $a$ of $n$ is restricted.
Indeed, $n-m(a)m$ must be divisible by $h$. In particular, if $n$ is divisible
by $h$, then $m(a)m$ is  divisible by $h$, and so $m(a)$ must be divisible
by $h$. Therefore, the parameter $m(a)$ is not balanced. Indeed, if $n$
is divisible by $h$, and we choose
$q=h$, then $p_{n,0}=1$, and $p_{n,r}=0$ for $r\neq 0$, while if $n$ is
not divisible by $h$ and $q=h$, then $p_{n,0}=0$.

So for $m(a)$ to be a balanced parameter, it is {\em necessary} for $S$ to
satisfy the condition that its smallest $k-1$ elements do not have
a proper common divisor. In the rest of this paper, we prove that this
condition is at the same time {\em sufficient} for $m(a)$ to be balanced.

{\em Unless otherwise stated}, let $S$ be a finite set of positive integers
 with at least two elements, and let $S=\{s_1,s_2,\cdots ,s_k=m\}$, where
the $s_i$ are listed in increasing order. So $m$ is the largest element of
$S$. {\em Unless otherwise stated}, let us also assume  that no integer
larger than 1 is a divisor of all of $s_1,s_2, \cdots ,s_{k-1}$.
Note that this means that if $|S|=2$, then $s_1=1$.

 For a fixed positive integer $n$, let
$A_{S,n}(x)$ be the ordinary generating function of all compositions of the
integer $n$ into parts in $S$ according to their number of parts equal to
$m$. In other words, \[A _{S,n}(x)=\sum_a x^{m(a)}=
\sum_{d}a_{S,n,d}x^d,\] where
$a$ ranges over all compositions of $n$ into parts in $S$, and $m(a)$ is
the multiplicity of $m$ as a part in $a$. On the far right, $a_{S,n,d}$ is
the number of compositions $a$ of $n$ into parts in $S$ so that $m(a)=d$.

\begin{example} Let $S=\{1,3\}$. Then the first few polynomials $A_n(x)=
A_{S,n}(x)$
are as follows.
\begin{itemize}
\item $A_0(x)=A_1(x)=A_2(x)=1$,
\item $A_3(x)=x$, $A_4(x)=2x+1$, $A_5(x)=3x+1$.
\item $A_6(x)=x^2+4x+1$, $A_7(x)=3x^2+5x+1$, $A_8(x)=6x^2+6x+1$.
\end{itemize}
\end{example}

Let $q\geq 2$ be a positive integer, and let $0\leq r\leq q-1$.
 Let $A_{S,n,r}$ be the number of compositions $a$ of  $n$ with parts in $S$
so that $m(a)$ is congruent
 to $r$ modulo $q$. So
\[A_{S,n,r}=a_{S,n,r}+a_{S,n,q+r}+\cdots +a_{S,n,\lfloor n/q \rfloor q+r} .\]
 In order to simplify the presentations of our results,
we will first discuss the special case when $n$ is divisible by $q$ and
$r=0$. Let $w$ be a primitive $q$th root of unity. Then
\begin{eqnarray}  \label{unitroots}
\sum_{t=0}^{q-1} A_{S,n}(w^t) & = & \sum_{t=0}^{q-1}
\sum_{d= 0}^{n/m} a_{S,n,d} w^{td} \\
 & = & \sum_{d=0}^{n/m} \sum_{t=0}^{q-1}
a_{S,n,d} w^{td} \\
 & = & \sum_{d=0}^{n/m} a_{S,n,d} \sum_{t=0}^{q-1} w^{td}.
\end{eqnarray}

Using the summation formula of a geometric progression, we get that
\[  \sum_{t=0}^{q-1} (w^{d})^t= \left\{ \begin{array}{l@{\ }l}
0 \hbox{ if $w^d\neq 1$, that is,  if $q\nmid d$}, \\
\\ q \hbox{ if $w^d =1$, that is, if $q|d$.
}
\end{array}\right.
\]
Therefore, (\ref{unitroots}) reduces to
\[
\sum_{t=0}^{q-1} A_{S,n}(w^t)  =
q\cdot \sum_{j=1}^{n/q} a_{S,n,jq}
 =  qA_{S,n,0},\]
\begin{equation}
\label{forma0} \frac{1}{q} \sum_{t=0}^{q-1} A_{S,n}(w^t)=A_{S,n,0}.
 \end{equation}

So in order to find the approximate value of $A_{S,n,0}$, it suffices to
find the approximate values of $A_n(w^t)$, for $0\leq t\leq r-1$, and
for a primitive root of unity $w$. The number $A_{S,n}$
 of {\em all} compositions
of $n$ into parts in $S$ is equal to $A_n(1)$, so we will need that value
as well, in order to compute the ratio $A_{S,n,0}/A_{S,n}$.

Finally, note that if $n$ is not divisible by $q$, but $r=0$,
then the same argument
applies, and $\frac{1}{q} \sum_{t=0}^{q-1} A_{S,n}(w^t)
=A_{S,n,0}$ still holds.
If $r\neq 0$, then instead of computing $\sum_{t=0}^{q-1} A_{S,n}(w^t)$, we
compute \[T_n(w)=\sum_{t=0}^{q-1} A_{S,n}(w^t)w^{-rt}
=\sum_{d= 0}^{n/m} a_{S,n,d} \sum_{t=0}^{q-1} w^{t(d-r)}.\]
This shows that the coefficient of $w^k$ in $T_n(w)$ is 0, unless
$w^{d-r}=1$, that is, unless $d-r$ is divisible by $q$. If $d-r$ is divisible
by $q$, then this coefficient is $q$. This shows again that
 \begin{equation}
\label{generalc} \frac{1}{q} \sum_{t=0}^{q-1} A_{S,n}(w^t)w^{-tr}=A_{S,n,r}.
\end{equation}
Therefore, computing $A_{S,n}(w^t)$
 will be useful in the general case as well.

\section{Linear Recurrence Relations}
In order to compute the values of $A_{S,n}(x)$ for various values of $x$,
we can keep $x$ fixed, and let $n$ grow.
The connection among the polynomials $A_{S,n}(x)$
 is explained by the following Proposition.

\begin{proposition} Let $S=\{s_1,s_2,\cdots ,s_k=m\}$, with $k\geq 2$. Then
for all positive integers $n\geq 3$, the polynomials $A_{S,n}(x)$ satisfy the
recurrence relation
\begin{equation} \label{recur}
A_{S,n}(x)=A_{S,n-s_1}(x)+A_{S,n-s_2}(x)+\cdots +A_{S,n-s_{k-1}}(x)+
xA_{S,n-m}(x).
\end{equation}
\end{proposition}

\begin{proof}
Let $a$ be a composition of $n$ with parts in $S$. If the first part of
$a$ is $s_i$, for some $i\in [1,k-1]$, then the rest of $a$ forms a
composition of $n-s_i$ with parts in $S$ in which the multiplicity of $m$ as
a part is still $m(a)$. These compositions of $n$
 are counted by $A_{S,n-s_i}(x)$.
If the first part of $a$ is $m$, then the rest of $a$ forms a
composition of $n-m$ with parts in $S$ in which the multiplicity of $m$ as
a part is $m(a)-1$. These compositions of $n$  are counted by
$xA_{S,n-m}(x)$.
\end{proof}

\begin{example} If $S=\{1,3\}$, then (\ref{recur}) reduces to
\begin{equation} \label{srecur} A_{S,n}(x)= A_{S,n-1}(x)+x A_{S,n-3}(x).
\end{equation}
\end{example}

For a {\em fixed} real number $x$, the recurrence relation (\ref{recur})
becomes a recurrence relation on real numbers. The solutions of such
 recurrence relations are described by the following well-known theorem.
(See, for instance, \cite{rosen}, Section 7.2.
)
\begin{theorem} \label{recthe}
Let \begin{equation}
\label{grec} a_n=c_1a_{n-1}+c_2a_{n-2}+\cdots +c_ka_{n-k} \end{equation}
 be a recurrence
relation,
where the $c_i$ are complex constants. Let $\alpha_1,\alpha_2,\cdots,
\alpha_t$ be the distinct roots of the characteristic
equation \begin{equation} \label{chareq}
z^k-c_1z^{k-1}-c_2z^{k-2}-\cdots -c_k=0,\end{equation}
and let $M_i$ be the multiplicity of $\alpha_i$.
Then the sequence $a_0,a_1,\cdots $ of complex numbers satisfies (\ref{grec})
if and only if there exist constants $b_1,b_2, \cdots ,b_k$ so that for
all $n\geq 0$, we have
\begin{equation}
\label{fgrec} a_n=b_1\alpha_1^n+b_2 n\alpha_1^n+ \cdots
+b_{M_1}n^{M_1-1}\alpha_1^n+ b_{M_1+1}\alpha_2^n, \cdots
\end{equation}  \begin{equation}
\cdots +b_{M_1+M_2}n^{M_2-1}\alpha_2^n, \cdots, \cdots
+b_kn^{M_k-1}\alpha_k^n.\end{equation}

In other words, the solutions of (\ref{grec}) form a $k$-dimensional vector
space.
\end{theorem}

We will need the following consequence of Theorem \ref{recthe}.
\begin{corollary} \label{nonzero}
Let us assume that the sequence $\{a_n\}$ is a solution of (\ref{grec}) and
that there is no linear recurrence relation of a degree less than
$k$ that is satisfied by $ \{a_n\}$. Let us further assume that the
characteristic equation (\ref{chareq}) of (\ref{grec}) has a unique
root $\alpha_1$ of largest modulus. Then there is a {\em nonzero}
 constant $C$ so that
\[a_n=C\alpha_1^n + o(\alpha_1^n).\]
\end{corollary}

\begin{proof}
As $\{a_n\}$ does not satisfy a recurrence relation of a degree less than
$k$, we must have $c_1\neq 0$. As $|\alpha_1|>|\alpha_i|$ for $i\neq 1$,
the statement follows.
\end{proof}

Let us now apply Theorem \ref{recthe} to find
 the solution of (\ref{recur}) for a fixed $x$. The characteristic
equation of (\ref{recur}) is

\begin{equation} \label{polyeq}
f(z)=z^{m}-\sum_{i=1}^{k-1}  z^{m-s_{i}} - x
=0.\end{equation}

As explained in Section 2, we will need to compute $A_{S,n}(1)$ and also,
$A_{S,n}(w^t)$ for the case when $w\neq 1$ is a $q$th primitive
root of unity.  To that
end, we need to find the roots of the corresponding characteristic
equations. That is, we will compare the root of largest modulus
 of the characteristic
equation
\begin{equation} \label{forone}
f_1(z)=
z^{m}-\sum_{i=1}^{k-1}  z^{m-s_{i}} - 1=0
\end{equation}
and the root(s) of the largest modulus of the characteristic equation
\begin{equation} \label{forw}
f_w(z)=
z^{m}-\sum_{i=1}^{k-1}  z^{m-s_{i}} - w=0
\end{equation}

The following lemma, helping to compute root of largest
modulus of $f_1(z)$,
 is a special case of Exercise III.16 in \cite{polya}.

\begin{lemma} \label{first}
 The polynomial  $f_1(z)=z^{m}-\sum_{i=1}^{k-1}
 z^{m-s_{i}} - 1$
 has a unique positive real root $\alpha$.
\end{lemma}

\begin{proof} Let $\alpha$ be the smallest positive real root of $f_1(z)$.
 We know such a root exists since $f(0)=-1$ and $\lim_{z\rightarrow
\infty}f(z)=\infty$. We claim that then $f$ is strictly monotone increasing
on $[\alpha, \infty )$, implying that $f$ cannot have another positive
real root. Indeed, if $r>1$, then
\begin{eqnarray*} f(r\alpha)+1 & = & (r\alpha)^{m} - \sum_{i=1}^{k-1}
 (r\alpha)^{m-s_{i}}\\
& > & r^{m} \left(\alpha^{m}-\sum_{i=1}^{k-1}  \alpha^{m-s_{i}} \right ) \\
& = & r^m \\
& > & 1,
\end{eqnarray*}
and so $f(r\alpha)>0$.
\end{proof}

Now we address the problem of finding the roots of the characteristic
equation (\ref{polyeq}) in the case when $w\neq 1$ is a root of unity.
It turns out that it suffices to assume that $|w|=1$. (The following
Lemma is similar to Exercise III.17 in \cite{polya}.)

\begin{lemma} Let $\alpha$ be defined as in Lemma \ref{first}.
Let $w$ be any complex number satisfying  $w\neq 1$ and $|w|=1$.
Then all  roots of the polynomial $f_w(z)$ are of smaller modulus than
 $\alpha$.
\end{lemma}

\begin{proof} Let $y$ be a root of $f$. Then
\begin{eqnarray*} |y|^m & = & \left |w + \sum_{i=1}^{k-1} y^{m-s_i}
\right |\\
& \leq & 1+  \sum_{i=1}^{k-1} \left | y^{m-s_i} \right |.
\end{eqnarray*}

 Therefore, $f(|y|)\leq 0$. This implies that $|y|\leq \alpha$ since we have
seen in the proof of Lemma \ref{first} that $f(t)>0$ if $t>\alpha$.

Furthermore, in the last displayed inequality, the inequality is strict
unless for all $i$ so that $1\leq i\leq k-1$,
 the complex numbers $y^{m-s_i}$ have the same argument as $w$, and that
argument is the same as the argument of $y^m$. That happens only if
the complex numbers $y^{s_1},y^{s_2-s_1},\cdots y_{s_{k-1}-s_{k-2}}$ all
have argument 0, that is, when these numbers are positive real numbers.
However, that happens precisely when $s_1, s_2-s_1, \cdots, s_{k-1}-s_{k-2}$
are all multiples of the multiplicative order $o_y$
 of $y/|y|$ as a complex number.
That implies that $s_1,s_2,\cdots ,s_{k-1}$ are all divisible by $o_y$,
contradicting our hypothesis on $S$.
\end{proof}

The previous two lemmas show that the largest root of the
 characteristic equation for the
sequence $\{A_{S,n}(1)\}_{n\geq 0}$ is larger than the largest root(s) of
the characteristic equation for the sequence $\{A_{S,n}(w)\}_{n\geq 0}$ for
any complex number $w\neq 1$ with absolute value 1. Given formula
(\ref{fgrec}), in order to see that
the first sequence indeed grows faster than the second,
all we need to show is that the {\em coefficient} $b_1$ of $\alpha^n$
in (\ref{grec}) is not 0. (Here $\alpha$, the largest root of $f_1(z)$,
plays the role of $\alpha_1$ in (\ref{fgrec})). This is the content of the
next lemma.

\begin{lemma} \label{noshorter} Let $S=\{s_1,s_2,\cdots ,s_{k-1}\}$
 be any finite set of positive
integers (so for this Lemma, we do not require that $s_1,s_2,\cdots s_{k-1}$
 do not have a proper common divisor).
Then the sequence $\{A_{S,n}\}_{n\geq 0}=\{A_{S,n}(1)\}_{n\geq 0}$
 does not satisfies a linear recurrence
relation with constant coefficients and
less than $|S|+1$ terms. In other words, if $|S|=k$, then
 there do not exist
constants $c_2,c_3,\cdots ,c_k$ and positive integers $j_1,j_2,\cdots ,
j_{k-1}$ so that for all $n\geq 0$,
\[A_{S,n}=\sum_{i=1}^{k-1} c_i A_{S,n-j_i}.\]
\end{lemma}

\begin{proof}
Let us assume that $S$ is a minimal counterexample. It is then straightforward
to verify that $|S|>2$. Let $S'=S-m$, that is, the set obtained from
$S$ by removing the largest element of $S$. Then
\begin{equation} \label{reduction}
A_{S',n}=\sum_{i=1}^{k-1} c_i' A_{S',n-s_i},\end{equation}
and there is no shorter recurrence satisfied by $\{A_{S',n}\}$.

Now crucially, $A_{S',n}=A_{S,n}$ for all $n$ satisfying $0\leq n< m$.
So these sequences agree in $m-1\geq k-1$ values. So if $\{A_{S,n}\}$
satisfied a linear recurrence relation of degree $k-1$, that would have
to be the recurrence relation (\ref{reduction}). Indeed, by Theorem
\ref{recthe}, the solutions of (\ref{reduction}) form a $k$-dimensional
vector space, so knowing $k$ elements of a solution determines the whole
solution.
However, $\{A_{S,n}\}$ does not satisfy (\ref{reduction}) since
$A_{S,m}=A_{S',m}+1\neq A_{S',m}$, where the difference is caused by the
one-part composition $m$.
\end{proof}

Now we are in position to express the growth rate of $A_{S,n}=A_{S,n}(1)$.
\begin{proposition}
Let $\alpha$ be defined as in Lemma \ref{first}. Then
\[A_{S,n}(1)=C \alpha^n + o(\alpha^n),\] for some  nonzero
constant $C$.
\end{proposition}

\begin{proof} Immediate from Corollary \ref{nonzero} and
Lemma \ref{noshorter}.
\end{proof}

We can now compare the growth rates of $A_{S,n}(w)$ and $A_{S,n}(1)$.

\begin{lemma} \label{grrate}
Let $w\neq 1$ be any complex number so that $|w|=1$. Then
\[\lim_{n\rightarrow \infty} \frac{A_{S,n}(w)}{A_{S,n}(1)} =0. \]
\end{lemma}

\begin{proof}
Lemma (\ref{first}) shows that the unique positive root of the characteristic
equation (\ref{forone}) is larger than the absolute value of all
roots of the characteristic
equation (\ref{forw}). Therefore, $A_{S,n}(w)=O(n^k\beta^k)$, with
$\beta < \alpha$.
\end{proof}

Finally, we can use the results of this section to prove the balanced
properties of the numbers $A_{S,n,r}$.

\begin{theorem}
Let $q\geq 2$ be an integer, and let $r$ be an integer satisfying $0\leq r
\leq q-1$. Then
\[\lim_{n\rightarrow \infty} p_{n,r}=\lim_{n\rightarrow \infty}
\frac{A_{S,n,r}}{A_{S,n}}=\frac{1}{q}.\]
\end{theorem}

\begin{proof} Let us first address the case of $r=0$.
Dividing (\ref{forma0}) by $A_{S,n}$, we get  that
 \[A_{S,n,0}=\frac{1}{q}
 \sum_{t=0}^{q-1}\frac{A_{S,n}(w^t)}{A_{S,n}}.\]
However, Lemma \ref{grrate} shows that all but one of the $q$ summands
on the right-hand side converge to 0, and the
remaining one (the first summand) is equal to 1.

For general $r$, the only change is that instead of dividing both sides
of (\ref{forma0}) by $A_{S,n}$, we divide both sides of (\ref{generalc})
by $A_{S,n}$. As $|w|=1$, the result follows in the same way.
\end{proof}

\section{Further Directions}
Let $S=\{1,3\}$. Numerical evidence suggest that for all $n$, the polynomials
$A_{S,n}(x)$ have real roots only. Furthermore, numerical evidence also
suggests that the sequences of polynomials $\{A_{S,3n+r}(x)\}_{n\geq 0}$ form
a Sturm sequence for each of  $r=0,1,2$. (See \cite{wilfb} for the definition
and importance of Sturm sequences.) This raises the following intriguing
questions.

\begin{question} For which sets $S$ is it true that  the polynomials
$A_{S,n}(x)$ have real roots only?
\end{question}

\begin{question} For which sets $S$ is it true that  the polynomials
$A_{S,n}(x)$ can be partitioned into a few Sturm sequences?
\end{question}

Herb Wilf \cite{wilf}
proved that the set $S=\{1,2\}$ does have both of these properties.

If $A_{S,n}(x)$ has real roots only, then its coefficients form a log-concave
(and therefore, unimodal) sequence. (See Chapter 8 of
\cite{bonaint} for an
introduction into into unimodal and log-concave sequences.)  This raises the
 following questions.

\begin{question} Let us assume that $A_{S,n}(x)$ has real roots only.
Is there a combinatorial proof for the log-concavity of its coefficients?
\end{question}

\begin{question}  Let us assume that $A_{S,n}(x)$ has real roots only.
Where is the peak (or peaks) of the unimodal sequence of its coefficients?
\end{question}

Another interesting question is the following.
\begin{question}
For what $S$ and $n$ does the equality $A_{S,n}(-1)=0$ hold? When it does,
the number of compositions of $n$ with parts in $S$ and with $m(a)$ even
equals the number of compositions of $n$ with parts in $S$ and with $m(a)$
odd. Is there a combinatorial proof of that fact?
\end{question}

Finally, our methods rested on the finiteness of $S$, but we can still
ask what can be said for {\em infinite} sets of allowed parts.

\begin{center} {\bf Acknowledgment}

I am grateful to Herb Wilf for valuable discussions and advice.
\end{center}

On a Balanced Property of Compositions


Cutoff for product replacement on finite groups

Abstract: We analyze a Markov chain, known as the product replacement chain, on the set of generating $n$-tuples of a fixed finite group $G$. We show that as $n \rightarrow \infty$, the total-variation mixing time of the chain has a cutoff at time $\frac{3}{2} n \log n$ with window of order $n$. This generalizes a result of Ben-Hamou and Peres (who established the result for $G = \mathbb{Z}/2$) and confirms a conjecture of Diaconis and Saloff-Coste that for an arbitrary but fixed finite group, the mixing time of the product replacement chain is $O(n \log n)$.

\section{Introduction}

Let $G$ be a finite group, and let $[n] := \{1, 2, \dots, n\}$. We
consider the set $G^n$ of all functions $\sigma: [n] \to G$ (or
``configurations''). We may define a Markov chain $(\sigma_t)_{t\ge0}$ on
$G^{n}$ as follows: if we have a current state $\sigma$, then uniformly at
random, choose an ordered pair $(i, j)$ of distinct integers in $[n]$,
and change the value of $\sigma(i)$ to $\sigma(i) \sigma(j)^{\pm 1}$, where the
signs are chosen with equal probability.

We will restrict the chain $(\sigma_t)_{t\ge 0}$ to the space of {\it
  generating $n$-tuples}, i.e.\ the set of $\sigma$ whose values generate
$G$ as a group:
\[ {\mathcal S} := \left\{ \sigma \in G^n \ : \ \langle \sigma(1), \dots, \sigma(n) \rangle=G \right\}. \]
It is not hard to see that for fixed $G$ and large enough $n$, the
chain on ${\mathcal S}$ is irreducible (see \cite[Lemma 3.2]{DSC96}). We will
always assume $n$ is large enough so that this irreducibility
holds. Note that the chain is also symmetric, and it is aperiodic
because it has holding on some states. Thus, the chain has a uniform
stationary distribution $\pi$ with $\pi(\sigma)=1/|{\mathcal S}|$.

This Markov chain was first considered in the context of computational
group theory---it models the \emph{product replacement algorithm} for
generating random elements of a finite group introduced in
\cite{Celler}. By running the chain for a long enough time $t$ and
choosing a uniformly random index $k \in [n]$, the element $\sigma_t(k)$
is a (nearly) uniformly random element of $G$. The product replacement
algorithm has been found to perform well in practice \cite{Celler,
  Holt-Rees}, but the question arises: how large does $t$ need to be
in order to ensure near uniformity?

One way of answering the question is to estimate the mixing time of
the Markov chain. It was shown by Diaconis and Saloff-Coste that for
any fixed finite group $G$, there exists a constant $C_G$ such that
the $\ell^2$-mixing time is at most $C_G n^2 \log n$ \cite{DSC96,
  DSC98} (see also Chung and Graham \cite{Chung-Graham-Group} for a
simpler proof of this fact with a different value for $C_G$).

In another line of work, Lubotzky and Pak \cite{Lubotzky-Pak} analyzed
the mixing of the product replacement chain in terms of Kazhdan
constants (see also subsequent quantitative estimates for Kazhdan
constants by Kassabov \cite{Kassabov}). We also mention a result of
Pak \cite{Pak} which shows mixing in $\text{polylog}(|G|)$ steps when
$n = \Theta(\log |G| \log \log |G|)$. The reader may consult the
survey \cite{Pak-Survey} for further background on the product
replacement algorithm.

Diaconis and Saloff-Coste conjectured that the mixing time bound can
be improved to $C_G n \log n$ \cite[Remark 2, Section 7,
  p.\ 290]{DSC98}, based on the observation that at least $n \log n$
steps are needed by the classical coupon-collector's problem. This was
confirmed in the case $G = {\mathbb Z}/2$ by Chung and Graham
\cite{Chung-Graham-Cube} and recently refined by Ben-Hamou and Peres,
who show that when $G={\mathbb Z}/2$, the chain in fact exhibits a cutoff at
time $\frac{3}{2}n \log n$ in total-variation with window of order $n$
\cite{BHP}.

In this paper, we extend the result of Ben-Hamou and Peres to all
finite groups. Note that this also verifies the conjecture of Diaconis
and Saloff-Coste for a fixed finite group. To state the result, let us
denote the total variation distance between $\Pr_\sigma(\sigma_t \in \cdot
\ )$ and $\pi$ by
\[ d_\sigma(t) := \max_{A \subseteq {\mathcal S}}|\Pr_\sigma(\sigma_t \in A)- \pi (A)|. \]

\begin{theorem}\label{Thm:main}
Let $G$ be a finite group. Then, the Markov chain $(\sigma_t)_{t \ge 0}$ on the set of
generating $n$-tuples of $G$ has a total-variation cutoff at time
$\frac{3}{2}n\log n$ with window of order $n$. More precisely, we have
\begin{equation}\label{Eq:UB}
  \lim_{\b\to\infty} \limsup_{n\to\infty} \max_{\sigma \in {\mathcal S}}
  d_\sigma\left(\frac{3}{2}n\log n + \b n\right) = 0
\end{equation}
and
\begin{equation}\label{Eq:LB}
  \lim_{\b\to\infty} \liminf_{n\to\infty} \max_{\sigma \in {\mathcal S}}
  d_\sigma\left(\frac{3}{2}n\log n - \b n\right) = 1.
\end{equation}
\end{theorem}

\subsection{A connection to cryptography}

We mention another motivation for studying the product replacement
chain in the case $G=({\mathbb Z}/q)^m$ for a prime $q \ge 2$ and integers $m \ge
1$. It comes from a public-key authentication protocol proposed by
Sotiraki \cite{Sotiraki}, which we now briefly describe. In the
protocol, a verifier wants to check the identity of a prover based on
the time needed to answer a challenge.

First, the prover runs the Markov chain with $G = ({\mathbb Z}/q)^m$ and $n = m$,
which can be interpreted as performing a random walk on $SL_n({\mathbb Z}/q)$,
where $\sigma(k)$ is viewed as the $k$-th row of a $n \times n$
matrix. (In each step, a random row is either added to or subtracted
from another random row.)

After $t$ steps, the prover records the resulting matrix $A \in
SL_n({\mathbb Z}/q)$ and makes it public. To authenticate, the verifier gives
the prover a vector $x \in ({\mathbb Z}/q)^n$ and challenges her to compute $y :=
Ax$. The prover can perform this calculation in $O(t)$ operations by
retracing the trajectory of the random walk.

Without knowing the trajectory, if $t$ is large enough, an adversary
will not be able to distinguish $A$ from a random matrix and will be
forced to perform the usual matrix-vector multiplication (using $n^2$
operations) to complete the challenge. Thus, the question is whether
$t \ll n^2$ is large enough for the matrix $A$ to become sufficiently
random, so that the prover can answer the challenge much faster than
an adversary.

Note that when $n > m$, the product replacement chain on $G = ({\mathbb Z}/q)^m$
amounts to the projection of the random walk on $SL_n({\mathbb Z}/q)$ onto the
first $m$ columns. Thus, Theorem \ref{Thm:main} shows that when $m$ is
fixed and $n \rightarrow \infty$, the mixing time for the first $m$
columns is around $\frac{3}{2} n \log n$. One then hopes that the
mixing of several columns is enough to make it computationally
intractable to distinguish $A$ from a random matrix; this would
justify the authentication protocol, as $n \log n \ll n^2$.

We remark that when $t$ is much larger than the mixing time of the
random walk on $SL_n({\mathbb Z}/q)$ generated by row and additions and
subtractions, it is information theoretically impossible for an
adversary to distinguish $A$ from a random matrix. However, the
diameter of the corresponding Cayley graph on $SL_n({\mathbb Z}/q)$ is known to
be of order $\Theta\left( \frac{n^2}{\log_q n} \right)$ \cite{AHM,
  Christofides}, so a lower bound of the same order necessarily holds
for the mixing time. Diaconis and Saloff-Coste \cite[Section 4,
  p.\ 420]{DSC96} give an upper bound of $O(n^4)$, which was
subsequently improved to $O(n^3)$ by Kassabov \cite{Kassabov}. Closing
the gap between $n^3$ and $\frac{n^2}{\log n}$ remains an open
problem.

\subsection{Outline of proof}

The proof of Theorem \ref{Thm:main} analyzes the mixing behavior in
several stages:
\begin{itemize}
\item an initial ``burn-in'' period lasting around $n \log n$ steps,
  after which the group elements appearing in the configuration are
  not mostly confined to any proper subgroup of $G$;
\item an averaging period lasting around $\frac{1}{2} n \log n$ steps,
  after which the counts of group elements become close to their
  average value under the stationary distribution; and
\item a coupling period lasting $O(n)$ steps, after which our chain
  becomes exactly coupled to the stationary distribution with high
  probability.
\end{itemize}
The argument is in the spirit of \cite{BHP}, but a more elaborate
analysis is required in the second and third stages. To analyze the
first stage, for a fixed proper subgroup $H$, the number of group
elements in $H$ appearing in the configuration is a birth-and-death
process whose transition probabilities are easy to estimate. The
analysis of the resulting chain is the same as in \cite{BHP}, and we
can then union bound over all proper subgroups $H$.

In the second stage, for a given starting configuration $\sigma_0 \in
{\mathcal S}$, we consider quantities $n_{a,b}(\sigma)$ counting the number of
sites $k$ where $\sigma_0(k) = a$ and $\sigma(k) = b$. A key observation
(which also appears in \cite{BHP}) is that by symmetry, projecting the
Markov chain onto the values $(n_{a,b}(\sigma_t))_{a, b \in G}$ does
not affect the mixing behavior. Thus, it is enough to understand the
mixing behavior of the counts $n_{a,b}$.

One expects these counts to evolve towards their expected value
${\mathbb E}_{\sigma \sim \pi} n_{a,b}(\sigma)$ as the chain mixes. To carry out the
analysis rigorously, we write down a stochastic difference equation
for the $n_{a,b}$ and analyze it via the Fourier
transform. Intuitively, as $n \rightarrow \infty$, the process
approaches a ``hydrodynamic limit'' so that it becomes approximately
deterministic. It turns out that after about $\frac{1}{2} n \log n$
steps, the $n_{a,b}$ are likely to be within $O(\sqrt{n})$ of their
expected value. Our analysis requires a sufficiently ``generic''
initial configuration, which is why the first stage is necessary.

Finally, in the last stage, we show that if the $(n_{a,b}(\sigma))_{a,b\in
  G}$ and $(n_{a,b}(\sigma'))_{a,b\in G}$ for two configurations are
within $O(\sqrt{n})$ in $\ell^1$ distance, they can be coupled to be
exactly the same with high probability after $O(n)$ steps of the
Markov chain. A standard argument involving coupling to the stationary
distribution then implies a bound on the mixing time.

The main idea to prove the coupling bound is that even if the $\ell^1$
distance evolves like an unbiased random walk, there is a good chance
that it will hit $0$ due to random fluctuations. A similar argument is
used to prove cutoff for lazy random walk on the hypercube \cite[Chapter
  18]{LevinPeresWilmer}. However, some careful accounting is necessary
in our setting to ensure that in fact the $\ell^1$ distance does not
increase in expectation and to ensure sufficient fluctuations.

\subsection{Organization of the paper}

The rest of the paper is organized as follows. In Section
\ref{Sec:UB}, we state (without proof) the key lemmas describing the
behavior in each of the three stages and use these to prove the upper
bound \eqref{Eq:UB} in Theorem \ref{Thm:main}. Sections
\ref{Sec:DE-Proofs} and \ref{Sec:Coupling-Proof} contain the proofs of
these lemmas. Finally, in Section \ref{Sec:LB}, we prove the lower
bound \eqref{Eq:LB} in Theorem \ref{Thm:main}; this is mostly a matter
of verifying that the estimates used in the upper bound were tight.

\subsection{Notation}

Throughout this paper, we use $c, C, C', \dots$, to denote
absolute constants whose exact values may change from line to line,
and also use them with subscripts, for instance, $C_G$ to specify its
dependency only on $G$. We also use subscripts with big-$O$ notation,
e.g.\ we write $O_G(\,\cdot\,)$ when the implied constant depends only
on $G$.

\section{Proof of Theorem \ref{Thm:main} (\ref{Eq:UB})}\label{Sec:UB}

Let us fix a finite group $G$ and denote its cardinality by ${\mathcal Q} :=
|G|$. For a configuration $\sigma \in {\mathcal S}$, let $n_a(\sigma)$ denote the
number of sites having group element $a$, i.e.,
\[ n_a(\sigma) := |\{i \in [n] \ : \ \sigma(i)=a\}|. \]

\subsection{The burn-in period}

For a proper subgroup $H \subseteq G$, let
\[ n_{non}^H(\sigma) := \sum_{a \in G \setminus H} n_a(\sigma) \]
denote the number of sites not in $H$, and define for $c \in (0, 1)$
the set
\[ \Snon{c} := \{\sigma \in {\mathcal S} \ : \ n_{non}^H(\sigma) \ge cn \text{ for all proper subgroups $H \subseteq G$} \}. \]
Thus, $\Snon{c}$ is the set of states $\sigma$ where the group elements
appearing in $\sigma$ are not mostly confined to any particular proper
subgroup of $G$. The next lemma shows that we reach $\Snon{1/3}$
in about $n \log n$ steps, and once we reach $\Snon{1/3}$, we
remain in $\Snon{1/6}$ for $n^2$ steps with high probability. Note
that $n^2$ is much larger than the overall mixing time, so we may
essentially assume that we are in $\Snon{1/6}$ for all of the
later stages.

\begin{lemma}\label{Lem:Burn}
  Let $\t_{1/3} := \min\{t \ge 0 : \sigma_t \in \Snon{1/3}\}$ be the
  first time to hit $\Snon{1/3}$. Then for all large enough $n$
  and for all large enough $\b > 0$,
  \[ \max_{\sigma \in {\mathcal S}} \Pr_\sigma(\t_{1/3} > n \log n + \b n) \le \frac{120 {\mathcal Q}}{\b^2}. \]
  Moreover, there exists a constant $C_G$ depending only on $G$ such that
  \[ \max_{\sigma \in \Snon{1/3}}\Pr_\sigma \left(\sigma_t \notin \Snon{1/6} \ \text{\rm for some $t \le n^2$}\right) \le C_G n^2e^{-n/10}. \]
\end{lemma}

\begin{proof}
  Fix a proper subgroup $H \subset G$, and consider what happens to
  $n_{non}^H(\sigma_t)$ at time $t$. Suppose our next step is to replace
  $\sigma(i)$ with $\sigma(i)\sigma(j)$.

  If $\sigma(j) \in H$, then $n_{non}^H(\sigma_{t+1}) = n_{non}^H(\sigma_t)$. If
  $\sigma(j) \not\in H$ and $\sigma(i) \in H$, then $n_{non}^H(\sigma_{t+1}) =
  n_{non}^H(\sigma_t) - 1$. Finally, if $\sigma(j), \sigma(i) \not\in H$, then
  $\sigma(i)\sigma(j)$ may or may not be in $H$, so $n_{non}^H(\sigma_{t+1}) \ge
  n_{non}^H(\sigma_t) - 1$.

  Let $(N_t)_{t \ge 0}$ be the birth-and-death chain with the
  following transition probabilities for $1 \le k \le n$:
  \begin{align*}
    \Pr(N_{t+1} = k+1 \mid N_t = k) &= \frac{k(n-k)}{n(n-1)} \\
    \Pr(N_{t+1} = k-1 \mid N_t = k) &= \frac{k(k-1)}{n(n-1)} \\
    \Pr(N_{t+1} = k \mid N_t = k) &= \frac{n-k}{n}.
  \end{align*}
  We start this chain at $N_0 = n^H_{non}(\sigma_0)$; note that because
  the elements appearing in $\sigma_0$ generate $G$, we are guaranteed
  to have $n^H_{non}(\sigma_0) > 0$.

  The above birth-and-death chain corresponds to the behavior of
  $(n^H_{non}(\sigma_t))$ if whenever $\sigma(j), \sigma(i) \not\in H$, it always
  happened that $\sigma(i)\sigma(j) \in H$. Thus, $(n^H_{non}(\sigma_t))$
  stochastically dominates $(N_t)$.

  The chain $(N_t)$ is precisely what is analyzed in \cite{BHP} for
  the case $G = {\mathbb Z}/2$. Let
  \[T_k := \min\{t \ge 0 : N_t=k\}.\]
  Then, we have ${\mathbb E}_{k-1}T_k \le \frac{n^2}{k(n-2k)}$ \cite[(2) in the
    proof of Lemma 1]{BHP} and thus ${\mathbb E}_1 (T_{n/3})
  =\sum_{k=2}^{n/3}{\mathbb E}_{k-1}T_k \le n \log n + n$.  On the other hand,
  setting $v_k={\rm Var}_{k-1}(T_k)$, we have $v_2 \le n^2$,
  \[v_{k+1}\le \frac{k}{n-k}v_k + \frac{54 n^2}{k^2},\]
  and ${\rm Var}_1 (T_{n/3}) = \sum_{k=2}^{n/3}v_k \le 110 n^2$ \cite[The proof of Lemma 1]{BHP}.
  Hence by Chebyshev's inequality for
  all large enough $\b > 0$,
  \[\Pr_1(T_{n/3} > n \log n + \b n) \le \frac{120}{\b^2}.\]
  Moreover, we have $\Pr_{n/3} \left( T_{n/6} \le n^2 \right) \le n^2e^{-n/10}$.
  Indeed, this follows from the fact that for $m<k$, we have
  \[\Pr_k(T_m \le n^2) \le n^2\frac{\pi_{\rm BD}(m)}{\pi_{\rm BD}(k)},\]
  where $\pi_{\rm BD}(k)={n \choose k}/(2^n-1)$ \cite[(5) and the
    following in the proof of Proposition 2]{BHP}.

    We now take a union bound over all the
  proper subgroups $H$.
\end{proof}

\subsection{The averaging period}

In the next stage, the counts $n_a(\sigma_t)$ go toward their average
value. We actually analyze this stage in two substages, looking at a
``proportion vector'' and ``proportion matrix'', as described below.

\subsubsection{Proportion vector chain}

For a configuration $\sigma \in {\mathcal S}$, we consider the ${\mathcal Q}$-dimensional
vector $(n_a(\sigma)/n)_{a \in G}$, which we call the {\it proportion
  vector} of $\sigma$. One may check that for a typical $\sigma \in {\mathcal S}$, each
$n_a(\sigma)/n$ is about $1/{\mathcal Q}$. For each $\d > 0$, we define the
$\d$-{\it typical set}
\[ {\mathcal S}_\ast(\d) := \left\{\sigma \in {\mathcal S} \ : \ \left\|\left(\frac{n_a(\sigma)}{n}\right)_{a \in G} - \left(\frac{1}{{\mathcal Q}}\right)_{a \in G}\right\| \le \d \right\}, \]
where $\| \cdot \|$ denotes the $\ell^2$-norm in ${\mathbb R}^G$.

The following lemma implies that starting from $\sigma \in \Snon{1/3}$, we
reach $\Sast{\d}$ in $O_\d(n)$ steps with high
probability. The proof is given in Section \ref{Subsec:vector}.

\begin{lemma}\label{Lem:1stDE}
  Consider any $\sigma \in \Snon{1/3}$ and any constant $\d>0$. There exists a constant $C_{G, \d}$
  depending only on $G$ and $\d$ such that for any $T \ge C_{G, \d} n$, we have
  \[ \Pr_\sigma\left(\sigma_T \notin \Sast{\d} \right) \le \frac{1}{n} \]
  for all large enough $n$.
\end{lemma}

\subsubsection{Proportion matrix chain}

We actually need a more precise averaging than what is provided by
Lemma \ref{Lem:1stDE}. Fix a configuration $\sigma_0 \in {\mathcal S}$. For any $\sigma
\in {\mathcal S}$ and for any $a, b \in G$, define
\[ n_{a,b}^{\sigma_0}(\sigma) := |\{i \in [n] \ : \ \sigma_0(i)=a, \sigma(i)=b \}|. \]
If we run the Markov chain $(\sigma_t)_{t\ge 0}$ with initial state
$\sigma_0$, then $n_{a,b}^{\sigma_0}(\sigma_t)$ is the number of sites that
originally contained the element $a$ (at time 0) but now contain $b$
(at time $t$). Note that
\[ \sum_{b \in G} n_{a,b}^{\sigma_0}(\sigma) = n_a(\sigma_0) \quad \text{and} \quad \sum_{a \in G} n_{a,b}^{\sigma_0}(\sigma) = n_b(\sigma).\]

We can then associate with $(\sigma_t)_{t \ge 0}$ another Markov chain
$\left(n_{a,b}^{\sigma_0}(\sigma_t)/n_a(\sigma_0)\right)_{a, b \in G}$ for $t \ge
0$, which we call the {\it proportion matrix chain} ({\it with respect to
  $\sigma_0$}). The state space for the proportion matrix chain is $\{0,
1, \dots, n\}^{G \times G}$, and the transition probabilities depend
on $\sigma_0$.

The proportion matrix acts like a ``sufficient statistic'' for
analyzing our Markov chain started at $\sigma_\ast$, because of the
permutation invariance of our dynamics. In fact, as the following
lemma shows, the distance to stationarity of the proportion matrix
chain is equal to the distance to stationarity of the original chain.

\begin{lemma}\label{Lem:nij}
  Let $\sigma_\ast \in {\mathcal S}$ be a configuration. For the Markov chain
  $(\sigma_t)_{t \ge 0}$ with initial state $\sigma_\ast$, we consider
  $\left(n_{a, b}^{\sigma_\ast}(\sigma_t)\right)_{a, b \in G}$. Let
  $\overline{\pi}^{\sigma_\ast}$ be the stationary measure for the Markov
  chain $\{(n_{a, b}^{\sigma_\ast}(\sigma_t))_{a, b \in G}\}_{t \ge 0}$ on
  $\left\{0, 1, \dots, n\right\}^{G \times G}$. Then, for every $t \ge
  0$, we have
  \[ \normTV{ \Pr_{\sigma_\ast}(\sigma_t \in \cdot \ ) - \pi } = \normTV{ \Pr_{\sigma_\ast}\left( (n_{a, b}^{\sigma_\ast}(\sigma_t))_{a, b \in G} \in \cdot \ \right) - \overline{\pi}^{\sigma_\ast} }. \]
\end{lemma}
\begin{proof}
  For any matrix $N = (N_{a,b})_{a,b \in G} \in \{0, 1, \ldots ,
  n\}^{G \times G}$, write
  \[ {\mathcal X}_{(N)} := \left\{\sigma \in {\mathcal S} \ : \ n_{a, b}^{\sigma_\ast}(\sigma)=N_{a, b} \ \text{for all $a, b \in G$}\right\} \]
  for the set of configurations with $N$ as their proportion matrix.

  Since the distribution of $\sigma_t$ is invariant under permutations on
  sites $i \in [n]$ preserving the set $\{ i : \sigma_\ast(i) = a\}$ for
  every $a \in G$, the conditional probability measures
  $\Pr_{\sigma_\ast}\left(\sigma_t \in \cdot \mid \sigma_t \in {\mathcal X}_{(N)} \right)$ and
  $\pi( \ \cdot \mid {\mathcal X}_{(N)})$ are both uniform on ${\mathcal X}_{(N)}$.
  This implies that for each $\sigma \in {\mathcal X}_{(N)}$,
  \[ |\Pr_{\sigma_\ast}(\sigma_t =\sigma) - \pi(\sigma)| = \frac{1}{\left|{\mathcal X}_{(N)}\right|}\left|\Pr_{\sigma_\ast}\left((n^{\sigma_\ast}_{a, b}(\sigma_t))_{a, b \in G}=N\right)- \overline{\pi}^{\sigma_\ast}(N) \right|, \]
  and summing over all $\sigma \in {\mathcal X}_{(N)}$ and all $N$, we obtain the
  claim.
\end{proof}

For $\sigma_0 \in {\mathcal S}$ and $r > 0$, define the set of configurations
\[ \Sast{\sigma_0, r} := \left\{ \sigma \in {\mathcal S} \ : \ \left\| \left(\frac{n^{\sigma_0}_{a, b}(\sigma)}{n_a(\sigma_0)}\right)_{b \in G} - \left(\frac{1}{{\mathcal Q}}\right)_{b \in G}\right\| \le r \text{ for all $a
  \in G$} \right\}. \]
Roughly speaking, the following lemma shows that starting from a
typical configuration $\sigma_\ast \in \Sast{\frac{1}{4{\mathcal Q}}}$, we need
about $\frac{1}{2}n \log n$ steps to reach $\Sast{\sigma_\ast,
  \frac{R}{\sqrt{n}}}$, where $R$ is a constant. We show this fact in
a slightly more general form where the initial state need not be
$\sigma_\ast$; the proof is given in Section \ref{Subsec:matrix}.

\begin{lemma}\label{Lem:2ndDE}
  Consider any $\sigma_\ast, \sigma'_\ast \in \Sast{\frac{1}{4{\mathcal Q}}}$, and let
  $T := \ceil{\frac{1}{2} n \log n}$. There exists a constant $C_G >
  0$ depending only on $G$ such that for any given $R > 0$, we have
  \[ \Pr_{\sigma'_\ast}\left(\sigma_T \notin \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}\right) \le C_G e^{-R} + \frac{1}{n} \]
  for all large enough $n$.
\end{lemma}

\subsection{The coupling period}

After reaching $\Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}$, we show that only
$O(n)$ additional steps are needed to mix in total variation
distance. The main ingredient in the proof is a coupling of proportion
matrix chains so that they coalesce in $O(n)$ steps when they both
start from configurations $\sigma, \tilde\sigma \in \Sast{\sigma_\ast,
  \frac{R}{\sqrt{n}}}$. We construct such a coupling and prove the
following lemma in Section \ref{Sec:Coupling-Proof}.

\begin{lemma}\label{Lem:RW}
  Consider any $\sigma_\ast \in \Sast{\frac{1}{5{\mathcal Q}^3}}$, and let $R > 0$. Suppose $\sigma, \tilde
  \sigma \in \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}$. Then, there exists a
  coupling $(\sigma_t, \tilde \sigma_t)$ of the Markov chains with initial
  states $(\sigma, \tilde \sigma)$ such that for a given $\b > 0$ and all
  large enough $n$,
  \[ \Pr_{\sigma, \tilde \sigma}(\t > \b n) \le \frac{32{\mathcal Q}^2 R}{\sqrt{\b}}, \]
  where $\t:=\min\{t \ge 0 : n^{\sigma_\ast}_{a, b}(\sigma_t) =
  n^{\sigma_\ast}_{a, b}(\tilde \sigma_t) \ \text{\rm for all $a, b \in G$}\}$.
\end{lemma}

To translate this coupling time into a bound on total variation
distance, we need also the simple observation that the stationary
measure $\pi$ concentrates on $\Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}$
except for probability $O(1/R^2)$, as given in the next lemma.

\begin{lemma}\label{Lem:stationary}
  For the stationary distribution $\pi$ of the chain $(\sigma_t)_{t \ge 0}$,
  for every $R>0$ and for all $n > m$,
  \[ \pi\left(\sigma \notin \Sast{\frac{R}{\sqrt{n}}}\right) \le \frac{C_G}{R^2}. \]
  Moreover for every $\d<1/(2{\mathcal Q})$, for every $R>0$ and for all $n>m$,
  \[ \max_{\sigma_\ast \in \Sast{\d}}\pi\left(\sigma \notin \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}\right) \le \frac{2 C_G {\mathcal Q}}{R^2}, \]
  where $C_G$ and $m$ are constants depending only on $G$.
\end{lemma}

\begin{proof}
  Observe that since the stationary distribution $\pi$ is uniform on
  ${\mathcal S}$, it is given by the uniform distribution ${\rm Unif}$ on $G^n$
  conditioned on ${\mathcal S}$. Note that we can always generate $G$ using
  each of its $|G|$ elements, so we have an easy lower bound of $|{\mathcal S}|
  \ge |G|^{n - |G|}$. Consequently, we have
  \begin{align*}
  \pi\left(\sigma \notin \Sast{\frac{R}{\sqrt{n}}}\right)
  &\le |G|^{|G|} {\rm Unif}\left(\sigma \notin \Sast{\frac{R}{\sqrt{n}}}\right) \\
  &\le |G|^{|G|} \sum_{a \in G}{\rm Unif}\left(\left| \frac{n_a(\sigma)}{n}-\frac{1}{{\mathcal Q}}\right| \ge \frac{R}{\sqrt{n}}\right) \le \frac{|G|^{|G|}}{R^2}\left(1-\frac{1}{{\mathcal Q}}\right).
  \end{align*}
  Concerning the second assertion, we note that $n_a(\sigma_\ast) \ge
  (1/{\mathcal Q} -\d)n$ for each $a \in G$; the rest follows similarly, so we
  omit the details.
\end{proof}

\begin{remark}\label{Rem:1}
  In Lemma \ref{Lem:stationary} above, we have given a very loose
  bound on $C_G$ for sake of simplicity. Actually, it is not hard to
  see that holding $G$ fixed, we have $\lim_{n\rightarrow \infty}
  |{\mathcal S}|/|G|^n = 1$. See also \cite[Section 6.B.]{DSC98} for more
  explicit bounds for various families of groups.
\end{remark}

Together, Lemmas \ref{Lem:2ndDE}, \ref{Lem:RW}, and \ref{Lem:stationary}
imply the following bound for total variation distance.

\begin{lemma} \label{Lem:coupling-tv}
  Let $\b > 0$ be given, and let $T := \ceil{\frac{1}{2} n \log n} +
  \ceil{\b n}$. Then, for any $\sigma_\ast \in \Sast{\frac{1}{5{\mathcal Q}^3}}$,
  we have
  \[ \normTV{ \Pr_{\sigma_\ast}(\sigma_T \in \cdot \ ) - \pi } \le \frac{C_G}{\b^{1/4}}, \]
  where $C_G$ is a constant depending only on $G$.
\end{lemma}
\begin{proof}
  Let $\tilde\sigma$ be drawn from the stationary distribution
  $\pi$. Define
  \[ \tau = \min \left\{ t \ge 0 : n^{\sigma_\ast}_{a,b}(\sigma_t) = n^{\sigma_\ast}_{a,b}(\tilde\sigma_t) \text{ for all $a, b \in G$}\right\}, \]
  where $(\tilde\sigma_t)$ is a Markov chain started at $\tilde\sigma$. Let
  $\overline{\pi}^{\sigma_\ast}$ denote the stationary distribution for
  the proportion matrix with respect to $\sigma_\ast$. Since $\tilde\sigma$
  was drawn from $\pi$, the proportion matrix of $\tilde\sigma_t$ remains
  distributed as $\overline{\pi}^{\sigma_\ast}$ for all $t$.

  We first run $\sigma$ and $\tilde\sigma$ independently up until time $T_1 :=
  \ceil{\frac{1}{2} n \log n}$. For a parameter $R$ to be specified
  later, consider the events
  \[ {\mathcal G} := \left\{ \sigma_{T_1} \in \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}} \right\}, \qquad \tilde{{\mathcal G}} := \left\{ \tilde\sigma_{T_1} \in \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}} \right\}. \]
  Lemma \ref{Lem:2ndDE} implies that $\Pr({\mathcal G}^{\sf c}) \le
  C_G e^{-R} + \frac{1}{n}$, and Lemma \ref{Lem:stationary} implies that
  $\Pr(\tilde{\mathcal G}^{\sf c}) \le \frac{2 C_G {\mathcal Q}}{R^2}$.

  Let $T_2 := \ceil{\b n}$. Starting from time $T_1$, as long as both
  ${\mathcal G}$ and $\tilde{\mathcal G}$ hold, we may use Lemma \ref{Lem:RW} to form a
  coupling $(\sigma_t, \tilde\sigma_t)$ so that
  \[ \Pr_{\sigma_\ast, \sigma_\ast} \Big( n^{\sigma_\ast}_{a, b}(\sigma_{T_1+T_2}) \ne n^{\sigma_\ast}_{a, b}(\tilde\sigma_{T_1+T_2}) \text{ for some $a, b \in G$} \,\Big|\, {\mathcal G} \cap \tilde{\mathcal G} \Big) \le \frac{C{\mathcal Q}^2 R}{\sqrt{\b}}. \]
  Setting $R = \b^{1/4}$, we conclude that
  \begin{align*}
    &\Pr_{\sigma_\ast, \sigma_\ast} \Big( n^{\sigma_\ast}_{a, b}(\sigma_{T_1+T_2}) \ne n^{\sigma_\ast}_{a, b}(\tilde\sigma_{T_1+T_2}) \text{ for some $a, b \in G$} \Big) \le \frac{C{\mathcal Q}^2 R}{\sqrt{\b}} + \Pr({\mathcal G}^{\sf c}) + \Pr(\tilde{\mathcal G}^{\sf c}) \\
    &\qquad\le \frac{C{\mathcal Q}^2 R}{\sqrt{\b}} + \left(C_G e^{-R} + \frac{1}{n}\right) + \frac{2C_G {\mathcal Q}}{R^2} = O_G\left(\frac{1}{\b^{1/4}}\right).
  \end{align*}
  We have $T = T_1 + T_2$, and recall that the proportion matrix for
  $\tilde\sigma$ is stationary for all time. This yields
  \[ \normTV{ \Pr_{\sigma_\ast}\left( (n^{\sigma_\ast}_{a, b}(\sigma_T))_{a, b \in G} \in \cdot \ \right) - \overline{\pi}^{\sigma_\ast} } = O_G\left(\frac{1}{\b^{1/4}}\right). \]
  The result then follows by Lemma \ref{Lem:nij}.
\end{proof}

\subsection{Proof of the main theorem}

We now combine the lemmas from the burn-in, averaging, and coupling
periods to complete the proof of the upper bound in Theorem
\ref{Thm:main}.

\begin{proof}[Proof of Theorem \ref{Thm:main} (\ref{Eq:UB})]
  Define $T_1 := \ceil{n \log n + \b n}$, $T_2 := \ceil{\b n}$, and
  $T_3 := \ceil{\frac{1}{2} n \log n} + \ceil{\b n}$.

  Let $\tau_{1/3}$ be the first time to hit $\Snon{1/3}$ as in
  Lemma \ref{Lem:Burn}. Then, Lemma \ref{Lem:Burn} implies that for
  any $\sigma_1 \in {\mathcal S}$ and any $t \ge 0$, we have
  \begin{align}
    d_{\sigma_1}(T_1 + t) &\le \Pr_{\sigma_1}\left( \tau_{1/3} > T_1 \right) + \max_{\sigma \in \Snon{1/3}} d_\sigma(t) \nonumber \\
    &\le \frac{120 {\mathcal Q}}{\b^2} + \max_{\sigma \in \Snon{1/3}} d_\sigma(t). \label{Eq:stage1}
  \end{align}
  Next, by Lemma \ref{Lem:1stDE}, for any $\sigma_2 \in \Snon{1/3}$
  and when $\b$ and $n$ are sufficiently large, we have that
  $\Pr_{\sigma_2} \left( \sigma_{T_2} \not\in \Sast{\frac{1}{5{\mathcal Q}^3}} \right)
  \le \frac{1}{n}$. Consequently, for $\sigma_2 \in \Snon{1/3}$, we
  have
  \begin{equation} \label{Eq:stage2}
    d_{\sigma_2}(T_2 + t) \le \frac{1}{n} + \max_{\sigma_\ast \in \Sast{\frac{1}{5{\mathcal Q}^3}}} d_{\sigma_\ast}(t).
  \end{equation}
  Finally, Lemma \ref{Lem:coupling-tv} states that
  \begin{equation} \label{Eq:stage3}
    \max_{\sigma_\ast \in \Sast{\frac{1}{5{\mathcal Q}^3}}} d_{\sigma_\ast}(T_3) \le \frac{C_G}{\b^{1/4}}.
  \end{equation}

  Thus, combining \eqref{Eq:stage1}, \eqref{Eq:stage2}, and
  \eqref{Eq:stage3}, we obtain for any $\sigma \in {\mathcal S}$ that
  \begin{align*}
    d_\sigma\left(\frac{3}{2} n \log n + 4 \b n \right) &\le d_\sigma\left(T_1 + T_2 + T_3 \right) \nonumber \\
    &\le \frac{120 {\mathcal Q}}{\b^2} + \frac{1}{n} + \frac{C_G}{\b^{1/4}}
  \end{align*}
  sending $n \rightarrow \infty$ and then $\b \rightarrow \infty$
  yields \eqref{Eq:UB}.
\end{proof}

\section{Proofs for the averaging period} \label{Sec:DE-Proofs}

In this section, we prove Lemmas \ref{Lem:1stDE} and
\ref{Lem:2ndDE}. The proofs are based on analyzing stochastic
difference equations satisfied by the Fourier transform of the
proportion vector or matrix.

\subsection{The Fourier transform for $G$}

We first establish some notation and preliminaries for the Fourier
transform. Let $G^\ast$ be a complete set of non-trivial irreducible
representations of $G$. In other words, for each $\r \in G^\ast$, we
have a finite dimensional complex vector space $V_\r$ such that $\r: G
\to GL(V_\r)$ is a non-trivial irreducible representation, and any
non-trivial irreducible representation of $G$ is isomorphic to some unique $\r \in
G^\ast$. Moreover, we may equip each $V_\r$ with an inner product for
which $\r \in G^\ast$ is unitary.

For a configuration $\sigma \in {\mathcal S}$ and for each $\r \in G^\ast$, we
consider the matrix acting on $V_\r$ given by
\[ x_\r(\sigma) := \sum_{a \in G} \frac{n_a(\sigma)}{n}\r(a), \]
so that $x_\r(\sigma)$ is the Fourier transform of the proportion vector
at the representation $\r$. We write $x(\sigma):=(x_\r(\sigma))_{\r \in
  G^\ast}$.

Let $\wt{V} := \bigoplus_{\r \in G^\ast}{\rm End}_{\mathbb C}(V_\r)$, and write
$d_\r := \dim_{\mathbb C} V_\r$. For an element $x = (x_\r)_{\r \in G^\ast} \in
\wt{V}$, we define a norm $\| \cdot \|_{\wt{V}}$ given by
\begin{equation*}\label{Eq:norm}
\|x\|_{\wt{V}}^2 := \frac{1}{{\mathcal Q}}\sum_{\r \in G^\ast}d_\r \|x_\r\|_{{\rm HS}}^2,
\end{equation*}
where $\langle A, B \rangle_{{\rm HS}} =\Tr(A^\ast B)$ denotes the
Hilbert-Schmidt inner product in ${\rm End}_{\mathbb C}(V_\r)$ and $\|\cdot\|_{{\rm HS}}$
denotes the corresponding norm. (Note that $\langle \cdot, \cdot
\rangle_{{\rm HS}}$ and $\|\cdot\|_{{\rm HS}}$ depend on $\r$, but for sake of
brevity, we omit the $\r$ when there is no danger of confusion.)

The Peter-Weyl theorem \cite[Chapter 2]{DiaconisRep} says that
\begin{equation*} \label{Eq:Peter-Weyl}
  L^2(G) \cong {\mathbb C} \oplus \wt{V},
\end{equation*}
where the isomorphism is given by the Fourier transform. The
Plancherel formula then reads
\begin{equation}\label{Eq:Plancherel}
\|x(\sigma)\|_{\wt{V}}^2 = \left\|\left(\frac{n_a(\sigma)}{n}\right)_{a \in G} - \left(\frac{1}{{\mathcal Q}}\right)_{a \in G}\right\|^2.
\end{equation}
Thus, in order to show that $\sigma \in \Sast{\d}$, it
suffices to show that $\|x(\sigma)\|_{\wt{V}}$ is small. A similar
argument may be applied to the proportion matrix instead of the
proportion vector.

Finally, for an element $A \in {\rm End}_{\mathbb C}(V_\r)$, we will at times also
consider the \emph{operator norm} $\|A\|_{op} := \sup_{v \in V_\r, v \neq 0}
\|Av\| / \|v\|$. We will also sometimes use the following (equivalent)
variational characterization of the operator norm:
\begin{align*}
  \sup_{\substack{X \in {\rm End}_{\mathbb C}(V_\r) \\ \|X\|_{{\rm HS}} = 1}} \|XA\|^2_{{\rm HS}} &= \sup_{\substack{X \in {\rm End}_{\mathbb C}(V_\r) \\ \|X\|_{{\rm HS}} = 1}} \Tr (XAA^*X^*) = \sup_{\substack{X \in {\rm End}_{\mathbb C}(V_\r) \\ \|X\|_{{\rm HS}} = 1}} \Tr (X^*XAA^*) \\
  &= \sup_{\substack{Y \in {\rm End}_{\mathbb C}(V_\r) \\ Y = Y^*, \;\; \Tr Y = 1}} \langle Y , AA^* \rangle_{{\rm HS}} = \|AA^*\|_{op} = \|A\|_{op}^2.
\end{align*}

\subsubsection{The special case of $G = {\mathbb Z}/q$}

On a first reading of this section, the reader may wish to consider
everything for the special case of $G = {\mathbb Z}/q$ for some integer $q \ge 2$. In
that case, each representation is one-dimensional, and the
representations can be indexed by $\ell = 0, 1, 2, \ldots , q - 1$. The
Fourier transform is then particularly simple: the coefficients are
scalar values
\[ x_\ell(\sigma) = \sum_{a = 0}^{q - 1} \frac{n_a(\sigma)}{n} \o^{a \ell}, \]
where $\o := e^{\frac{2\pi i}{q}}$ is a primitive $q$-th root of
unity.

This special case already illustrates most of the main ideas while
simplifying the estimates in some places (e.g.\ matrix inequalities we
use will often be immediately obvious for scalars).

\subsection{A stochastic difference equation for the $n_a$}

For $a \in G$, we next analyze the behavior of $n_a(\sigma_t)$ over
time. For convenience, we write $n_a(t) = n_a(\sigma_t)$. Let ${\mathcal F}_t$
denote the $\sigma$-field generated by the Markov chain $(\sigma_t)_{t \ge 0}$
up to time $t$. Then, our dynamics satisfy the equation
\begin{equation}\label{Eq:DE0}
  {\mathbb E}[n_a(t+1)-n_a(t) \mid {\mathcal F}_t] = \sum_{b \in G} \frac{n_{ab^{-1}}(t) n_b(t) }{2n(n-1)}+\sum_{b \in G} \frac{n_{ab}(t) n_b(t)}{2n(n-1)} - \frac{n_a(t)}{n}.
\end{equation}
Note that $|n_a(t + 1) - n_a(t)| \le 1$ almost surely. Thus, for each
$a \in G$, we can write the above as a stochastic difference equation
\begin{equation}\label{Eq:DE}
  n_a(t+1) - n_a(t) = \sum_{b \in G} \frac{n_{ab^{-1}}(t) n_b(t) }{2n(n-1)}+\sum_{b \in G} \frac{n_{ab}(t) n_b(t)}{2n(n-1)} - \frac{n_a(t)}{n} + M_a(t+1),
\end{equation}
where ${\mathbb E}[M_a(t+1) \mid {\mathcal F}_t] = 0$ and $|M_a(t)| \le 2$ almost
surely.

It is easiest to analyze this equation through the Fourier
transform. Writing $x_\r(t) = x_\r(\sigma_t)$, we calculate from \eqref{Eq:DE} that
\[ x_\r(t+1) - x_\r(t) = \frac{1}{n-1}x_\r(t) \left(\frac{x_\r(t) + x_\r(t)^\ast}{2} -\frac{n-1}{n}\right) + \wh{M}_\r(t+1), \]
where $\wh{M}_\r(t) := \frac{1}{n}\sum_{a \in G}M_a(t) \r(a)$.
For convenience, write
\[X_\r(t) = \frac{1}{n - 1}\left(\frac{x_\r(t) + x_\r(t)^\ast}{2} - \frac{n-1}{n}\right),\]
so that our equation becomes
\begin{equation} \label{Eq:DEfourier}
  x_\r(t+1) - x_\r(t) = x_\r(t) X_\r(t) + \wh{M}_\r(t+1).
\end{equation}
Note that we have
\[ \|x_\r(t)\|_{{\rm HS}} \le \sqrt{d_\r}, \qquad {\mathbb E}[\wh{M}_\r(t+1) \mid {\mathcal F}_t] = 0, \qquad\text{and}\qquad \|\wh{M}_\r(t)\|_{{\rm HS}} \le \frac{2 {\mathcal Q}\sqrt{d_\r}}{n}, \]
and thus,
\[ \|x(t)\|_{\wt{V}} \le 1 \qquad\text{and}\qquad \|\wh{M}(t)\|_{\wt{V}}\le \frac{2{\mathcal Q}}{n},\]
where $\wh{M}=(\wh{M}_\r)_{\r \in G^\ast}$.

\subsection{A general estimate for stochastic difference equations}

Before proving Lemma \ref{Lem:1stDE}, we also need a technical lemma
for controlling the behavior of stochastic difference equations, which
will be used to analyze \eqref{Eq:DEfourier} as well as other similar
equations.

\begin{lemma} \label{Lem:GenDE}
  Let $(z(t))_{t \ge 0}$ be a sequence of $[0,1]$-valued random
  variables adapted to a filtration $({\mathcal F}_t)_{t \ge 0}$. Let $\varepsilon \in
  (0, 1)$ be a small constant, and let $\varphi : {\mathbb R}^+ \to (0,1]$ be a
non-decreasing function.

  Suppose that there are ${\mathcal F}_t$-measurable random variables $M(t)$
  for which
  \begin{equation} \label{Eq:zDE}
    z(t+1) - z(t) \le -\varepsilon\varphi(t+1) z(t) + M(t+1)
  \end{equation}
  and which, for some constant $D$, satisfy the bounds
  \[ {\mathbb E}[ M(t+1) \mid {\mathcal F}_t ] \le D\varepsilon\sqrt{\varepsilon}, \qquad |M(t)| \le D\varepsilon. \]
  Then, for each $t$ and each $\lambda > 0$, we have
  \[ \Pr\left( z(t) \ge \lambda \sqrt{\varepsilon} + e^{- \varepsilon \int_0^t \varphi(s)\,ds} \cdot z(0) \right) \le C_{D,\varphi} e^{-c_{D,\varphi} \lambda^2} \]
  for constants $c_{D,\varphi}, C_{D,\varphi}$ depending only on $D$ and $\varphi$.
\end{lemma}
\begin{proof}
  Let us define for integers $t \ge 1$,
  \[ \Phi(t) := \varepsilon^{-1} \sum_{k = 1}^t \log \frac{1}{1-\varepsilon \varphi(k)}, \qquad \text{and} \qquad \Phi(0) := 0.\]
  Taking conditional expectations in
  the inequality relating $z(t+1)$ to $z(t)$, we have
  \[ {\mathbb E}[ z(t+1) \mid {\mathcal F}_t ] \le (1 - \varepsilon \varphi(t+1)) z(t) + D \varepsilon\sqrt{\varepsilon}. \]
  Rearranging and using the fact that $\varphi(t)$ is non-decreasing, we have
  \begin{align*}
    {\mathbb E}[ z(t+1) \mid {\mathcal F}_t ] - \frac{D\sqrt{\varepsilon}}{\varphi(0)} &\le (1 - \varepsilon \varphi(t+1)) z(t) - \frac{D\sqrt{\varepsilon}(1 - \varepsilon \varphi(t+1))}{\varphi(0)} \\
    &\le (1 - \varepsilon \varphi(t+1)) \left( z(t) - \frac{D\sqrt{\varepsilon}}{\varphi(0)} \right).
  \end{align*}
  Consequently,
  \[ Z_t := e^{\varepsilon \Phi(t)} \left( z(t) - \frac{D\sqrt{\varepsilon}}{\varphi(0)} \right) \]
  is a supermartingale, and its increments are bounded by
  \begin{equation} \label{Eq:Z-increment-bound}
    |Z_{t+1}-Z_t| \le e^{\varepsilon \Phi(t+1)}\left(|M(t+1)|+D \varepsilon\right) \le 2D\varepsilon e^{\varepsilon\Phi(t+1)}.
  \end{equation}
  Recall that $\varphi$ is non-decreasing, so that for all $t \ge s \ge 0$,
  we have
  \[ \Phi(t) = \Phi(s) + \varepsilon^{-1} \sum_{k = s + 1}^t \log \frac{1}{1 - \varepsilon \varphi(k)} \ge \Phi(s) + (t - s) \varphi(0). \]
  Using this with \eqref{Eq:Z-increment-bound}, we see that the sum of
  the squares of the first $t$ increments is at most
  \begin{align*}
    \sum_{s = 1}^{t} 4D^2 \varepsilon^2 e^{2\varepsilon\Phi(s)} &\le 4D^2\varepsilon^2 \sum_{s = 1}^t e^{2\varepsilon \Phi(t) - 2\varepsilon\varphi(0)(t - s)} \le 4D^2\varepsilon^2 e^{2\varepsilon\Phi(t)} \cdot \frac{1}{1 - e^{-2\varepsilon\varphi(0)}} \\
    &\le 4D^2\varepsilon^2 e^{2\varepsilon\Phi(t)} \cdot \frac{1}{1 - (1 - \frac{1}{2}\varepsilon\varphi(0))} = \frac{8D^2 \varepsilon}{\varphi(0)} \cdot e^{2\varepsilon\Phi(t)}.
  \end{align*}
  By the Azuma-Hoeffding inequality, this yields
  \[ \Pr\left(Z_t \ge \lambda \sqrt{\varepsilon} e^{\varepsilon\Phi(t)} + Z_0 \right) \le \exp\left( - \frac{\varphi(0) \lambda^2 \varepsilon \cdot e^{2\varepsilon\Phi(t)}}{16D^2 \varepsilon \cdot e^{2\varepsilon\Phi(t)}} \right) = \exp\left( -\frac{\varphi(0) \lambda^2}{16D^2} \right), \]
  which in turn implies
  \[ \Pr\left( z(t) \ge \frac{D\sqrt{\varepsilon}}{\varphi(0)} + e^{-\varepsilon\Phi(t)} z(0) + \lambda \sqrt{\varepsilon} \right) \le \exp\left( -\frac{\varphi(0)\lambda^2}{16D^2} \right). \]
  Finally, observe that $\Phi(t) \ge \sum_{k = 1}^t \varphi(k) \ge \int_0^t
  \varphi(s)\, ds$. The result then follows upon shifting and rescaling of
  $\lambda$.
\end{proof}

\subsection{Proportion vector chain: Proof of Lemma \ref{Lem:1stDE}}\label{Subsec:vector}

We first prove a bound for the Fourier coefficients $x_\r(t)$.

\begin{lemma} \label{Lem:1stDE-fourier}
  Consider any $\sigma \in \Snon{1/3}$ and any $\r \in G^\ast$. We have a
  constant $c_G$ depending only on $G$ for which
  \[ \Pr_\sigma\left( \bigcup_{t = 1}^{n^2} \left\{ \|x_\r(t)\|_{{\rm HS}} \ge \frac{1}{n^{1/8}} + e^{-c_G t/n}\cdot \|x_\r(0)\|_{{\rm HS}} \right\} \right) \le \frac{1}{n^3}. \]
  for all large enough $n$.
\end{lemma}

This immediately implies Lemma \ref{Lem:1stDE}.

\begin{proof}[Proof of Lemma \ref{Lem:1stDE}]
  With $c_G$ defined as in Lemma \ref{Lem:1stDE-fourier}, take $C_{G, \d}$
  large enough so that for any $T \ge C_{G, \d} n$,
  \[ \frac{1}{n^{1/8}} + e^{-c_G T/n}\sqrt{d_\r} \le \d. \]
  Then, Lemma \ref{Lem:1stDE-fourier} and Plancherel's formula yield
  \begin{align*}
    \Pr_\sigma\left(\sigma_T \notin \Sast{\d} \right) &\le \Pr_\sigma\left( \|x_\r(T)\|_{{\rm HS}} \ge \d \text{ for some $\r \in G^\ast$} \right) \\
    &\le \frac{{\mathcal Q}}{n^3} \le \frac{1}{n},
  \end{align*}
  for large enough $n$, as desired.
\end{proof}

We are now left with proving Lemma \ref{Lem:1stDE-fourier}, which
relies on the following bound on the operator norm.

\begin{lemma} \label{Lem:gap}
There exists a positive constant $\gamma_G$ depending on $G$ such that for
  any $\r \in G^\ast$ and any $\sigma \in \Snon{1/6}$,
    \[ \|I_{d_\r}+X_\r(\sigma)\|_{op}  \le 1-\frac{\gamma_G}{n}. \]
\end{lemma}

\begin{proof}
  Let $\Delta_G$ denote the set of all probability distributions on
  $G$, and for $c \in (0, 1)$, let $\Delta_G(c) \subset \Delta_G$
  denote the set of all probability distributions $\mu$ such that
  $\mu(H) \le 1 - c$ for all proper subgroups $H \subset G$.

  Consider a representation $\r \in G^\ast$, and consider the function
  $h : \Delta_G(1/6) \to {\rm End}_{\mathbb C}(V_\r)$ given by
  \[ h(\mu) = \sum_{a \in G} \mu(a) \frac{\r(a)+\r(a)^\ast}{2}. \]
  Then, $h(\mu)$ is hermitian, and since $\r$ is unitary, we clearly have
  \[\l(\mu):=\max_{v \in V_\r, \|v\|=1}\langle h(\mu)v, v\rangle \le 1. \]
  We claim that $\l(\mu) < 1$ for each $\mu \in \Delta_G(c)$.
  Indeed, suppose the contrary.
  Then, there exists a non-zero vector $v \in V_\r$ such that $\Re \langle \r(a)v, v \rangle=1$ for all $a \in G$ with $\mu(a)>0$.
  This implies that the support of $\mu$ is included in the subgroup
  \[H=\{a \in G \ : \ \r(a)v=v\}.\]
  Since $\r$ is a (non-trivial) irreducible representation, $H$ is a
  proper subgroup of $G$, and thus $\mu(H) \le 1- c$, contradicting the
  assumption that $\mu \in \Delta_G(c)$.

  Note that $\mu \mapsto \l(\mu)$ is continuous.
  We may define
  \[\gamma_\r:=\max_{\mu \in \Delta_G(1/6)}\l(\mu) < 1
  \qquad\text{and}\qquad \tilde \gamma_G:=\max_{\r \in G^\ast}\gamma_\r<1.\]
  Then, we have for any $\sigma \in \Snon{1/6}$,
  \[ \frac{x_\r(\sigma) + x_\r(\sigma)^\ast}{2} = \sum_{a \in G} \frac{n_a(\sigma)}{n}\frac{\r(a)+\r(a)^\ast}{2}
  \preceq \tilde \gamma_G I_{d_\r}. \]
  Taking $0<\gamma_G < 1-\tilde \gamma_G$, and
  plugging this into the definition of $X_\r$ gives
  $X_\r(\sigma) \preceq -\frac{\gamma_G}{n}I_{d_\r}$.
  Note that $X_\r(\sigma) \succeq -\frac{2}{n-1}I_{d_\r}$.
  Combining these together gives the result.
\end{proof}

\begin{remark}
  A much more direct approach is possible in the case $G = {\mathbb Z}/q$. The
  condition $\sigma \in \Snon{1/6}$ implies that $n_0(\sigma) \le
  \frac{5}{6}$. Then, we have
  \[ \Re x_\ell(\sigma) := \Re \sum_{a = 0}^{q - 1} \frac{n_a(\sigma)}{n} \o^{a \ell} \le \frac{5}{6} + \frac{1}{6} \max_{1 \le a \le q - 1} \Re \o^{a \ell} = \frac{5}{6} + \frac{1}{6} \cos \frac{2\pi}{q} < 1 - \gamma_G \]
  for some positive $\gamma_G$. Some rearranging of equations then yields
  the desired result.
\end{remark}

\begin{proof}[Proof of Lemma \ref{Lem:1stDE-fourier}]
  Fix $\r \in G^\ast$. Let ${\mathcal G}_t$ denote the event where for all $0
  \le s \le t$, we have $\|I_{d_\r}+X_\r(s)\|_{op} \le 1 - \frac{\gamma_G}{n}$,
  where $\gamma_G$ is taken as in Lemma \ref{Lem:gap}. Since our chain
  starts at $\sigma \in \Snon{1/3}$, Lemmas \ref{Lem:Burn} and
  \ref{Lem:gap} together imply that
  \[ \Pr_\sigma({\mathcal G}_{n^2}^{\sf c}) \le C_G n^2 e^{-n/10}. \]
  Next, we turn to \eqref{Eq:DEfourier}. Rearranging
  \eqref{Eq:DEfourier} and squaring, we have
  \begin{align}
    \|x_\r(t+1)\|_{{\rm HS}}^2 &= \|x_\r(t)(I_{d_\r} + X_\r(t))\|_{{\rm HS}}^2 + \|\wh{M}_\r(t+1)\|_{{\rm HS}}^2 \nonumber \\
    &\hphantom{==} + 2\Re \langle x_\r(t)(I_{d_\r} + X_\r(t)), \wh{M}_\r(t+1) \rangle_{{\rm HS}} \label{Eq:1stDEsquared}
  \end{align}

  Let $z_t := {\bf 1}_{{\mathcal G}_t} \|x_\r(t)\|_{{\rm HS}}^2$ and
  \[ M'(t+1) := \|\wh{M}_\r(t+1)\|_{{\rm HS}}^2 + 2\Re \langle x_\r(t)(I_{d_\r} + X_\r(t)), \wh{M}_\r(t+1) \rangle_{{\rm HS}}. \]
  Substituting into \eqref{Eq:1stDEsquared}, we obtain
  \[ z_{t+1} \le \|I_{d_\r} + X_\r(t)\|_{op}^2 \cdot z_t +  {\bf 1}_{{\mathcal G}_t} M'(t+1) \le \left(1 - \frac{\gamma_G}{n}\right)^2 z_t +  {\bf 1}_{{\mathcal G}_t} M'(t+1). \]
  Note that we have the bounds
  \[ {\mathbb E}[ M'(t+1) \mid {\mathcal F}_t ]= {\mathbb E}[ \|\wh{M}_\r(t+1)\|_{{\rm HS}}^2 \mid {\mathcal F}_t ] \le \frac{4{\mathcal Q}^2d_\r}{n^2} \]
  \[ |M'(t+1)| \le \|\wh{M}_\r(t+1)\|_{{\rm HS}}^2 + 2\sqrt{d_\r}\left(1+\frac{1}{n(n-1)}\right) \|\wh{M}_\r(t+1)\|_{{\rm HS}} \le \frac{6{\mathcal Q}^2 d_\r}{n}. \]

  We now apply Lemma \ref{Lem:GenDE} with $\varepsilon = \frac{1}{n}$, $\varphi(t) =
  \gamma_G$, $D = 6{\mathcal Q}^2 d_\r$, and $\lambda = n^{1/4}$. This yields
  \[ \Pr\left( z_t \ge n^{-1/4} + e^{-\gamma_G t/n}\cdot z_0 \right) \le C'_G e^{-c'_G \sqrt{n}}. \]
  Consequently,
  \[ \Pr_\sigma\left( \|x_\r(t)\|_{{\rm HS}} \ge n^{-1/8} + e^{-\gamma_G t/2n} \cdot \|x_\r(0)\|_{{\rm HS}} \right) \le C'_G e^{-c'_G \sqrt{n}} + C_G n^2 e^{-n/10}. \]
  The lemma with $c_G = \gamma_G/2$ then follows from union bounding over
  all $1 \le t \le n^2$ and taking $n$ sufficiently large.
\end{proof}

\subsection{Proportion matrix chain: Proof of Lemma \ref{Lem:2ndDE}}\label{Subsec:matrix}

We carry out a similar albeit more refined strategy to analyze the
proportion matrix. Throughout this section, we assume our Markov chain
$(\sigma_t)_{t \ge 0}$ starts at an initial state $\sigma_\ast \in
\Sast{\frac{1}{4{\mathcal Q}}}$. We again write $n_a(t)=n_a(\sigma_t)$ and
$n_{a, b}(t)=n^{\sigma_\ast}_{a, b}(\sigma_t)$, and similar to before, the
$n_{a, b}(t)$ satisfy the difference equation
\begin{equation}\label{Eq:de2}
n_{a, b}(t+1)-n_{a, b}(t)=\sum_{c \in G}\frac{n_{a, bc^{-1}}(t)n_c(t)}{2n(n-1)}+\sum_{c \in G}\frac{n_{a, bc}(t)n_c(t)}{2n(n-1)} - \frac{n_{a, b}(t)}{n} + M_{a, b}(t+1),
\end{equation}
where ${\mathbb E}[M_{a, b}(t+1) \mid {\mathcal F}_t]=0$ and $|M_{a, b}(t)| \le 2$ for all $t \ge 0$.

We can again analyze this equation via the Fourier transform. In this
case, for each $a \in G$, we take the Fourier transform of
$\left(n_{a, b}(t)/n_a(\sigma_\ast)\right)_{b \in G}$. For $\r \in G^\ast$,
let
\[ y_{a,\r}(t) = y_{a,\r}^{\sigma_\ast}(t) := \sum_{b \in G}\frac{n_{a, b}(t)}{n_a(\sigma_\ast)}\r(b) \]
denote the Fourier coefficient at $\r$. Let $\wh{M}_{a, \r}(t) :=
\frac{1}{n_a(\sigma_\ast)}\sum_{b \in G}M_{a, b}(t)\r(b)$. Then, \eqref{Eq:de2} becomes
\begin{equation}\label{Eq:DEy}
y_{a, \r}(t+1) - y_{a, \r}(t) = y_{a, \r}(t) X_\r(t) + \wh{M}_{a, \r}(t+1).
\end{equation}
Note that ${\mathbb E}_\sigma[\wh{M}_{a, \r}(t+1) \mid {\mathcal F}_t]=0$. Also, since we
assumed $\sigma_\ast \in \Sast{\frac{1}{4{\mathcal Q}}}$, it follows that $\frac{n_a(\sigma_\ast)}{n}
\ge \frac{1}{2{\mathcal Q}}$. Thus, we also know
$\|\wh{M}_{a, \r}(t+1)\|_{{\rm HS}} \le \frac{4{\mathcal Q}^2\sqrt{d_\r}}{n}$.

Again, our main step is a bound on the Fourier coefficients
$y_{a, \r}(t)$, which will also be useful later in proving Lemma
\ref{Lem:RW}.

\begin{lemma} \label{Lem:2ndDE-fourier}
  Consider any $\sigma_\ast, \sigma'_\ast \in \Sast{\frac{1}{4{\mathcal Q}}}$. There
  exist constants $c_G, C_G > 0$ depending only on $G$ such that for
  all large enough $n$, we have
  \[ \Pr_{\sigma'_\ast}\left( \|y^{\sigma_\ast}_{a, \r}(t)\|_{{\rm HS}} \ge R \left(\frac{1}{\sqrt{n}} + e^{-t/n} \|y^{\sigma_\ast}_{a, \r}(0)\|_{{\rm HS}} \right) \right) \le e^{-\O_G(R^2) + O_G(1)} + \frac{2}{n^3} \]
  for all $t$ and $R > 0$.
\end{lemma}

The above lemma directly implies Lemma \ref{Lem:2ndDE}.

\begin{proof}[Proof of Lemma \ref{Lem:2ndDE}]
  We apply Lemma \ref{Lem:2ndDE-fourier} to each $a \in G$ and $\r \in
  G^\ast$. Recall that $T = \ceil{\frac{1}{2} n \log n}$, so that
  \[ \frac{1}{\sqrt{n}} + e^{-T/n} \|y^{\sigma_\ast}_{a, \r}(0)\|_{{\rm HS}} \le \frac{2\sqrt{d_\r}}{\sqrt{n}}. \]
  Then, Lemma \ref{Lem:2ndDE-fourier} implies
  \[ \Pr_{\sigma'_\ast}\left( \|y^{\sigma_\ast}_{a, \r}(T)\|_{{\rm HS}} \ge \frac{R}{\sqrt{n}} \right) \le e^{-\O_G(R^2) + O_G(1)} + \frac{2}{n^3}. \]
  Union bounding over all $a \in G$ and $\r \in G^\ast$ and using the
  Plancherel formula, this yields
  \begin{align*}
    &\Pr_{\sigma'_\ast}\left( \sigma_\ast \not\in \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}} \right) \le \Pr_{\sigma'_\ast}\left( \max_{a, \r} \|y^{\sigma_\ast}_{a, \r}(T)\|_{{\rm HS}} \ge \frac{R}{\sqrt{n}} \right) \\
    &\qquad\qquad \le e^{-\O_G(R^2) + O_G(1)} + \frac{2 {\mathcal Q}^2}{n^3} \le C_G e^{-R} + \frac{1}{n}
  \end{align*}
  for sufficiently large $C_G$ and $n$.
\end{proof}

We now prove Lemma \ref{Lem:2ndDE-fourier}. Before proceeding with the
main proof, we need the following routine estimate as a preliminary
lemma.

\begin{lemma} \label{Lem:theta}
  Let $\theta_n : {\mathbb R}^d \to {\mathbb R}^+$ be the function given by $\theta_n(x)
  = \|x\| + \frac{1}{\sqrt{n}}e^{-\sqrt{n}\|x\|} -
  \frac{1}{\sqrt{n}}$. Then, we have the inequalities
  \[ \|\nabla \theta_n(x)\| \le 1, \qquad \theta_n(x + h) \le \theta_n(x) + \langle h, \nabla \theta_n(x) \rangle + \frac{\sqrt{n}}{2} \|h\|^2. \]
\end{lemma}
\begin{proof}
  We can write $\theta_n(x) = f(\|x\|)$, where $f(r) = r +
  \frac{1}{\sqrt{n}} e^{-\sqrt{n} r} - \frac{1}{\sqrt{n}}$. By
  spherical symmetry, we have
  \[ \|\nabla \theta_n(x)\| = f'(\|x\|) = 1 - e^{-\sqrt{n}\|x\|} \le 1, \]
  which is the first inequality. Again by spherical symmetry, the
  eigenvalues of the Hessian $\nabla^2 \theta_n(x)$ can be directly
  computed to be $f''(\|x\|)$ and $f'(\|x\|) / \|x\|$. But these are
  bounded by
  \[ f''(r) \le \sqrt{n} e^{-\sqrt{n}r} \le \sqrt{n}, \qquad f'(r)/r \le \frac{1 - e^{-\sqrt{n}r}}{r} \le \sqrt{n}. \]
  Thus, $\nabla^2 \theta_n(x) \preceq \sqrt{n} I$, and the second inequality
  follows from Taylor expansion.
\end{proof}

\begin{proof}[Proof of Lemma \ref{Lem:2ndDE-fourier}]
  Let $\gamma_G$ and $c_G$ be the constants from Lemmas \ref{Lem:gap} and
  \ref{Lem:1stDE-fourier}, respectively. Define the events
  \[ {\mathcal G}_t := \bigcap_{s = 0}^t \left\{ X_\r(\sigma_s) \preceq -\frac{\gamma_G}{n} \right\}, \qquad {\mathcal G}'_t := \bigcap_{s = 0}^t \left\{ X_\r(\sigma_s) \preceq -\frac{1 - \sqrt{d_\r} e^{-c_G s/n} - 2n^{-1/8}}{n} \right\}. \]

  Note that $\sigma'_\ast \in \Sast{\frac{1}{4{\mathcal Q}}} \subseteq
  \Snon{1/3}$. Hence, by Lemmas \ref{Lem:Burn} and \ref{Lem:gap}, we
  have $\Pr({\mathcal G}^{\sf c}_{n^2}) \le C_G n^2 e^{-n/10}$. We also have
  \begin{align*}
    X_\r(s) &= \frac{1}{n - 1} \left( \frac{x_\r(s)+x_\r(s)^\ast}{2} - \frac{n - 1}{n}I_{d_\r} \right) \preceq -\frac{1}{n}\left( 1 - \frac{n \|x_\r(s)\|_{{\rm HS}}}{n - 1} \right) I_{d_\r} \\
    &\preceq -\frac{1}{n}\left( 1 - \|x_\r(s)\|_{{\rm HS}} - \frac{\sqrt{d_\r}}{n-1} \right) I_{d_\r},
  \end{align*}
  where we have used the fact that
  $\left\|\frac{x_\r(s)+x_\r(s)^\ast}{2}\right\|_{op} \le \|x_\r(s)\|_{op} \le
  \|x_\r(s)\|_{{\rm HS}}$.

  Lemma \ref{Lem:1stDE-fourier} then implies that $\Pr({\mathcal G}'^{\sf
    c}_{n^2}) \le \frac{1}{n^3}$. Thus, setting
  \[ \varphi(t) := \max(\gamma_G, 1 - \sqrt{d_\r} e^{-c_G t/n} - 2 n^{-1/8}), \]
  \[ {\mathcal H}_t := {\mathcal G}_t \cap {\mathcal G}'_t = \bigcap_{s = 0}^t \left\{ X_\r(\sigma_s) \preceq -\frac{\varphi(t)}{n} \right\}, \]
  we conclude that
  \[ \Pr({\mathcal H}^{\sf c}_{n^2}) \le \Pr({\mathcal G}^{\sf c}_{n^2}) + \Pr({\mathcal G}'^{\sf c}_{n^2}) \le \frac{2}{n^3} \]
  for all large enough $n$.

  Next, we turn to \eqref{Eq:DEy} and apply $\theta_n$ to both sides,
  where we identify ${\mathbb C}^{d_\r^2}$ with ${\mathbb R}^{2d_\r^2}$. Using Lemma \ref{Lem:theta} and
  taking the conditional expectation, we obtain
  \begin{align*}
    {\mathbb E}\left[ \theta_n\left( y_{a, \r}(t+1) \right) \,\middle|\, {\mathcal F}_t \right] &\le \theta_n\left( y_{a, \r}(t) (I_{d_\r} + X_\r(t))\right) + \frac{8 {\mathcal Q}^4 d_\r}{n \sqrt{n}} \\
    &\le \theta_n(\|I_{d_\r} + X_\r(t)\|_{op} \cdot y_{a, \r}(t)) + \frac{8 {\mathcal Q}^4 d_\r}{n \sqrt{n}} \\
    &\le \|I_{d_\r} + X_\r(t)\|_{op} \cdot \theta_n(y_{a, \r}(t)) + \frac{8 {\mathcal Q}^4 d_\r}{n \sqrt{n}},
  \end{align*}
  where the second inequality follows from the variational formula for
  operator norm (i.e. that $\|BA\|_{{\rm HS}} \le \|A\|_{op} \|B\|_{{\rm HS}}$),
  and the third inequality follows from the fact that $\theta_n$ is
  convex with $\theta_n(0) = 0$.  Thus, we may write
  \[ \theta_n(y_{a, \r}(t+1)) \le \|I_{d_\r} + X_\r(t)\|_{op} \cdot \theta_n(y_{a, \r}(t)) + M'(t+1) \]
  where
  \[ {\mathbb E}[ M'(t+1) \mid {\mathcal F}_t ] \le \frac{8 {\mathcal Q}^4 d_\r}{n\sqrt{n}}, \qquad |M'(t+1)| \le \frac{8{\mathcal Q}^2 \sqrt{d_\r}}{n}. \]

  Now, let $z_t := {\bf 1}_{{\mathcal H}_t} \theta_n(y_{a, \r}(t))$, and note that
  since $X_\r(\sigma) \succeq -\frac{2}{n-1} I_{d_\r}$, we have
  $\|I_{d_\r} + X_\r(t)\|_{op} \le 1-\frac{\varphi(t)}{n}$ whenever ${\mathcal H}_t$
  holds. Thus,
  \[ z_{t+1} \le \|I_{d_\r} + X_\r(t)\|_{op} \cdot z_t + {\bf 1}_{{\mathcal H}_t}M'(t+1) \le \left(1 - \frac{1}{n} \varphi(t) \right) z_t + {\bf 1}_{{\mathcal H}_t}M'(t+1). \]

  We may then apply Lemma \ref{Lem:GenDE} with $\varepsilon = \frac{1}{n}$ and
  $D = 8{\mathcal Q}^4 d_\r$. Note that
  \begin{align*}
    \int_0^t \varphi(s) \,ds &\ge \left(1 - 2 n^{-\frac{1}{8}}\right)t - \sqrt{d_\r} \int_0^\infty e^{-\frac{c_G s}{n}} \,ds \ge t - O_G(n)
  \end{align*}
  for all large enough $n$. Thus, Lemma \ref{Lem:GenDE} implies that
  \begin{equation} \label{Eq:2ndDE-fourier-z_t}
    \Pr\left(z_t \ge \frac{\lambda}{\sqrt{n}} + C_G e^{-t/n} \cdot z_0 \right) \le C'_G e^{-c'_G \lambda^2}.
  \end{equation}
  Consequently,
  \begin{align*}
    &\Pr\left( \|y_{a, \r}(t)\|_{{\rm HS}} \ge R\left( \frac{1}{\sqrt{n}} + e^{-\frac{t}{n}}\|y_{a, \r}(0)\|_{{\rm HS}} \right) \right) \\
    &\qquad\qquad \le \Pr\left( \theta_n(y_{a, \r}(t)) \ge \frac{R - 1}{\sqrt{n}} + Re^{-\frac{t}{n}}\|y_{a, \r}(0)\|_{{\rm HS}} \right) \\
    &\qquad\qquad \le \Pr\left( \theta_n(y_{a, \r}(t)) \ge \frac{R - 1}{\sqrt{n}} + Re^{-\frac{t}{n}}\theta_n(y_{a, \r}(0)) \right) \\
    &\qquad\qquad \le \Pr\left( z_t \ge \frac{R - 1}{\sqrt{n}} + Re^{-\frac{t}{n}}z_0 \right) + \Pr({\mathcal H}^{\sf c}_{n^2}) \\
    &\qquad\qquad \le e^{-\O_G(R^2) + O_G(1)} + \frac{2}{n^3},
  \end{align*}
  as desired.
\end{proof}

\section{Construction of the coupling: Proof of Lemma \ref{Lem:RW}}\label{Sec:Coupling-Proof}

For each $\d>0$, we define a subset of $\{0, 1, \dots, n\}^{G \times
  G}$ by
\[ \Mc_\d:=\left\{ (n_{a, b})_{a, b \in G} \ : \ n_{a, b} \ge \frac{(1-\d) n}{{\mathcal Q}^2} \ \text{for every $a, b \in G$ and}\ \sum_{a, b \in G}n_{a, b}=n\right\}. \]

\begin{lemma}\label{Lem:coupling}
  Consider a configuration $\sigma_\ast \in {\mathcal S}$ and a constant $0<\d \le
  \frac{1}{2{\mathcal Q}^2}$, and assume that $(1 - \d)n/{\mathcal Q}^2$ is an
  integer. Let $(\sigma_t)_{t \ge 0}$ and $(\tilde\sigma_t)_{t \ge 0}$ be two
  product replacement chains started at $\sigma$ and $\tilde\sigma$,
  respectively. Then, there exists a coupling $(\sigma_t, \tilde \sigma_t)$ of
  the Markov chains satisfying the following:

  Let
  \[ D_t:=\frac{1}{2}\sum_{a, b \in G}|n^{\sigma_\ast}_{a, b}(\sigma_t) - n^{\sigma_\ast}_{a, b}(\tilde \sigma_t)|. \]
  Then, on the event $\{(n^{\sigma_\ast}_{a, b}(\sigma_t))_{a, b \in G}, (n^{\sigma_\ast}_{a, b}(\tilde \sigma_t))_{a, b \in G} \in \Mc_\d\}$ and $\{D_t > 0\}$, one has
  \begin{align}
    {\mathbb E}_{\sigma, \tilde \sigma}[D_{t+1}-D_t \mid \sigma_t, \tilde \sigma_t] &\le 0, \label{Eq:D-drift-0} \\
    \Pr_{\sigma, \tilde \sigma}\left(D_{t+1} - D_t \neq 0 \mid \sigma_t, \tilde \sigma_t \right) &\ge \frac{(1-\d)^2}{4{\mathcal Q}^3}. \label{Eq:D-fluctuate}
  \end{align}
\end{lemma}

\begin{proof}
  Let us abbreviate $n_{a, b}(t) = n^{\sigma_\ast}_{a, b}(\sigma_t)$ and $\tilde
  n_{a, b}(t) = n^{\sigma_\ast}_{a, b}(\tilde \sigma_t)$. Let
  $m_{a, b}(t):=\min(n_{a, b}(t), \tilde n_{a, b}(t))$. For each $a \in G$, we
  define the quantity
  \[ d_a(t) := \frac{1}{2}\sum_{b \in G} |n_{a, b}(t) - \tilde{n}_{a, b}(t)| = \sum_{b \in G} n_{a, b}(t) - \sum_{b \in G} m_{a, b}(t), \]
  so that $D_t = \sum_{a \in G} d_a(t)$.

  For accounting purposes, it is helpful to introduce two sequences
  \[ (x_1, x_2, \ldots , x_n) \text{ and } (\tilde{x}_1, \tilde{x}_2, \ldots , \tilde{x}_n) \]
  of elements of $G \times G$. These sequences are chosen so that
  the number of $x_k$ equal to $(a, b)$ is exactly $n_{a, b}$, and
  similarly the number of $\tilde{x}_k$ equal to $(a, b)$ is
  $\tilde{n}_{a, b}$.  Moreover, we arrange their indices in a coordinated
  fashion, as described below.

  We define three families of disjoint sets: $P_{a, b}$, $Q_a$, and $R_a
  \subset [n]$.
  \begin{itemize}
  \item For each $a, b \in G$, let $P_{a, b}$ be a set of size $(1 - \delta)n/{\mathcal Q}^2$ such that for any $k \in P_{a, b}$, we have $x_k
    =\tilde{x}_k = (a, b)$. (This is possible provided that $(n_{a, b}(t)), (\tilde n_{a, b}(t)) \in \Mc_\d$
    holds.)
  \item For each $a \in G$, let $Q_a$ be a set of size $\sum_{b \in G}(m_{a, b} - |P_{a, b}|)$ such that for any $k \in Q_a$, $x_k
    =\tilde{x}_k= (a, b)$ for some $b$. (Note that $Q_a$ may be empty.)
  \item For each $a \in G$, let $R_a$ be a set of size $d_a$ such
    that for any $k \in R_a$, $x_k$ and $\tilde{x}_k$ both have $a$ as
    their first coordinate. (This $R_a$ is well-defined since $\sum_b
    n_{a, b} = \sum_b \tilde n_{a, b}$ for each $a$; it may also be empty.)
  \end{itemize}
  Define
  \[
  P := \bigsqcup_{a, b \in G} P_{a, b}, \qquad Q := \bigsqcup_{a \in G} Q_a, \qquad R := \bigsqcup_{a \in G} R_a.
  \]
  Suppose that $D_t > 0$, so that for some $a_*, b_*, b_*' \in G$ we
  have $n_{a_*, b_*} > \tilde{n}_{a_*, b_*}$ and $n_{a_*, b'_*} <
  \tilde{n}_{a_*, b'_*}$. Let us consider all possible ways to sample a
  pair of indices and a sign $(k, l, s) \in \{1, 2, \ldots , n\}^2
  \times \{\pm 1\}$ with $k \neq l$.

  Suppose $x_k = (a_k, b_k)$ and $x_l = (a_l, b_l)$. We think of $(k, l,
  +1)$ as corresponding to a move on $(n_{a, b}(t))$ where $n_{a_k, b_k}$ is
  decremented and $n_{a_k, (b_k \cdot b_l)}$ is incremented. Similarly, $(k, l,
  -1)$ corresponds to a move where $n_{a_k, b_k}$ is decremented and
  $n_{a_k, (b_k \cdot b_l^{-1})}$ is incremented. We may also think of $(k, l, \pm
  1)$ as corresponding to moves on $(\tilde n_{a, b}(t))$ in an analogous way.

  \begin{figure}
    \includegraphics{pqr_figure.pdf}
    \caption{Illustration of cases (i) through (iv).}
    \label{fig:coupling-cases}
  \end{figure}

  We now analyze four cases, as illustrated in Figure \ref{fig:coupling-cases}.

  \paragraph{(i) {\bf Case $(k, l) \in (P \sqcup Q) \times (P \sqcup Q)$.}}

  For all but an exceptional situation described below, we apply the
  move corresponding to $(k, l, s)$ to both states $(n_{a, b}(t))$ and
  $(\tilde n_{a, b}(t))$. In these cases, $D_{t+1}=D_t$.

  We now describe the exceptional situation. Define
  \[ S = P_{a_*, b_*} \times \left(\bigsqcup_{c \in G} P_{c, (b_*^{-1} \cdot b'_*)}\right) \qquad\text{and}\qquad S' = P_{a_*, b'_*} \times \left(\bigsqcup_{c \in G} P_{c, {\rm id}}\right). \]
  Then, the exceptional situation occurs when $s = +1$ and $(k, l) \in
  S \sqcup S'$.

  Take any bijection $\tau$ from $S$ to $S'$. If $(k, l) \in S$, then
  we apply $(k, l, +1)$ to $(n_{a, b}(t))$ while applying $(\tau(k, l),
  +1)$ to $(\tilde n_{a, b}(t))$. This increments $n_{a_*, b'_*}$,
  decrements $n_{a_*, b_*}$, and has no effect on the
  $(\tilde{n}_{a, b}(t))$. The overall effect is that $D_{t+1} = D_t - 1$.

  If instead $(k, l) \in S'$, then we apply $(k, l, +1)$ to $(n_{a,
    b}(t))$ and $(\tau^{-1}(k, l), +1)$ to $(\tilde n_{a, b}(t))$. A
  similar analysis shows that in this case $D_{t+1} = D_t + 1$.

  The exceptional event occurs with probability $\frac{(1 -
    \d)^2}{2{\mathcal Q}^3}$, and when it occurs, $D_t$ increases or decreases
  by $1$ with equal probability. Thus, the exceptional situation plays
  the role of introducing some unbiased fluctuation in $D_t$ and gives
  us \eqref{Eq:D-fluctuate}. \\

  \paragraph{(ii) {\bf Case $(k, l) \in (Q \sqcup R) \times (Q \sqcup R)$ but $(k, l) \not\in Q \times Q$.}}

  This occurs with probability
  \[ \frac{1}{n(n-1)}\left((|Q|+|R|)(|Q|+|R|-1)-|Q|(|Q|-1)\right) \]
  which is at most
  \[ \frac{2}{n(n - 1)}(|Q| + |R|) |R|  = \frac{2 \d}{n - 1}D_t. \]
  Apply the move corresponding to $(k, l, s)$ to both states.  This
  increases $D_t$ by at most $1$. We will see later that the effect of
  this case is small compared to the other cases. \\

  \paragraph{(iii) {\bf Case $(k, l) \in P \times R$.}}

  This occurs with probability
  \[ \frac{1}{n(n - 1)}|P| |R| =\frac{1 - \d}{n - 1}D_t. \]
  Apply the move corresponding to $(k, l, s)$ to both states. Again,
  this increases $D_t$ by at most $1$, but there is also a chance not
  to increase.

  Suppose that $x_l = (a_1, b_1)$ and $\tilde{x}_l = (a_1,
  \tilde{b}_1)$, and suppose that $k \in P_{a_2, b_2}$. Then the move has
  the effect of decreasing $n_{a_2, b_2}$ and $\tilde{n}_{a_2, b_2}$ while
  increasing $n_{a_2, (b_2 \cdot b_1^s)}$ and $\tilde{n}_{a_2,
    (b_2\cdot \tilde{b}_1^s)}$.
  Note that conditioned on this case happening, $(a_2, b_2)$ is
  distributed uniformly over $G \times G$. When $(a_2,
  (b_2\cdot \tilde{b}_1^s)) = (a_*, b_*)$ or $(a_2,
  (b_2\cdot b_1^s)) = (a_*, b'_*)$, the move does not increase
  $D_t$. Therefore there is at least a $2/{\mathcal Q}^2$ chance that $D_t$ is
  actually not increased. Hence, the probability that $D_t$ is
  increased by $1$ is at most
  \[ \left(1 - \frac{2}{{\mathcal Q}^2}\right)\frac{1 - \d}{n-1}D_t. \]

  \paragraph{(iv) {\bf Case $(k, l) \in R \times P$.}}

  This occurs with probability
  \[ \frac{1}{n(n - 1)} |R| |P| = \frac{1 - \d}{n - 1} D_t. \]
  Suppose that $x_k = (a, b)$ and $\tilde{x}_k = (a, \tilde{b})$.  Let
  $\tau$ be a permutation of $P$ such that for $l \in P_{a, c}$, one has
  $\tau(l) \in P_{a, \tilde{b}^{-1}\cdot b \cdot c^s}$.  Then apply $(k, l, s)$ to $(n_{a, b}(t))$
  and apply $(k, \tau(l), s)$ to $(\tilde n_{a, b}(t))$.  This always decreases
  $D_t$ by $1$. \\

  Let us now summarize what we know when $(n_{a, b}(t)), (\tilde n_{a, b}(t)) \in \Mc_\d$ and $D_t>0$.
  From Cases (i), (ii), and (iii), we have
  \[ \Pr_{\sigma, \tilde \sigma}(D_{t+1} = D_t + 1 \mid \sigma_t, \tilde \sigma_t) \le \left(1-\frac{2(1-\d)}{{\mathcal Q}^2}+\d\right)\frac{D_t}{n-1} + \frac{(1 - \d)^2}{4{\mathcal Q}^3}. \]
  From Cases (i) and (iv), we have
  \[ \Pr_{\sigma, \tilde \sigma}(D_{t+1} = D_t - 1 \mid \sigma_t, \tilde \sigma_t) \ge (1-\d)\frac{D_t}{n-1}  + \frac{(1 - \d)^2}{4{\mathcal Q}^3}. \]
  Therefore, if $0<\d \le \frac{1}{2{\mathcal Q}^2}$, then
  \[{\mathbb E}_{\sigma, \tilde \sigma}[D_{t+1}-D_t \mid \sigma_t, \tilde \sigma_t] \le 0, \]
  verifying \eqref{Eq:D-drift-0}.

  To fully define the coupling, when $D_t = 0$, we can couple $\sigma_t$
  and $\sigma_t$ to be identical, and if either $(n_{a, b}(t)) \notin
  \Mc_\d$ or $(\tilde n_{a, b}(t)) \notin \Mc_\d$, we may run the two
  chains independently.
\end{proof}

\begin{proof}[Proof of Lemma \ref{Lem:RW}]
  Since $\sigma \in \Sast{\sigma_\ast, \frac{R}{\sqrt{n}}}$, we must have for
  each $a \in G$ and $\r \in G^\ast$ that $\|y^{\sigma_\ast}_{a, \r}(\sigma)\|_{{\rm HS}}
  \le \frac{R}{\sqrt{n}}$. Note that for large enough $n$, we have
  $\Sast{\sigma_\ast, \frac{R}{\sqrt{n}}} \subseteq
  \Sast{\frac{1}{5{\mathcal Q}^3}}$. Thus, we may apply Lemma
  \ref{Lem:2ndDE-fourier} to obtain
  \begin{equation} \label{Eq:RW-1}
    \Pr\left( \bigcup_{t = 0}^{n^2} \left\{ \|y^{\sigma_\ast}_{a, \r}(\sigma_t)\|_{{\rm HS}} \ge \frac{1}{5{\mathcal Q}^3} \right\} \right) \le n^2 \left( e^{-\O_{G}(n) + O_{G}(1)} + \frac{2}{n^3} \right) \le \frac{3}{n}
  \end{equation}
  for large enough $n$. Define the event
  \[ {\mathcal G}_t := \left\{ \sigma_s \in \Sast{\sigma_\ast, \frac{1}{5{\mathcal Q}^3}} \text{ for all $1 \le s \le t$} \right\}. \]
  The Plancherel formula applied to \eqref{Eq:RW-1} implies that
  $\Pr({\mathcal G}^{\sf c}_{n^2}) \le \frac{3{\mathcal Q}^2}{n}$. We may analogously define
  an event $\tilde{\mathcal G}_t$ for $\tilde\sigma$ and let ${\mathcal A}_t := {\mathcal G}_t \cap
  \tilde{\mathcal G}_t$. Thus, $\Pr({\mathcal A}_{n^2}^{\sf c}) \le \frac{6{\mathcal Q}^2}{n}$.

  Pick $\d' \in \left(\frac{2}{5{\mathcal Q}^2}, \frac{3}{7{\mathcal Q}^2}\right)$ so that $(1 - \d')n/{\mathcal Q}^2$ is an
  integer. Note that when ${\mathcal A}_t$ holds, we have
  \[ \sigma_t \in \Sast{\sigma_\ast, \frac{1}{5{\mathcal Q}^3}} \quad \text{and} \quad \sigma_\ast \in \Sast{\frac{1}{5{\mathcal Q}^3}}\implies (n_{a, b}(t)) \in \Mc_{\frac{2}{5{\mathcal Q}^2}} \subseteq \Mc_{\d'}, \]
  and similarly $\tilde\sigma_t \in \Mc_{\d'}$.

  Thus, we may invoke Lemma \ref{Lem:coupling} to give a coupling
  between $\sigma$ and $\tilde\sigma$ where on the event ${\mathcal A}_t$, the quantity
  $D_t$ is more likely to decrease than increase. Letting ${\bf
    D}_t:={\bf 1}_{{\mathcal A}_t}D_t$, we see that $({\bf D}_t)$ is a
  supermartingale with respect to $({\mathcal F}_t)$.

  Define
  \[ \t := \min\{ t \ge 0 : D_t=0 \}, \qquad {\tilde \t} := \min\{ t \ge 0 : {\bf D}_t=0 \}. \]
  Then, Lemma \ref{Lem:coupling} ensures that on the event $\{\tilde\t
  > t\}$, we have ${\rm Var}({\bf D}_{t+1}\mid {\mathcal F}_t) \ge \a^2$, where
  $\a^2:=\left(1- \frac{1}{{\mathcal Q}^2}\right)\frac{(1-\d')^2}{4{\mathcal Q}^3}$. By
  \cite[Proposition 17.20]{LevinPeresWilmer}, for every $u > 12/\a^2$,
  \begin{equation}\label{Eq:RW}
    \Pr(\tilde \t > u) \le \frac{4 {\bf D}_0}{\a\sqrt{u}}.
  \end{equation}
  Recall that $T = \ceil{\b n}$ and $D_0 \le \sqrt{{\mathcal Q}} R\sqrt{n}$. As
  long as $\b$ is large enough, we may apply \eqref{Eq:RW} with $u =
  T$ to get
  \[ \Pr_{\sigma, \tilde \sigma}(\t > T) \le \frac{16 {\mathcal Q}^2 R}{(1-\d')\sqrt{\b}} + \Pr({\mathcal A}_T^{\sf c}) \le \frac{32 {\mathcal Q}^2 R}{\sqrt{\b}} \]
  for all large enough $n$, as desired.
\end{proof}

\section{Proof of Theorem \ref{Thm:main} (\ref{Eq:LB})}\label{Sec:LB}

\newcommand{n^{\{\id\}}_{non}}{n^{\{{\rm id}\}}_{non}}

The lower bound is proved essentially by showing that the estimates of
Lemmas \ref{Lem:Burn} and \ref{Lem:2ndDE} cannot be improved. Let
$a_1, a_2, \ldots , a_k$ be a set of generators for $G$. Let $\sigma_\star
\in {\mathcal S}$ be the configuration given by
\[ \sigma_\star(i) = \begin{cases}
  a_i & \text{if $i \le k$,} \\
  0 & \text{otherwise}.
\end{cases} \]
We will analyze the Markov chain started at $\sigma_\star$ and show that
it does not mix too fast.

Recall from Section \ref{Sec:UB} the notation
\[ n^{\{\id\}}_{non}(\sigma) = |\{ i \in [n] : \sigma(i) \ne {\rm id} \}| \]
for the number of sites in $\sigma$ that do not contain the identity. We
first show that if we run the chain for slightly less than $n \log n$
steps, most of the sites will still contain the identity.

\begin{lemma} \label{Lem:Burn-lower}
  Let $T := \floor{n \log n - Rn}$. Then,
  \[ \Pr_{\sigma_\star}\left( n^{\{\id\}}_{non}(\sigma_T) \ge \frac{n}{3} \right) \le \frac{4{\mathcal Q}^2}{R^2}. \]
\end{lemma}
\begin{proof}
  Recall that in one step of our Markov chain, we pick two indices $i,
  j \in [n]$ and replace $\sigma(i)$ with $\sigma(i) \cdot \sigma(j)$ or $\sigma(i) \cdot \sigma(j)^{-1}$.
  The only way for $n^{\{\id\}}_{non}(\sigma_t)$ to increase after this step is
  if $\sigma(j) \ne {\rm id}$. Thus,
  \begin{equation} \label{Eq:nnon}
    \Pr( n^{\{\id\}}_{non}(\sigma_{t+1}) = n^{\{\id\}}_{non}(\sigma_t) + 1 \mid n^{\{\id\}}_{non}(\sigma_t)) \le \frac{n^{\{\id\}}_{non}(\sigma_t)}{n}.
  \end{equation}

  Let $\t := \min\{ t \ge 0 : n^{\{\id\}}_{non}(\sigma_t) \ge \frac{n}{3} \}$ be the
  first time that $n^{\{\id\}}_{non}(\sigma_t)$ is at least $\frac{n}{3}$. We have
  that $n^{\{\id\}}_{non}(\sigma_\star) = k$, so it follows from \eqref{Eq:nnon} that
  $\t$ stochastically dominates the sum
  \[ G := \sum_{s = k}^{\floor{n/3}} G_s, \]
  where the $G_s$ are independent geometric variables with success
  probability $\frac{s}{n}$. Note that we have the bounds
  \[ {\mathbb E} G = \sum_{s = k}^{\floor{n/3}} \frac{n}{s} \ge n \left( \log \floor{\frac{n}{3}} - \log k \right), \qquad {\rm Var}(G) = \sum_{s = k}^{\floor{n/3}} \frac{n(n-s)}{s^2} \le n^2. \]
  Hence,
  \begin{align*}
    \Pr(\t < T) &\;\le\; \Pr(G < T) \;\le\; \Pr(G < {\mathbb E} G + n\log(3k) - Rn ) \\
    &\;\le\; \frac{n^2}{n^2(R - \log(3k))^2} \le \frac{4}{R^2}
  \end{align*}
  for $R \ge 2 {\mathcal Q} \ge 2 \log (3k)$. On the other hand, the bound
  claimed in the lemma is trivial for $R \le 2 {\mathcal Q}$, so we have
  completed the proof.
\end{proof}

Next, we show that it really takes about $\frac{1}{2}n \log n$ steps
for the Fourier coefficients $x_\r$ to decay to $O\left(
\frac{1}{\sqrt{n}} \right)$, as suggested by Lemma
\ref{Lem:2ndDE}. Note that it suffices here to analyze the $x_\r$
instead of the $y_{a, \r}$, which simplifies our analysis.  Actually,
it suffices to consider (the real part of) the trace of $x_\r$.  Here
the orthogonality of characters reads $\frac{1}{{\mathcal Q}}\sum_{a \in G} \Tr
\r(a)=0$, and it takes about $\frac{1}{2}n \log n$ steps for $\Re \Tr
x_\r(t)$ to decay to $O\left( \frac{1}{\sqrt{n}} \right)$.

\begin{lemma} \label{Lem:DE-lower}
  Consider any $\r \in G^\ast$ and any $R > 5$. Let $T :=
  \floor{\frac{1}{2}n \log n - Rn}$, and suppose that $\sigma \in {\mathcal S}$
  satisfies $n^{\{\id\}}_{non}(\sigma) \le \frac{n}{3}$. Then,
  \[ \Pr_{\sigma}\left( \|x_\r(\sigma_T)\|_{{\rm HS}} \le \frac{R}{\sqrt{n}} \right) \le \frac{4{\mathcal Q}^2}{R^2}. \]
\end{lemma}
\begin{proof}
  Let $z(t) := (1/d_\r) \Tr (x_\r(t)+x_\r(t)^\ast)/2$.
  Then, noting that \eqref{Eq:DEfourier} also holds for $x_\r(t)^\ast$ since $x_{\r^\ast}(t)=x_\r(t)^\ast$,
  we have
  \[ z(t+1)-z(t) = \frac{1}{n-1}\frac{1}{d_\r} \Tr \left(\frac{x_\r(t)+x_\r(t)^\ast}{2}\right)^2-\frac{1}{n}z(t) + M(t+1), \]
  where
  \[ {\mathbb E}[M(t+1) \mid {\mathcal F}_t]=0 \qquad \text{and} \qquad |M(t)| \le \frac{2{\mathcal Q}}{n}. \]
  Here we have
  \[\frac{1}{d_\r} \Tr \left(\frac{x_\r(t)+x_\r(t)^\ast}{2}\right)^2 \ge z(t)^2.\]
  We compare $z(t)$ to another process $(w(t))_{t \ge 0}$ defined by
  $w(0) := \frac{1}{3}$ and
  \begin{equation} \label{Eq:w-equation}
    w(t+1) := \left(1-\frac{1}{n}\right)w(t) + M(t+1).
  \end{equation}

  We will show by induction that $z(t) \ge w(t)$ for all $t$. For the
  base case, note that since $n^{\{\id\}}_{non}(\sigma) \le \frac{n}{3}$, we have
  \[ z(0) = \frac{1}{d_\r}\Re \Tr \sum_{a \in G} \frac{n_a(t)}{n} \cdot \r(a) \ge \frac{2}{3} - \frac{1}{3} = \frac{1}{3}. \]
  Suppose now that $z(t) \ge w(t)$. Then,
  \begin{align*}
    z(t+1) &\ge z(t) + \frac{1}{n-1}z(t)^2-\frac{1}{n}z(t) + M(t+1) \\
      		&\ge \left(1 - \frac{1}{n}\right) w(t) + M(t+1) = w(t+1),
  \end{align*}
  completing the induction.

  It now suffices to lower bound $w(T)$. To this end, we first note
  that applying \eqref{Eq:w-equation} repeatedly and taking
  expectations, we obtain
  \[ {\mathbb E} w(T) = \left(1 - \frac{1}{n}\right)^T \cdot \frac{1}{3} \ge \frac{e^R}{6\sqrt{n}} \ge \frac{2R}{\sqrt{n}}. \]
  In order to calculate the variance of $w(T)$, we can also square
  \eqref{Eq:w-equation} and take the expectation, which gives us
  \begin{align*}
    {\rm Var}(w(T)) &= {\mathbb E} w(T)^2 - ({\mathbb E} w(T))^2 \\
    &\le \left(1 - \frac{1}{n}\right)^{2T} \cdot \frac{1}{9} + n \cdot \left( \frac{2{\mathcal Q}}{n} \right)^2  -  \left( \left(1 - \frac{1}{n}\right)^T
  \cdot \frac{1}{3} \right)^2 \\
    &= \frac{4{\mathcal Q}^2}{n}.
  \end{align*}
  Then, by Chebyshev's inequality, we have
  \begin{align*}
    \Pr_{\sigma}\left( \|x_\r(\sigma_T)\|_{{\rm HS}} \le \frac{R}{\sqrt{n}} \right) &\le \Pr\left( z(T) \le \frac{R}{\sqrt{n}} \right) \le \Pr\left( w(T) \le \frac{R}{\sqrt{n}} \right) \\
    &\le \frac{4{\mathcal Q}^2/n}{(R/\sqrt{n})^2} = \frac{4{\mathcal Q}^2}{R^2},
  \end{align*}
  as desired.
\end{proof}

\begin{proof}[Proof of Theorem \ref{Thm:main} (\ref{Eq:LB})]
  Let $T = T_1 + T_2$, where $T_1 := \floor{n \log n - \b n}$ and $T_2
  := \floor{\frac{1}{2} n \log n - \b n}$. Fix any $\r \in G^\ast$. By
  Lemma \ref{Lem:Burn-lower} followed by Lemma \ref{Lem:DE-lower}, we
  have for large enough $\b$ that
  \[ \Pr_{\sigma_\star} \left(\sigma_T \in \Sast{\frac{\b}{\sqrt{n}}} \right) \le \Pr_{\sigma_\star}\left( \|x_\r(T)\|_{{\rm HS}} \le \sqrt{\frac{{\mathcal Q}}{d_\r}}\frac{\b}{\sqrt{n}} \right) \le \frac{8{\mathcal Q}^2}{\b^2}. \]
  On the other hand, Lemma \ref{Lem:stationary} tells us that
  \[ \pi\left({\mathcal S}_\ast\left(\frac{\b}{\sqrt{n}}\right)\right) \ge 1-
  \frac{c_G}{\b^2}. \]
  Consequently,
  \[ d_{\sigma_\star}(T) \ge 1 - \frac{c_G}{\b^2} - \frac{8{\mathcal Q}^2}{\b^2}, \]
  which tends to $1$ as $\b \rightarrow \infty$, establishing
  \eqref{Eq:LB}.
\end{proof}

\section*{Acknowledgements}

This work was initiated while R.T. and A.Z. were visiting Microsoft
Research in Redmond. They thank Microsoft Research for the
hospitality. R.T. was also visiting the University of Washington in
Seattle and thanks Professor Christopher Hoffman for making his visit
possible. R.T. is supported by JSPS Grant-in-Aid for Young
Scientists (B) 17K14178. A.Z. is supported by a Stanford Graduate Fellowship.

\bibliographystyle{alpha}