On Improved Bounds on Bounded Degree Spanning Trees for Points in Arbitrary Dimension

Abstract: Given points in Euclidean space of arbitrary dimension, we prove that there exists a spanning tree having no vertices of degree greater than 3 with weight at most 1.559 times the weight of the minimum spanning tree. We also prove that there is a set of points such that no spanning tree of maximal degree 3 exists that has this ratio be less than 1.447. Our central result is based on the proof of the following claim:Given $n$ points in Euclidean space with one special point $V$, there exists a Hamiltonian path with an endpoint at $V$ that is at most 1.559 times longer than the sum of the distances of the points to $V$.These proofs also lead to a way to find the tree in linear time given the minimal spanning tree.

\section{Abstract}

Given points in Euclidean space of arbitrary dimension, we prove that there exists a spanning tree having no vertices of degree greater than 3 with weight at most 1.559 times the weight of the minimum spanning tree.  We also prove that there is a set of points such that no spanning tree of maximal degree 3 exists that has this ratio be less than 1.447.  Our central result is based on the proof of the following claim:

Given $n$ points in Euclidean space with one special point $v$, there exists a Hamiltonian path with an endpoint at $v$ that is at most 1.559 times longer than the sum of the distances of the points to $v$.

These proofs also lead to a way to find the tree in linear time given the minimal spanning tree.

\section{Introduction}
\label{sec:problem}
The minimum spanning tree (MST) problem in graphs is perhaps one of the most basic problems in graph algorithms.  An MST is a spanning tree with minimal sum of edge weights.  Efficient algorithms for finding an MST are well known.

One variant on the MST problem is the bounded degree MST problem, which consists of finding a spanning tree satisfying given upper bounds on the degree of each vertex and with minimal sum of edges weights subject to these degree bounds.

In general, this problem is NP-hard \cite{NPhard}, so no efficient algorithm exists.  However, there are certain achievable results.  For undirected graphs, Singh and Lau \cite{SinghLau} found a polynomial time algorithm to generate a spanning tree with total weight no more than that of the bounded degree MST and with each vertex having degree at most one greater than that vertex's bound.  If the graph is undirected and satisfies the triangle inequality, Fekete and others \cite{network} bound the ratio of the total weight of the bounded-degree MST to that of any given tree, with a polynomial-time algorithm for generating a spanning tree satisfying the degree constraints and this ratio bound.

The Euclidean case, with vertices being points in Euclidean space and edge weights being Euclidean distances, also has a rich history.  We denote (following Chan in \cite{1.633}) by $\tau_k^d$ the supremum, over all sets of points in $d$-dimensional Euclidean space, of the ratio of the weight of the bounded degree MST with all degrees at most $k$ to the weight of the MST with no restrictions on degrees ($\tau_k^\infty$ is the supremum of $\tau_k^d$ over all $d$).  For $k=2$, the bounded-degree MST problem becomes the Traveling Salesman Problem and $\tau_2^d=2$ \cite{network}, thus making $k=3$ the first unsolved case.

Papadimitriou and Vazirani \cite{NPhard} showed that finding the degree-3 MST is NP-hard.  Khuller, Raghavachari, and Young \cite{1.67} showed that $1.104\approx (\sqrt{2}+3)/4 \leq \tau_3^2 \leq 1.5$ and $1.035< \tau_4^2 \leq 1.25$. Chan \cite{1.633} improved the upper bounds to 1.402 and 1.143, respectively.
Jothi and Raghavachari \cite{degree4} showed that $\tau_4^2 \leq (2+\sqrt{2})/3\approx 1.1381$.  $\tau_5^2=1$ since there is always an MST with maximal degree 5 or less \cite{degree5}.

These same papers also studied the problem in higher dimensions.  Khuller, Raghavachari, and Young \cite{1.67} gave an upper bound on $\tau_3^\infty$ of $5/3\approx 1.667$, which Chan \cite{1.633} improved to $2\sqrt{6}/3\approx 1.633$.  These two followed the same approach, proving these bounds on a certain ratio, which we will call $r$. $r$ is the maximum ratio between the shortest path through a collection of points starting at a special point and the size of a star centered at that point. It is conjectured to actually be 1.5.

Khuller, Raghavachari, and Young \cite{1.67} showed that $\tau_3^\infty \leq r$.  This is achieved in linear time as follows:
\begin{enumerate}
\item root the original tree
\item treating the root as $v$, find a Hamiltonian path with ratio at most $r$ through its children.
\item repeat recursively on each child.
\end{enumerate}
Each vertex then has at most 3 neighbors: two as a child and one as a parent.

We improve previous upper bounds on $r$, and thus $\tau_3^\infty$, to 1.559. The proof leads to a linear time algorithm for generating the path and thus the bounded degree tree.  Our approach is based on Chan's, but we weigh paths differently and select the number of points to remove when performing the induction based on the distances of points to $v$.

We also find, by construction, a non-trivial lower bound of about 1.447 on $\tau_3^\infty$.

In Section \ref{sec:lemmas}, we go over $r$ a bit more carefully as well as refering to a useful paper and discuss how we will use it.  In Section \ref{sec:upperbound}, we improve the upper bound on $r$ to 1.559, and in Section \ref{sec:lowerbound} we improve the lower bound on $\tau_3^\infty$ to 1.447.

\section{Preliminaries}
\label{sec:lemmas}
$r$ is properly defined as follows:

Given point $v$ and $m$ points $a_1,a_2,\ldots,a_m$ in a Euclidean space of arbitrary finite dimension, let $\displaystyle S=\sum_{i=1}^n d(v,a_i)$ and let $L$ be the length of the shortest possible path that starts at $v$ and goes around the other points in some order (it does not go back to $v$).  Then $r$ is the supremum of the possible values of $L/S$ over all arrangements of points in any number of dimensions.  $r=1.5$ is achieved for $m=2$ in one dimension by the points $v=0,a_1=1,a_2=-1$.

We use the results of Young \cite{distsum} multiple times in order to bound certain sums of distances.
This paper deals with the maximum of weighted sums (with weights $w_{i,j}$) of lengths between $n$ points in $n-1$ dimensional Euclidean space, given that each point $a_i$ is specified as being no further than some distance $l_i$ from the origin.
\begin{equation}\label{eq: Young}
\max \left(\sum_{1\leq i<j\leq n} w_{i,j} d(a_i,a_j) \right)=\min \left(\sqrt{\sum_{1\leq i<j\leq n} \frac{w^2_{\displaystyle i,j}}{x_ix_j}}\sqrt{\sum_{i=1}^n l^2_ix_i}\sqrt{\sum_{i=1}^n x_i}\right)
\end{equation}
where the maximum is taken over all arrangements of points and the minimum is taken over all nonnegative $x_i$.

Furthermore, Young specifies a relationship between the optimal arrangement and the values of $x_i$ where equality is achieved.  Thus one can iteratively approximate the optimal arrangement using the same method as in \cite{experimental}, and then calculate $x_i$ values from it.

Whenever \eqref{eq: Young} is used to give an upper bound on some weighted sum of distances, the values for $x_i$ used are given in Appendix \ref{sec: xis}.

\section{Main proof of upper bound on $r$}
\label{sec:upperbound}
Let $r^*=1.559$. We will prove that $L\leq r^*S$ (as $L$ and $S$ are defined in the introduction), thus showing that $r<1.559$.

We will prove this by strong induction on the number of points. Given $m$ vectors $a_1, a_2,\ldots,a_m$ with norms $d_1 \geq d_2 \geq d_3 \geq \ldots \geq d_m > 0$, respectively, we will try to induct by removing $a_1,\ldots,a_n$ for various values of $n$. We will try to traverse the other points, ending at $a_{n+1}$ or $a_{n+2}$. We will then add in the removed points, projected onto a sphere, and look at the average length of a path traversing them and ending at $a_1$ or $a_2$. We will then move them out in stages, seeing how this average path length changes at each stage, in order to bound the final average path length in terms of the values $d_k$. Since the average is an upper bound on the minimum, this gives us a linear inequality on the $d_k$ which is a sufficient condition for the inductive step to work. We then use linear programming to show that one of these inequalities is satisfied and thus that induction is possible. For the algorithm, we will then follow the induction to split the points up into blocks, choose the starting and ending vertex for one block at a time, using brute force to find the shortest path that goes through all the block's points.

We start by defining $a_k=0$ and $d_k=0$ for all $k>m$. Introducing these new points does not affect the distance sum or the traversing path length, as the traversing path can go to them first.

We will prove the following claim:
\begin{claim}\label{claim}
There exist two paths $P_1$ and $P_2$ ending at $a_1$ and $a_2$, respectively, such that the average of the lengths of these paths is at most $r^*S$
\end{claim}

This clearly implies that $L\leq r^*S$.

We will proceed by strong induction on $m$. To induct, remove $a_1$ through $a_n$ (where $n\geq 3$ may vary), use the inductive hypothesis to find two paths $P_1$ and $P_2$ through the other $m-n$ points, ending at $a_{n+1}$ and $a_{n+2}$, respectively. We will then try to find four paths $Q_{11}, Q_{12}, Q_{21}, Q_{22}$ with path $Q_{ij}$ going from $a_{n+i}$ to $a_j$ and going through all points $a_1, \ldots, a_n$, so that the average length of these four paths is at most $r^*(\sum_{i=1}^n d_i)$.

We will assume that this is impossible, generate a set of conditions on the values $d_n$, then prove that one of the conditions must be violated.

\subsection{Given $n \geq 3$}
\label{subsec:given}
In this section, we will assume $n>3$ to be a given value.  We will select it in Section \ref{sec: recombine}.

\begin{figure}[htbp]
\centering
\includegraphics[scale=.5]{LandSn.png}
\label{fig:thickseg}
\caption{The thick segments contribute to $L(a_n, \ldots, a_1)$; the dotted segments contribute to $S_n$.}
\end{figure}

Let $\displaystyle S_n=\sum_{i=1}^n d_i$.
Let $L(u_{n+1}, \ldots, u_1)$ be the shortest length of a path $u_{s_{n+1}}, \ldots, u_{s_1}$ where $s$ is a permutation of $1, \ldots, n+1$ so that $s_{n+1}=n+1$ and $\{s_1,s_2\}=\{1,2\}$.
Let $\overline{L(u_{n+1}, \ldots, u_1)}$ be the average length over all such paths $u_{s_{n+1}}, \ldots, u_{s_1}$. Then
\begin{equation} \label{eq: L()}
	\begin{split}
		L(u_{n+1}, \ldots, u_1)&=\frac{1}{n-1}d(u_1,u_2)+\frac{1}{2(n-1)}d(u_1,u_{n+1})+\frac{1}{2(n-1)}d(u_2,u_{n+1})\\
		+&\sum_{i=3}^n \frac{3}{2(n-1)}d(u_1,u_i)+\sum_{i=3}^n \frac{3}{2(n-1)}d(u_2,u_i)\\
		+&\sum_{i=3}^n \frac{1}{n-1}d(u_i,d_{n+1})+\sum_{3\leq i<j\leq n} \frac{2}{n-1}d(u_i,u_j)
	\end{split}
\end{equation}

We wish to find upper bounds on $\overline{L(a_{n+1}, \ldots, a_1)}$ and $\overline{L(a_{n+2},a_n,a_{n-1}, \ldots, a_1)}$. For $1\leq~i\leq~n$, let
\begin{equation} \label{eq: defineDnik}
	\begin{split}
		D_{n,i,1}&=\overline{L \left( a_{n+1}, \ldots, a_{i+1},a_i, \frac{d_i}{d_{i-1}}a_{i-1}, \ldots, \frac{d_i}{d_1}a_1 \right)} \\
		&- \overline{L \left( a_{n+1}, \ldots, a_{i+1}, \frac{d_{i+1}}{d_{i}}a_i, \frac{d_{i+1}}{d_{i-1}}a_{i-1}, \ldots, \frac{d_{i+1}}{d_1}a_1 \right)}.
	\end{split}
\end{equation}
and let
\begin{equation} \label{eq: defineDnk}
D_{n,1}=\overline{L \left( a_{n+1},\frac{d_{n+1}}{d_n}a_n, \ldots, \frac{d_{n+1}}{d_1}a_1 \right)}.
\end{equation}
$D_{n,i,2}$  and $D_{n,2}$ are defined identically, except $a_{n+1}$ and $d_{n+1}$ are replaced with $a_{n+2}$ and $d_{n+2}$.
For $k=1$ or $k=2$,
\begin{equation} \label{eq: decompose}
\overline{L(a_{n+k},a_n,a_{n-1},\ldots,a_1)}=D_{n,k}+\sum_{i=1}^n  D_{n,i,k}
\end{equation}
Intuitively, we are setting all points at distance $d_{n+k}$, then moving out $n$ points to distance $d_n$, then moving out $n-1$ points, and so on.

We will now find values $B_{n,i}$ and $B_n$ independent of the arrangement of $a_1,a_2,\ldots$ satisfying
\begin{equation}\label{eq: UgeqD}
	\begin{split}
		B_{n,i}(d_i-d_{i+1}) &\geq D_{n,i,1}\\
		B_n d_n &\geq D_{n,1}
	\end{split}
\end{equation}
The corresponding equations (substituting $a_{n+2}$ for $a_{n+1}$ and $d_{n+2}$ for $d_{n+1}$) will then hold for $D_{n,i,2}$ and $D_{n,2}$.

\subsubsection{$B_n$}
Define $g(n)$ as the maximum value of
\begin{align*}
	d(u_1,u_2)+\frac{1}{2}d(u_1,u_{n+1})+\frac{1}{2}d(u_2,u_{n+1})+\sum_{j=3}^n\frac{3}{2}d(u_1,u_j)+\\
	+\sum_{j=3}^n\frac{3}{2}d(u_2,u_j)+\sum_{j=3}^n d(u_j,u_{n+1})+\sum_{3\leq j<k\leq n} 2d(u_j,u_k)
\end{align*}
over unit vectors $u_1,\ldots,u_{n+1}$.

We use equation \eqref{eq: Young} to obtain upper bounds on $g(n)$, which we then use to find numerical values for $B_n$.

Substituting in $\eqref{eq: defineDnk}$ and $\eqref{eq: L()}$, we get that
\[D_{n,1}\leq d_n\frac{g(n)}{n-1}\]
and similarly for $D_{n,2}$. Thus we can set
\begin{equation} \label{eq:Bn}
B_n=\frac{g(n)}{n-1}
\end{equation}

\subsubsection{$B_{n,i}$}
For $i<j<k$,
\begin{equation} \label{eq: bothstay}
	d(a_j,a_k)-d(a_j,a_k)=0.
\end{equation}
For $j\leq i< k$
\begin{equation} \label{eq: onemoves}
	d \left( \frac{d_i}{d_j}a_j,a_k \right)-d \left(\frac{d_{i+1}}{d_j}a_j,a_k \right)\leq d_i-d_{i+1}.
\end{equation}
For $k<j\leq i$
\begin{equation} \label{eq: bothmove}
	d \left( \frac{d_i}{d_j}a_j,\frac{d_i}{d_j}a_k \right)-d \left( \frac{d_{i+1}}{d_j}a_j,\frac{d_{i+1}}{d_j}a_k \right)\leq (d_i-d_{i+1}) d\left( \frac{a_j}{d_j},\frac{a_k}{d_k} \right).
\end{equation}

Define also $f(i)$ as the maximum value of
\begin{align*}
	d(u_1,u_2)+\sum_{j=3}^i\frac{3}{2}d(u_1,u_j)+\sum_{j=3}^i\frac{3}{2}d(u_2,u_j) +\sum_{3\leq j<k\leq i}2d(u_j,u_k)
\end{align*}
over unit vectors $u_1,\ldots,u_i$.

We use equation \eqref{eq: Young} to obtain upper bounds on $f(i)$, which we then use to find numerical values for $B_{n,i}$.

For $i>2$, substituting $\eqref{eq: bothstay}$, $\eqref{eq: onemoves}$, $\eqref{eq: bothmove}$, and  $\eqref{eq: L()}$ into $\eqref{eq: defineDnik}$, we get that
\begin{align*}
	D_{n,i,1} &\leq (d_i-d_{i+1})\frac{f(i)}{n-1}+ (d_i-d_{i+1})\left(\frac{1+3(n-i)+(i-2)+2(n-i)(i-2)}{n-1} \right)\\
	D_{n,i,1} &\leq (d_i-d_{i+1})\left(\frac{f(i)}{n-1}+2\frac{(n-i)(i-1)}{n-1}+1 \right)
\end{align*}
and similarly for $D_{n,i,2}$. So we set
\begin{equation} \label{eq:Bni}
	B_{n,i}=\frac{f(i)}{n-1}+2\frac{(n-i)(i-1)}{n-1}+1
\end{equation}

For $i=2$, the same substition gives us $B_{n,2}=3$. For $i=1$, the same substition gives us $B_{n,1}=1.5$.

If there do not exist four paths $Q_{11}, Q_{12}, Q_{21}, Q_{22}$, then the average length of a path is too great, namely
\begin{align*}
\frac{1}{2} \left( \overline{L(a_{n+1}, \ldots, a_1)}+\overline{L(a_{n+2},a_n,a_{n-1}, \ldots, a_1)} \right) &> r^*S_n\\
\frac{d_{n+1}+d_{n+2}}{2}B_n + \left(d_n-\frac{d_{n+1}+d_{n+2}}{2}\right)B_{n,n}+\sum_{i=1}^{n-1} \left(d_i-d_{i+1}\right)B_{n,i} &> \sum_{i=1}^{n} r^*d_n
\end{align*}

\subsection{$n=3$}
If $d_4 \leq 0.541d_3$, then, by \eqref{eq: Young},
\[\overline{L \left( a_4, a_3, \frac{d_3}{d_2}a_2, \frac{d_3}{d_1}a_1 \right)}\leq 4.677d_3=3r^*d_3.\]
Then, since $B_{3,2}=3<2r^*$ and $B_{3,1}=1.5<r^*$ as in the last section,
\[\overline{L(a_4, a_3, a_2, a_1)}\leq r^*(d_1+d_2+d_3).\]
Similarly,
\[\overline{L(a_5, a_3, a_2, a_1)}\leq r^*(d_1+d_2+d_3)\]
so the induction works. Thus for $n=3$ we have the constraint
$d_4>0.541d_3$, which is stronger than the one obtained for $n=3$ in the previous section.

\section{Choosing $n$} \label{sec: recombine}
We obtained linear constraints for various values of $n\leq 10$. These, together with the constraints $d_i\geq d_{i+1}$, make a linear program (given in Appendix \ref{sec: lp}), which is unsatisfiable. Thus one of the constraints must not hold, so the induction works for some $n$.

\section{Algorithm}
We repeatedly use the inductive step to obtain a sequence of indices $0=n_0<n_1<n_2<~\ldots$. At stage $j$, we remove $n_j-n_{j-1}$ points.
The intermediate ending points are then of the form $n_j+k_j$ where each $k_j$ is 1 or 2.
Since we are only using $n\leq 10$, we can find all the paths $Q_{11}, Q_{12}, Q_{21}, Q_{22}$ by brute force in linear time.
Now, for both possible values of $k_1$, we find which value of $k_0$ gives the shorter path. Then, for both possible values of $k_2$, we find which value of $k_1$ will make the total path after $a_{n_2+k_2}$ shorter. We repeat until we get to some $n_j>m$, at which point we have two paths and choose the shorter one. This whole algorithm is linear.

\section{Lower bound on degree-3 tree ratios}
\label{sec:lowerbound}
Denote by $\sigma$ the sum of edge weights of the minimal spanning tree and by $\sigma_3$ the sum of edge weights of a minimal degree 3 tree. Denote by $(x_1,x_2,\ldots,x_n)$ the coordinates of a point in $n$ dimensions.

In six dimensions, let $O$ be the origin and let $v_1,v_2,\ldots,v_7$ be the vertices of a simplex with center  at $O$ and radius $\sqrt{6}$.  Let the coordinates of $v_i$ be $(v_{i,1},v_{i,2},v_{i,3},v_{i,4},v_{i,5},v_{i,6})$.  Note that $d(v_i,v_j)=\sqrt{2*7/6}\sqrt{6}=\sqrt{14}$.

Now, given natural $N$ and $0<\alpha < 1$, take the following tree in $7N$ dimensions:
\begin{enumerate}
\item The origin, $O$, is the root.
\item Its $N$ children are $p_1,p_2,\ldots,p_N$.  $p_i$ has coordinates 0 except $x_{7i}=1-\alpha$.
\item Each $p_i$ has seven children, $q_{i,1},q_{i,2},\ldots,q_{i,7}$  The coordinates of $q_{i,j}$ are all 0 except $x_{7i}=1$ and, for $k$ from 1 to 6, $x_{7i-k}=v_{j,k}$.
\end{enumerate}

Then $q_{i,1},q_{i,2},\ldots,q_{i,7}$ form a simplex with center distance $\alpha$ from $p_i$ and with each vertex distance $\sqrt{6}$ from the center.

It is easy to check that
\begin{align*}
d(p_i,p_h)&=\sqrt{2}(1-\alpha) \text{ for } i\neq h\\
d(q_{i,j},q_{i,k})&=\sqrt{14}=d(q_{i,j},q_{h,k}) \text{ for } j\neq k,h\neq i\\
d(p_i,q_{i,j})&=\sqrt{6+\alpha^2}\\
\sigma&=N(1-\alpha+7\sqrt{6+\alpha^2}).
\end{align*}

Then we can pick
\[\alpha=-1-\sqrt{7}+\sqrt{4+4\sqrt{7}},\]
which gives us $d(q_{i,j},q_{h,k})+d(p_i,p_h)=2d(p_i,q_{i,j})$.

Then we can define function $c$ on the vertices so that $c(O)=0, c(p_i)=d(p_i,p_h)/2$ and $c(q_{i,j})=d(q_{i,j},q_{h,k})/2$.
In that case, the length of edge $AB$ is at least $c(A)+c(B)$, so $c$ can be thought of a half-edge length.
Then, since there are $8N+1$ vertices, there are $8N$ edges, so there is a total of $16N$ edge endpoints.  At most 3 of them contribute 0 to $\sigma_3$, at most $3N$ contribute $(1-\alpha)/\sqrt{2}$, and the remainder contribute $\sqrt{14}/2$. Thus
\[\sigma_3\geq 3N\left(\frac{1}{2}\sqrt{2}(1-\alpha)\right)+(13N-3)\left(\frac{1}{2}\sqrt{14}\right)\]
\[\frac{\sigma_3}{\sigma}=\frac{3N\left(\frac{1}{2}\sqrt{2}(1-\alpha)\right)+(13N-3)\left(\frac{1}{2}\sqrt{14}\right)}{N\left(1-\alpha+7\sqrt{6+\alpha^2}\right)}\]
\[\lim_{N \to \infty}\frac{\sigma_3}{\sigma}=\frac{3\left(\frac{1}{2}\sqrt{2}(1-\alpha)\right)+13\left(\frac{1}{2}\sqrt{14}\right)}{1-\alpha+7\sqrt{6+\alpha^2}}\approx 1.4473\]
Thus $\tau_3^\infty \geq 1.447$.
\section{Acknowledgements}
I thank Samir Khuller for suggesting that I work on this problem and Timothy Chan for improved notation and organization.

Computational Geometry (cs.CG); Combinatorics (math.CO)


A simple upper bound for the perimeter of an ellipse

Abstract: We propose a simple derivation of an upper bound for the perimeter of an ellipse. The procedure, which relies on the use of elliptic integrals, consists in introducing, via inequalities and convexity properties, specific integrals which can be calculated analytically.

\section{Introduction}

The perimeter of an ellipse is the length of the continuous line forming its boundary. Computing accurate approximations to the perimeter of an ellipse has been a subject of interest for mathematicians for a long time \cite{Adlaj2012,Borwein1987,Bailey2016}. Many approaches have been used to estimate its value, such as approximation formulas (two formulas derived by Ramanujan are particularly famous \cite{Almkvist1988,Villarino} and we provide a non-exhaustive list in Appendix B) \cite{Hiriart,collect,Barnard2001}, infinite series (Maclaurin, Gauss-Kummer, Euler, etc.), hypergeometric functions \cite{Abbott2009,Chandrupatla2010} (see Appendix C), and of course (elliptic) integrals \cite{Abel1827,Abramowitz1972,Arfken1985,Prudnikov1986,Whittaker1990}.

It is well known that the perimeter of an ellipse can be expressed exactly as a complete elliptic integral of the second kind. Approximate formulas can, of course, be obtained by truncating the series representations of exact formulas. The difficulty of calculating exactly the perimeter has aroused a particular interest in the search for bounds. A long time ago, Kepler found the geometric mean of semi-major and semi-minor axis, as a lower bound for the perimeter, and many sophisticated approaches were developed along the years (see for instance \cite{Pfiefer1988,Wang2012,Gusic2015}). The subject is still an active field of research \cite{He2020,Zhao2022}, important for astrophysical applications (trajectories of planets, satellites, \emph{etc.}).

An ellipse with semi-major axis $a$ and semi-minor axis $b$ is parametrized by $x=a\cos\phi$ and $y=b\sin\phi$. The infinitesimal variation of curvilinear abscissa $s$ is $ds=\sqrt{dx^2+dy^2}$. The length of a quarter of the ellipse is provided by the elliptic integral
\begin{equation}\label{def}
L=\int_0^{\pi/2}ds=\int_0^{\pi/2}\sqrt{a^2\sin^2\phi+b^2\cos^2\phi}~d\phi.
\end{equation}
We will use the latter formula to derive the upper bound. The total perimeter of the ellipse will therefore be
\begin{equation}
L_e=4L.
\end{equation}
The well-known simple lower bound $2\pi\sqrt{ab}$ and upper bound $4a+\pi b$ are recalled in sections \ref{sec1} and \ref{sec2} respectively, and the main result of the present work, which is an upper bound more accurate than the one of section \ref{sec2}, but still rather simple, is presented in section \ref{sec3}.

\section{The geometric-mean lower bound}\label{sec1}

Let us consider two parametrizations of an ellipse
\begin{equation}
\begin{array}{l}
x_1=a\cos\phi,\\
y_1=b\sin\phi
\end{array}
\end{equation}
and
\begin{equation}
\begin{array}{l}
x_2=b\cos\phi,\\
y_2=a\sin\phi.
\end{array}
\end{equation}
The two associated lengths of the quarter of the ellipse are
\begin{equation}
L_1=\int_0^{\pi/2}\sqrt{a^2\cos^2\phi+b^2\sin^2\phi}~d\phi
\end{equation}
and
\begin{equation}
L_2=\int_0^{\pi/2}\sqrt{b^2\cos^2\phi+a^2\sin^2\phi}~d\phi.
\end{equation}
Since $t\rightarrow\sqrt{t}$ is a concave function, one has
\begin{equation}
\sqrt{\lambda t_1+(1-\lambda)t_2}\geq\lambda\sqrt{t_1}+(1-\lambda)\sqrt{t_2},
\end{equation}
yielding ($\lambda=\cos^2\phi$, $t_1=a^2$ and $t_2=b^2$):
\begin{equation}
L_1\geq\int_0^{\pi/2}\left(a\cos^2\phi+b\sin^2\phi\right)d\phi
\end{equation}
and
\begin{equation}
L_2\geq\int_0^{\pi/2}\left(b\cos^2\phi+a\sin^2\phi\right)d\phi
\end{equation}
and therefore
\begin{equation}
L=L_1=L_2=\frac{1}{2}\left(L_1+L_2\right)\geq (a+b)\frac{\pi}{4}.
\end{equation}
Setting $ab=R^2$, the minimum of $a+b=a+R/a^2=f(a)$ is obtained for $R=\sqrt{ab}$ and is equal to $2R$. In other words, the ellipse which has the minimum perimeter for a given area is the circle of radius $R$. This yields the well-known result
\begin{align}
\frac{\pi}{2}\sqrt{ab}\leq L
\end{align}
and for the total perimeter of the ellipse
\begin{equation}
2\pi\sqrt{ab}\leq L_e.
\end{equation}

\begin{figure}
\centering
\includegraphics[width=10cm]{fig23_ellipse.eps}
\caption{quarters of ellipse with semi-major and semi-minor axis $(a=2,b=\sqrt{3})$ and $(a=2,b=\sqrt{3})$ respectively.}
\label{fig23}
\end{figure}

\section{First upper bound}\label{sec2}

Equation (\ref{def}) is equivalent to
\begin{equation}
L=a\int_0^{\pi/2}\sqrt{1-\frac{\left(a^2-b^2\right)}{a^2}\cos^2\phi}~d\phi
\end{equation}
which can be put in the form
\begin{equation}
L=a\frac{\pi}{2}\mathscr{F}\left(\frac{b^2-a^2}{a^2}\right)\cos^2\phi~ d\phi=a\frac{\pi}{2}\mathscr{F}\left(-e^2\right)
\end{equation}
where we have introduced the function
\begin{equation}
\mathscr{F}(t)=\frac{2}{\pi}\int_0^{\pi/2}\sqrt{1+t\cos^2\phi}~d\phi
\end{equation}
where $e$ represents the eccentricity of the ellipse defined by
\begin{equation}
\frac{b^2}{a^2}=1-e^2.
\end{equation}
The asymptotic behaviour of the function $\mathscr{F}$ is studied in Appendix A. If $-1<t<0$, then $-t/(1+t)>0$ and
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)=\frac{2}{\pi}\int_0^{\pi/2}\left(\sqrt{1+t\cos^2\phi}-\sqrt{1-\cos^2\phi}\right)d\phi.
\end{equation}
Since
\begin{equation}
\sqrt{u}-\sqrt{v}=\frac{u-v}{\sqrt{u}+\sqrt{v}},
\end{equation}
we get
\begin{equation}\label{inter}
\mathscr{F}(t)-\mathscr{F}(-1)=\frac{2}{\pi}\int_0^{\pi/2}\frac{(1+t)\cos^2\phi}{\sqrt{1+t\cos^2\phi}+\sqrt{1-\cos^2\phi}}d\phi.
\end{equation}
and
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)\leq\frac{2}{\pi}(1+t)\int_0^{\pi/2}\frac{\cos^2\phi}{\sqrt{1+t}}d\phi,
\end{equation}
\emph{i.e.}
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)\leq\frac{2}{\pi}\sqrt{1+t}~\frac{\pi}{4}.
\end{equation}
because
\begin{equation}
\int_0^{\pi/2}\cos^2\phi~d\phi=\frac{\pi}{4}.
\end{equation}
This enables us to write
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)\leq\frac{\sqrt{1+t}}{2}.
\end{equation}
Since $-1<-e^2<0$, we get
\begin{equation}
\mathscr{F}(-e^2)-\mathscr{F}(-1)\leq\frac{\sqrt{1-e^2}}{2},
\end{equation}
leading to
\begin{equation}\label{firs}
L\leq a+\frac{\pi}{4}b.
\end{equation}
and for the total perimeter of the ellipse
\begin{empheq}[box=\fbox]{align}\label{firstup}
L_e\leq 4a+\pi b.
\end{empheq}

\begin{figure}
    \centering
    \includegraphics[width=10cm]{fig1_ellipse.eps}
    \caption{Comparison between the first and second upper bounds of the perimeter of a quarter of an ellipse with semi-major axis $a=2$. On has $f_1(a,b)=a+\frac{\pi}{4}b$ (Eq. (\ref{firs})) and $f_2(a,b)=a+\frac{b^2}{a^2}\left[\sqrt{a^2-b^2}\ln\left(\frac{a+\sqrt{a^2-b^2}}{b}\right)+\frac{\pi}{2}b-a\right]$ (Eq. (\ref{seco})).}
    \label{fig1}
\end{figure}

\section{Second upper bound}\label{sec3}

Let us go back to Eq. (\ref{inter}):
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)=\frac{2}{\pi}\int_0^{\pi/2}\frac{(1+t)\cos^2\phi}{\sqrt{1+t\cos^2\phi}+\sqrt{1-\cos^2\phi}}d\phi.
\end{equation}
We have
\begin{equation}
\mathscr{F}(t)-\mathscr{F}(-1)\leq\frac{2}{\pi}(1+t)\int_0^{\pi/2}\frac{\cos^2\phi}{\sqrt{1+t}+\sin\phi}d\phi,
\end{equation}
where we introduce, for $0<\xi<1$, the following integral
\begin{equation}
\mathscr{K}(\xi)=\int_0^{\pi/2}\frac{\cos^2\phi}{\xi+\sin\phi}d\phi.
\end{equation}
We obtain
\begin{eqnarray}
\mathscr{K}(\xi)&=&\int_0^{\pi/2}\frac{1-\sin^2\phi}{\xi+\sin\phi}d\phi=\int_0^{\pi/2}\frac{\left(\xi^2-\sin^2\phi\right)+\left(1-\xi^2\right)}{1+\sin\phi}d\phi\nonumber\\
&=&\int_0^{\pi/2}\left(\xi-\sin\phi\right)d\phi+\left(1-\xi^2\right)\int_0^{\pi/2}\frac{d\phi}{\xi+\sin\phi}\nonumber\\
&=&\xi\frac{\pi}{2}-1+\left(1-\xi^2\right)\mathscr{J}(\xi)
\end{eqnarray}
with
\begin{equation}
\mathscr{J}(\xi)=\int_0^{\pi/2}\frac{d\phi}{\xi+\sin\phi}.
\end{equation}
Setting $t=\tan(\phi/2)$, we have
\begin{equation}
dt=\frac{1}{2}\left[1+\tan^2\left(\frac{\phi}{2}\right)\right]d\phi,
\end{equation}
\emph{i.e.} $d\phi=2dt/(1+t^2)$. Then, since
\begin{equation}
\sin\phi=\frac{2t}{\left(1+t^2\right)},
\end{equation}
we get
\begin{equation}
\frac{d\phi}{\xi+\sin\phi}=\frac{2dt}{\xi t^2+2t+\xi}.
\end{equation}
Writing
\begin{equation}
\frac{2}{\xi t^2+2t+\xi}=\frac{2}{\xi(t-\chi_1)(t-\chi_2)}
\end{equation}
with
\begin{equation}
\chi_1=\frac{-1-\sqrt{1-\xi^2}}{\lambda}
\end{equation}
and
\begin{equation}
\chi_2=\frac{-1+\sqrt{1-\xi^2}}{\lambda},
\end{equation}
enables us to write
\begin{eqnarray}
\mathscr{J}(\xi)&=&\frac{2}{\xi}\int_0^1\frac{dt}{(t-\chi_1)(t-\chi_2)}=\frac{2}{\xi(\chi_1-\chi_2)}\left[\frac{1}{t-\chi_1}-\frac{1}{t-\chi_2}\right]dt\nonumber\\
&=&\frac{2}{\xi(\chi_1-\chi_2)}\left|\ln\left(\frac{t-\chi_1}{t-\chi_2}\right)\right|_0^1=\frac{2}{\xi(\chi_1-\chi_2)}\ln\left[\frac{(1-\chi_1)\chi_2}{(1-\chi_2)\chi_1}\right].
\end{eqnarray}
We have also
\begin{equation}
\chi_1\chi_2=1
\end{equation}
and
\begin{equation}
\chi_1-\chi_2=-\frac{2}{\xi}\sqrt{1-\xi^2}
\end{equation}
and thus
\begin{eqnarray}
\mathscr{J}(\xi)&=&-\frac{1}{\sqrt{1-\xi^2}}\ln\left(\frac{\chi_2-1}{\chi_1-1}\right)=\frac{1}{\sqrt{1-\xi^2}}\ln\left(\frac{\xi+1+\sqrt{1-\xi^2}}{\xi+1-\sqrt{1-\xi^2}}\right)\nonumber\\
&=&\frac{1}{\sqrt{1-\xi^2}}\ln\left(\frac{\sqrt{\xi+1}+\sqrt{1-\xi}}{\sqrt{\xi+1}-\sqrt{1-\xi}}\right)\nonumber\\
&=&\frac{1}{\sqrt{1-\xi^2}}\ln\left(\frac{1+\sqrt{1-\xi^2}}{\xi}\right),
\end{eqnarray}
yielding
\begin{equation}
\mathscr{K}(\xi)=\frac{\xi\pi}{2}-1+\sqrt{1-\xi^2}~\ln\left(\frac{1+\sqrt{1-\xi^2}}{\xi}\right).
\end{equation}
Therefore
\begin{equation}
L=\frac{\pi}{2}a\mathscr{F}\left(\frac{b^2}{a^2}-1\right)\leq\frac{\pi}{2}a\left[\mathscr{F}(-1)+\frac{2}{\pi}\left(\frac{b^2}{a^2}\right)\mathscr{K}\left(\frac{b}{a}\right)\right],
\end{equation}
or finally
\begin{equation}\label{seco}
L\leq a+\frac{b^2}{a^2}\left[\sqrt{a^2-b^2}\ln\left(\frac{a+\sqrt{a^2-b^2}}{b}\right)+\frac{\pi}{2}b-a\right]
\end{equation}
and for the total perimeter of the ellipse
\begin{empheq}[box=\fbox]{align}
L_e\leq 4a+4\frac{b^2}{a^2}\left[\sqrt{a^2-b^2}~\ln\left(\frac{a+\sqrt{a^2-b^2}}{b}\right)+\frac{\pi}{2}b-a\right].
\end{empheq}
This upper bound is better than the first one in Eq. (\ref{firstup}) (see figure \ref{fig1}).

\section{Conclusion}

In this article, we obtained an upper bound for the perimeter of an ellipse. The calculation consists in simplifying elliptic integrals by applying inequalities and convexity properties. It is hoped that the method may stimulate the derivation of further bounds, improving the compromise between simplicity and accuracy.


A note relating ridge regression and OLS p-values to preconditioned sparse penalized regression

Abstract: When the design matrix has orthonormal columns, "soft thresholding" the ordinary least squares (OLS) solution produces the Lasso solution [Tibshirani, 1996]. If one uses the Puffer preconditioned Lasso [Jia and Rohe, 2012], then this result generalizes from orthonormal designs to full rank designs (Theorem 1). Theorem 2 refines the Puffer preconditioner to make the Lasso select the same model as removing the elements of the OLS solution with the largest p-values. Using a generalized Puffer preconditioner, Theorem 3 relates ridge regression to the preconditioned Lasso; this result is for the high dimensional setting, p > n. Where the standard Lasso is akin to forward selection [Efron et al., 2004], Theorems 1, 2, and 3 suggest that the preconditioned Lasso is more akin to backward elimination. These results hold for sparse penalties beyond l1; for a broad class of sparse and non-convex techniques (e.g. SCAD and MC+), the results hold for all local minima.

\section{}
\begin{abstract}
When the design matrix has orthonormal columns, ``soft thresholding" the ordinary least squares (OLS) solution produces the Lasso solution \citep{tibshirani1996regression}.  If one uses the Puffer preconditioned Lasso \citep{jia2012preconditioning}, then this result generalizes from orthonormal designs to full rank designs (Theorem 1).  Theorem 2 refines the Puffer preconditioner to make the Lasso select the same model as removing  the elements of the OLS solution with the largest p-values.
Using a generalized Puffer preconditioner, Theorem 3 relates ridge regression to the preconditioned Lasso; this result is for the high dimensional setting, $p>n$.
Where the standard Lasso is akin to forward selection \citep{efron2004least}, Theorems 1, 2, and 3 suggest that the preconditioned Lasso is more akin to backward elimination.  These results hold for sparse penalties beyond $\ell_1$;  for a broad class of sparse and non-convex techniques (e.g. SCAD and MC+), the results hold for all local minima.

\end{abstract}

\section{Introduction}

Preconditioning is a classical computational technique in numerical linear algebra that creates fast algorithms.  Several papers have recently proposed and studied the ``preconditioned" Lasso \citep{paul2008preconditioning, huang2011variable, rauhut2011sparse, jia2012preconditioning, qian2012pattern, wauthier2013comparative}. Instead of accelerating standard Lasso algorithms, preconditioning the Lasso creates a new statistical estimator that retains several properties of the Lasso while making the solution less sensitive to the correlation between the columns of the design matrix.  This paper demonstrates how penalized least squares estimators with various forms of preconditioning are equivalent to classical quantities in linear regression--the OLS estimator, OLS p-values, and ridge regression.

 The theorems below do not make any assumptions on the design matrix beyond full rank.  Nor do they assume a linear model $Y = \ensuremath{{\mathbf{X}}} \beta + \epsilon$, or assume some conditions on an error term $\epsilon$.  Instead of studying the statistical estimation properties of preconditioned penalized least squares problems, the following theorems study the preconditioned Lasso estimators as functions of data $(\ensuremath{{\mathbf{X}}}, Y)$ that return a vector $\hat \beta$.  The theorems compare these new functions to classical ``functions" like $ols$ and ridge regression.

\subsection{Preliminaries}
While the theorems below do not require the linear model, it is the linear model that motivates the estimators (i.e. functions) studied in this paper. The linear model is
\begin{equation}\label{linearModel}
Y = \ensuremath{{\mathbf{X}}} \beta + \epsilon,
\end{equation}
where $Y \in R^n$ and $\ensuremath{{\mathbf{X}}} \in R^{n \times p}$ are observed, and $\epsilon \in R^n$ is random noise satisfying $E(\epsilon) = 0$ and $E(\epsilon \epsilon') = \sigma^2 I_p$.  The goal is to estimate $\beta \in R^p$ with $\ensuremath{{\mathbf{X}}}$ and $Y$.

Throughout the paper, we will assume that $\ensuremath{{\mathbf{X}}}$ is full rank (when $n<p$, it is full row rank). Define $\|x\|_q = (\sum_i x_i^q)^{1/q}$.
For $n>p$, define
 \[\hat \beta^{ols} =  \arg \min_{b\in R^p} \|  Y -   \ensuremath{{\mathbf{X}}} b\|_2^2 = (\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}})^{-1} \ensuremath{{\mathbf{X}}}' Y \in R^p\]
 as the standard OLS estimator.
\begin{definition}
Define the function $Lasso_\lambda$,
\[Lasso_\lambda(  \ensuremath{{\mathbf{X}}},   Y) = \arg \min_{b\in R^p} \|  Y -   \ensuremath{{\mathbf{X}}} b\|_2^2 + \lambda \|b\|_1.\]
\end{definition}

For $a\in R$, define $\textrm{sign}(a)$ as $-1, 0,$ or $1$, depending on whether $a$ is negative, zero, or positive.  Define $(a)^+$ as equal to $a$ if $a$ is nonnegative and equal to zero if $a$ is negative.  The soft-thresholding function is defined as
\[t_\lambda(x) = sign(x) (|x| - \lambda)^+.\]
To apply $t_\lambda$ to a vector $x$, apply it element-wise, $[t_\lambda(x)]_j = t_\lambda(x_j)$.

\begin{lemma}\label{classicalRelation} (Equation 3 in \citep{tibshirani1996regression})
If the design matrix $\ensuremath{{\mathbf{X}}}$ is orthonormal,
\begin{equation} \label{classical}
Lasso_\lambda(\ensuremath{{\mathbf{X}}}, Y) = t_\lambda(\ensuremath{{\hat \beta^{ols}}}).
\end{equation}
\end{lemma}

\section{Preconditioning the Lasso}
Sections 2 and 3 study the low dimensional setting $n > p$.  For these sections, let   $\ensuremath{{\mathbf{X}}} = UDV'$ be the ``skinny" SVD; $U \in R^{n \times p}$ and $V\in R^{p \times p}$ have orthonormal columns and $D$ is a diagonal matrix.  The Puffer transform is defined as $\ensuremath{F} = U D^{-1} U'$ \citep{jia2012preconditioning}.  After preconditioning, Equation \eqref{linearModel} becomes
\begin{equation} \label{precondModel}
\ensuremath{F} Y = (\ensuremath{F} \ensuremath{{\mathbf{X}}}) \beta + \ensuremath{F} \epsilon.
\end{equation}
 While $(\ensuremath{F} \ensuremath{{\mathbf{X}}}) =U V'$ is an orthonormal matrix, it is not orthogonalized by rotating the columns as in a QR decomposition; this would correspond to \textit{right} multiplying $\ensuremath{{\mathbf{X}}}$ by some matrix.  Rotating the columns would create a new basis and make the Lasso penalize in the incorrect basis.  By \textit{left} multiplying, each row of $\ensuremath{F} \ensuremath{{\mathbf{X}}}$ is a linear combination of the rows in $\ensuremath{{\mathbf{X}}}$.
Importantly, the regression estimators that use $(\ensuremath{F} \ensuremath{{\mathbf{X}}}, \ensuremath{F} Y)$ instead of $(\ensuremath{{\mathbf{X}}}, Y)$ still estimate the same vector $\beta$ and the Lasso penalizes in the correct basis; the original regression model in Equation \eqref{linearModel} contains the exact same $\beta$ as Equation \eqref{precondModel}. Define
\begin{eqnarray*}
\textit{Puffer}(\ensuremath{{\mathbf{X}}}, Y) &=& (\ensuremath{F} \ensuremath{{\mathbf{X}}} , \ensuremath{F} Y) \in \ensuremath{{\mathbb{R}}}^{n \times p} \times \ensuremath{{\mathbb{R}}}^n \ \mbox{ and } \\
Lasso_\lambda(\textit{Puffer}(\ensuremath{{\mathbf{X}}}, Y)) &=& Lasso_\lambda(\ensuremath{F} \ensuremath{{\mathbf{X}}}, \ensuremath{F} Y).
\end{eqnarray*}
\cite{jia2012preconditioning} showed that if the smallest singular value of $\ensuremath{{\mathbf{X}}}$ is bounded from below, then with $p$ fixed and $n\rightarrow \infty$, the preconditioned Lasso, $Lasso_\lambda(\textit{Puffer}(\ensuremath{{\mathbf{X}}}, Y))$, is sign consistent.  Importantly, the Puffer preconditioned Lasso does not require the Irrepresentable Condition from \cite{zhao2006model}.  The next theorem shows how this estimator relates to the classical estimator $\hat \beta^{ols} \in R^p$, computed on the full model.  This  extends the relationship in Lemma \ref{classicalRelation} from the case where $\ensuremath{{\mathbf{X}}}$ is orthonormal, to the case where $\ensuremath{{\mathbf{X}}}$ is full rank.

\begin{theorem}\label{theorem1}
If $\ensuremath{{\mathbf{X}}}$ is full rank and $n>p$, then
\begin{equation}\label{threshold}
Lasso_\lambda(\textit{Puffer}(\ensuremath{{\mathbf{X}}},  Y)) = t_\lambda(\hat \beta^{ols} ).
\end{equation}
\end{theorem}
  In general, the relationship in Equation \eqref{threshold} does not hold without \textit{Puffer}.
All proofs are contained in the Appendix.

\section{Correcting for the heterogeneous variability in $\hat \beta^{ols}$}

Under the linear model,
\begin{equation}\label{olsSE}
cov(\ensuremath{{\hat \beta^{ols}}}) = \sigma^2 (\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}})^{-1}.
\end{equation}
Define
\[\Sigma^{ols} = cov(\ensuremath{{\hat \beta^{ols}}}). \ \mbox{ Then, } variance(\ensuremath{{\hat \beta_j^{ols}}}) = \Sigma_{jj}^{ols}.\]
If the diagonal elements of $\Sigma^{ols}$ are not all equal, then some elements of $\ensuremath{{\hat \beta^{ols}}}$ will have greater variability than others.  Classical confidence intervals and p-values account for this uncertainty.  However, the preconditioned Lasso estimator above (i.e. $t_\lambda( \ensuremath{{\hat \beta^{ols}}}) $) does not account for this known heteroskedasticity  of $\ensuremath{{\hat \beta^{ols}}}$ by applying a stronger penalty to terms with larger variance.

Classical versions of model selection test the following null hypotheses in various ways:
\[H_{0,j}: \beta_j = 0,  \ \mbox{ for } j \in 1, \dots, p.\]
Under the linear model \eqref{linearModel} and $H_{0,j}$,
\begin{equation}\label{teststat}
Z_j = \sqrt{n} \frac{\hat \beta_j^{ols}}{\sqrt{\sigma^2 (\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}})_{jj}^{-1}}} \stackrel{d}{\Rightarrow}N(0,1) \ \mbox{ as $n \rightarrow \infty$}.
\end{equation}
The classical ``marginal" p-values are defined as
\begin{equation}\label{pvalue}
p_j = 2(1 - \Phi(|Z_j|)), \mbox{ for $j = 1, \dots, p$
\end{equation}
where $\Phi$ is the cdf of the standard normal distribution.

\subsection{Scaling matters}

Standard Lasso packages normalize the columns of $\ensuremath{{\mathbf{X}}}$ so that they all have equal $\ell_2$ length. This ensures that each element of the Lasso estimator is equally penalized.  However, it does not ensure that the elements of the estimator are equally variable. This section  \textit{right} preconditions $\ensuremath{{\mathbf{X}}}$ with a diagonal matrix $N$, making the column lengths heterogeneous.   The column lengths are chosen so that the penalty strengths align with the heteroskedasticity of $\ensuremath{{\hat \beta^{ols}}}$, ensuring equal variability among the elements of the estimator $\hat \beta$.

After left and right preconditioning with $\ensuremath{F}$ and $N$ respectively, the regression equation becomes
\[\ensuremath{F} Y = (\ensuremath{F} \ensuremath{{\mathbf{X}}} N)(N^{-1}\beta) + \ensuremath{F} \epsilon.\]
Define $\nu \in R^p$ as the diagonal elements of $(\ensuremath{{\mathbf{X}}}' \ensuremath{{\mathbf{X}}})^{-1}$ and let $N \in R^{ p \times p}$ be a diagonal matrix with
$N_{jj} = \sqrt{\nu_j}. $
Because $\ensuremath{{\mathbf{X}}}$ is assumed full column rank and $N_{jj}$ is proportional to the standard error of $\ensuremath{{\hat \beta_j^{ols}}}$, $N_{jj}$ exists and is strictly positive.  This ensures that $\textrm{sign}(N^{-1}\beta) = \textrm{sign}(\beta).$ So, inferences in the transformed space carry over to the original model.

Take the SVD of $\ensuremath{{\mathbf{X}}} N = U_N D_N V_N'$. Then, define $\ensuremath{F}_N = U_N D_N^{-1} U_N'$ and
\[\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y) = (\ensuremath{F}_N \ensuremath{{\mathbf{X}}} N, \ensuremath{F}_N Y) \in \ensuremath{{\mathbb{R}}}^{n \times p} \times \ensuremath{{\mathbb{R}}}^n.\]
Theorem \ref{pvalueLasso} shows that
if $\lambda = 1.96 \sigma/\sqrt{n}$, then $Lasso_\lambda(\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y))$ selects the same variables as fitting the standard OLS estimator and removing any variables with p-value greater than $.05$.

\begin{theorem} \label{pvalueLasso}
If $\ensuremath{{\mathbf{X}}}$ is full rank and $n>p$, denote
\[\hat \beta^N(\lambda) = Lasso_\lambda(\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y)).\]
Let $Z_j$ be the classical test statistic defined in Equation \eqref{teststat}, let $p_j$ be the classical p-value defined in Equation \eqref{pvalue}, and let $\Phi$ represent the cdf of the standard normal distribution.
\[\hat \beta_j^N(\lambda) \ne 0 \ \  \ \Leftrightarrow  \ \ \ |Z_j| > \lambda \sqrt{n} / \sigma \ \ \ \Leftrightarrow \ \ \ p_j \le 2\left(1-\Phi(\lambda \sqrt{n} /\sigma)\right).\]

\end{theorem}

Importantly, this is an algebraic equivalence between $Lasso_\lambda(\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y))$ and the classical OLS p-values in Equation \eqref{pvalue}.  This requires only a full rank design matrix $\ensuremath{{\mathbf{X}}}$ and does not make any assumption on the error distribution or the elements of $\beta$.  Theorem 2 asserts nothing about the statistical reliability of these p-values.  If one wishes to use the p-value for statistical inference, then Theorem \ref{pvalueLasso} needs additional assumptions on the error term, $\epsilon$.

\subsection{Generalizing to other methods}

For simplicity, Theorems 1 and 2 are stated for the Lasso.  However, both theorems hold more generally.
For $\lambda \ge 0$ and penalty function $\ensuremath{\mbox{pen}}: \ensuremath{{\mathbb{R}}} \rightarrow \ensuremath{{\mathbb{R}}}_+$, define the function $sparse(\ensuremath{{\mathbf{X}}},Y,\lambda, \ensuremath{\mbox{pen}})$ as
\begin{equation}
sparse(\ensuremath{{\mathbf{X}}},Y,\lambda, \ensuremath{\mbox{pen}}) = \arg \min_b \frac{1}{2}\|Y - \ensuremath{{\mathbf{X}}} b\|_2^2 + \lambda \sum_j \ensuremath{\mbox{pen}}(b_j).
\end{equation}
Whenever $\ensuremath{\mbox{pen}}$ has a thresholding function $\tilde t_\lambda$ that satisfies a version of Lemma \ref{classicalRelation} with orthonormal designs, i.e.
\[sparse(\ensuremath{{\mathbf{X}}}, Y,\lambda, \ensuremath{\mbox{pen}}) = \tilde t_\lambda(\ensuremath{{\hat \beta^{ols}}}) \ \mbox{ for orthonormal } \ensuremath{{\mathbf{X}}},\]
then modified versions of Theorems 1 and 2 also apply for this penalty:
\begin{eqnarray*}
&sparse(\textit{Puffer}(\ensuremath{{\mathbf{X}}}, Y),\lambda, \ensuremath{\mbox{pen}}) &= \tilde t_\lambda(\ensuremath{{\hat \beta^{ols}}})  \\
&\left[ sparse(\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y),\lambda, \ensuremath{\mbox{pen}})\right]_j &= \tilde t_\lambda(  \sigma Z_j / \sqrt{n} ).
\end{eqnarray*}
For example, $\ensuremath{\mbox{pen}}$ could be an $\ell_q$ penalty for $q \in [0,1]$, the elastic net penalty, or a concave penalty such as SCAD or MC+ \citep{fan2001variable, zhang2010nearly}.  After preconditioning, a broad class of penalties  select the same sequence of models as the Lasso.

\section{Relating to ridge regression}

For $p > n$, take the ``skinny" SVD of
$\ensuremath{{\mathbf{X}}} = UDV'$, where $U \in R^{n \times n}, V \in R^{p \times n}, D \in R^{n \times n}$.  Consider a generalized Puffer transformation
\[\ensuremath{F}_\tau = U(D^2 + \tau I)^{-1/2}U' \ \mbox{ and } \textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y) = (\ensuremath{F}_\tau \ensuremath{{\mathbf{X}}}, \ensuremath{F}_\tau Y).\]
Theorem \ref{mainresult} relates $sparse(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y), \lambda, \ensuremath{\mbox{pen}})$  to  the ridge estimator \citep{hoerl1970ridge}.  For $\tau > 0$, the ridge estimator is
\begin{eqnarray}
\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}} &=& \arg \min_b \|Y - \ensuremath{{\mathbf{X}}} b\|_2^2 + \tau \|b\|_2^2 \label{ridge} \\
 &=& (\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}} + \tau I)^{-1} \ensuremath{{\mathbf{X}}}' Y. \nonumber
\end{eqnarray}
Define $\hat \beta_{ridge}(0)$ as the Moore-Penrose estimator
\begin{eqnarray*}
\hat \beta_{ridge}(0) &=&\arg \min_{b: \ensuremath{{\mathbf{X}}} b = Y}  \|b\|_2^2 \\
 &=& (\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}})^{+} \ensuremath{{\mathbf{X}}}' Y,
\end{eqnarray*}
where $(\ensuremath{{\mathbf{X}}}'\ensuremath{{\mathbf{X}}})^{+}$ is the Moore-Penrose pseudoinverse, $VD^{-2}V'$.\footnote{In what follows, replace any matrix inversion with the Moore-Penrose pseudoinverse if $\tau =0$ and the true inverse does not exist.}

Define $\ensuremath{\mathscr{P}_\tau}: \ensuremath{{\mathbb{R}}}^p \rightarrow \ensuremath{{\mathbb{R}}}^p$ as
\begin{eqnarray}
\ensuremath{\mathscr{P}_\tau}(v) &=& \ensuremath{{\mathbf{X}}}'(\ensuremath{{\mathbf{X}}}\X' + \tau I)^{-1} \ensuremath{{\mathbf{X}}} v. \label{proj}
\end{eqnarray}
For $\tau = 0$, $\ensuremath{\mathscr{P}_0}$ projects onto the row space of $\ensuremath{{\mathbf{X}}}$.  Under the linear model (Equation \ref{linearModel}), $\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}}$ is an unbiased estimator of $\ensuremath{\mathscr{P}_\tau}(\beta)$.

The following theorem assumes that $sparse$ uses a penalty function satisfying the following assumptions.
\begin{definition} \label{sparsepen}
Define a function $\ensuremath{\mbox{pen}}$ as a \textbf{regular sparse penalty} if it is
\begin{enumerate}
\item non-differentiable at zero and differentiable everywhere else;
\item symmetric, $\ensuremath{\mbox{pen}}(a) = \ensuremath{\mbox{pen}}(-a)$;
\item monotonically increasing away from zero, $\ensuremath{\mbox{pen}}(a) \ge \ensuremath{\mbox{pen}}(b)$ if $|a| > |b|$;
\item Lasso derivative in the neighborhood of zero,
\[\lim_{x \rightarrow 0} |\ensuremath{\mbox{pen}}'(x)| = 1,\]
where $\ensuremath{\mbox{pen}}'$ is the first derivative of $\ensuremath{\mbox{pen}}$.
\end{enumerate}
\end{definition}
This includes the Lasso, elastic net, SCAD, and MC+.  However, $\ell_q$ penalties fail condition (4) when $q < 1$.  Such penalties have an unbounded derivative in the neighborhood of zero which creates discontinuities in the solution path;  previous research has also excluded such penalties (e.g. \cite{zhang2012general, loh2013regularized}).

Theorem \ref{mainresult} says that if $\ensuremath{\mbox{pen}}$ is a regular sparse penalty, then any local minimum for the objective function in $sparse(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y), \lambda, \ensuremath{\mbox{pen}} )$, transformed by $\ensuremath{\mathscr{P}_\tau}$, is close to the ridge estimator $\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}}$.  Moreover, $\lambda$ controls the distance between these estimators.

\begin{theorem} \label{mainresult}
Let $\ensuremath{\mbox{pen}}$ be a regular sparse penalty (Definition \ref{sparsepen}) and let $p\ge n$.  Let
\[\hat \beta = sparse(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y), \lambda, \ensuremath{\mbox{pen}} ).\]
If $\hat \beta_j \ne 0$, then
\[\ensuremath{{\ensuremath{\hat \beta_{ridge,j}(\tau)     }}} - \ensuremath{\mathscr{P}_\tau}(\hat \beta)_j = \lambda \ensuremath{\mbox{pen}}'(\hat \beta_j),\]
where $\ensuremath{\mbox{pen}}'$ is the derivative of $\ensuremath{\mbox{pen}}$.  If $\hat \beta_j = 0$, then
\[|\ensuremath{{\ensuremath{\hat \beta_{ridge,j}(\tau)     }}} - \ensuremath{\mathscr{P}_\tau}(\hat \beta)_j| \le \lambda.\]
Moreover, these results still hold if $\hat \beta$ is any local minimizer of the objective function for $sparse(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y), \lambda, \ensuremath{\mbox{pen}} )$.
\end{theorem}

Importantly, $sparse$ has been computed with the preconditioned data, $\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y)$, while $\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}}$ is the traditional estimator computed with the original data.  When $\lambda$ is small, these two estimators are aligned in the row space of $\ensuremath{{\mathbf{X}}}$.  Determining the statistically appropriate scale of $\lambda$ requires some care because after preconditioning, the scale of the problem changes; $\|\ensuremath{{\mathbf{X}}}\|_F^2 = O(np)$, but $\|\ensuremath{F}_\tau \ensuremath{{\mathbf{X}}}\|_F^2 = O(n)$.  A forthcoming revision to \cite{jia2012preconditioning},  shows that $sparse(\textit{Puffer}_0(\ensuremath{{\mathbf{X}}}, Y), \lambda, \| \cdot \|_1)$ is sign consistent when $\min_j \beta_j$ is larger than $\log n / \sqrt{n}$ and
\[\lambda = O\left(\left(\frac{\log n \log p}{p} \right)^{1/2}\right).\]
In the high dimensional setting, this $\lambda$ is clearly converging to zero.

If Theorem 3 held without $\ensuremath{\mathscr{P}_\tau}$, then it would say that $Lasso_\lambda(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y)) = t_\lambda(\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}})$, which would make a clear analogy to backward elimination.  However, the inclusion of $\ensuremath{\mathscr{P}_\tau}$ stains the analogy to backward elimination. In fact, it is not even true that $\ensuremath{\mathscr{P}_\tau}\left(Lasso_\lambda(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y))\right) = t_\lambda(\ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}})$; the left side is in the row space of $\ensuremath{{\mathbf{X}}}$, but the right side is not. That said,
typical algorithms to compute the Lasso  solution path (e.g. \cite{efron2004least}) start at $\lambda =\infty$ with $Lasso_{\lambda= \infty}(\ensuremath{{\mathbf{X}}}, Y) = 0$ and decrease $\lambda$, increasing the number of terms in the model.  This resembles forward selection.
The results of Theorem 3 are ``backwards" in the sense that for $\lambda \rightarrow 0$, $\ensuremath{\mathscr{P}_\tau}(Lasso_\lambda(\textit{Puffer}_\tau(\ensuremath{{\mathbf{X}}}, Y))) \rightarrow \ensuremath{{\ensuremath{\hat \beta_{ridge}(\tau)     }}}$.

Suppose $\hat \beta^{(1)}$ and $\hat \beta^{(2)}$ are both local minima of $sparse(\textit{Puffer}_0(\ensuremath{{\mathbf{X}}}, Y),\lambda, \ensuremath{\mbox{pen}})$ for a regular sparse penalty $\ensuremath{\mbox{pen}}(x)$ that is concave on $\{x:x>0\}$.  By concavity and the definition of regular sparse penalty, it follows that $|\ensuremath{\mbox{pen}}'(x)| \le 1$ for $x\ne0$.  So, the triangle inequality around $\hat \beta_{ridge,j}(0)$ yields
\begin{equation}\label{closeMins}
\ensuremath{\mathscr{P}_0}( \hat \beta^{(1)}_j  - \hat \beta^{(2)}_j ) \le 2 \lambda.
\end{equation}
So after preconditioning, local minima are exceedingly similar in the row space of $\ensuremath{{\mathbf{X}}}$.  Moreover, even if $\hat \beta^{(1)}$ and $\hat \beta^{(2)}$ are (local) minima from different penalty functions, then Equation \ref{closeMins} holds so long as (i) both $\ensuremath{\mbox{pen}}$ functions are regular and concave and (ii) both are computed with the same tuning parameter $\lambda$.\footnote{If they have different tuning parameters $\lambda_1$ and $\lambda_2$, then the bound $2 \lambda$ is replaced by $\lambda_1 + \lambda_2$.}  In general, $\beta$ is not identifiable when $p>n$; only the projection of $\beta$ into the row space of $\ensuremath{{\mathbf{X}}}$ is identifiable \citep{shao2012estimation}.  This suggests that, after preconditioning, it is difficult statistically distinguish the difference between local minima or the difference between penalty functions.

\section{Discussion}

Several previous papers have studied different preconditioners for the Lasso \citep{paul2008preconditioning, huang2011variable, rauhut2011sparse, jia2012preconditioning, qian2012pattern, wauthier2013comparative}.
This paper connects two types of preconditioning techniques to the classical OLS solution.  When performing model selection in the classical setting of $n>>p$, the marginal p-values from OLS are typically considered more informative than the absolute sizes of the elements in $\hat \beta^{ols}$; the p-values account for the potentially heterogeneous standard errors across $\hat \beta^{ols}$.  This suggests that $Lasso_\lambda(\textit{Puffer}_N(\ensuremath{{\mathbf{X}}}, Y))$ should be preferred to $Lasso_\lambda(\textit{Puffer}(\ensuremath{{\mathbf{X}}}, Y))$.  However, classical intuitions also suggest that two variables might have statistically insignificant p-values because these variables are correlated with each other(see Section 10.1 in \cite{weisberg2014applied}).  As such, a backward procedure should have multiple steps.  On each step, remove the variable with the largest p-value and refit the OLS with the remaining predictors.  Neither of the preconditioning techniques above  creates a Lasso solution path that is equivalent to this multi-step approach.  A preconditioner that is designed to match this path must be a function of $Y$ and $\lambda$, thus becoming much more complicated.

 The Lasso (without preconditioning) is akin to forward selection \citep{efron2004least}.  Classical methods of forward selection are not model selection consistent, unless the columns of the design matrix are only  weakly correlated \citep{tropp2007signal}.  Similarly, the Lasso is not sign consistent unless the design matrix satisfies the Irrepresentable Condition \citep{zhao2006model}.  After preconditioning with the (generalized) Puffer transformation, Theorems 1, 2, and 3 show that  the Lasso is akin to backward elimination.   This preconditioner makes the design orthogonal when $n>p$, trivially satisfying all consistency conditions (e.g. Irrepresentable Condition, RIP, etc.).  As such, Theorem 1 implies that a one step backward elimination is also sign consistent.

This suggests that backward procedures are sign consistent when  forward procedures are not.
 However, backward procedures are not a panacea. They will become unstable whenever the OLS p-values are incorrect (e.g. when $p$ is large compared to $n$  and the CLT does not hold for the test statistic $Z_j$ in Equation \eqref{teststat}).

\bibliographystyle{plainnat}

Machine Learning (stat.ML); Methodology (stat.ME)