Why do the singular values of $A$ scale by $\sqrt{k}$ as I compute the SVD on $k$ copies of $A$? This is a research question I have been stuck on for some time. 
Consider $U\Sigma V^T = A$. Suppose I compute $A_2 = [A;A] = U_2 \Sigma_2 V_2^T$, then $\Sigma_2 = \sqrt{2}\cdot\Sigma$. Similarly, if I compute $A_3 = [A;A;A] = U_3 \Sigma_3 V_3^T$, then $\Sigma_3 = \sqrt{3}\cdot\Sigma$.
If I continue on and compute the SVD of $A_k = \underbrace{[A;A;\ldots;A]}_{k}$, then $\Sigma_k = \sqrt{k}\cdot\Sigma$.
Why does this happen? I have observed this in numerical experiments  but I do not understand the theory. 

The singular values are the square roots of the eigenvalues of $A A^T$.
With $A_2=\begin{bmatrix} A& A\end{bmatrix}$ you have $A_2^T=\begin{bmatrix} A^T\\ A^T\end{bmatrix}$ so:
$$A_2 A_2^T=\begin{bmatrix} A& A\end{bmatrix} \cdot \begin{bmatrix} A^T\\ A^T\end{bmatrix}=\begin{bmatrix} A A^T+A A^T \end{bmatrix}= \begin{bmatrix} 2A A^T\end{bmatrix}$$
so the eigenvalues of $A_2 A_2^T$ are $2$ times the ones of $A A^T$ and taking the square root the factor $\sqrt{2}$ appears.
Similarly you have:
$$A_k A_k^T=\begin{bmatrix} A & \cdots& A\end{bmatrix} \cdot \begin{bmatrix} A^T \\ \vdots\\A^T\end{bmatrix}=\begin{bmatrix} k A A^T \end{bmatrix}$$
so the eigenvalues ofre the ones of $A A^T$ times $k$ and the singular values are the one of $A$ times $\sqrt{k}$.
