What kind of matrix is it that when multiplied with its transpose produces the identity? If $A^TA = I$, where $A$ is a lower triangular matrix, does that mean $A$ has to be an identity matrix (and nothing else)? In general, which kind of matrix $A$ must be for that equality to hold?

OK, edits complete!
Setting lower triangularity aside for the moment, take any $m$ mutually orthonormal vectors $\vec n_1, \vec n_2, \ldots,  \vec n_m \in \Bbb R^m$ so that $\langle \vec n_j, \vec n_k \rangle = \delta_{jk}$, and turn them into a matrix $N$ by using them as columns, so we may write $N$ in columnar form as
$N = \begin{bmatrix} \vec n_1 & \vec n_2 & \ldots & \vec n_m \end{bmatrix}; \tag{1}$
then the rows of $N^T$ are the columns of $N$, each transposed:
$N^T = \begin{bmatrix} \vec n_1^T \\ \vec n_2^T \\ \vdots \\ \vec n_m^T \end{bmatrix}. \tag{2}$
Thus
$N^TN = \begin{bmatrix} \vec n_1^T \\ \vec n_2^T \\ \vdots \\ \vec n_m^T \end{bmatrix} \begin{bmatrix} \vec n_1 & \vec n_2 & \ldots & \vec n_m \end{bmatrix} = [\langle \vec n_j, \vec n_k \rangle ]  = [\delta_{jk}] = I, \tag{3}$
for $1 \le j, k \le m$.  This shows there are very many matrices satisfying $A^TA = I$.  Furthermore, the columns of $N$ are linearly independent, since any system of orthonormal vectors is independent (and I'll take this as known); thus $\det N \ne 0$ and $N$ is invertible.  Or one can argue from (3) that
$(\det N^T)(\det N) = \det N^TN = 1, \tag{4}$
showing $\det N \ne 0$ and the existence of $N^{-1}$ follows.  Having $N^{-1}$ we obtain from (3)
$N^T = N^TI = N^T(N N^{-1}) = (N^T N)N^{-1} = IN^{-1} = N^{-1}, \tag{5}$
so that $N^T = N^{-1}$; thus (3) yields
$NN^T = NN^{-1} = I \tag{6}$
as well.  So in the general case we see that
$\bullet$ $N = I$ needn't hold;
$\bullet$ there are many other possibilities, one for each orthornormal frame $\vec n_1, \vec n_2, \ldots, \vec n_m \in \Bbb R^m$;
$\bullet$ the key property is that the rows be orthonormal, as the columns be; in algebraic terms, this translates directly to $N^TN = NN^T = I$.
Returning finally to the case $N$ lower triangular, we see in this case $N^T$ is upper triangular.  In this situation, $N$ takes the form of a diagonal matrix with diagonal entries $\pm 1$; to see this, we can write out the matrix equation $N^TN = I$ in terms of the components of the vectors $\vec n_i = (n_{i1}, n_{i2}, \ldots, n_{im})^T$; we see that the result is a system of quadratic equations in the $m(m + 1)/2$ unknowns $n_{ij}$ where $1 \le j \le i$ by the lower triangularity of $N$; the remaining $n_{ij} = 0$, $1 \le i < j$; since $(N^T)_{jl} = N_{lj} = n_{lj}$ the equations may be cast in the form
$\sum_{l = 1}^m n_{lj}n_{lk} = \delta_{jk}, \tag{7}$
and since the matrix $N$ is lower triangular, we always have $n_{lk} = 0$ for $ l < k$, so (7) reduces to
$\sum_{l = k}^m n_{lj}n_{lk} = \delta_{jk}. \tag{8}$
The system of equations (8) may apparently be solved by starting with $j = k = m$ whereby (8) becomes $n_{mm}^2 = 1$; thus $n_{mm} = \pm 1$ and thus $n_{mj} = 0$, $j < m$, since taking $k = m$, $j < m$, (8) becomes simply
$n_{mj} = n_{mj}n_{mm} = \delta_{jm} = 0, \tag{9}$
whence $n_{mj} = 0$ since $n_{mm} = \pm 1$.  Having the $m$-th row of $N$, we may work our way downwards in $k$, successively finding the remaining rows of $N$; for example, with $j = k = m - 1$ we have
$n_{m - 1 \; m - 1}^2 + n_{m \; m - 1}^2 = \sum_{l = m - 1}^m n_{l \; m - 1} n_{l \; m - 1} = \delta_{m - 1 \; m - 1} = 1; \tag{10}$
now using $n_{m \; m - 1} = 0$ we see that $n_{m - 1 \; m - 1} = \pm 1$ and we proceed as before, eventually working our way down to $k = 1$ and accruing the results $n_{kk} = \pm 1$, $n_{jk} = 0$ in the process; $N$ is seen to be a diagonal matrix whose only nonzero entries are the $n_{kk} = \pm 1$, $1 \le k \le m$.
There are $2^m$ such matrices.
As a final note, I address the OP's question from the comment stream:  if $L$ and $M$ are lower triangular, with $L$ invertible and $(L^{-1}M)(L^{-1}M)^T = I$, then $L^{-1}$ is lower triangular (see   Show that A is invertible and that it is Lower Triangular.) and hence $N = L^{-1}M$ is lower triangular as well; lower triangular matrices are closed under multiplication.  Then $N = L^{-1}M$ satisfies $NN^T = I$, so $(\det N)(\det N^T) = 1$, so $\det N \ne 0$ and $N$ is invertible; thus we may write $N^{-1} = N^T$, and hence $N^TN = I$ as well.  From what we have seen above, we may conclude that $N$ is a diagonal matrix with all non-zero entries $\pm 1$; no diagonal entry $n_{jj} = 0$.  $L^{-1}M = N$ implies $M= LN$;  $L = M$ if and only if $N = I$.  The effect of right mutiplying $L$ by $N$ is to reverse the sign of column $j$ of $L$ if and only if $n_{jj} = -1$; the remaning column vectors are left invariant.  
Hope this helps.  Cheerio,
and as always,
Fiat Lux!!!
