When is an upper triangular matrix invertible? I did the following (the exercise assumes $2\times 2$ matrices):
$$\begin{pmatrix}
{a}&{b}\\ 
{0}&{d}
\end{pmatrix} \begin{pmatrix}
{A}&{B}\\ 
{C}&{D}
\end{pmatrix} = \begin{pmatrix}
{1}&{0}\\ 
{0}&{1}
\end{pmatrix}$$
And then, for it to have an inverse would be the same of finding $A,B,C,D$ on:
\begin{eqnarray*}
  {aA+bC}&=&{1} \\ 
  {aB+bD}&=&{0} \\ 
  {dC}&=&{0} \\ 
  {dD}&=&{1} 
\end{eqnarray*}
Here, If $dC=0$ then $d=0$ or $D=0$, $d\neq 0$ otherwise $dD=0$ then $C=0.$ With this, we found $C$ and we can proceed to:
\begin{eqnarray*}
  {aA}&=&{1} \\ 
  {aB+bD}&=&{0} \\  
  {dD}&=&{1} 
\end{eqnarray*}
And then: $a=\frac{1}{A}$ ,now:
$$\begin{eqnarray*}
  {\frac{B}{A}+bD}&=&{0} \\  
  {dD}&=&{1} 
\end{eqnarray*}\tag{1}$$
And then, It is invertible if $C=0$ and we can find $A,B,D$ such that $(1)$ is satisfied. Is this correct? I think it's still too complicated and perhaps wrong.

An upper triangular matrix is invertible if and only if it has no zeros on the main diagonal. Here are some ways to see this:


*

*The determinant of such a matrix is the product of the diagonal entries, and is non-zero if and only if the condition above holds.

*The matrix has full rank whenever there are no zeros on the diagonal.

*The inverse of the matrix can be explicitly computed via row operations. Use the bottom row to clean out the last column, the second to bottom row to clean out the second to last column, and so on.
Now in your case, it's a bit simpler; there's a general form for finding the inverse of a $2 \times 2$ matrix by switching around elements, and the inverse is
$$\left[\begin{array}{cc} a & b \\ 0 & d\end{array}\right]^{-1} = \frac{1}{ad} \left[\begin{array}{cc} d & -b\\ 0 & a\end{array}\right]$$
