Give all values of h for which the matrix A fails to be invertible Can someone please help me out with this please.  I know the answer is h = 8 and I know the determinant is 21h - 168 and I even know the steps to find those answers.  For some reason this is giving me fits and I must be making silly mathematical mistakes that I keep missing.  First I have been simply using cofactor expansion and cannot come up with the correct answer. Then I used row reduction where I had a seven and two zeros in the first column.
I came up with 21h - 28 at one point and 119h + 28 at one point.
Thanks to anyone taking a look at this.  I know this is a pretty simple problem but I'm not getting something right.  I am studying for a test and fill like I am understanding everything pretty well, but this one problem is making me crazy!
First I use two row reductions.  R2 becomes R1(-2)+R2 and R3 becomes R1(-1)+R3
So I get {{7,-5,3},{0,3,-5},{0,-3,h-3}}


You may be forgetting to change the sign, depending on the row/column index sum:
If we expand along the first column, e.g., we have $$7 \left|\begin{matrix}-7&1\\-8&h\end{matrix} \right| - 14\left|\begin{matrix} -5 & 3 \\ -8 & h\end{matrix} \right| + 7\left|\begin{matrix} -5 & 3\\-7&1\end{matrix}\right|\tag{1}$$
$$ = 7 \Bigg(\left|\begin{matrix}-7&1\\-8&h\end{matrix} \right| - 2\left|\begin{matrix} -5 & 3 \\ -8 & h\end{matrix} \right| + \left|\begin{matrix} -5 & 3\\-7&1\end{matrix}\right|\Bigg)\tag{2}$$
$$\det A = 7\Big( (-7h+8) -2(-5h+24)+ (-5+21)\Big) = 7(3h -24) = 21(h - 8). $$
Hence, the determinant is zero if and only if $\;(h - 8) = 0 \iff  h = 8$. That means that the given matrix is invertible for all values of $h$ except at $h = 8$.
There are a lot of places where arithmetic errors can creep in, especially in keeping straight the signs.
Also notice the subtraction of the middle cofactor in $(1)$: since the entry $14$ has an index sum of $2+ 1=3$, it is odd, so that term needs to be subtracted.
