I'm confused how to get this result using a different definition of an adjoint operator. In the problem we're given that $ \hat{A}|0 \rangle = |0\rangle -2i|1 \rangle$ and that $ \hat{A}|1 \rangle = 2|0 \rangle -i |1 \rangle$.
We want to know what $\hat{A}^{\dagger}|0\rangle, \hat{A}^{\dagger}|1\rangle$ are using the definition of an adjoint operator,
$$\langle \psi \mid A^{\dagger}\phi \rangle = \langle \phi\mid A\psi \rangle^* $$
When I try to use this definition I get lost, but I have a value for A via my own methods on the information in sentence 1. How would I use the definition of adjoint to get the same answer as below?
I got, $$\hat{A} = \left(\begin{matrix} -i & 2i \\ 2 & 1\end{matrix}\right),    \hat{A}^{\dagger} = \left(\begin{matrix} 1 & -2i \\ -2 & -i\end{matrix}\right) $$

Rewriting the definition so it can be used in our case as follows
$$
\langle \psi |A^{\dagger}\phi \rangle = \langle \phi|A\psi \rangle^*
\iff
\langle \psi | A^{\dagger}|\phi \rangle = \langle \phi| A|\psi \rangle^*
$$
using the definition you have to find the matrix elements $A^\dagger_{j,k}$ of the operator $\hat A^\dagger$
$$
A^\dagger_{j,k} = \langle j | A^{\dagger}|k \rangle = \langle k| A|j \rangle^*
$$
Since we have $\hat{A}|0 \rangle = |0\rangle -2i|1 \rangle$ and $\hat{A}|1 \rangle = 2|0 \rangle -i |1 \rangle$ you can find, for example
$$
A^\dagger_{0,1} = \langle 0 | A^{\dagger}|1 \rangle = \langle 1| A|0 \rangle^* = (-2i)^* = 2i
$$
and the matrix of $\hat A^\dagger$ is
$$
\hat{A}^{\dagger} = \left(\begin{matrix} 1 & 2i \\ 2 & i\end{matrix}\right)
$$
The operator in the dyadic form
$$
\hat A^\dagger = |0\rangle\langle0| + 2i|0\rangle\langle1| + 2|1\rangle\langle0| + i|1\rangle\langle1|
$$
Now you can find $\hat{A}^{\dagger}|0\rangle , \hat{A}^{\dagger}|1\rangle$
