How can we prove that the rank of a matrix is a non-convex function of that matrix? How can we prove that  $\operatorname{rank}(\mathbf{X})= 1$ is a non-convex function of $\mathbf{X}$.

Rank of a matrix is not convex as shown by the above answer.  However, it is  worth noting that if the matrix is positive semidefinite, it is a quasiconcave function. The modified Jensen's inequality for quasiconcave functions is $$f(\theta x+(1-\theta)y) \geq min(f(x),f(y))$$
The proof follows from the fact that $$rank(A+B)\geq min(rank(A),rank(B)) ,  A,B \in S_n^+$$
