If $G$ is a finite group s.t. $|G|=4$, is it abelian ? If $G$ is a finite group s.t. $|G|=4$, is it abelian ? To me it's isomorphic to $\mathbb Z/2\mathbb Z\times \mathbb Z/2\mathbb Z$ or $\mathbb Z/4\mathbb Z$, but a friend of me said that they are not the only group of order 4, and there exist some non-abelian. Do you have such example ?

Without relying on orders of an element or anything else, you can prove this from the ground up by applying the sheer definition of a group.
Assume that $G$ is a group of order $4$, say $G=\{e,a,b,c\}$, where $e$ denotes the identity element and all elements are different. Assume that $G$ is not abelian, then (maybe after renaming the elements) we can assume that $ab \neq ba$, that is $a$ and $b$ do not commute. Now consider the element $ab$ which must be in $G$, since the set is closed under group multiplication.
If $ab=e$, then $a=b^{-1}$, and hence $a$ and $b$ commute, which is not the case.
If $ab=a$, then $b=e$, which is not the case. 
If $ab=b$, then $a=e$, which is not the case. 
Hence $ab=c$. But also $ba \in G$. It cannot be equal to $ab$ by assumption. If $ba=e$, then again $a=b^{-1}$, which is not the case.
If $ba=a$, then $b=e$, which is not the case. 
If $ba=b$, then $a=e$, which is the final contradiction.
So after all, $G$ must be abelian.
Note with a similar slightly more subtle reasoning you can prove that a group of order $5$ must be abelian. Try it, or see here.
