Is $G= \mathbb{Z}$ a group with binary operation defined as $a \cdot b \equiv a - b$? 
Is $G= \mathbb{Z}$  a group with binary operation defined as $a \cdot b \equiv a - b$?

I am pretty sure the answer is NO but I would like some verification on my reasoning below: 
Associativity fails as in:
Let $a,b,c \in G$ then $a \cdot (b \cdot c) = a \cdot (b-c) = a - (b-c) = a - b + c$.
But $(a \cdot b) \cdot c = (a-b) \cdot c = a - b - c \neq a \cdot (b \cdot c)$.
Identity
Let $a \in G$. Then $a \cdot 0 = a - 0 = a$, so $0$ is the additive identity of $a$, hence the identity element in $G$. But $0 \cdot a = 0 - a = -a \neq a \cdot 0$
Inverse
Let $a \in G \backslash \{0\}$. Assume $G$ is a group, then $a$ has an inverse $a^{-1}$. So $a \cdot a^{-1} = a - a^{-1} = a - (-a) \neq 0$.
I am confident about associativity, but I am not sure if my arguments about the inverse and identity are justified. 

You don't need to disprove every property for $(\mathbb Z,-)$ not to be a group, it suffices to show that at least one property does not hold.
Indeed, as you rightly pointed out, associativity does not work, so it is not a group.
If I tell you that every cat has whiskers, a tail and four paws, it suffices for you to show me that a human doesn't have whiskers in order to deduce that a human is not a cat (you don't have to bother with tail and paws). 
