Would it be possible to form such a group using the ADD instruction and the NOT instruction?
Can you form a group with assembly instructions under the MIPS-32 architecture?
group theory
Sure.  The integers modulo $2^{32}$ form a group.  The group operation is addition modulo $2^{32}$, which can be implemented by the ADD instruction.  You don't need the NOT instruction.There are other groups you could form, such as the integers modulo 2, and many more.  I recommend you read the definition of a group and play around with some examples.