Why definition of Out(G) does actually make sense? I would like to ask - why does $\operatorname{Out}(G)$ := ${\operatorname{Aut}(G) / \operatorname{Inn}(G)}$ make sense? As $\operatorname{Out}(G)$ is outer automorphism group, I would expect it to include outer automorphisms, but for nontrivial $\operatorname{Inn}(G)$ the quotient is a group of cosets which are sets of functions, not functions. Isomorphism would be totally intuitive for me but equality I do not understand.

Thank you in advance!
The set of outer automorphisms (taking an "outer automorphism" of a group to be an automorphism that is not inner) does not give a group. The easiest way to see this is to consider that the identity automorphism is an inner automorphism. But there can also be other cases where the composition of two outer automorphisms gives a nontrivial inner automorphism.

Thus it makes sense to consider equivalence classes of outer automorphisms.  Looking at $\mathrm{Aut}(G)/\mathrm{Inn}(G)$, you have 1: an actual group, and 2: every nonzero equivalence class consists entirely of outer automorphisms.
Since the definition is everything that survives modding out by Inn, the definition is precisely as those equivalence classes of automorphisms which are not inner.