What exactly does conjugation mean? In group theory, the mathematical definition for "conjugation" is:
$$
(g, h) \mapsto g h g^{-1}
$$
But what exactly does this mean, like in laymans terms?

The following is equivalent to the second paragraph of Marc van Leeuwen's answer, but I think it might help emphasize how natural conjugation really is.  With notation as in Marc's answer, let me write $h'$ for the conjugate $ghg^{-1}$.  Then $h'$ is obtained by shifting $h$ along $g$ in the sense that, whenever $h$ sends an element $x\in X$ to another element $y$, then $h'$ sends $g(x)$ to $g(y)$.  If, as people sometimes do, one regards a function $h$ as a set of ordered pairs, then $h'$ is obtained by applying $g$ to both components in all those ordered pairs.  
