Sum of the supremum and supremum of a sum Consider two real-valued functions of $\theta$, $f(\cdot): \Theta \subset\mathbb{R}\rightarrow \mathbb{R}$ and $g(\cdot):\Theta \subset \mathbb{R}\rightarrow \mathbb{R}$. 
Is there any relation between
(1) $\sup_{\theta \in \Theta} (f(\theta)+g(\theta))$
and 
(2) $\sup_{\theta \in \Theta} f(\theta)+\sup_{\theta \in \Theta} g(\theta)$
?
Could you provide some informal proof or intuition behind your answer?

There is a relationship, and here's the intuition. It works best if we assume that $f$ and $g$ attain their supremums on $\Theta$, but you can extend the intuition to the general case. Let $\theta_f^*$ and $\theta_g^*$ be the values in $\Theta$ at which $f$ and $g$ achieve their supremums, respectively:
$$
f(\theta_f^*) = \sup_{\theta \in \Theta} f(\theta), \qquad g(\theta_g^*) = \sup_{\theta \in \Theta} g(\theta).
$$
If $f$ and $g$ attain their supremums at the same point $\theta_f^* = \theta_g^* =: \theta^*$, then to maximize $f(\theta) + g(\theta)$, we should pick $\theta = \theta^*$; we know this is right because picking any $\theta$ other than $\theta^*$ will decrease both $f$ and $g$, and hence $f+g$. So in this case,
$$
\sup_{\theta \in \Theta} [f(\theta) + g(\theta)] = f(\theta^*) + g(\theta^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta).
$$
If $f$ and $g$ do not attain their supremums at the same point, that is, $\theta_f^* \neq \theta_g^*$, then again, the best possible value for $(f + g)(\theta)$ that we could hope for is $f(\theta_f^*) + g(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta)$. But we can only evaluate $f+g$ at one particular $\theta$; maybe we should pick $\theta_f^*$ or maybe $\theta_g^*$ or some other $\theta$, but since $\theta_f^* \neq \theta_g^*$, we cannot simultaneously maximize $f$ and $g$ to maximize $f+g$. Hence, in this case,
$$
\sup_{\theta \in \Theta} [f(\theta) + g(\theta)] \leq f(\theta_f^*) + f(\theta_g^*) = \sup_{\theta \in \Theta} f(\theta) + \sup_{\theta \in \Theta} g(\theta).
$$
This is the inequality which holds most generally. @Siddharth Joshi shows how to prove it (without assuming $f$ or $g$ attains its supremum).
