Existence of closed ball outside of closed convex set. Let $H$ be a Hilbert space, and $K\subseteq H$ a closed convex subset, with $p\in \partial K$. Is there $B$, a closed ball, with $B \cap K=\emptyset$ and $\text{conv}(B,p)\cap K=\{p\}$?

This is not true. It should be clear from looking at the drawing you made. Try moving the point $p$ (in your drawing  ! )along the boundary of $K$ and you will clearly see that for some $p$ the cone $Conv(B,p)$ intersects $K$ at more than one point.
Another counterexample :
Consider $H = \mathbb R^2$,  $K = [0,1]^2$ , $p = (0,0)$ and $B_\epsilon$ the closed ball centered at $(2,2)$ with radius $\epsilon$.
For $\epsilon$ small enough $B_\epsilon \cap K = \varnothing $ but Conv$(B_\epsilon,p) \cap K$ will always contain the diagonal $[(0,0),(1,1)]$ of the square K.
