Like 1-D, is the force always directed towards the minimum of the potential in 3-D? Assertion Consider the motion of a particle of mass $m$ under the influence of a potential $V(x)$ in 1-dimension. The force on that particle is given by $\textbf{F}(x)=-\frac{dV}{dx}\hat{\textbf{x}}$. From this, it is clear that the $\textbf{F}(x)$ is always directed downhill at any point on $x$-axis i.e., towards a local minimum of the potential. This is true independent of the form of the potential $V(x)$. 

Question In 3-D, $\textbf{F}(x,y,z)=-\boldsymbol{\nabla} V(x,y,z)$. What can we say in this case about the direction of the force at any point $(x,y,z)$? Again I'm looking for the most general conclusion that can be drawn for an arbitrary V(x,y,z) but reasonably well-behaved.
Let $V: \mathbb{R}^3 \rightarrow \mathbb{R}$ be a potential function.

The force is now $F = -\nabla V$, that is, it is minus the gradient of $V$. The gradient is the direction of steepest ascent. Hence, minus the gradient is the direction of steepest descent.

Now there are two claims:


Following the force (or minus the gradient) infinitesimally will give you a local minimum, provided that the potential is bounded below$^1$. 
The gradient need not point in the direction of a global or local minimum.


For the second point I invite you to think about a hilly landscape, here we can define a function $h: \mathbb{R}^2 \rightarrow \mathbb{R}$ to be the height of the current position. Now, a river will flow along the direction of steepest descent (that is $-\nabla h$), but if you are standing on the bank of a river and looking downstream there is no guarantee that you are looking in the direction of the lake where the river ends up. Indeed the river may meander back and forth, and may even stream in the opposite direction before doubling back on itself!

The first point is the justification for the method of gradient descent for finding minima/maxima of functions.

A proof would presumably use the gradient theorem, but I don't have time to write anything up right now.

(1): A local minimum could still be located "at infinity", if this bothers you, you could ask for the domain of $V$ to be compact.
$$F_x=-\dfrac{\partial V(x,y,z)}{\partial(x)}$$
$$F_y=-\dfrac{\partial V(x,y,z)}{\partial(y)}$$
$$F_z=-\dfrac{\partial V(x,y,z)}{\partial(z)}$$
So the total force is:
$F=F_x+F_y+F_z$
Obviously this is a vector sum. As you said the $V(x,y,z)$ 
must be a reasonably well behaved potential. In that case the direction of the force is directed in a direction orthogonal to the tangent plane of $V(x,y,z)$
in any point $P(x,y,z)$