1.
First, following the deep learning approach in [1]}, we shall use neural networks functions \(\mathrm {a}_\theta \)  \(:\)  \([0,T]\times {\mathbb {R}}^d\)  \(\rightarrow \)  \(A\)  \(\subset \)  \({\mathbb {R}}^q\) , to approximate the optimal feedback control \(\hat{b}\) , by maximizing over parameters \(\theta \)  the objective function
\( \begin{split}  J(\theta ) &= \; {\mathbb {E}}\Big [ g(X_T^\theta ) + \int _0^T f(X_t^\theta ,\mathrm {a}_\theta (t,X_t^\theta )) {\rm d}t \Big ],\end{split} \) 
where \(X^\theta \)  solves
\( \begin{split} dX_t^\theta &= \; \mathrm {b}(X_t^\theta ,\mathrm {a}_\theta (t,X_t^\theta )) {\rm d}t + \sigma (X_t^\theta ,\mathrm {a}_\theta (t,X_t^\theta )) {\rm d}W_t, \quad 0 \le t \le T,\end{split} \) 
with initial condition \(X_0^\theta \)  distributed acccording to some law \(\mu _0\)  on \({\mathbb {R}}^d\) .
We denote by \(\theta ^*\)  the “optimal parameter" that maximizes \(J(\theta )\) , and set \(\mathrm {a}^*\)  \(=\)  \(\mathrm {a}_{\theta ^*}\) .
We denote by \(X^*\)  \(=\)  \(X^{\theta ^*}\)  an approximation of the optimal state process \(\hat{X}\) . For the numerical implementation, we discretize in time the process \(X^\theta \)  and the integral over \(f\)  in (REF ), and apply a stochastic gradient ascent algorithm based on samples of \(X^\theta \) .
The pseudo-code is presented in Algorithm REF .
2.
Once we get an approximation of the optimal feedback control, we could in principle compute \(u(t,x)\)  from the Feynman-Kac representation (REF ) by Monte-Carlo simulations of \(X^*\) . However, with the purpose of solving the PDE (REF ) on the whole domain, this has to be performed for every point \((t,x)\)  \(\in \)  \([0,T]\times {\mathbb {R}}^d\) , which is not feasible in practice. Instead, we apply
three types of differential learning
methods for approximating simultaneously the value function \(u\) , as well as its derivative:
(i) the first one, called differential regression learning, is directly inspired from the original approach
in [2]}, and
gives an approximation of \(u\)  and its first derivative \(D_x u\)  from the minimization of two loss functions
based on least-square regressions,
(ii) the second one in the spirit of [3]}, [4]}, which uses a contingent claim hedging strategy as a Monte Carlo control variate,
approximates the value function and its first derivative from the minimization of a single loss function based on the martingale representation in (REF ), and is refered to as pathwise martingale learning method.
(iii) the third one, called pathwise differential learning, provides in addition an accurate approximation of the second derivative \(D_x^2 u\)  of \(u\) .
We develop these three methods and present their pseudo-codes in the next sections.

