The actor network is updated to produce actions which maximize expected returns. Therefore, loss in this case is simply mean of \(Q\) -values for states. The critic network is updated with simple TD loss. As in DQN [1]}, [2]} DDPG also uses replay buffer and target network (in this case for both actor and ctitic) to stabilize learning.
<FIGURE>