A2C is a synchronous, deterministic variant of asynchronous advantage actor-critic (A3C) [1]}. For autonomous driving, A2C maintains a policy \(\pi (a_t|s_t;\theta )\)  and an estimate of the value function \(V(s_t;\theta _v)\) , which uses the same mix of \(n\) -step returns to update the policy and the value function. The policy and the value function are updated every \(t_{max}\)  actions or when a terminal state is reached. The update performed by A2C can be denoted as \(\nabla _{\theta ^{\prime }} \log \pi \left(a_{t} s_{t} ; \theta ^{\prime }\right) A\left(s_{t}, a_{t} ; \theta , \theta _{v}\right)\) , and \(A(s_t,a_t;\theta ,\theta _v)\)  denotes the advantage function as follows:
\(\scalebox {1}{A(s_t,a_t;\theta ,\theta _v)=\sum _{i=0}^{k-1}\gamma ^i r_{t+i}+\gamma ^kV(s_{t+k};\theta _v)-V(s_t;\theta _v)}\) 
