We follow [1]} to soft-update the target-network parameter and the \(Q_{avg}\)  value,
\(\begin{split}{\mathbf {\phi }}^{\prime } & \leftarrow \beta {\mathbf {\phi }}+ (1-\beta ) {\mathbf {\phi }}^{\prime }, \\{\mathbf {\theta }}^{\prime }_j &\leftarrow \beta {\mathbf {\theta }}_j + (1-\beta ){\mathbf {\theta }}^{\prime }_j, \quad \forall \, j=1,2, \\Q_{avg} &\leftarrow \beta \frac{1}{\left|{\mathcal {B}}\right|} \sum _{s \in {\mathcal {B}}, a \sim \pi _{\mathbf {\phi }}(\cdot \,|\,s)} \left|\min _{j=1,2} Q_{{\mathbf {\theta }}_j}\left(s, a\right)\right| + (1 - \beta ) Q_{avg}.\end{split}\) 
