where \(\tau _{i}\)  is the torque output of the \(i\) th motor, \(\bar{v}_{\text{base}}\)  and \(v_{\text{base}}\)  are desired and actual base velocities respectively. We use \(c=4\)  as the survival bonus, and \(w_{v}=1\) , \(w_{\tau }=0.0025\)  for all of our experiments. Note that we use total torques \(\sum _{i=1}^{12}\tau _{i}^2\)  as an approximation for energy because the actual energy consumption on the robot depends on the detailed circuit and motor design and is hard to estimate. Also note that this reward function is much simpler than those in prior works that only uses RL, which need careful reward shaping [1]}. Because in our framework, a significant portion of heavy lifting for locomotion control is handled by the low-level controller (Section ).
