To evaluate performance versus wall clock time, we model the time
that each node takes to calculate a fixed local minibatch of size
\(b\)  as random variable that follows a shifted exponential distribution.
Exponential and shifted exponential distributions are widely used
in the literature to model variable compute times [1]}, [2]}.
We use \(T_{i}(t)\)  to denote the time taken by worker \(i\)  to calculate
\(b\)  gradients in the \(t\) -th epoch. The probability density function
of \(T_{i}(t)\)  is
\(f_{T_{i}}(\tau )=\lambda e^{-\lambda (\tau -\xi )},\,\,\tau \ge \xi \ge 0\) 
