
FaaS can be more cost-efficient than “serverful” libraries such as PyTorch [1]} for models that quickly converge. Although
indirect communication severely penalizes FaaS-based ML training, MLLess ameliorates its impact
with the aid of its two main optimizations, being 15X faster while \(6.3\) X cheaper than PyTorch. It must be noted that for dense models, though, the benefits
of MLLess are narrower, which suggests that where FaaS-based ML training excels is in sparse models.

Specializing distributed ML training to FaaS is crucial to yield a higher cost-efficiency.
This requires dealing with low-level issues such as high gradient sparsity, filtering out non-significant updates, or
dynamically scaling down the pool of workers, i.e., abilities that are not always available in VM-based ML systems such as PyTorch.

Filtering non-significant updates is better than bounded staleness for FaaS-based ML training. While SSP has proven to
be very effective for distributed “serverful” ML training, it renders only a marginal benefit for model training over FaaS. Since the communication
of updates is the major bottleneck in FaaS, the flexibility to delay the propagation of an update until it
eventually becomes significant is more effective than tolerating some amount of staleness.

