THECOMPUTATIONAL LIMITS OF DEEPLEARNING
Neil C. Thompson1, Kristjan Greenewald2, Keeheon Lee3, Gabriel F. Manso4
1MIT Computer 
The Computational Limits of Deep Learning
Deep learning's recent history has been one of achievement: from triumphing
over humans in the game of Go to world-leading performance in image
classification, voice recognition, translation, and other tasks. But this
progress has come with a voracious appetite for computing power. This article
catalogs the extent of this dependency, showing that progress across a wide
variety of applications is strongly reliant on increases in computing power.
Extrapolating forward this reliance reveals that progress along current lines
is rapidly becoming economically, technically, and environmentally
unsustainable. Thus, continued progress in these applications will require
dramatically more computationally-efficient methods, which will either have to
come from changes to deep learning or from moving to other machine learning
methods.
Neil C. Thompson Kristjan Greenewald Keeheon Lee Gabriel F. Manso