The Deep Learning Compiler: A Comprehensive Survey <s> • Lambda expression: TVM represents operation IR by a lambda expression(shown in Fig- <s> There is an increasing need to bring machine learning to a wide diversity of hardware devices. Current frameworks rely on vendor-specific operator libraries and optimize for a narrow range of server-class GPUs. Deploying workloads to new platforms -- such as mobile phones, embedded devices, and accelerators (e.g., FPGAs, ASICs) -- requires significant manual effort. We propose TVM, a compiler that exposes graph-level and operator-level optimizations to provide performance portability to deep learning workloads across diverse hardware back-ends. TVM solves optimization challenges specific to deep learning, such as high-level operator fusion, mapping to arbitrary hardware primitives, and memory latency hiding. It also automates optimization of low-level programs to hardware characteristics by employing a novel, learning-based cost modeling method for rapid exploration of code optimizations. Experimental results show that TVM delivers performance across hardware back-ends that are competitive with state-of-the-art, hand-tuned libraries for low-power CPU, mobile GPU, and server-class GPUs. We also demonstrate TVM's ability to target new accelerator back-ends, such as the FPGA-based generic deep learning accelerator. The system is open sourced and in production use inside several major companies. <s> BIB001 </s> The Deep Learning Compiler: A Comprehensive Survey <s> • Lambda expression: TVM represents operation IR by a lambda expression(shown in Fig- <s> Deep learning models with convolutional and recurrent networks are now ubiquitous and analyze massive amounts of audio, image, video, text and graph data, with applications in automatic translation, speech-to-text, scene understanding, ranking user preferences, ad placement, etc. Competing frameworks for building these networks such as TensorFlow, Chainer, CNTK, Torch/PyTorch, Caffe1/2, MXNet and Theano, explore different tradeoffs between usability and expressiveness, research or production orientation and supported hardware. They operate on a DAG of computational operators, wrapping high-performance libraries such as CUDNN (for NVIDIA GPUs) or NNPACK (for various CPUs), and automate memory allocation, synchronization, distribution. Custom operators are needed where the computation does not fit existing high-performance library calls, usually at a high engineering cost. This is frequently required when new operators are invented by researchers: such operators suffer a severe performance penalty, which limits the pace of innovation. Furthermore, even if there is an existing runtime call these frameworks can use, it often doesn't offer optimal performance for a user's particular network architecture and dataset, missing optimizations between operators as well as optimizations that can be done knowing the size and shape of data. Our contributions include (1) a language close to the mathematics of deep learning called Tensor Comprehensions offering both imperative and declarative styles, (2) a polyhedral Just-In-Time compiler to convert a mathematical description of a deep learning DAG into a CUDA kernel with delegated memory management and synchronization, also providing optimizations such as operator fusion and specialization for specific sizes, (3) a compilation cache populated by an autotuner. [Abstract cutoff] <s> BIB002
ure 3(a)) BIB001 , an index formula expression, and TVM offers a language called tensor expression through the lambda expression. Lambda expression describes calculation by variable binding and substitution, which widely exists in programming languages such as Python and C++11. In TVM, computational operators in tensor expression are composed of two parts: the shape of output tensor and the lambda expression of computing rules. • Einstein notation: The Einstein notation, also called the summation convention, is adopted by TC. The Einstein notation is a notation to express summation. Take the definition of matrix-vector multiplication (shown in Figure 3 (b)) BIB002 for example, temporary variables for index do not need to be defined. TC IR can figure out the actual expression by the occurrence of undefined variables. For example, in the second line of Figure 3 (b), the variable k only exists on the right, so the operator + will perform the reduction operator over the dimension represented by k. In the Einstein notation expression, the operator needs to be associative and commutative, and operators such as max are not supported. This restriction promises the reduction operator can be executed by any order, making it possible for further parallelization.