The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> The polyhedral model is known to be a powerful framework to reason about high level loop transformations. Recent developments in optimizing compilers broke some generally accepted ideas about the limitations of this model. First, thanks to advances in dependence analysis for irregular access patterns, its applicability which was supposed to be limited to very simple loop nests has been extended to wide code regions. Then, new algorithms made it possible to compute the target code for hundreds of statements while this code generation step was expected not to be scalable. Such theoretical advances and new software tools allowed actors from both academia and industry to study more complex and realistic cases. Unfortunately, despite strong optimization potential of a given transformation for e.g., parallelism or data locality, code generation may still be challenging or result in high control overhead. This paper presents scalable code generation methods that make possible the application of increasingly complex program transformations. By studying the transformations themselves, we show how it is possible to benefit from their properties to dramatically improve both code generation quality and space/time complexity, with respect to the best state-of-the-art code generation tool. In addition, we build on these improvements to present a new algorithm improving generated code performance for strided domains and reindexed schedules. <s> BIB001 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> This paper presents a new polyhedra scanning system called CodeGen+ to address the challenge of generating high-performance code for complex iteration spaces resulting from compiler optimization and autotuning systems. The strength of our approach lies in two new algorithms. First, a loop overhead removal algorithm provides precise control of trade-offs between loop overhead and code size based on actual loop nesting depth. Second, an if-statement simplification algorithm further reduces the number of comparisons in the code. These algorithms combined with the expressive power of Presburger arithmetic enable CodeGen+ to support complex optimization strategies expressed in iteration spaces. We compare with the state-of-the-art polyhedra scanning tool CLooG on five loop nest computations, demonstrating that CodeGen+ generates code that is simpler and up to 1.15x faster. <s> BIB002 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> This paper describes a loop transformation framework that extends a polyhedral representation of loop nests to represent and transform computations with non-affine index arrays in loop bounds and subscripts via a new interface between compile-time and run-time abstractions. Polyhedra scanning code generation, which historically applies an affine mapping to the subscript expressions of the statements in a loop nest, is modified to apply non-affine mappings involving index arrays that are represented at compile time by uninterpreted functions; non-affine loop bounds involving index arrays are also represented. When appropriate, an inspector is utilized to capture the non-affine subscript mappings, and a generalized loop coalescing transformation is introduced as a non-affine transformation to support non-affine loop bounds. With this support, complex sequences of new and existing transformations can then be composed. We demonstrate the effectiveness of this framework by optimizing sparse matrix vector multiplication operations targeting GPUs for different matrix structures and parallelization strategies. This approach achieves performance that is comparable to or greater than the hand-tuned CUSP library; for two of the implementations it achieves an average 1.14× improvement over CUSP across a collection of sparse matrices, while the third performs on average within 8% of CUSP. <s> BIB003 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> The Julia programming language is gaining enormous popularity. Julia was designed to be easy and fast. Most importantly, Julia shatters deeply established notions widely held in the applied community: $\begin{array}{l} \mbox{1. High-level, dynamic code has to be slow by some sort of law of nature. } \\ \mbox{2. It is sensible to prototype in one language and then recode in another language.} \\ \mbox {3. There are parts of a system for the programmer, and other parts best left untouched as they are built by the experts.} \end{array}$ Julia began with a deep understanding of the needs of the scientific programmer and the needs of the computer in mind. Bridging cultures that have often been distant, Julia combines expertise from computer science and computational science creating a new approach to scientific computing. This note introduces the programmer to the language and the underlying design theory. It invites the reader to rethink the fundamental foundations of numerical computing systems. In particular, there is the fascinating dance between specialization and abstraction. Specialization allows for custom treatment. We can pick just the right algorithm for the right circumstance and this can happen at runtime based on argument types (code selection via multiple dispatch). Abstraction recognizes what remains the same after differences are stripped away and ignored as irrelevant. The recognition of abstraction allows for code reuse (generic programming). A simple idea that yields incredible power. The Julia design facilitates this interplay in many explicit and subtle ways for machine performance and, most importantly, human convenience. <s> BIB004 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> The Canonical Polyadic Decomposition (CPD) of tensors is a powerful tool for analyzing multi-way data and is used extensively to analyze very large and extremely sparse datasets. The bottleneck of computing the CPD is multiplying a sparse tensor by several dense matrices. Algorithms for tensor-matrix products fall into two classes. The first class saves floating point operations by storing a compressed tensor for each dimension of the data. These methods are fast but suffer high memory costs. The second class uses a single uncompressed tensor at the cost of additional floating point operations. In this work, we bridge the gap between the two approaches and introduce the compressed sparse fiber (CSF) a data structure for sparse tensors along with a novel parallel algorithm for tensor-matrix multiplication. CSF offers similar operation reductions as existing compressed methods while using only a single tensor structure. We validate our contributions with experiments comparing against state-of-the-art methods on a diverse set of datasets. Our work uses 58% less memory than the state-of-the-art while achieving 81% of the parallel performance on 16 threads. <s> BIB005 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> This paper introduces three new compiler transformations for representing and transforming sparse matrix computations and their data representations. In cooperation with run-time inspection, our compiler derives transformed matrix representations and associated transformed code to implement a variety of representations targeting different architecture platforms. This systematic approach to combining code and data transformations on sparse computations, which extends a polyhedral transformation and code generation framework, permits the compiler to compose these transformations with other transformations to generate code that is on average within 5% and often exceeds manually-tuned, high-performance sparse matrix libraries CUSP and OSKI. Additionally, the compiler-generated inspector codes are on average 1.5 faster than OSKI and perform comparably to CUSP, respectively. <s> BIB006 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> The aim of the CTU Prague Relational Learning Repository is to support machine learning research with multi-relational data. The repository currently contains 50 SQL databases hosted on a public MySQL server located at relational.fit.cvut.cz. A searchable meta-database provides metadata (e.g., the number of tables in the database, the number of rows and columns in the tables, the number of foreign key constraints between tables). <s> BIB007 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> In the last decade, machine-learning-based compilation has moved from an obscure research niche to a mainstream activity. In this paper, we describe the relationship between machine learning and compiler optimization and introduce the main concepts of features, models, training, and deployment. We then provide a comprehensive survey and provide a road map for the wide variety of different research areas. We conclude with a discussion on open issues in the area and potential research directions. This paper provides both an accessible introduction to the fast moving area of machine-learning-based compilation and a detailed bibliography of its main achievements. <s> BIB008 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Google's Cloud TPUs are a promising new hardware architecture for machine learning workloads. They have powered many of Google's milestone machine learning achievements in recent years. Google has now made TPUs available for general use on their cloud platform and as of very recently has opened them up further to allow use by non-TensorFlow frontends. We describe a method and implementation for offloading suitable sections of Julia programs to TPUs via this new API and the Google XLA compiler. Our method is able to completely fuse the forward pass of a VGG19 model expressed as a Julia program into a single TPU executable to be offloaded to the device. Our method composes well with existing compiler-based automatic differentiation techniques on Julia code, and we are thus able to also automatically obtain the VGG19 backwards pass and similarly offload it to the TPU. Targeting TPUs using our compiler, we are able to evaluate the VGG19 forward pass on a batch of 100 images in 0.23s which compares favorably to the 52.4s required for the original model on the CPU. Our implementation is less than 1000 lines of Julia, with no TPU specific changes made to the core Julia compiler or any other Julia packages. <s> BIB009 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Artificial intelligence (AI) has undergone a renaissance recently, making major progress in key domains such as vision, language, control, and decision-making. This has been due, in part, to cheap data and cheap compute resources, which have fit the natural strengths of deep learning. However, many defining characteristics of human intelligence, which developed under much different pressures, remain out of reach for current approaches. In particular, generalizing beyond one's experiences--a hallmark of human intelligence from infancy--remains a formidable challenge for modern AI. The following is part position paper, part review, and part unification. We argue that combinatorial generalization must be a top priority for AI to achieve human-like abilities, and that structured representations and computations are key to realizing this objective. Just as biology uses nature and nurture cooperatively, we reject the false choice between"hand-engineering"and"end-to-end"learning, and instead advocate for an approach which benefits from their complementary strengths. We explore how using relational inductive biases within deep learning architectures can facilitate learning about entities, relations, and rules for composing them. We present a new building block for the AI toolkit with a strong relational inductive bias--the graph network--which generalizes and extends various approaches for neural networks that operate on graphs, and provides a straightforward interface for manipulating structured knowledge and producing structured behaviors. We discuss how graph networks can support relational reasoning and combinatorial generalization, laying the foundation for more sophisticated, interpretable, and flexible patterns of reasoning. As a companion to this paper, we have released an open-source software library for building graph networks, with demonstrations of how to use them in practice. <s> BIB010 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Graph Neural Networks (GNNs) are an effective framework for representation learning of graphs. GNNs follow a neighborhood aggregation scheme, where the representation vector of a node is computed by recursively aggregating and transforming representation vectors of its neighboring nodes. Many GNN variants have been proposed and have achieved state-of-the-art results on both node and graph classification tasks. However, despite GNNs revolutionizing graph representation learning, there is limited understanding of their representational properties and limitations. Here, we present a theoretical framework for analyzing the expressive power of GNNs to capture different graph structures. Our results characterize the discriminative power of popular GNN variants, such as Graph Convolutional Networks and GraphSAGE, and show that they cannot learn to distinguish certain simple graph structures. We then develop a simple architecture that is provably the most expressive among the class of GNNs and is as powerful as the Weisfeiler-Lehman graph isomorphism test. We empirically validate our theoretical findings on a number of graph classification benchmarks, and demonstrate that our model achieves state-of-the-art performance. <s> BIB011 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics system, learning molecular fingerprints, predicting protein interface, and classifying diseases require a model to learn from graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures, like the dependency tree of sentences and the scene graph of images, is an important research topic which also needs graph reasoning models. Graph neural networks (GNNs) are connectionist models that capture the dependence of graphs via message passing between the nodes of graphs. Unlike standard neural networks, graph neural networks retain a state that can represent information from its neighborhood with arbitrary depth. Although the primitive GNNs have been found difficult to train for a fixed point, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful learning with them. In recent years, systems based on variants of graph neural networks such as graph convolutional network (GCN), graph attention network (GAT), gated graph neural network (GGNN) have demonstrated ground-breaking performance on many tasks mentioned above. In this survey, we provide a detailed review over existing graph neural network models, systematically categorize the applications, and propose four open problems for future research. <s> BIB012 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Frameworks for writing, compiling, and optimizing deep learning (DL) models have recently enabled progress in areas like computer vision and natural language processing. Extending these frameworks to accommodate the rapidly diversifying landscape of DL models and hardware platforms presents challenging tradeoffs between expressivity, composability, and portability. We present Relay, a new compiler framework for DL. Relay's functional, statically typed intermediate representation (IR) unifies and generalizes existing DL IRs to express state-of-the-art models. The introduction of Relay's expressive IR requires careful design of domain-specific optimizations, addressed via Relay's extension mechanisms. Using these extension mechanisms, Relay supports a unified compiler that can target a variety of hardware platforms. Our evaluation demonstrates Relay's competitive performance for a broad class of models and devices (CPUs, GPUs, and emerging accelerators). Relay's design demonstrates how a unified IR can provide expressivity, composability, and portability without compromising performance. <s> BIB013 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> TensorFlow Eager is a multi-stage, Python-embedded domain-specific language for hardware-accelerated machine learning, suitable for both interactive research and production. TensorFlow, which TensorFlow Eager extends, requires users to represent computations as dataflow graphs; this permits compiler optimizations and simplifies deployment but hinders rapid prototyping and run-time dynamism. TensorFlow Eager eliminates these usability costs without sacrificing the benefits furnished by graphs: It provides an imperative front-end to TensorFlow that executes operations immediately and a JIT tracer that translates Python functions composed of TensorFlow operations into executable dataflow graphs. TensorFlow Eager thus offers a multi-stage programming model that makes it easy to interpolate between imperative and staged execution in a single package. <s> BIB014 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> Deep learning has revolutionized many machine learning tasks in recent years, ranging from image classification and video processing to speech recognition and natural language understanding. The data in these tasks are typically represented in the Euclidean space. However, there is an increasing number of applications, where data are generated from non-Euclidean domains and are represented as graphs with complex relationships and interdependency between objects. The complexity of graph data has imposed significant challenges on the existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. In this article, we provide a comprehensive overview of graph neural networks (GNNs) in data mining and machine learning fields. We propose a new taxonomy to divide the state-of-the-art GNNs into four categories, namely, recurrent GNNs, convolutional GNNs, graph autoencoders, and spatial-temporal GNNs. We further discuss the applications of GNNs across various domains and summarize the open-source codes, benchmark data sets, and model evaluation of GNNs. Finally, we propose potential research directions in this rapidly growing field. <s> BIB015 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> The huge computation demand of deep learning models and limited computation resources on the edge devices calls for the cooperation between edge device and cloud service by splitting the deep models into two halves. However, transferring the intermediates results from the partial models between edge device and cloud service makes the user privacy vulnerable since the attacker can intercept the intermediate results and extract privacy information from them. Existing research works rely on metrics that are either impractical or insufficient to measure the effectiveness of privacy protection methods in the above scenario, especially from the aspect of a single user. In this paper, we first present a formal definition of the privacy protection problem in the edge-cloud system running DNN models. Then, we analyze the-state-of-the-art methods and point out the drawbacks of their methods, especially the evaluation metrics such as the Mutual Information (MI). In addition, we perform several experiments to demonstrate that although existing methods perform well under MI, they are not effective enough to protect the privacy of a single user. To address the drawbacks of the evaluation metrics, we propose two new metrics that are more accurate to measure the effectiveness of privacy protection methods. Finally, we highlight several potential research directions to encourage future efforts addressing the privacy protection problem. <s> BIB016 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> A wide variety of deep neural applications increasingly rely on the cloud to perform their compute-heavy inference. This common practice requires sending private and privileged data over the network to remote servers, exposing it to the service provider and potentially compromising its privacy. Even if the provider is trusted, the data can still be vulnerable over communication channels or via side-channel attacks in the cloud. To that end, this paper aims to reduce the information content of the communicated data with as little as possible compromise on the inference accuracy by making the sent data noisy. An undisciplined addition of noise can significantly reduce the accuracy of inference, rendering the service unusable. To address this challenge, this paper devises Shredder, an end-to-end framework, that, without altering the topology or the weights of a pre-trained network, learns additive noise distributions that significantly reduce the information content of communicated data while maintaining the inference accuracy. The key idea is finding the additive noise distributions by casting it as a disjoint offline learning process with a loss function that strikes a balance between accuracy and information degradation. The loss function also exposes a knob for a disciplined and controlled asymmetric trade-off between privacy and accuracy. While keeping the DNN intact, Shredder divides inference between the cloud and the edge device, striking a balance between computation and communication. In the separate phase of inference, the edge device takes samples from the Laplace distributions that were collected during the proposed offline learning phase and populates a noise tensor with these sampled elements. Then, the edge device merely adds this populated noise tensor to the intermediate results to be sent to the cloud. As such, Shredder enables accurate inference on noisy intermediate data without the need to update the model or the cloud, or any training process during inference. We also formally show that Shredder maximizes privacy with minimal impact on DNN accuracy while the tradeoff between privacy and accuracy is controlled through a mathematical knob. Experimentation with six real-world DNNs from text processing and image classification shows that Shredder reduces the mutual information between the input and the communicated data to the cloud by 74.70% compared to the original execution while only sacrificing 1.58% loss in accuracy. On average, Shredder also offers a speedup of 1.79x over Wi-Fi and 2.17x over LTE compared to cloud-only execution when using an off-the-shelf mobile GPU (Tegra X2) on the edge. <s> BIB017 </s> The Deep Learning Compiler: A Comprehensive Survey <s> CONCLUSION AND FUTURE DIRECTIONS <s> We present and evaluate Deep Private-Feature Extractor (DPFE) , a deep model which is trained and evaluated based on information theoretic constraints. Using the selective exchange of information between a user's device and a service provider, DPFE enables the user to prevent certain sensitive information from being shared with a service provider, while allowing them to extract approved information using their model. We introduce and utilize the log-rank privacy, a novel measure to assess the effectiveness of DPFE in removing sensitive information and compare different models based on their accuracy-privacy trade-off. We then implement and evaluate the performance of DPFE on smartphones to understand its complexity, resource demands, and efficiency trade-offs. Our results on benchmark image datasets demonstrate that under moderate resource utilization, DPFE can achieve high accuracy for primary tasks while preserving the privacy of sensitive information. <s> BIB018
In this survey, we present a thorough analysis of the existing DL compilers. First, we provide a comprehensive comparison of the existing DL compilers from various aspects, which can serve as the guideline for users to choose the suitable DL compiler for their customized scenarios. Then, we take a deep dive into the common design adopted in the existing DL compilers including the multi-level IR, the frontend, and the backend. We present the design philosophy and reference implementation of each component in detail, with the emphasis on the unique IRs and optimizations specific to DL compilers. We summarize the findings in this survey and highlight the future directions in DL compiler as follows, which we hope can shed the light to encourage more researchers and practitioners contribute to this field. Dynamic shape and pre/post processing -Dynamic model becomes more and more popular in the field of deep learning, whose input shape or even model itself may change in execution. Particularlly, dynamic shape is the major concern of dynamic models, which is partially supported by TVM and other DL compilers. In the DL community, especially in natural language processing (NLP), models may accept inputs of various shapes, which is challenging for DL compilers since the shape of data is unknown until runtime. Existing DL compilers require more research efforts to support dynamic shape efficiently for emerging dynamic models. In addition, as future DL models become more complex, their entire control flow may inevitably include complicated pre/post-processing procedures. Currently, most DL compilers use Python as their programming language, the pre/post-processing could become a performance bottleneck when it is executed by the Python interpreter. Such potential performance bottleneck has not yet been considered by existing DL compilers. Supporting the entire control flow in DL compiler enables express and optimize the pre/post-processing along with DL models, which opens up new opportunities for performance acceleration in model deployment. Advanced auto-tuning -Existing auto-tuning techniques focus on the optimization of individual operators. However, the combination of the local optimal does not lead to global optimal. For example, two adjacent operators that apply on different data layouts can be performance tuned together without introducing extra memory transformation nodes in between. Besides, with the rise of edge computing, execution time is not only the optimization objective for DL compilers. New optimization targets should also be considered in the auto-tuning such as less memory footprint and lower energy consumption. For ML-based auto-tuning techniques, there are several directions worth further exploring. First, the ML techniques should be applied in other stages of auto-tuning, other than the cost model. For example, in the stage of selecting compiler options and optimization schedules, instead of developing a cost model, ML techniques can be used to predict the expectant possibility directly and develop algorithm to determine the final result. Additionally, various computation characteristics and optimization objectives of different DL models require their specific tuning models trained with different datasets and objective functions rather than sharing a universal auto-tuning model. Second, the ML-based auto-tuning techniques can also be improved based on the domain knowledge of DL models. For example, incorporating the feature engineering (selecting features to represent program) BIB008 in auto-tuning techniques could be a potential direction for achieving better tuning results. Besides, in additional to the static code feature gathered from IR, other computation characteristics are also expressive and strongly correlated with performance. For example, the features of the computational graph can represent data dependency and memory movement more intuitively and enable better execution time prediction. Polyhedral model -As described in Section 4.5.2, the auto-tuning can be applied to minimize the overhead of polyhedral JIT compilation by reusing the previous configurations. At the other hand, the polyhedral model can be used to perform auto-scheduling, which can reduce the search space of auto-tuning. It is a promising research direction to further apply the combination of the polyhedral model and the auto-tuning to the design of DL compilers for efficiency. Another challenge of the polyhedral model is to support the sparse tensor. In general, the format of a sparse tensor such as CSF BIB005 expresses the loop indices with index arrays (e.g., a[b[i]]) that is no longer linear. Such indirect index addressing leads to non-affine subscript expressions and loop bounds, which prohibits the loop optimization of the polyhedral model BIB002 BIB001 . Fortunately, the polyhedral community has made progress in supporting sparse tensor BIB006 BIB003 , and integrating the latest advancement of the polyhedral model can increase the performance opportunities for future DL compilers. Subgraph partitioning -A DL compiler supporting subgraph partitioning can partition the computation graph into several subgraphs. By this approach, the computation graph can no longer be treated as a whole, and the subgraphs can be processed in different manners. The subgraph partitioning can bring at least two advantages to DL compilers. First, it opens up the possibility to integrate more libraries that apply graph optimizations. Take the nGraph with DNNL for example, DNNL is a DL optimized libraries on CPU, which can apply layer fusion and other graph optimizations by leveraging its diverse collection of highly optimized kernels. This integration enables DNNL to optimize and execute the most of compatible subgraphs, while leaves the remaining graph to nGraph. The integration of TensorFlow with TensorRT adopts a similar approach. Other DL compilers however, fail to exploit such integration. Take TVM for example, TVM supports invoking DNNL library, but treats it as a regular BLAS library without utilizing the subgraph optimizations. Similar problem happens on dedicated DL accelerators, where they rely on customized graph optimization libraries. In sum, the integration of libraries with graph optimizations can provide an alternative approach to improve the performance of the code generated from the DL compilers. Besides, the integration also provides a new approach to support hardware targets by invoking graph optimized libraries. Second, it opens up the possibility of heterogeneous and parallel execution. Currently, the scenarios at two extreme scales such as edge devices and data centers for deploying DL models all exhibit the trend of heterogeneous and parallel execution. Once the computation graph is partitioned into subgraphs, the execution of different subgraphs can be assigned to heterogeneous hardware targets at the same time. Take the edge device for example, its computation units may consist of ARM CPU, Mail GPU, DSP, and probably NPU. Generating code from the DL compilers that utilizes all computation units efficiently can deliver significant speedup of the DL tasks, such as face recognition and voice assistant. Quantization -Quantization is a well-known optimization technique in DL models that can reduce the burden of computation and memory by lowing the precision of operational data. With reduced bit-width, the data can be stored and calculated with less resource at the expense of slightly reduced model accuracy. The major challenge of quantization is to design the strategies that tradeoff between the benefits of low precision and loss of model accuracy. Traditional quantization strategies applied in DL frameworks are based on a set of fixed schemes and datatypes with little customization for codes running on different hardware. Whereas, supporting quantization in DL compilers can leverage more optimization information during compilation to derive more efficient quantization strategies. For example, Relay BIB013 provides a quantization rewriting flow that can automatically generate code for various schemes. To support quantization, there are more challenges to be solved in the DL compilers. The first challenge is how to implement new operators with reduced precision without heavy engineering efforts. To reuse the quantization implementation in Relay, [15] proposes a new dialect to implement new operator with basic instructions instead of implementing the new operator from scratch, which eliminates the engineering efforts of re-implementing the graph-level and operator-level optimizations. The interaction between quantization and other optimizations during compilation is another challenge. For example, determining the appropriate place for quantization and collaborating with optimizations such as operator fusion require future research investigations. Meanwhile, quantization also impacts the hardware-specific optimizations in DL compilers, which can be re-designed to leverage the performance opportunity that less resource is required with the low-precision operators and data. Unified optimizations -Although existing DL compilers adopt similar design in both computation graph optimization and hardware-specific optimization, each of compiler has its own implementation with advantages in certain aspects. There is missing a way to share the state-of-theart optimizations, as well as support of emerging hardware targets across existing compilers. We advocate unifying the optimizations from existing DL compilers so that the best practices adopted in each DL compiler can be reused. In addition, unifying the optimizations across DL compilers can accumulate a strong force to impact the design of general-purpose and dedicated DL accelerators, and provide an environment for efficient co-design of DL compiler and hardware. Currently, Google MLIR is a promising initiative towards such direction. It provides the infrastructure of multi-level IRs, and contains IR specification and toolkit to perform transformations across IRs at each level. It also provides flexible dialects, so that each DL compiler can construct its customized dialects for both high-level and low-level IRs. Through transformation across dialects, optimizations of one DL compiler can be reused by another compiler. However, the transformation of dialects may need delicate tradeoffs and some engineering efforts. Differentiable programming -Differentiable programming is a programming paradigm, and the programs in the differentiable programming paradigm are differentiable thoroughly. It has attracted the attention of the DL compiler community recently. Many new compiler projects have replaced the computational graph with differentiable programming, such as Myia  and Swift for TensorFlow BIB014 . In addition, Flux  is one of the most promising ML stacks with its differential language, Julia BIB004 . Julia is a well-suited language for ML programming that is designed for mathematical and numerical computing, and Flux extends Julia for differentiable algorithms and acceleration realized by the compiler. Unfortunately, there is little support for differential languages in existing DL compilers. Supporting differential language is quite challenging for existing DL compilers. The difficulties come from not only data structure, but also language semantic. For example, to realize the transformation from Julia to XLA HLO IR, one of the challenges faced by the project BIB009 is that the control flow is different between the imperative language used by Julia and the symbolic language used by XLA. In order to use HLO IR efficiently, the compiler also needs to provide operation abstraction for Julia in order to support the particular semantic of XLA, such as MapReduce and broadcast. Moreover, the difference of the semantic of differentiation between Julia and XLA, also leads to significant changes of the automatic differentiation algorithm and corresponding designs. Graph neural network (GNN) -GNN has been a popular research direction in the field of deep learning in recent years BIB010 BIB015 BIB011 BIB012 . Traditional DL networks are effective at regularly structured data in Euclidean space (e.g., picture and speech), whereas behavior poorly on irregularly structured data (e.g., social and e-commerce). Taking the e-commerce for example, users, products, and advertisements can be regarded as nodes, and operations such as users purchasing goods and clicking advertisements can be regarded as edges. Nodes and edges form a large graph. GNN takes this large graph as input and obtains the representation of nodes, edges or subgraphs by aggregating neighbor information, so as to realize tasks such as classification, prediction and recommendation. It has been accepted that the design of GNN behaviors well on unstructured data. A large number of DL frameworks have already provided implementation of GNN, such as TensorFlow, PyTorch, MXNet, PaddlePaddle, and Theano. However, existing DL compilers have little support for GNNs. Only TVM has released a primitive tutorial of building a Graph Convolutional Network (GCN) with Relay and MXNet on the Cora dataset BIB007 . One of the reasons for the immature support of GNNs in existing DL compilers is due to its unique design. For example, GNNs are always shallow, most of which are no more than three layers BIB012 . This is because GNN merges the representation of neighboring nodes closer to each other, which results in fewer layers. However, stacking multiple GNN layers recklessly can easily result in over-smoothing. For DL compilers, how to use the graph-level optimizations to avoid over-smoothing of GNNs requires more research efforts. Privacy protection -As the edge devices such as sensors and mobile phones are widely used in our daily life, edge-cloud system is becoming pervasive to run DL models for intelligent tasks such as face detection and voice recognition. In edge-cloud system, the DL models are usually split into two halves with each partial model running on the edge device and cloud service respectively, which can provide better response latency and consume less communication bandwidth. However, one of the drawbacks with the edge-cloud system is that the user privacy becomes vulnerable. The reason is that the attackers can intercept the intermediate results sent from the edge devices to cloud, and then use the intermediate results to train another model that can reveal the privacy information deviated from the original user task BIB016 BIB017 BIB018 . To protect privacy in edge-cloud system, existing approaches BIB016 BIB017 BIB018 propose to add noise with special statistic properties to the intermediate results that can reduce the accuracy of the attacker model without severely deteriorating the accuracy of the original user model. However, the difficulty with the existing approaches is to determine the layer where the noise should be inserted, which is quite labor intensive to identify the optimal layer. The above difficulty presents a great opportunity for DL compilers to support privacy protection, because the compilers maintain rich information about the computation, communication and entropy of all layers, which can guide the noise generation across layers for better privacy protection automatically. We believe this could be an interesting research direction across the disciplines of security and compilation for DL community.