The current state of cancer therapeutics has been moving away from one-size-fits-all cytotoxic chemotherapy, and towards a more individualized and specific approach involving the targeting of each tumor's genetic vulnerabilities. Different tumors, even of the same type, may be more reliant on certain cellular pathways more than others. With modern advancements in our understanding of cancer genome sequencing, these pathways can be discovered. Investigating each of the millions of possible small molecule inhibitors for each kinase in vitro, however, would be extremely expensive and time consuming. This project focuses on predicting the inhibition activity of small molecules targeting 8 different kinases using multiple deep learning models. We trained fingerprint-based MLPs and simplified molecular-input line-entry specification (SMILES)-based recurrent neural networks (RNNs) and molecular graph convolutional networks (GCNs) to accurately predict inhibitory activity targeting these 8 kinases.
We will first directly feed the concatenated fingerprint feature representation into a multilayer perceptron (MLP) with a two hidden dense layers of size 128, each with ReLU activation. Before the output layer, we include a dropout layer for regularization. The output layer is a dense layer of size 2 - corresponding to the two possible labels - with softmax activation for a probability prediction for each label. The MLP architecture is summarized in Figure . We use binary cross entropy as the loss function, as given in the following equation: @math where @math is the loss, @math is the current set of model parameters, @math is the total number of data samples, @math is true label (0 or 1) for data sample @math , and @math is predicted probability output by the model that the label for data sample @math is 1. Adam is a gradient descent algorithm with an adaptive learning rate that in practice yields quicker model convergence than vanilla stochastic gradient descent, so we use Adam to optimize our model @cite_3 . We set the initial learning rate to @math , minibatch size to 32, and dropout probability to @math .
@cite_3
1522301498
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.