FIELD OF THE TECHNOLOGY DISCLOSED

The technology disclosed relates to artificial intelligence type computers and digital data processing systems and corresponding data processing methods and products for emulation of intelligence (i.e., knowledge based systems, reasoning systems, and knowledge acquisition systems); and including systems for reasoning with uncertainty (e.g., fuzzy logic systems), adaptive systems, machine learning systems, and artificial neural networks. The technology disclosed uses deep neural networks like long short-term memory (LSTM) networks for natural language processing tasks.

BACKGROUND

The subject matter discussed in this section should not be assumed to be prior art merely as a result of its mention in this section. Similarly, a problem mentioned in this section or associated with the subject matter provided as background should not be assumed to have been previously recognized in the prior art. The subject matter in this section merely represents different approaches, which in and of themselves can also correspond to implementations of the claimed technology.

Recent deep learning advances in natural language processing (NLP) have been focused on one goal—state-of-the-art performance no matter the computational cost. We propose a deep learning-based solution that reduces computational cost while retaining high accuracy provided by deep neural networks. By letting a fast baseline model decide which sentence is easy or difficult, we show that an inexpensive classifier, such as the bag of words, learns how to separate easy from difficult sentences. Our results find that thresholding probability (produced as a byproduct during classification) outperforms not having a guided strategy on accuracy versus time saved metric.

Deep learning models are getting bigger, better, and more expensive in the all-or-nothing hunt for reaching superhuman performance. In natural language processing (NLP), often new additions base their novel state-of-the-art design on a relatively narrow extension that solves a complex linguistic task really well. This includes incorporating semantic knowledge, reducing dictionary size, or even increasing dictionary size. Intuitively, it would make sense that such additions would only exert the new system to those data points having the specific challenges that the addition conquered. That is, however, often not the case as deep learning algorithms are mostly benchmarked with an invariant of being oblivious to the given data point and the difficulty hereof. Further, with the increased understanding that deeper means better, current state-of-the-art systems can be excessive and waste computation to secure a narrow accuracy improvement in a linguistic area that may not be relevant in most scenarios. For example,FIG. 3illustrates how a cheap model clusters sentences when it is trained for sentiment analysis (e.g., using backpropagation). InFIG. 3, some clusters are marked to show that some data points are easier than others and that supervised learning can reveal clustering of certain linguistic properties.

The technology disclosed proposes two different methods for understanding when a sentence is easy to classify and when it is difficult. The technology disclosed applies either a cheap bag of words (skimming) or an expensive LSTM (reading) to classify a sentence based on the decision of the methods. The first method uses the probability output of the bag of words system as a confidence measure. The second method builds a decision neural network to learn the relationship between when the bag of words might be correct and when the LSTM might be correct.

Both our methods outperform not having a guided baseline strategy, and can save significant computation resources for engineering teams.

