BACKGROUND

An anomaly in a textual item is a word or phrase that is appearing with a frequency that is greater than expected frequency according to a language model. Anomaly detection can be used for a variety of purposes such as determining the intent of a conversation, or for determining when to escalate a customer agent interaction in a call center.

Anomalies in textual items are typically detected using a count model or a snapshot model. Unfortunately, there can be a disconnect between what is considered an anomaly by the count model and what is considered an anomaly via a snapshot language model. For example, consider agent conversations with customers in a call center. Suppose that on average, the number of times the “gift certificates” intent is hit per hour of call time is 10, but during a particular hour, it reaches 30. This would most likely trigger a count based anomaly detection model. However, if the text of the conversations that triggered the intent are further analyzed, it may be the case that the hits are all “expected”. For example, all 30 texts could be some variation of “I need a gift certificate.” The snapshot language model could be used to rank these 30 texts, but none of the cross-entropy values would be very high, meaning that the text is not surprising given the past linguistic state.

As another example, suppose on hour y there are 10 texts (which is not unusual, count-wise). But one of those texts is “You guys are not helping! My gift certificate is not working!! You guys are idiots”. The snapshot language model would probably return a high cross-entropy value, but the count based model would not trigger an anomaly alert as the counts are not unusual. Thus, it is still necessary to consider both the count model and snapshot language model to detect anomalies.

SUMMARY

In an implementation, a method for detecting anomalies in textual items is provided. The method includes: receiving a first plurality of textual items by a computing device; training a language model using the received first plurality of textual items by the computing device; after training the language model, receiving a second plurality of textual items by the computing device; calculating a cross-entropy for each textual item in the second plurality of textual items by the computing device using the language model; and detecting an anomaly in at least one of the textual items of the second plurality of textual items by the computing device using the calculated cross-entropies.

Implementations may have some or all of the following features. The method may further include: determining that a number of textual items in the second plurality of textual items exceeds a threshold; and in response to the determination, sliding the language model forward using the second plurality of textual items. The method may further include: determining that an amount of time has passed since the language was last slid forward exceeds a threshold; and in response to the determination, sliding the language model forward using the second plurality of textual items. The textual item may include a plurality of words. Calculating the cross-entropy for each textual item in the second plurality of textual items may include calculating the cross-entropy using a subset of the plurality of words of the textual item. Calculating the cross-entropy for each textual item in the second plurality of textual items may include calculating an average cross-entropy for each textual item in the second plurality of textual items. The textual item may include a call transcript between an agent and a customer, a chat transcript between the agent and the customer, or messages in a social networking application. The method may further include: receiving a list of textual items that were determined not to be associated with anomalies; for each textual item of the second plurality of textual items: determining if the textual item is in the list of textual items; and if the textual item is in the list of textual items, reducing the calculated cross-entropy for the textual item. Determining if the textual item is in the list of textual items may include determining if the textual item is similar to a textual item in the list of textual items.

In an implementation, a method for detecting anomalies in textual items is provided. The method includes: receiving a first plurality of textual items by a computing device; training a language model using the received first plurality of textual items by the computing device; after training the language model, receiving a second plurality of textual items by the computing device; for each textual item of the second plurality of content items: calculating a cross-entropy for the textual item using the language model by the computing device; calculating a perplexity of the language model by the computing device; and if the calculated perplexity exceeds a threshold, sliding the language model forward using a subset of the textual items of the second plurality by the computing device; and detecting an anomaly in at least one textual item of the second plurality of textual items by the computing device using the calculated cross-entropies.

Implementations may include some or all of the following features. A textual item may include a plurality of words. Calculating the cross-entropy for the textual item may include calculating the cross-entropy using a subset of the plurality of words of the textual item. Calculating the cross-entropy for the textual item may include calculating an average cross-entropy. The textual item may include a call transcript between an agent and a customer, a chat transcript between the agent and the customer, or messages in a social networking application. The method may further include: receiving a list of textual items that were determined not to be associated with anomalies; for each textual item of the second plurality of textual items: determining if the textual item is in the list of textual items; and if the textual item is in the list of textual items, reducing the calculated cross-entropy for the textual item. Determining if the textual item is in the list of textual items comprises determining if the textual item is similar to a textual item in the list of textual items.

In an implementation, a system for detecting anomalies in textual items is provided. The system may include a least one processor; and a memory storing instructions that when executed by the at least one processor cause the at least one processor to: receive a first plurality of textual items; train a language model using the received first plurality of textual items; after training the language model, receive a second plurality of textual items; calculate a cross-entropy for each textual item in the second plurality of textual items using the language model; and detect an anomaly in at least one of the textual items of the second plurality of textual items using the calculated cross-entropies.

Implementations may include some or all of the following features. The instructions may further include instructions that when executed by the at least one processor cause the at least one processor to: determine that a number of textual items in the second plurality of textual items exceeds a threshold; and in response to the determination, slide the language model forward using the second plurality of textual items. The instructions may further include instructions that when executed by the at least one processor cause the at least one processor to: determine that an amount of time has passed since the language was last slid forward exceeds a threshold; and in response to the determination, slide the language model forward using the second plurality of textual items.

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

