<commit_before>---
services: hdinsight
platforms: dotnet
author: blackmist
---

# Event correlation with Apache Storm and HBase

This example demonstrates how to use Apache Storm and a persistent data store (Apache HBase in this case,) to correlate pieces of data that arrive at different times. This example was written and tested using HDInsight on Microsoft Azure.

This example is based on tracking session state. When a session begins, a START event is sent through the topology. This event, the time it was sent, and the Session ID are stored in HBase.

Later, an END event is received. The topology uses the Session ID to look up the previously received START event and calculate the duration (time elapsed between the two,) and then stores the END event and duration to HBase.

The SessionInfo project is a basic HBase client application that can be used to create, delete, or do some basic time range queries against the table in HBase.
<commit_msg>updating readme to add warning about using on newer, linux-based clusters<commit_after>---
services: hdinsight
platforms: dotnet
author: blackmist
---

# Event correlation with Apache Storm and HBase

__WARNING__: This example was created and tested with Windows-based HDInsight clusters. Starting with HDInsight version 3.4, Linux is the only operating system used for HDInsight. This example __will not work on HDInsight 3.4 or higher__ due to incompatibilities between the .NET components that it relies on and Mono (used to run .NET on Linux clusters).

This example demonstrates how to use Apache Storm and a persistent data store (Apache HBase in this case,) to correlate pieces of data that arrive at different times. This example was written and tested using HDInsight on Microsoft Azure.

This example is based on tracking session state. When a session begins, a START event is sent through the topology. This event, the time it was sent, and the Session ID are stored in HBase.

Later, an END event is received. The topology uses the Session ID to look up the previously received START event and calculate the duration (time elapsed between the two,) and then stores the END event and duration to HBase.

The SessionInfo project is a basic HBase client application that can be used to create, delete, or do some basic time range queries against the table in HBase.

{'fraction_non_alphanumeric': 0.03429061294470639, 'fraction_numerical': 0.0017145306472353193, 'mean_word_length': 4.453271028037383, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '843074', 'n_tokens_mistral': 561, 'n_tokens_neox': 534, 'n_words': 375}