<commit_before># OpenSOC-Streaming

Extensible set of Storm topologies and topology attributes for streaming, enriching, indexing, and storing telemetry in Hadoop.  General information on OpenSOC is available at www.getopensoc.com


## Build Instructions

If building via maven add the following dependency:

```
<dependency>
	<groupId>com.opensoc</groupId>
	<artifactId>OpenSOC-Streaming</artifactId>
	<version>${opensoc.streaming.version}</version>
</dependency>
```

To build the project in Eclipse do the following:

Check out via git plugin or git app
File->import->maven->exiting maven project

# Usage Instructions

## Message Parser Bolt
<commit_msg>Added FAQ section<commit_after># OpenSOC-Streaming

Extensible set of Storm topologies and topology attributes for streaming, enriching, indexing, and storing telemetry in Hadoop.  General information on OpenSOC is available at www.getopensoc.com

For OpenSOC FAQ please read the following wiki entry:  https://github.com/OpenSOC/opensoc-streaming/wiki/OpenSOC-FAQ


## Build Instructions

If building via maven add the following dependency:

```
<dependency>
	<groupId>com.opensoc</groupId>
	<artifactId>OpenSOC-Streaming</artifactId>
	<version>${opensoc.streaming.version}</version>
</dependency>
```

To build the project in Eclipse do the following:

Check out via git plugin or git app
File->import->maven->exiting maven project

# Usage Instructions

## Message Parser Bolt

{'fraction_non_alphanumeric': 0.08692628650904033, 'fraction_numerical': 0.0, 'mean_word_length': 5.424107142857143, 'pattern_counts': {'":': 0, '<': 19, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6142009', 'n_tokens_mistral': 440, 'n_tokens_neox': 396, 'n_words': 146}