<commit_before>distributed-graph-analytics
===========================

Distributed Graph Analytics (DGA) is a compendium of graph analytics written for Bulk-Synchronous-Parallel (BSP) processing frameworks such as Giraph and GraphX.

Currently, DGA supports the following analytics:

###### Giraph
- Weakly Connected Components
- Leaf Compression
- Page Rank
- High Betweenness Set Extraction

###### GraphX
- Louvain Modularity

Eventually, the following analytics will be supported:

- Louvain Modularity
- Weakly Connected Components
- High Betweenness Set Extraction
- Leaf Compression
- Page Rank


How to Build:
=============
gradle clean distDGA

The jar with config files will be output to the following directory: dga-giraph/build/dist/

Note: You must have the hadoop jars included elsewhere.  This only builds DGA and include Giraph Dependencies that are not included with
hadoop.<commit_msg>Updating the main readme and pointing it to the dga-giraph readme<commit_after>distributed-graph-analytics
===========================

Distributed Graph Analytics (DGA) is a compendium of graph analytics written for Bulk-Synchronous-Parallel (BSP) processing frameworks such as Giraph and GraphX.

Currently, DGA supports the following analytics:

###### Giraph
- Weakly Connected Components
- Leaf Compression
- Page Rank
- High Betweenness Set Extraction
- Louvain

###### GraphX
- Louvain Modularity (initial stage)

Eventually, the following analytics will be supported:

- Weakly Connected Components
- High Betweenness Set Extraction
- Leaf Compression
- Page Rank


###### dga-giraph
dga-giraph is the project that contains our Giraph implementation of DGA.  For more information, go here: (dga-giraph README.md)[https://github.com/Sotera/distributed-graph-analytics/tree/master/dga-giraph]
{'fraction_non_alphanumeric': 0.10210876803551609, 'fraction_numerical': 0.0, 'mean_word_length': 5.050335570469799, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1479216', 'n_tokens_mistral': 522, 'n_tokens_neox': 485, 'n_words': 197}