<commit_before># Shark (Hive on Spark)

Shark is a large-scale data warehouse system for Spark designed to be compatible with
Apache Hive. It can answer Hive QL queries up to 30 times faster than Hive without
modification to the existing data nor queries. Shark supports Hive's query language,
metastore, serialization formats, and user-defined functions.


# For current documentation, see the [Shark Project Wiki](https://github.com/amplab/shark/wiki)<commit_msg>Added a line in the readme on the requirement of running Shark.<commit_after># Shark (Hive on Spark)

Shark is a large-scale data warehouse system for Spark designed to be compatible with
Apache Hive. It can answer Hive QL queries up to 30 times faster than Hive without
modification to the existing data nor queries. Shark supports Hive's query language,
metastore, serialization formats, and user-defined functions.

Shark 0.2 requires Scala 2.9.2, Hive 0.9, and Spark 0.6.

## For current documentation, see the [Shark Project Wiki](https://github.com/amplab/shark/wiki)
{'fraction_non_alphanumeric': 0.06358381502890173, 'fraction_numerical': 0.012524084778420038, 'mean_word_length': 4.804469273743017, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '6008241', 'n_tokens_mistral': 306, 'n_tokens_neox': 284, 'n_words': 142}