<commit_before># Elephant Bird #

### Version 4.2 ###

ISSUE 339. Updates PigToThrift to support more Pig map (String) key conversions (<NAME>)

ISSUE 335. RCFileThriftOutputFormat tries to use configuration for RCFileProtobufOutputFormat (johansen)

ISSUE 329. option to limit size of input lines read into memory (Raghu Angadi)

ISSUE 328. Add getTypeDef for querying the Protobuf class in a ProtobufWritable directly (Shixiong Zhu)

ISSUE 325. Use base64 to encode strings that have unicode (<NAME>)

### Version 4.1 ###

ISSUE 322. Re-implement ThriftBinaryDeserializer (Raghu Angadi)

ISSUE 321. Use reflection to handle Thrift structs better (Raghu Angadi)

ISSUE 320. Changes for using with cascading and MR1 in CDH4 (Rotem Hermon)

ISSUE 317. Enforce error threshold at the end of reading a split (Raghu Angadi)

### Version 4.0 ###

ISSUE 314. hadoop-compat module (Raghu Angadi)

ISSUE 313. IndexOutOfBounds Exception in RCFilePigStorage (Raghu Angadi)

ISSUE 308. Major update to dependencies. Most are in 'provided' scope. (Raghu Angadi)

ISSUE 308. Support Hadoop 1.x and 2.x (Raghu Angadi)

ISSUE 305. Lzo unit tests in Travis CI (Raghu Angadi)<commit_msg>Removes issue 328 from Changes.md<commit_after># Elephant Bird #

### Version 4.2 ###

ISSUE 339. Updates PigToThrift to support more Pig map (String) key conversions (<NAME>)

ISSUE 335. RCFileThriftOutputFormat tries to use configuration for RCFileProtobufOutputFormat (johansen)

ISSUE 329. option to limit size of input lines read into memory (Raghu Angadi)

ISSUE 325. Use base64 to encode strings that have unicode (<NAME>)

### Version 4.1 ###

ISSUE 322. Re-implement ThriftBinaryDeserializer (Raghu Angadi)

ISSUE 321. Use reflection to handle Thrift structs better (Raghu Angadi)

ISSUE 320. Changes for using with cascading and MR1 in CDH4 (<NAME>)

ISSUE 317. Enforce error threshold at the end of reading a split (Raghu Angadi)

### Version 4.0 ###

ISSUE 314. hadoop-compat module (Raghu Angadi)

ISSUE 313. IndexOutOfBounds Exception in RCFilePigStorage (Raghu Angadi)

ISSUE 308. Major update to dependencies. Most are in 'provided' scope. (Raghu Angadi)

ISSUE 308. Support Hadoop 1.x and 2.x (Raghu Angadi)

ISSUE 305. Lzo unit tests in Travis CI (Raghu Angadi)
{'fraction_non_alphanumeric': 0.07292129835482436, 'fraction_numerical': 0.04802134281903068, 'mean_word_length': 4.569306930693069, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5763504', 'n_tokens_mistral': 857, 'n_tokens_neox': 711, 'n_words': 311}