<commit_before>Aspose For Hadoop
=================

Aspose for Hadoop project enables Hadoop developers to work with binary file formats. The Hadoop / MR developers can use this project to create and convert binary sequence files into text sequence files. The text can then be used for analysis purpose in MapReduce algorithms.

Packages
--------
com.aspose.hadoop.core

      Provides Aspose for Java wrapper classes to parse binary formats into text. The package also includes a couple of classes to override Hadoop input formats so as to be used for creating binary sequence files.
com.aspose.hadoop.examples 
      
      Provides mapper examples for converting binary sequence file(s) into text sequence file(s). Each mapper example takes a particular set of binary format as exaplained in the next section.

Mapper Examples Flow and Usage
------------------------------

CreateBinarySequence

      Picks up the set of files from an HDFS directory, create binary sequence file(s) and stores the binary sequence file(s) to an HDFS directory.
      Usage: [HADOOP_HOME]$ bin/hadoop jar aspose-hadoop.jar CreateBinarySequence <HDFS input directory> <HDFS output directory>
CreateDocumentTextSequence

      Picks up binary sequence file(s) generated from documents (MS Words / OpenOffice docs), parses text from the documents, creates a text sequence file and stores this text sequence file to an HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar CreateDocumentTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your documents in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file. Finally supply output directory of this mapper as an input here.
CreateSpreadSheetTextSequence

      Picks up binary sequence file(s) generated from spreadsheets (MS Excel / OpenOffice spreadsheets), parses text from the spreadsheets, creates a text sequence file and stores this text sequence file to an HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar CreateSpreadSheetTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your spreadsheets in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file. Finally supply output directory of this mapper as an input here.
CreatePresentationTextSequence

      Picks up binary sequence file(s) generated from presentation (MS PoerPoint PPTX presentations), parses text from the presentations, creates a text sequence file and stores this text sequence file to an HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar CreatePresentationTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your PPTX presentations in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file. Finally supply output directory of this mapper as an input here.
CreateEmailTextSequence

      Picks up binary sequence file(s) generated from emails (msg emails), parses text from the msg files, creates a text sequence file and stores this text sequence file to an HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar Aspose-hadoop.jar CreateEmailTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your msg files in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file. Finally supply output directory of this mapper as an input here.

Aspose Pty Ltd: www.aspose.com<commit_msg>Fixed usage commands in the mapper examples.<commit_after>Aspose For Hadoop
=================

Aspose for Hadoop project enables Hadoop developers to work with binary file formats. The Hadoop / MR developers can use this project to create and convert binary sequence files into text sequence files. The text can then be used for analysis purpose in MapReduce algorithms.

Packages
--------
com.aspose.hadoop.core

      Provides Aspose for Java wrapper classes to parse binary formats into text. The package also includes a couple of classes to override Hadoop input formats so as to be used for creating binary sequence files.
com.aspose.hadoop.examples 
      
      Provides mapper examples for converting binary sequence file(s) into text sequence file(s). Each mapper example takes a particular set of binary format as exaplained in the next section.

Mapper Examples Flow and Usage
------------------------------

CreateBinarySequence

      Picks up the set of files from an HDFS directory, create binary sequence file(s) and stores the binary sequence file(s) to an HDFS directory.
      Usage: [HADOOP_HOME]$ bin/hadoop jar aspose-hadoop.jar com.aspose.hadoop.examples.CreateBinarySequence <HDFS input directory> <HDFS output directory>
CreateDocumentTextSequence

      Picks up binary sequence file(s) generated by documents (MS Words / OpenOffice docs) from an input HDFS directory, parses text from the documents, creates text sequence(s) file to be stored on an output HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar com.aspose.hadoop.examples.CreateDocumentTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your documents in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file(s). Finally supply output directory of the mapper as an input here.
CreateSpreadSheetTextSequence

      Picks up binary sequence file(s) generated by spreadsheets (MS Excel / OpenOffice spreadsheets) from an input HDFS directory, parses text from the spreadsheets, creates text sequence file(s) to be stored on an output HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar com.aspose.hadoop.examples.CreateSpreadSheetTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your spreadsheets in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file(s). Finally supply output directory of the mapper as an input here.
CreatePresentationTextSequence

      Picks up binary sequence file(s) generated by presentations (MS PowerPoint PPTX presentations) from an input HDFS directory, parses text from the presentations, creates text sequence file(s) to be stored on an output HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar aspose-hadoop.jar com.aspose.hadoop.examples.CreatePresentationTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your PPTX presentations in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file(s). Finally supply output directory of the mapper as an input here.
CreateEmailTextSequence

      Picks up binary sequence file(s) generated by emails (msg emails) from an input HDFS directory, parses text from the msg files, creates text sequence file(s) to be stored on an output HDFS directory.
      Usage: [Hadoop_HOME]$ bin/hadoop jar Aspose-hadoop.jar com.aspose.hadoop.examples.CreateEmailTextSequence <HDFS input directory> <HDFS output directory>
      Tip: Put your msg files in an HDFS directory, use CreateBinarySequence mapper to generate binary sequence file(s). Finally supply output directory of the mapper as an input here.

Aspose Pty Ltd: www.aspose.com
{'fraction_non_alphanumeric': 0.05693207126948775, 'fraction_numerical': 0.0, 'mean_word_length': 4.361940298507463, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 0, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '5573606', 'n_tokens_mistral': 1810, 'n_tokens_neox': 1689, 'n_words': 909}