<commit_before>==========
 jubadump
==========

``jubadump`` converts saved jubatus internal model to json.``jubadump`` supports only classifier model and inverted_index model.

Requirements
============

- g++
- pkg-config
- jubatus_core
- msgpack


Compile and Install
===================

::

   $ ./waf configure
   $ ./waf
   $ ./waf install


Usage
=====

1. Call save RPC and save your model.
2. Execute jubadump.

    usage: jubadump --input=string [options] ... 
    options:
      -i, --input    Input file (string)
      -t, --type     Format type (string [=classifier])
      -?, --help     print this message


Format
======

- common

  - ``weights``

    - ``version_number`` : Version of model. This value will be updated by MIX.
    - ``document_frequencies`` : Frequency of each features in data inputted so far.
    - ``document_count`` : Number of all documents. This value will be used in calculation of global_weight(idf).
- classifier, regression

  - ``storage``

    - ``weight`` : Weights of each features and each labels in data inputted so far.
- recommender (inverted_index)

  - ``index``

    - ``storage``

      - ``inv``: IDs of each features
    - ``original``

      - ``inv`` : Features of each IDs
- anomaly (lof based on inverted_index)

  - ``storage``

    - ``lof_records``

      - ``entry``: kdist and lrd value of each IDs
    - ``nn_records`` : Refer to recommender for the list of values.


License
=======

LGPL 2.1<commit_msg>Fixed typo<commit_after>==========
 jubadump
==========

``jubadump`` converts saved jubatus internal model to json.``jubadump`` supports only classifier model and inverted_index model.

Requirements
============

- g++
- pkg-config
- jubatus_core
- msgpack


Compile and Install
===================

::

   $ ./waf configure
   $ ./waf
   $ ./waf install


Usage
=====

1. Call save RPC and save your model.
2. Execute jubadump.

    usage: jubadump --input=string [options] ... 
    options:
      -i, --input    Input file (string)
      -t, --type     Format type (string [=classifier])
      -?, --help     print this message


Format
======

- common

  - ``weights``

    - ``version_number`` : Version of model. This value will be updated by MIX.
    - ``document_frequencies`` : Frequency of each feature in data inputted so far.
    - ``document_count`` : Number of all documents. This value will be used in calculation of global_weight(idf).
- classifier, regression

  - ``storage``

    - ``weight`` : Weights of each feature and each label in data inputted so far.
- recommender (inverted_index)

  - ``index``

    - ``storage``

      - ``inv``: IDs of each feature
    - ``original``

      - ``inv`` : Features of each ID
- anomaly (lof based on inverted_index)

  - ``storage``

    - ``lof_records``

      - ``entry``: kdist and lrd value of each ID
    - ``nn_records`` : Refer to recommender for the list of values.


License
=======

LGPL 2.1
{'fraction_non_alphanumeric': 0.16048962937776268, 'fraction_numerical': 0.0027201632097925877, 'mean_word_length': 2.762148337595908, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4167944', 'n_tokens_mistral': 957, 'n_tokens_neox': 884, 'n_words': 316}