<commit_before># Parse Redis dump.rdb files, Analyze Memory, and Export Data to JSON #

Rdbtools is a parser for Redis' dump.rdb files. The parser generates events similar to an xml sax parser, and is very efficient memory wise.

In addition, rdbtools provides utilities to :

 1.  Generate a Memory Report of your data across all databases and keys
 2.  Convert dump files to JSON
 3.  Compare two dump files using standard diff tools

Rdbtools is written in Python, though there are similar projects in other languages. See [FAQs](https://github.com/sripathikrishnan/redis-rdb-tools/wiki/FAQs) for more information.

## Installing rdbtools ##

Pre-Requisites : 

1. python-lzf is optional but highly recommended to speed up parsing.
1. redis-py is optional and only needed to run test cases.

To install from PyPI (recommended) :

    pip install rdbtools python-lzf
    
To install from source : 

    git clone https://github.com/sripathikrishnan/redis-rdb-tools
    cd redis-rdb-tools
    sudo python setup.py install

# Command line usage examples

Every run of RDB Tool requires to specify a command to indicate what should be done with the parsed RDB data.
Valid commands are: json, diff, justkeys, justkeyvals and protocol.

JSON from a two database dump:
<commit_msg>Adding link to rdbtools.com<commit_after># Parse Redis dump.rdb files, Analyze Memory, and Export Data to JSON #

Rdbtools is a parser for Redis' dump.rdb files. The parser generates events similar to an xml sax parser, and is very efficient memory wise.

In addition, rdbtools provides utilities to :

 1.  Generate a Memory Report of your data across all databases and keys
 2.  Convert dump files to JSON
 3.  Compare two dump files using standard diff tools

Rdbtools is written in Python, though there are similar projects in other languages. See [FAQs](https://github.com/sripathikrishnan/redis-rdb-tools/wiki/FAQs) for more information.

See https://rdbtools.com for a gui to administer redis, commercial support, and other enterprise features.

## Installing rdbtools ##

Pre-Requisites : 

1. python-lzf is optional but highly recommended to speed up parsing.
1. redis-py is optional and only needed to run test cases.

To install from PyPI (recommended) :

    pip install rdbtools python-lzf
    
To install from source : 

    git clone https://github.com/sripathikrishnan/redis-rdb-tools
    cd redis-rdb-tools
    sudo python setup.py install

# Command line usage examples

Every run of RDB Tool requires to specify a command to indicate what should be done with the parsed RDB data.
Valid commands are: json, diff, justkeys, justkeyvals and protocol.

JSON from a two database dump:

{'fraction_non_alphanumeric': 0.05605381165919283, 'fraction_numerical': 0.0037369207772795215, 'mean_word_length': 3.867272727272727, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1282417', 'n_tokens_mistral': 788, 'n_tokens_neox': 748, 'n_words': 367}