<commit_before># djinn

[![Join the chat at https://gitter.im/MaxwellRebo/djinn](https://badges.gitter.im/MaxwellRebo/djinn.svg)](https://gitter.im/MaxwellRebo/djinn?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

Urban simulation and market dynamics toolkit with connectors to popular machine learning tools

This is still heavily a work-in-progress, but there are some examples you can run:

    cargo run --example basic
    cargo run --example multiple

The `basic` example also has a demo of a frontend client using websockets. Open `examples/ws.html` in a browser, then run the `basic` example to see it in action.

## Dependencies

- `redis >= 3.2`<commit_msg>performance note<commit_after># djinn

[![Join the chat at https://gitter.im/MaxwellRebo/djinn](https://badges.gitter.im/MaxwellRebo/djinn.svg)](https://gitter.im/MaxwellRebo/djinn?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

Urban simulation and market dynamics toolkit with connectors to popular machine learning tools

This is still heavily a work-in-progress, but there are some examples you can run:

    cargo run --example basic
    cargo run --example multiple

The `basic` example also has a demo of a frontend client using websockets. Open `examples/ws.html` in a browser, then run the `basic` example to see it in action.

## Dependencies

- `redis >= 3.2`

## A note on performance

Right now most of the overhead is from network communication with Redis. The `redis-rs` library does not have async IO yet (Rust is still working on it, see <https://github.com/mitsuhiko/redis-rs/pull/93>), but it probably will eventually, which should speed things up.
{'fraction_non_alphanumeric': 0.09526627218934912, 'fraction_numerical': 0.0035502958579881655, 'mean_word_length': 4.655518394648829, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 6, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '548896', 'n_tokens_mistral': 543, 'n_tokens_neox': 521, 'n_words': 189}