<commit_before>strepsirrhini
=============

Strepsirrhini, a modular composable toolkit in scala for retrieval, reranking, and expansion with and without entity annotations.

The main contribution of this project is a set of independent modules that are easy to string together in an information retrieval application to your liking. Different components are in different github-repositories to minimize clutter of dependencies if only a subset of the components is required (check out components from the respective repository). If you want the whole package, checkout this project and compile with maven, it will automatically compile all components into one jar.

Components
----------

- strepsi-tools: basic utilities for sequence and map processing, used by all other strepsirrhini components.
- strepsimur: scala integration with Galago from Lemur project
- strepsirank: discriminative learning-to-rank component (useful for feature-based re-ranking of galago output). Forked from RankLib project.
- strepsipand: several query expansion models or base classes to implement your own relevance model variant.
- strepsent: parsing of different entity linking annotations, including Google's FACC1 annptations and output of KB-Bridge.


Installation
------------

compile with maven:
    mvn compile assembly:single

This produces one uber-jar .../target/strepsirrhine-s2.10.2-g3.5-1.2-with-dependencies.jar containing all dependencies required

To produce individual jars, follow the instructions in the respective component repositories.<commit_msg>Adding Links to README.md<commit_after>strepsirrhini
=============

Strepsirrhini, a modular composable toolkit in scala for retrieval, reranking, and expansion with and without entity annotations.

The main contribution of this project is a set of independent modules that are easy to string together in an information retrieval application to your liking. Different components are in different github-repositories to minimize clutter of dependencies if only a subset of the components is required (check out components from the respective repository). If you want the whole package, checkout this project and compile with maven, it will automatically compile all components into one jar.

Components
----------

- [strepsi-tools](http://github.com/laura-dietz/strepsi-tools): basic utilities for sequence and map processing, used by all other strepsirrhini components.
- [strepsimur](http://github.com/laura-dietz/strepsimur): scala integration with Galago from Lemur project
- [strepsirank](https://github.com/laura-dietz/strepsirank): discriminative learning-to-rank component (useful for feature-based re-ranking of galago output). Forked from RankLib project.
- [strepsipand](https://github.com/laura-dietz/strepsipand): several query expansion models or base classes to implement your own relevance model variant.
- [strepsent](https://github.com/laura-dietz/strepsent): parsing of different entity linking annotations, including Google's FACC1 annptations and output of [KB-Bridge](http://ciir.cs.umass.edu/~jdalton/kbbridge/).


Installation
------------

compile with maven:
    mvn compile assembly:single

This produces one uber-jar .../target/strepsirrhine-s2.10.2-g3.5-1.2-with-dependencies.jar containing all dependencies required

To produce individual jars, follow the instructions in the respective component repositories.
{'fraction_non_alphanumeric': 0.0774896876841485, 'fraction_numerical': 0.005303476723629935, 'mean_word_length': 4.833333333333333, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4903443', 'n_tokens_mistral': 888, 'n_tokens_neox': 823, 'n_words': 380}