<commit_before>#############
``libfnl``\ ™
#############

Introduction
============

**libfnl** is an API and colleciton of CLIs to facilitate data mining text by providing data management tools for this task.
The library is designed to play ball with Python 3.2+ and *only* targets Python 3k.
It is another piece in the txtfnnl_ text mininig library, the gnamed_ gene name repository daemon, and the medic_ PubMed mirroring tool collection, all by the same author.

The library contains the following packages:

``fnl.couch``
    provides a Py3k Apache CouchDB_ client for managing JSON_ data (when this project was started, there was no Py3k CouchDB_ client around, so this project has its own);
``fnl.nlp``
    currently, only contains a Python wrapper for the GENIA_ Tagger_, a handler for the GENIA_ corpus, and a module collecting all Penn tags
``fnl.stat``
    a module to evaluate inter-rator Kappa scores and a module to develop text classifiers based on Scikit-Learn_
``fnl.text``
    modules to extract, tokenize, segment sentences (based on NLTK_), and annotate text (strings)
``fnl.utils``<commit_msg>corrections<commit_after>#############
``libfnl``\ ™
#############

Introduction
============

**libfnl** is an API and CLI facilitating data mining of text by providing a collection of easy-to-use tools.
The library is designed to play ball with Python 3.2+ and *only* targets Python 3k.
It is another piece in the txtfnnl_ text mining library, the gnamed_ gene name repository daemon, and the medic_ PubMed mirroring tool collection, all by the same author.

The library contains the following packages:

``fnl.couch``
    provides a Py3k Apache CouchDB_ client for managing JSON_ data (when this project was started, there was no Py3k CouchDB_ client around, so this project has its own);
``fnl.nlp``
    currently, only contains a Python wrapper for the GENIA_ Tagger_, a handler for the GENIA_ corpus, and a module collecting all Penn tags
``fnl.stat``
    a module to evaluate inter-rater Kappa scores and a module to develop text classifiers based on Scikit-Learn_
``fnl.text``
    modules to extract, tokenize, segment sentences (based on NLTK_), and annotate text (strings)
``fnl.utils``
{'fraction_non_alphanumeric': 0.09502262443438914, 'fraction_numerical': 0.004524886877828055, 'mean_word_length': 4.727979274611399, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '699245', 'n_tokens_mistral': 629, 'n_tokens_neox': 608, 'n_words': 316}