<commit_before>* This requires [Python 3](https://www.python.org/downloads/) or higher
* Be sure to create the following directories to hold incoming and created files:
    * ./out
    *  ./raw
* The basic verification step requires (this can be turned off)
[owltools](https://code.google.com/p/owltools/wiki/InstallOWLTools) be available on your path

* Required python packages:
    * [psycopg2](http://initd.org/psycopg/)
    * [rdflib](https://code.google.com/p/rdflib/)

* right now this runs completely through the __init__() method.  you can select the sources to process
by commenting them on/off.

* the general strategy here is that there is one class per data source.  we define the files to be fetched,
any file scrubbing, and then the parsing methods.  as the files are parsed, they are turned into triples and
loaded into a graph.  this graph is then typically dumped into triples (currently coupled to the parsers,
but will be moved into a separate method in the future to be controlled from main.

* The following sources have been partially mapped:
    * Human Phenotype Ontology Annotations (Disease-Phenotype associations)
    * BioGrid (Gene-gene interactions)
    * ZFIN (Genotypes (with their parts), and Genotype-Phenotype associations)
    * OMIM (Diseases, labels, and descriptions)<commit_msg>noting 2 more dependencies<commit_after>* This requires [Python 3](https://www.python.org/downloads/) or higher
* Be sure to create the following directories to hold incoming and created files:
    * ./out
    *  ./raw
* The basic verification step requires (this can be turned off)
[owltools](https://code.google.com/p/owltools/wiki/InstallOWLTools) be available on your path

* Required python packages:
    * [psycopg2](http://initd.org/psycopg/)
    * [rdflib](https://code.google.com/p/rdflib/)
    * isodate
    * roman

* right now this runs completely through the __init__() method.  you can select the sources to process
by commenting them on/off.

* the general strategy here is that there is one class per data source.  we define the files to be fetched,
any file scrubbing, and then the parsing methods.  as the files are parsed, they are turned into triples and
loaded into a graph.  this graph is then typically dumped into triples (currently coupled to the parsers,
but will be moved into a separate method in the future to be controlled from main.

* The following sources have been partially mapped:
    * Human Phenotype Ontology Annotations (Disease-Phenotype associations)
    * BioGrid (Gene-gene interactions)
    * ZFIN (Genotypes (with their parts), and Genotype-Phenotype associations)
    * OMIM (Diseases, labels, and descriptions)
{'fraction_non_alphanumeric': 0.07769891669779604, 'fraction_numerical': 0.0018677624206200972, 'mean_word_length': 4.033834586466165, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 6, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '7402151', 'n_tokens_mistral': 754, 'n_tokens_neox': 719, 'n_words': 332}