<commit_before># Hail

[![Join the chat at https://gitter.im/broadinstitute/hail](https://badges.gitter.im/broadinstitute/hail.svg)](https://gitter.im/broadinstitute/hail?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

Hail is a framework for scalable genetic data analysis.  Hail is
pre-alpha software and under active development.  Hail is written in
Scala (mostly) and uses Apache [Spark](http://spark.apache.org/) and
other Apache Hadoop projects.  If you are interested in getting
involved in Hail development, email <EMAIL>.

## Documentation

 - [Building](docs/Building.md)
 - [Representation](docs/Representation.md)
 - [Hail's expression language](docs/HailExpressionLanguage.md)
 - [Importing VCF files](docs/Importing.md)
 - [Importing sites-only text files](docs/commands/ImportAnnotations.md)
 - [Splitting Multiallelic Variants](docs/Splitmulti.md)
 - [Renaming Samples](docs/RenameSamples.md)
 - [Annotating Variants](docs/commands/AnnotateVariants.md)
 - [Annotating Samples](docs/commands/AnnotateSamples.md)
 - [Annotating Global](docs/commands/AnnotateGlobal.md)
 - [Quality Control](docs/QC.md)
 - [PCA](docs/PCA.md)
 - [Annotating with the Variant Effect Predictor](docs/VEP.md)
 - [Computing aggregate statistics on intervals](docs/commands/AggregateIntervals.md)
 - [Filtering](docs/Filtering.md)
 - [Querying using SQL](docs/SQL.md)
 - [Linear regression](docs/LinearRegression.md)
 - [Mendel errors](docs/MendelErrors.md)
 - [Exporting to TSV](docs/ExportTSV.md)
 - [Exporting to VCF](docs/ExportVCF.md)
 - [Exporting to Plink](docs/ExportPlink.md)
 - [Persist](docs/Persist.md)

## Citing Hail

If you use Hail for published work, please cite both the software:

 - Hail, [https://github.com/broadinstitute/hail](https://github.com/broadinstitute/hail)

and the forthcoming manuscript describing Hail:

 - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>.  Hail: An Open-Source Framework for Scalable Genetic Data Analysis.  In preparation.

of the following paper which includes a brief introduction to Hail in
the online methods:

 - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>. _Ultra-rare disruptive and damaging mutations influence educational attainment in the general population_.  doi: http://dx.doi.org/10.1101/050195.

## Roadmap

Here is a rough list of features currently planned or under
development:

 - generalized query language
 - better interoperability with other Hadoop projects
 - kinship estimation from GRM
 - LMM
 - burden tests, SKAT
 - logistic regression
 - dosage
 - posterior (PP)
 - LD pruning
 - sex check
 - TDT
 - BGEN
 - Kaitlin Samocha's de novo caller<commit_msg>small changes to readme - fixed typo - added Daniel King to paper - deleted some forthcoming methods that are done<commit_after># Hail

[![Join the chat at https://gitter.im/broadinstitute/hail](https://badges.gitter.im/broadinstitute/hail.svg)](https://gitter.im/broadinstitute/hail?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)

Hail is a framework for scalable genetic data analysis.  Hail is
pre-alpha software and under active development.  Hail is written in
Scala (mostly) and uses Apache [Spark](http://spark.apache.org/) and
other Apache Hadoop projects.  If you are interested in getting
involved in Hail development, email <EMAIL>.

## Documentation

 - [Building](docs/Building.md)
 - [Representation](docs/Representation.md)
 - [Hail's expression language](docs/HailExpressionLanguage.md)
 - [Importing VCF files](docs/Importing.md)
 - [Importing sites-only text files](docs/commands/ImportAnnotations.md)
 - [Splitting Multiallelic Variants](docs/Splitmulti.md)
 - [Renaming Samples](docs/RenameSamples.md)
 - [Annotating Variants](docs/commands/AnnotateVariants.md)
 - [Annotating Samples](docs/commands/AnnotateSamples.md)
 - [Annotating Global](docs/commands/AnnotateGlobal.md)
 - [Imputing Sex](docs/ImputeSex.md)
 - [Quality Control](docs/QC.md)
 - [PCA](docs/PCA.md)
 - [Annotating with the Variant Effect Predictor](docs/VEP.md)
 - [Computing aggregate statistics on intervals](docs/commands/AggregateIntervals.md)
 - [Filtering](docs/Filtering.md)
 - [Querying using SQL](docs/SQL.md)
 - [Linear regression](docs/LinearRegression.md)
 - [Mendel errors](docs/MendelErrors.md)
 - [Exporting to TSV](docs/ExportTSV.md)
 - [Exporting to VCF](docs/ExportVCF.md)
 - [Exporting to Plink](docs/ExportPlink.md)
 - [Persist](docs/Persist.md)

## Citing Hail

If you use Hail for published work, please cite both the software:

 - Hail, [https://github.com/broadinstitute/hail](https://github.com/broadinstitute/hail)

and the forthcoming manuscript describing Hail (if possible):

 - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>.  Hail: An Open-Source Framework for Scalable Genetic Data Analysis.  In preparation.

or the following paper which includes a brief introduction to Hail in the online methods:

 - <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>, <NAME>. _Ultra-rare disruptive and damaging mutations influence educational attainment in the general population_.  doi: http://dx.doi.org/10.1101/050195.

## Roadmap

Here is a rough list of features currently planned or under
development:

 - generalized query language
 - better interoperability with other Hadoop projects
 - kinship estimation from GRM
 - LMM
 - burden tests, SKAT
 - logistic regression
 - posterior (PP)
 - LD pruning
 - TDT
 - Kaitlin Samocha's de novo caller
{'fraction_non_alphanumeric': 0.13837985956499402, 'fraction_numerical': 0.004110292858366158, 'mean_word_length': 4.822532402791625, 'pattern_counts': {'":': 0, '<': 80, '<?xml version=': 0, '>': 80, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1385704', 'n_tokens_mistral': 1948, 'n_tokens_neox': 1840, 'n_words': 521}