<commit_before>Databaker Jupyter notebook tool for converting data that is laid out in a formatted excel 
spreadsheet into a normalized form for use by databases.  
It depends on [okfn/messytables](https://github.com/okfn/messytables) and 
[sensiblecodeio/xypath](https://github.com/sensiblecodeio/xypath)

See the documentation is in the Jupyter notebooks in
**[tutorial](databaker/tutorial)**.

## Starting up

To install for development, go into the virtualenv with 

`source bin/activate`

and then type

`pip install -e git+https://github.com/sensiblecodeio/databaker.git#egg=databaker`

This will install the code into src/databaker where you can edit and commit it.  
Then type:

`jupyter notebook` 

to start it up.  

## Authors

Made by the [Sensible Code Company](http://sensiblecode.io) on behalf of the 
[Office of National Statistics](https://www.ons.gov.uk/) (UK).

The current documentation is in the form of Jupyter notebooks located
inside the [tutorial](databaker/tutorial) directory.

You can access these directly by creating a new Jupyter notebook and
running the following in a Jupyter cell:

```
from databaker.tutorial import tutorial
tutorial()
```

which will copy the tutorials to your current directory and provide
links to these copied notebooks.<commit_msg>Improve README<commit_after># Databaker

Jupyter notebook tool for converting data that is laid out in a formatted Excel 
spreadsheet into a normalized form for use by databases.

It depends on [okfn/messytables](https://github.com/okfn/messytables) and 
[sensiblecodeio/xypath](https://github.com/sensiblecodeio/xypath)

## Starting up

### For development

To install for development, the easiest way is create a virtualenv,
activate it:

`source bin/activate`

and then type

`pip install -e git+https://github.com/sensiblecodeio/databaker.git#egg=databaker`

This will install the code into `src/databaker` where you can edit and commit it.  

### For normal use

Install with `pip install databaker`

## Usage

Launch a Jupyter notebook:

`jupyter notebook` 

and then follow the tutorials as described below. 

## Documentation

The current documentation is in the form of Jupyter notebooks located
inside the [tutorial](databaker/tutorial) directory.

You can access these directly by creating a new Jupyter notebook and
running the following in a Jupyter cell:

```
from databaker.tutorial import tutorial
tutorial()
```

which will copy the tutorials to your current directory and provide
links to these copied notebooks.

## Authors

Made by the [Sensible Code Company](http://sensiblecode.io) on behalf of the 
[Office of National Statistics](https://www.ons.gov.uk/) (UK).
{'fraction_non_alphanumeric': 0.07896706586826348, 'fraction_numerical': 0.0, 'mean_word_length': 4.760775862068965, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2384564', 'n_tokens_mistral': 844, 'n_tokens_neox': 776, 'n_words': 319}