<commit_before># LATBIN #

This python package is used for binning onto lattices in multiple dimensions. 

## Installation ##

In the terminal you can install this in the usual way.

    python setup.py install

## Basic Example ##

This example shows two data sets which have various lengths but both have two dimensions. We create an A2 (aka honeycomb) lattice which we histogram both data sets onto. Then we can make comparisons (e.g. by subtraction) of the two and analyze the result. 

    import latbin
    import numpy as np

    # create fake data sets
    dat1 = np.random.uniform(-10,10,size=(2000, 2))
    dat2 = np.random.normal(size=(2300, 2))*8.0
    
    # create an A2 Lattice (honeycomb lattice)
    lat = latbin.ALattice(2)
    
    # histogram the data sets
    h1 = lat.histogram(dat1)
    h2 = lat.histogram(dat2)
    
    # do some operation to the data sets
    diff = h1-h2
    
    # get the data centers
    centers = diff.centers()
    x_centers = centers[:, 0]
    y_centers = centers[:, 1]
    
    # show the results
    import matplotlib.pylab as plt
    plt.scatter(x_centers, y_centers, c=diff.values())
    plt.show()
<commit_msg>added syntax highlighting to README<commit_after># LATBIN #

This python package is used for binning onto lattices in multiple dimensions. 

## Installation ##

In the terminal you can install this in the usual way.

```bash
    python setup.py install
```

## Basic Example ##

This example shows two data sets which have various lengths but both have two dimensions. We create an A2 (aka honeycomb) lattice which we histogram both data sets onto. Then we can make comparisons (e.g. by subtraction) of the two and analyze the result. 

```python
    import latbin
    import numpy as np

    # create fake data sets
    dat1 = np.random.uniform(-10,10,size=(2000, 2))
    dat2 = np.random.normal(size=(2300, 2))*8.0
    
    # create an A2 Lattice (honeycomb lattice)
    lat = latbin.ALattice(2)
    
    # histogram the data sets
    h1 = lat.histogram(dat1)
    h2 = lat.histogram(dat2)
    
    # do some operation to the data sets
    diff = h1-h2
    
    # get the data centers
    centers = diff.centers()
    x_centers = centers[:, 0]
    y_centers = centers[:, 1]
    
    # show the results
    import matplotlib.pylab as plt
    plt.scatter(x_centers, y_centers, c=diff.values())
    plt.show()
```
{'fraction_non_alphanumeric': 0.08673684210526315, 'fraction_numerical': 0.024421052631578948, 'mean_word_length': 2.8950819672131147, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1105890', 'n_tokens_mistral': 777, 'n_tokens_neox': 732, 'n_words': 284}