<commit_before><img height=40 src='http://my5C.umassmed.edu/images/3DG.png' title='3D-Genome' />
&nbsp;&nbsp;
<img height=30 src='http://my5C.umassmed.edu/images/dekkerlabbioinformatics.gif' />
&nbsp;&nbsp;
<img height=30 src='http://my5C.umassmed.edu/images/umasslogo.gif' />

# c-world-encode

h5 helper scripts for encode/dekkerlab hi-c data.

convert hdf5 (h5) files into tsv (tab seperated value) text (matrix) files.

```
c-world-encode/
	hdf2tab.py - Extract c-data from HDF5 file into TXT (matrix.gz)
```

## Full Documentation

See the [Wiki](https://github.com/blajoie/c-world-encode/wiki) for full documentation of encode/dekkerlab Hi-C HDF5 file format.
	Datasets
	Attributes
	
See the [Wiki](https://github.com/blajoie/c-world-encode/wiki) for full documentation, examples, operational details and other information.

## Communication

- [<NAME>](https://github.com/blajoie)
- [<NAME>](https://github.com/NoamKaplan)
- Twitter: [@my5C](https://twitter.com/my5C)

## What does it do?<commit_msg>added links to wiki, hdf spec etc<commit_after><img height=40 src='http://my5C.umassmed.edu/images/3DG.png' title='3D-Genome' />
&nbsp;&nbsp;
<img height=30 src='http://my5C.umassmed.edu/images/dekkerlabbioinformatics.gif' />
&nbsp;&nbsp;
<img height=30 src='http://my5C.umassmed.edu/images/umasslogo.gif' />

# c-world-encode

h5 helper scripts for encode/dekkerlab hi-c data.

convert hdf5 (h5) files into tsv (tab seperated value) text (matrix) files.

```
c-world-encode/
	hdf2tab.py - Extract c-data from HDF5 file into TXT (matrix.gz)
```

## Full Documentation

See the Wiki for full documentation of encode/dekkerlab Hi-C HDF5 file format.
[c-world-encode wiki](https://github.com/blajoie/c-world-encode/wiki)

[HDF5 spec](https://github.com/blajoie/c-world-encode/wiki/H5-Spec)

[HDF5 helper script (python), hdf2tab.py](https://github.com/blajoie/c-world-encode)
numpy/scipy/h5py required. Python 2.7+

[hdf2tab.py usage](https://github.com/blajoie/c-world-encode#usage</a>)

Hi-C data is pushed through the dekkerlab standard pipeline (to be available on GIT soon) and binned at multiple bin sizes (resolutions)
10mb
2.5mb
1mb
500kb
250kb
100kb
40kb

See the [Wiki](https://github.com/blajoie/c-world-encode/wiki) for full documentation, examples, operational details and other information.

## Communication

- [<NAME>](https://github.com/blajoie)
- [<NAME>](https://github.com/NoamKaplan)
- Twitter: [@my5C](https://twitter.com/my5C)

## What does it do?
{'fraction_non_alphanumeric': 0.15683104284559418, 'fraction_numerical': 0.025060630557801132, 'mean_word_length': 4.612244897959184, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 13, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2575665', 'n_tokens_mistral': 958, 'n_tokens_neox': 917, 'n_words': 205}