<commit_before># pyCProject
Provides basic function to read a ContentMine [CProject](https://github.com/ContentMine/workshop-resources/blob/master/software-tutorials/cproject/README.md) and CTrees into python datastructures.

Main use is to read in all results.xml created by ami, and to be relate them to papers/metadata.


# Installation

* You can install it from PyPI with

```
pip install pycproject
```

* Another option is to install it into a [virtualenv](http://docs.python-guide.org/en/latest/dev/virtualenvs/):

```
source activate YOURVIRTUALENV
pip install pycproject
```

* Download and unzip the [source from github](https://github.com/ContentMine/pyCProject/archive/master.zip), change into the unzipped folder, then run

```
python setup.py build
python setup.py install
```

# converting to json-dumps

If your cproject is in `PATH/TO/CPROJECT/CPROJECTNAME`, call the script with
```
python3 pycproject/convert2elasticdump.py --raw PATH/TO/CPROJECT --name CPROJECTNAME --output PATH/TO/OUTPUTFOLDER
```

# Usage

You can then read a generated ContentMine-project in with

```
from pycproject.readctree import CProject
MYPROJECT = CProject("path_to_cproject", "cproject_name")
```

You can work with a pandas DataFrame after creating it with
```
df = MYPROJECT.get_dataframe()
```

# DEPRECATION WARNING

Visualization / network analysis in factnet.py will is no longer supported or maintained in this package as it is not part of the core functionality, and will be removed.<commit_msg>deprecation warning<commit_after># pyCProject
Provides basic function to read a ContentMine [CProject](https://github.com/ContentMine/workshop-resources/blob/master/software-tutorials/cproject/README.md) and CTrees into python datastructures.

Main use is to read in all results.xml created by ami, and to be relate them to papers/metadata.

# DEPRECATION WARNING

Visualization / network analysis in factnet.py will is no longer supported or maintained in this package as it is not part of the core functionality, and will be removed.


# Installation

* You can install it from PyPI with

```
pip install pycproject
```

* Another option is to install it into a [virtualenv](http://docs.python-guide.org/en/latest/dev/virtualenvs/):

```
source activate YOURVIRTUALENV
pip install pycproject
```

* Download and unzip the [source from github](https://github.com/ContentMine/pyCProject/archive/master.zip), change into the unzipped folder, then run

```
python setup.py build
python setup.py install
```

# converting to json-dumps

If your cproject is in `PATH/TO/CPROJECT/CPROJECTNAME`, call the script with
```
python3 pycproject/convert2elasticdump.py --raw PATH/TO/CPROJECT --name CPROJECTNAME --output PATH/TO/OUTPUTFOLDER
```

# Usage

You can then read a generated ContentMine-project in with

```
from pycproject.readctree import CProject
MYPROJECT = CProject("path_to_cproject", "cproject_name")
```

You can work with a pandas DataFrame after creating it with
```
df = MYPROJECT.get_dataframe()
```

{'fraction_non_alphanumeric': 0.09286898839137644, 'fraction_numerical': 0.0013266998341625207, 'mean_word_length': 4.833655705996131, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 2}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2383887', 'n_tokens_mistral': 954, 'n_tokens_neox': 899, 'n_words': 320}