<commit_before>phoxy
=====

Download, ingest, and analyze the Phoenix event dataset.

The Phoenix event data set records social and political events
automatically extracted from news reporting. `phoxy` provides
functions to make acquiring and analyzing this event data easier. For more
information about Phoenix, see its website here:
[http://phoenixdata.org/](http://phoenixdata.org/) and the website of the Open Event Data
Alliance here: [http://openeventdata.org/](http://openeventdata.org).

`phoxy`, like the Phoenix event dataset, is still under beta development and will
change greatly. Bug reports are welcome.

Installation
------------
`devtools::install_github("ahalterman/phoxy")`

Usage
-----

Currently, phoxy includes two basic functions.

* `download_phoenix` takes a directory path and downloads the entire Phoenix
  dataset from [http://phoenixdata.org/data](http://phoenixdata.org/data) into
  that directory. 

* `ingest_phoenix` reads in each daily text file from that directory, does some
  basic column name and class fixing, and combines them all into one large
  dataframe for easy analysis. 

Example:

```
download_phoenix("~/OEDA/phoxy_test/")
events <- ingest_phoenix("~/OEDA/phoxy_test/")

str(events)

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   351386 obs. of  26 variables:
 $ EventID             : int  313122 313123 313124 313125 313126 313127 313128 313129 313130 313131 ...
 $ Date                : int  20140619 20140620 20140619 20140620 20140620 20140620 20140620 20140620 20140620 20140617 ...<commit_msg>Update documentation for new function<commit_after>phoxy
=====

Download, ingest, and analyze the Phoenix event dataset.

The Phoenix event data set records social and political events
automatically extracted from news reporting. `phoxy` provides
functions to make acquiring and analyzing this event data easier. For more
information about Phoenix, see its website here:
[http://phoenixdata.org/](http://phoenixdata.org/) and the website of the Open Event Data
Alliance here: [http://openeventdata.org/](http://openeventdata.org).

`phoxy`, like the Phoenix event dataset, is still under beta development and will
change greatly. Bug reports are welcome.

Installation
------------
`devtools::install_github("ahalterman/phoxy")`

Usage
-----

Currently, phoxy includes three basic functions.

* `download_phoenix` takes a directory path and downloads the entire Phoenix
  dataset from [http://phoenixdata.org/data](http://phoenixdata.org/data) into
  that directory. 

* `ingest_phoenix` reads in each daily text file from that directory, does some
  basic column name and class fixing, and combines them all into one large
  dataframe for easy analysis. 

* `convert_cameo` will translate from CAMEO event codes to the human readable
  definitions taken from the CAMEO codebook.

Example:

```
download_phoenix("~/OEDA/phoxy_test/")
events <- ingest_phoenix("~/OEDA/phoxy_test/")

str(events)

Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   351386 obs. of  26 variables:
 $ EventID             : int  313122 313123 313124 313125 313126 313127 313128 313129 313130 313131 ...
 $ Date                : int  20140619 20140620 20140619 20140620 20140620 20140620 20140620 20140620 20140620 20140617 ...
{'fraction_non_alphanumeric': 0.0931523750771129, 'fraction_numerical': 0.09130166563849476, 'mean_word_length': 4.1394611727416795, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '35137', 'n_tokens_mistral': 1154, 'n_tokens_neox': 914, 'n_words': 359}