<commit_before># API for California State Code

This is an API for the California state code that is available as JSON. Currently the state of [California mades it difficult to access the laws programmatically](https://github.com/tylerpearson/state-code-scrapers/blob/master/ca/scrape.rb), so this is an attempt at making that easier.

## Endpoints

### Codes

A list of codes is available at https://api.calilaws.com/v1/codes

### Sections

A list of sections of state code is available at https://api.calilaws.com/v1/sections.

The laws can be sorted by `code`, `division`, `chapter`, and `article` by passing it along as parameters in the URL. An example would be https://api.calilaws.com/v1/sections?code=ELEC&division=1. There is information on pagination in `meta` in the JSON response.

For a direct link to the JSON for a law, pass the id in the URL. For example https://api.calilaws.com/v1/sections/ELEC1500.20048177

## Database setup

### Locally

There is a MySQL database of California state codes available through FTP at ftp://www.leginfo.ca.gov/pub/bill/. Inside the directory are [instructions on importing the tables](https://s3.amazonaws.com/cali-laws/pubinfo_Readme.pdf) that are needed.

Here's a [direct link to the converted and imported MySQL database](https://s3.amazonaws.com/cali-laws/calaws.mysql) that I put up on S3 if you prefer not to deal with that. It's for the 2015 state code with a last modified listed as June 2015.

### Heroku

The database can be converted to Postgres for easy installation on Heroku. One option is to dump the database and convert with [Lanyrd's MySQL to PostgreSQL conversion script](https://github.com/lanyrd/mysql-postgresql-converter).

- `mysqldump --compatible=postgresql --default-character-set=utf8 -r calaws.mysql -u root capublic`
- `python dbconverter.py calaws.mysql calaws.psql`

It can be loaded from a local dump with `heroku pg:psql < db-dumps/calaws.psql`. You'll likely need to comment out the creation of the `schema_migrations` table.

<commit_msg>Add todos in README<commit_after># API for California State Code

This is an API for the California state code that is available as JSON. Currently the state of [California mades it difficult to access the laws programmatically](https://github.com/tylerpearson/state-code-scrapers/blob/master/ca/scrape.rb), so this is an attempt at making that easier.

## Endpoints

### Codes

A list of codes is available at https://api.calilaws.com/v1/codes

### Sections

A list of sections of state code is available at https://api.calilaws.com/v1/sections.

The laws can be sorted by `code`, `division`, `chapter`, and `article` by passing it along as parameters in the URL. An example would be https://api.calilaws.com/v1/sections?code=ELEC&division=1. There is information on pagination in `meta` in the JSON response.

For a direct link to the JSON for a law, pass the id in the URL. For example https://api.calilaws.com/v1/sections/ELEC1500.20048177

## Database setup

### Locally

There is a MySQL database of California state codes available through FTP at ftp://www.leginfo.ca.gov/pub/bill/. Inside the directory are [instructions on importing the tables](https://s3.amazonaws.com/cali-laws/pubinfo_Readme.pdf) that are needed.

Here's a [direct link to the converted and imported MySQL database](https://s3.amazonaws.com/cali-laws/calaws.mysql) that I put up on S3 if you prefer not to deal with that. It's for the 2015 state code with a last modified listed as June 2015.

### Heroku

The database can be converted to Postgres for easy installation on Heroku. One option is to dump the database and convert with [Lanyrd's MySQL to PostgreSQL conversion script](https://github.com/lanyrd/mysql-postgresql-converter).

- `mysqldump --compatible=postgresql --default-character-set=utf8 -r calaws.mysql -u root capublic`
- `python dbconverter.py calaws.mysql calaws.psql`

It can be loaded from a local dump with `heroku pg:psql < db-dumps/calaws.psql`. You'll likely need to comment out the creation of the `schema_migrations` table.

## Next steps

Things that can be done next:

- Build a UI so it can be browsed online as an alternative to the official [California Legislative Information](http://leginfo.legislature.ca.gov/faces/codes.xhtml) site.
- Automate updates with the latest laws. This would be downloading from FTP, converting, and uploading.
- Index with something like Elasticsearch or a search-as-a-service platform like [Algolia](https://www.algolia.com/), so it easier to for people to find specific laws.
{'fraction_non_alphanumeric': 0.08820941487021558, 'fraction_numerical': 0.012758468983721953, 'mean_word_length': 4.330597889800703, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 3, 'https://': 17, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '2906829', 'n_tokens_mistral': 1383, 'n_tokens_neox': 1308, 'n_words': 570}