<commit_before>---
layout: post
title: "Blog Post 1"
date: 2017-03-08
categories:
---
##### Big idea

##### Dataset information

For our final project, we are using a dataset from the EPA (https://aqsdr1.epa.gov/aqsweb/aqstmp/airdata/download_files.html) that contains details about pollution in the US from 2000 to 2016 daily. The dataset contains information about emission rates by various pollutants by different locations. For each pollutant (NO2, O3, SO2, and O3), the dataset contains information about:
- Site info
- Date
- Units of Measure
- Mean
- Air quality index (AQI)
- Maximum value (max value for a day)
- Maximum hour (hour when max was recorded)

These data are available for most locations across the United States, at a variety of scales. The dataset provides information on each observation site:
- State code
- County code
- Site number
- Latitude
- Longitude
- Address
- State
- County
- City
- Date local
- etc.


##### Cleaning
<img src="../images/first" alt="image" height="42" width="42"><commit_msg>try image<commit_after>---
layout: post
title: "Blog Post 1"
date: 2017-03-08
categories:
---
##### Big idea

##### Dataset information

For our final project, we are using a dataset from the EPA (https://aqsdr1.epa.gov/aqsweb/aqstmp/airdata/download_files.html) that contains details about pollution in the US from 2000 to 2016 daily. The dataset contains information about emission rates by various pollutants by different locations. For each pollutant (NO2, O3, SO2, and O3), the dataset contains information about:
- Site info
- Date
- Units of Measure
- Mean
- Air quality index (AQI)
- Maximum value (max value for a day)
- Maximum hour (hour when max was recorded)

These data are available for most locations across the United States, at a variety of scales. The dataset provides information on each observation site:
- State code
- County code
- Site number
- Latitude
- Longitude
- Address
- State
- County
- City
- Date local
- etc.


##### Cleaning
<img src="../images/first" alt="image">
{'fraction_non_alphanumeric': 0.0957079427725703, 'fraction_numerical': 0.023680315737543166, 'mean_word_length': 3.946341463414634, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '4716676', 'n_tokens_mistral': 619, 'n_tokens_neox': 556, 'n_words': 270}