<commit_before>---
layout: post
title: World Air Quality Map
date: 2017-04-09
excerpt: Mapping air quality data from around the world.
image: /images/posts/aggregate_data.png
project: true
tags: scraper pollution Mapbox
---

[![image](/images/posts/aggregate_data.png)](/projects/AirQuality/World_Data)
<!--(/projects/AirQuality/AQmap_static.html)-->
***click the map for an interactive version***

Data does not tell its own story. My graduate thesis project, towards which I worked meticulously for over five years, essentially boils down to around 8 billion numbers in various arrays. The mere record of these numbers would surely not have persuaded my thesis committee to grant me my PhD. The point is, of course, not that I *have* the numbers, but what the numbers mean: These numbers, in this arrangement, arriving in this order means-- something. It's a human task to turn data into information.
<commit_msg>AQ plot caveat<commit_after>---
layout: post
title: World Air Quality Map
date: 2017-04-09
excerpt: Mapping air quality data from around the world.
image: /images/posts/aggregate_data.png
project: true
tags: scraper pollution Mapbox
---

[![image](/images/posts/aggregate_data.png)](/projects/AirQuality/World_Data)
<!--(/projects/AirQuality/AQmap_static.html)-->
***click the map for an interactive version (NOTE: data for this project is still being processed- the data presented above is not accurate- yet)***

Data does not tell its own story. My graduate thesis project, towards which I worked meticulously for over five years, essentially boils down to around 8 billion numbers in various arrays. The mere record of these numbers would surely not have persuaded my thesis committee to grant me my PhD. The point is, of course, not that I *have* the numbers, but what the numbers mean: These numbers, in this arrangement, arriving in this order means-- something. It's a human task to turn data into information.

{'fraction_non_alphanumeric': 0.08122090015519917, 'fraction_numerical': 0.009311950336264873, 'mean_word_length': 4.654970760233918, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'dir': 'git-commits-cleaned', 'id': '1999697', 'n_tokens_mistral': 524, 'n_tokens_neox': 489, 'n_words': 263}