<jupyter_start><jupyter_text>Validation on Predictions This notebook generates intermediate datasets for validation. It collects and summarizes the steps taken in the git branch `validation/geih-metro-areas`.  Imports<jupyter_code>import sys
sys.path.insert(0, '../utils')

import modelutils
from settings import *<jupyter_output>/opt/conda/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)<jupyter_text>Compare 2020 and 2018 features by departmentTo find what drove the WASH access prediction, we aggregate features by departmentTo ensure comparability, only grids present in 2018 were included in the aggregate.<jupyter_code>df = modelutils.aggregate_predictions(by = 'department')
df.to_csv(data_dir + 'wash_agg.csv', index = False)<jupyter_output><empty_output><jupyter_text>Check trends of actual vs predicted by metro areaSince we are limited to metro areas for validation<jupyter_code>df = modelutils.aggregate_predictions(by = 'metro_area')
df.to_csv(data_dir + 'metro_trends.csv', index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0784452296819788, 'fraction_numerical': 0.018374558303886925, 'mean_word_length': 6.4526315789473685, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '32897', 'license': 'MIT', 'n_tokens_mistral': 440, 'n_tokens_neox': 404, 'n_words': 138, 'path': 'notebooks/04_Validation.ipynb', 'repo_name': 'thinkingmachines/geoai-immap-wash', 'size': '1415'}