<jupyter_start><jupyter_text>Modelling via scikit-learn This notebook performs model fitting and evaluating on hold out areas, using models from scikit-learn.  Imports and Setup<jupyter_code>import pandas as pd
import numpy as np
from tqdm import tqdm
from math import sqrt
import pathlib
import os
from sklearn.externals import joblib

from sklearn.ensemble import RandomForestRegressor
from scipy.stats import pearsonr
# pd.set_option('display.float_format', lambda x: '%.2f' % x)
# pd.reset_option('^display')

# ignore warnings
import logging
import warnings
logging.getLogger().setLevel(logging.ERROR)
warnings.filterwarnings("ignore")

import sys
sys.path.insert(0, '../utils')
import modelutils
from settings import *<jupyter_output>/opt/conda/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=FutureWarning)<jupyter_text>File Locations<jupyter_code>dirs = [model_dir, scaler_dir, preds_dir]
for dir_ in dirs:
    with pathlib.Path(dir_) as path:
        if not path.exists():
            path.mkdir(parents=True, exist_ok=True)<jupyter_output><empty_output><jupyter_text>Load Data<jupyter_code># !gsutil cp gs://immap-wash-training/training/20200916_*.csv {data_dir}
df = pd.read_csv(data_dir + '20200916_dataset.csv')
df.shape<jupyter_output><empty_output><jupyter_text>Fit, Predict and Evaluate Models Using 5-fold Cross ValidationTrains a random forest model to predict each of the 3 WASH indicators, setting random 20% of the data (whichever the area) as test data and everything else as training data. The train-test split is done for 5 different folds. The data that was split into different folds includes **both Urban and Rural** data.<jupyter_code>from sklearn.ensemble import RandomForestRegressor
clf = RandomForestRegressor(random_state=42)

res = modelutils.fit_with_randomsplit(df, clf, features, indicators, scale = True, n_splits = 5, prefix = 'all_250mv2')
res = pd.read_csv(data_dir + 'all_250mv2_randomsplit_results.csv')
modelutils.summarize_metrics(res, iterate_over = 'split_id', prefix = 'all_250mv2')<jupyter_output>Access to water

Average Metrics
correlation: 0.7723
r2: 0.5967
rmse: 0.1291
{'fraction_non_alphanumeric': 0.08714859437751005, 'fraction_numerical': 0.026907630522088354, 'mean_word_length': 5.900277008310249, 'pattern_counts': {'":': 0, '<': 15, '<?xml version=': 0, '>': 15, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '272512', 'license': 'MIT', 'n_tokens_mistral': 853, 'n_tokens_neox': 786, 'n_words': 245, 'path': 'notebooks/02_Modelling.ipynb', 'repo_name': 'thinkingmachines/geoai-immap-wash', 'size': '2490'}