<jupyter_start><jupyter_text>Notebook 5 - Double ML (teacher notebook) Let's look at the "Twins" data, which was used in the academic paper: <NAME> et al. "The costs of low birth weight." The Quarterly Journal of Economics 120.3 (2005): 1031-1083.<jupyter_code>import doubleml as dml
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
from sklearn.linear_model import LinearRegression, LogisticRegression
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor

%matplotlib inline
plt.rcParams['figure.figsize'] = [10, 5]
# Windows users: replace the following line with `df = pd.read_csv(r"..\..\data\twins.csv.gz")`
df = pd.read_csv("../../data/twins.csv.gz", compression='gzip')<jupyter_output><empty_output><jupyter_text>This dataset has lots of columns in it. If you'd like to learn more about this data [you can visit this repo](https://github.com/AMLab-Amsterdam/CEVAE/blob/master/datasets/TWINS/ReadmeTwins).The outcome of interest:* `dbirwt_0` represents birthweight (in grams)The "treatment" of interest:* `tobacco` represents whether there was smoking present in the family while the mother was pregnantAll the covariates:* 'adequacy': 'adequacy of care',* 'alcohol': 'risk factor, alcohol use',* 'anemia': 'risk factor, Anemia',* 'birattnd': 'medical person attending birth',* 'birmon': 'birth month Jan-Dec',* 'bord_0': 'birth order of lighter twin',* 'brstate': 'state of residence NCHS',* 'brstate_reg': 'US census region of brstate',* 'cardiac': 'risk factor, Cardiac',* 'chyper': 'risk factor, Hypertension, chronic',* 'cigar6': 'num of cigarettes /day, quantiled',* 'crace': 'race of child',* 'csex': 'sex of child',* 'data_year': 'year: 1989, 1990 or 1991',* 'dfageq': 'octile age of father',* 'diabetes': 'risk factor, Diabetes',* 'dlivord_min': 'number of live births before twins',* 'dmar': 'married',* 'drink5': 'num of drinks /week, quantiled',* 'dtotord_min': 'total number of births before twins',* 'eclamp': 'risk factor, Eclampsia',* 'feduc6': 'education category',* 'frace': 'dad race',* 'gestat10': 'gestation 10 categories',* 'hemo': 'risk factor Hemoglobinopathy',* 'herpes': 'risk factor, Herpes',* 'hydra': 'risk factor Hvdramnios/Oliqohvdramnios',* 'incervix': 'risk factor, Incompetent cervix',* 'infant_id_0': 'infant id of lighter twin in original df',* 'lung': 'risk factor, Lung',* 'mager8': 'mom age',* 'meduc6': 'mom education',* 'mplbir': 'mom place of birth',* 'mplbir_reg': US census region of mplbir',* 'mpre5': 'trimester prenatal care begun, 4 is none',* 'mrace': 'mom race',* 'nprevistq': 'quintile number of prenatal visits',* 'orfath': 'dad hispanic',* 'ormoth': 'mom hispanic',* 'othermr': 'risk factor, Other Medical Risk Factors',* 'phyper': 'risk factor, Hypertension, preqnancy-associated',* 'pldel': 'place of delivery',* 'pre4000': 'risk factor, Previous infant 4000+ grams',* 'preterm': 'risk factor, Previos pre-term or small',* 'renal': 'risk factor, Renal disease',* 'rh': 'risk factor, RH sensitization',* 'stoccfipb': 'state of occurence FIPB',* 'stoccfipb_reg': 'US census region of stoccfipb',* 'uterine': 'risk factor, Uterine bleeding'  The `DoubleML` package requires an interesting workflow. First, you need to instantiate a "data model" in which you specify the outcome, the treatment, and the covariates<jupyter_code># We pick out a set of potential confounders to control for
features_base = [
    'mager8', 'csex', 'anemia', 
    'cardiac', 'lung', 'diabetes', 'herpes', 'eclamp',
    'preterm', 'renal', 'rh', 'alcohol', 'dfageq',
]

# Initialize DoubleMLData (data-backend of DoubleML)
data_dml_base = dml.DoubleMLData(
    df,
    y_col='dbirwt_0',
    d_cols='tobacco',
    x_cols=features_base
)

print(data_dml_base)<jupyter_output><empty_output><jupyter_text>Now that we have instantiated our "data model", we can setup our "learners". These learners are typically just scikit-learn models.  Remember, in Double ML we construct a model to predict the outcome given all covariates and the treatment, and we also construct a model to predict treatment status from all of the covariates (hence the "double" in Double ML). The outcome is continuous and the treatment in this case is binary, so we'll need one regression learner and one classification learner<jupyter_code>rf = RandomForestRegressor(n_estimators=500, max_depth=5)
rf_class = RandomForestClassifier(n_estimators=500, max_depth=5)

np.random.seed(123)
dml_plr_forest = dml.DoubleMLPLR(data_dml_base,
                                 ml_g = rf,
                                 ml_m = rf_class,
                                 n_folds = 3)

dml_plr_forest.fit(store_predictions=True)
dml_plr_forest.summary<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13035303948193022, 'fraction_numerical': 0.014205138917902652, 'mean_word_length': 5.4268456375838925, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '254765', 'license': 'MIT', 'n_tokens_mistral': 1676, 'n_tokens_neox': 1517, 'n_words': 530, 'path': 'notebooks/student/5_student_double_ML.ipynb', 'repo_name': 'ronikobrosly/causal_inference_intro', 'size': '4789'}