<jupyter_start><jupyter_text>Calculating the probabilities of necessary and sufficient causationConsider the causal effect of a voucher on customer conversion. We can distinguish between the following types of causation:* **Necessary**: If the customer doesn't get the voucher, they will not convert* **Sufficient**: If the customer gets the voucher, they will convert* **Necessary and sufficient**: The customer will convert if and only if they receive the voucherIn general, we would like many intervetions to be of the last type. If the voucher is not necessary for a given customer, we might be wasting money by targeting them; if the voucher is not sufficient, we may not fulfil the goal of the campaign, which is to cause customers to convert.[Tian and Pearl (2000)](https://ftp.cs.ucla.edu/pub/stat_ser/r271-A.pdf) provided a way to combine experimental and observational data to derive bounds for the probability of each of the above types of causation. In this notebook, we replicate the example from their paper.<jupyter_code>import numpy as np
import pandas as pd

from causalml.optimize import get_pns_bounds<jupyter_output><empty_output><jupyter_text>[Tian and Pearl (2000, p. 306)](https://ftp.cs.ucla.edu/pub/stat_ser/r271-A.pdf) imagine a setup where we have both experimental and observational data about the efficacy of a certain drug. The experimental data looks as follows:|           | Treatment | Control ||-----------|-----------|---------|| Deaths    | 16        | 14      || Survivals | 984       | 986     |Therefore, based on the experiment, it looks like there isn't much of a difference in the rate of deaths in the treatment and control groups. However, in addition to the experimental data, we also have the following data that is from an observational study, i.e. a study in which we simply observe the outcomes for those who choose to use the drug vs. those who don't:|           | Treatment | Control ||-----------|-----------|---------|| Deaths    | 2         | 28      || Survivals | 998       | 972     |Because people self-select to use the drug, the data shown in the table is very likely confounded. However, Tian and Pearl argue that the above two datasets can be combined to obtain information that is not visible by looking at either of the datasets independently, namely the probabilities of necessary and sufficient causation (PNS). More specifically, it is possible to derive bounds for PNS by combining the two data sources. To see how, let's generate the datasets:<jupyter_code>num_samples = 2000
half = int(num_samples / 2)
treatment = np.tile([0, 1], half)
recovery = np.zeros(num_samples)

df_rct = pd.DataFrame({'treatment': treatment, 'death': recovery})
df_obs = pd.DataFrame({'treatment': treatment, 'death': recovery})
# Set the label to `1' for 16 treatment and 14 control observations
df_rct.loc[df_rct.loc[df_rct['treatment'] == 1].sample(n=16).index, 'death'] = 1
df_rct.loc[df_rct.loc[df_rct['treatment'] == 0].sample(n=14).index, 'death'] = 1
df_rct.groupby('treatment')['death'].sum()
# Set the label to `1' for 2 treatment and 28 control observations
df_obs.loc[df_obs.loc[df_obs['treatment'] == 1].sample(n=2).index, 'death'] = 1
df_obs.loc[df_obs.loc[df_obs['treatment'] == 0].sample(n=28).index, 'death'] = 1
df_obs.groupby('treatment')['death'].sum()<jupyter_output><empty_output><jupyter_text>WIth these data, we can now use the `get_pns_bounds()' function to calculate the relevant bounds. Let's do it for each of the three types of bound:<jupyter_code>pns_lb, pns_ub = get_pns_bounds(df_rct, df_obs, 'treatment', 'death', type='PNS')
pn_lb, pn_ub = get_pns_bounds(df_rct, df_obs, 'treatment', 'death', type='PN')
ps_lb, ps_ub = get_pns_bounds(df_rct, df_obs, 'treatment', 'death', type='PS')
print(f'''
Bounds for the probability of necessary causation: [{round(pn_lb, 3)}, {round(pn_ub, 3)}]
Bounds for the probability of sufficient causation: [{round(ps_lb, 3)}, {round(ps_ub, 3)}]
Bounds for the probability of necessary and sufficient causation: [{round(pns_lb, 3)}, {round(pns_ub, 3)}]
''')<jupyter_output>Bounds for the probability of necessary causation: [1.0, 1]
Bounds for the probability of sufficient causation: [0.002, 0.031]
Bounds for the probability of necessary and sufficient causation: [0.002, 0.016]
{'fraction_non_alphanumeric': 0.1147120540918629, 'fraction_numerical': 0.021450221496852412, 'mean_word_length': 4.644736842105263, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '183997', 'license': 'Apache-2.0', 'n_tokens_mistral': 1374, 'n_tokens_neox': 1215, 'n_words': 522, 'path': 'examples/necessary_and_sufficient.ipynb', 'repo_name': 'saiwing-yeung/causalml', 'size': '4289'}