<jupyter_start><jupyter_text>Tutorial: Counterfactual explanations for scorecard with continuous target This tutorial shows how to generate counterfactual explanations on scorecard models with continuous target. The dataset for this tutorial is https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html.<jupyter_code>import pandas as pd

from sklearn.datasets import fetch_california_housing
from sklearn.linear_model import HuberRegressor

from optbinning import BinningProcess
from optbinning import Scorecard
from optbinning.scorecard import Counterfactual<jupyter_output><empty_output><jupyter_text>Load the dataset<jupyter_code>data = fetch_california_housing()

target = "target"
variable_names = data.feature_names
X = pd.DataFrame(data.data, columns=variable_names)
y = data.target<jupyter_output><empty_output><jupyter_text>Scorecard model First, we develop a scorecard following the steps presented in previous tutorials.<jupyter_code>binning_process = BinningProcess(variable_names)

estimator = HuberRegressor(max_iter=200)

scorecard = Scorecard(binning_process=binning_process,
                      estimator=estimator, scaling_method="min_max",
                      scaling_method_params={"min": 0, "max": 100},
                      reverse_scorecard=True,verbose=True)

scorecard.fit(X, y)<jupyter_output>2021-10-23 20:02:53,105 | INFO : Scorecard building process started.
2021-10-23 20:02:53,107 | INFO : Options: check parameters.
2021-10-23 20:02:53,109 | INFO : Dataset: continuous target.
2021-10-23 20:02:53,110 | INFO : Binning process started.
2021-10-23 20:02:54,929 | INFO : Binning process terminated. Time: 1.8181s
2021-10-23 20:02:54,930 | INFO : Fitting estimator.
2021-10-23 20:02:55,670 | INFO : Fitting terminated. Time 0.7399s
2021-10-23 20:02:55,672 | INFO : Scorecard table building started.
2021-10-23 20:02:55,830 | INFO : Scorecard table terminated. Time: 0.1575s
2021-10-23 20:02:55,831 | INFO : Scorecard building process terminated. Time: 2.7254s<jupyter_text>Generating counterfactual explanations As an input data point or query, we select the first sample. Note that a query must be either a dictionary of a pandas DataFrame.<jupyter_code>query = X.iloc[0, :].to_frame().T
query
scorecard.predict(query)<jupyter_output><empty_output><jupyter_text>The predicted outcome (house value) for this query (house) is 4.38. We want to generate counterfactual explanations to find out how to increase the house value to at least 4.5.<jupyter_code>cf = Counterfactual(scorecard=scorecard, verbose=True)
cf.fit(X)
cf.generate(query=query, y=4.5, outcome_type="continuous", n_cf=1,
            max_changes=3, hard_constraints=["min_outcome"])
cf.information()<jupyter_output>optbinning (Version 0.13.0)
Copyright (c) 2019-2021 <NAME>, Apache License 2.0

  Status  : OPTIMAL                         

  Solver statistics
    Type                                 mip
    Number of variables                   42
    Number of constraints                120
    Objective value                   7.6866
    Best objective bound              7.6866

  Objectives
    proximity                         0.7069
    closeness                         6.9798

  Timing
    Total time                          0.35 sec
    Fit                                 0.06 sec   ( 16.22%)
    Solver                              0.29 sec   ( 80.46%)
    Post-processing                     0.01 sec   (  4.13%)<jupyter_text>The generate counterfactual suggest increasing the block population, reduce the average house occupancy and change the house block longitude. None of them seems doable.<jupyter_code>cf.display(show_only_changes=True, show_outcome=True)<jupyter_output><empty_output><jupyter_text>Now, let's generate several counterfactuals aiming to limit the house value to 4.0.<jupyter_code>cf.generate(query=query, y=4.0, outcome_type="continuous", n_cf=3,
            max_changes=3,
            hard_constraints=["diversity_features", "max_outcome"],
            time_limit=30
           ).display(show_only_changes=True, show_outcome=True)<jupyter_output>2021-10-23 20:03:24,534 | INFO : Counterfactual generation started.
2021-10-23 20:03:24,535 | INFO : Options: check parameters.
2021-10-23 20:03:24,546 | INFO : Options: check objectives and constraints.
2021-10-23 20:03:24,548 | INFO : Optimizer started.
2021-10-23 20:03:24,548 | INFO : Optimizer: build model...
2021-10-23 20:03:24,701 | INFO : Optimizer: solve...
2021-10-23 20:03:44,240 | INFO : Optimizer terminated. Time: 19.6911s
2021-10-23 20:03:44,241 | INFO : Post-processing started.
2021-10-23 20:03:44,276 | INFO : Post-processing terminated. Time: 0.0334s
2021-10-23 20:03:44,277 | INFO : Counterfactual generation terminated. Status: OPTIMAL. Time: 19.7429s<jupyter_text>And the same generation enforcing diversity on feature values.<jupyter_code>cf.generate(query=query, y=3.0, outcome_type="continuous", n_cf=3,
            max_changes=3,
            hard_constraints=["diversity_features", "diversity_values", "max_outcome"],
            time_limit=30
           ).display(show_only_changes=True, show_outcome=True)<jupyter_output>2021-10-23 20:03:51,189 | INFO : Counterfactual generation started.
2021-10-23 20:03:51,191 | INFO : Options: check parameters.
2021-10-23 20:03:51,198 | INFO : Options: check objectives and constraints.
2021-10-23 20:03:51,199 | INFO : Optimizer started.
2021-10-23 20:03:51,201 | INFO : Optimizer: build model...
2021-10-23 20:03:51,377 | INFO : Optimizer: solve...
2021-10-23 20:04:15,261 | INFO : Optimizer terminated. Time: 24.0606s
2021-10-23 20:04:15,262 | INFO : Post-processing started.
2021-10-23 20:04:15,297 | INFO : Post-processing terminated. Time: 0.0341s
2021-10-23 20:04:15,299 | INFO : Counterfactual generation terminated. Status: OPTIMAL. Time: 24.1091s
{'fraction_non_alphanumeric': 0.10041053711939787, 'fraction_numerical': 0.11238453643516935, 'mean_word_length': 3.711522965350524, 'pattern_counts': {'":': 2, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '523028', 'license': 'Apache-2.0', 'n_tokens_mistral': 2284, 'n_tokens_neox': 1851, 'n_words': 472, 'path': 'doc/source/tutorials/tutorial_counterfactual_continuous_target.ipynb', 'repo_name': 'philip-khor/optbinning', 'size': '5864'}