<jupyter_start><jupyter_text>Controlling DataRobot Optimizer Through PythonThis code shows how to interact to DataRobot Optimizer App using the example Lending Club dataset **"Lending Club Sample 30.csv"**.In this example, we are trying to find the best combination of values for revol_util, inq_last_6mths, loan_amnt, and dti that minimizes the probability of a loan going bad.   ResultA dataframe with the original features, prediction, and the optimized features.  The optimized features have the prefix **opt_** Assumptions1. A model has been deployed 2. An Optimization App has been created3. The dataset and Jupyter notebook are in the same folder4. The outcome of the optimization is added to the original dataframe Steps1. Change key_dict     1. Get the URL from the application (see figure below)    2. Put the values in key_dict    3. Put the name of the dataset in ts_settings["filename"]2. Read file into a dataframe3. result_df = perform_optimization(data_df) Functions*get_optimization* : makes a post request to perform optimization.  It returns optimized values for the constraint features, and the predicted target*get_constraints*  : accesses the constraint features and their ranges from the Optimizer App*create_constrain_from_df* : if you want to decide which features to constrain, change the  "cfeatures" list in ts_settings and provide a file to estimate the min and the max for these features*set_optimizer* : prepares the elements required by the Optimizer App  Get the URL with application id, and the token id fromThe URL is available from the share icon in the top menu of the Optimization App.<jupyter_code>import requests  # you could also use aiohttp instead of requests to make calls asynchronously
import json
import pandas as pd

'''
You get the ID and Token from the Optimizer App by clicking the share triangle (in the top menu of the UI)
'''
#https://606c32b7462514a41c85c492.apps.datarobot.com/settings?token=xMdWLVsq2C2dNvMCTMlbDzbZ6IcF_GwA4E76iYtJTGs&lang=en&theme=light

key_dict = {
    'Complete_data': (
                 '',  # ID
                 '',  # Token
                ),
}
#Set constraint features, and filename

ts_settings = {"cfeatures" :['revol_util','inq_last_6mths','loan_amnt','dti'],
              "filename":"Lending Club Sample 30.csv"}

def get_optimization(app, constraints):
    '''
    Given the credentials and data, get the best target under the constrained features
    :param app: 'Complete_data' in key_dict
    :param constraints: a dict containing the constrained features, the data, and the optimization type
    :return: optimization performance, and values tried for each constrained feature
    '''
    app_id, token = key_dict[app]
    url = f'https://{app_id}.apps.datarobot.com/api/optimize'
    headers = {"Authorization": f"Bearer {token}"}
    return requests.post(url, json=constraints, headers=headers).json()

def get_constraints(key_dict):
    '''
    Get a list of constraint features and their minimum and maximum values from the Optimization App  
    :param key_dict: the dictionary with the required app id and token
    :return: a list of constraint features and their min and max
    '''
    app_id, token = key_dict['Complete_data']
    url = f'https://{app_id}.apps.datarobot.com/api/application'
    headers = {"Authorization": f"Bearer {token}"}
    constraints = requests.get(url,headers=headers).json()
    constrain_list = constraints['constraints']
    [x.pop('feature_type') for x in constrain_list]
    return constrain_list
#You can build your constraint list from full dataset
#The min and the max are calculated from the full dataset
def create_constrain_from_df(df_tmp,cfeatures):
    '''
    Create a list of constraint features and their minimum and maximum values.  We assume we get the minimum and maximum 
    of the constraint features from a dataset
    :param df_tmp: dataframe 
    :param cfeatures: the name of the constrained features
    :return: a list of constraint features and their min and max
    '''
    constrain_list = []
    for c_name in cfeatures:
        c_dict = {'feature': c_name,'info':{'min': df_tmp[c_name].min(),'max':df_tmp[c_name].max(),'is_int':False}}
        constrain_list.append(c_dict)
    return constrain_list
def set_optimizer(constrain_list,tmp1):
    '''
    Create the constraints dictionary:  It has to have the constraint list, and a row that need to be optimized
    :param constrain_list: list of features and their constraints
    :param tmp1: a row that has been changed to a JSON format
    :return: a dict
    '''
    constraints = {
        'settings': {
        # Specify if you want to maximize or minimize
            "targetDirection": 'max',

            # Specify your flex feature ranges
            'constraints': constrain_list,
        },
    #    'optimization': {'method': 'exhaust'},  # optional

        # Specify your fixed values
        'datapoint': tmp1,
    }
    return constraints
def perform_optimization(df):
    '''
    Perform a batch optimization.  Add the results to the original dataframe
    :param df: dataframe with the dataset to be optimized 
    :return: a dataframe with the original dataframe with the predicted and optimized features.  The optimized features  
             name are prefixed with opt_ 
    '''
    #create constrain_list
    constrain_list = get_constraints(key_dict)
    #create new features and set them to 0.00
    df["opt_prediction"] = 0.00
    for feature in ts_settings["cfeatures"]:
        df["opt_"+feature] = 0.00

    #For create a constraints for each row and request optimization from DataRobot Optimizer App
    for index, row in df.iterrows():
        tmp1 = json.loads(row.to_json())
        constraints =set_optimizer(constrain_list,tmp1)
        for app in key_dict:  # if using aiohttp, you could start multiple optimizations and have them run simultaneously
            results = get_optimization(app, constraints)
            #Get the optimized target: prediction, and the values of the constrained features that resulted in the optimized target
            tp = results['optimized_simulation']
            df.loc[index,"opt_prediction"] = tp['prediction']
            for feature in tp['features']:
                f_name = feature["name"]
                df.loc[index,"opt_"+f_name] = feature["value"]
    return df<jupyter_output><empty_output><jupyter_text>RUN THIS<jupyter_code>data_df = pd.read_csv(ts_settings["filename"])
result_df = perform_optimization(data_df)
result_df[['opt_prediction','opt_revol_util','opt_inq_last_6mths','opt_loan_amnt','opt_dti']]

# See example of output from the optimizer
results
df.loc[3,"opt_prediction"]
resuld_df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07762081784386617, 'fraction_numerical': 0.008475836431226767, 'mean_word_length': 3.797432239657632, 'pattern_counts': {'":': 4, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '446924', 'license': 'Apache-2.0', 'n_tokens_mistral': 1916, 'n_tokens_neox': 1853, 'n_words': 719, 'path': 'Advanced Tuning and Partitioning/Python/DataRobot_Optimizer/Optimizing Through API.ipynb', 'repo_name': 'ahjota/examples-for-data-scientists', 'size': '6725'}