<jupyter_start><jupyter_text>"Como calcular IV e WOE no Python"> Tutorial de como calcular o *Information Value* e *Weight of Evidence*.- toc: false- badges: true- comments: true- categories: [iv, woe, python]- image:<jupyter_code>#Bibliotecas necessárias
import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Importando a base de dados utilizada no exemplo.<jupyter_code>df = pd.read_excel("CasoUso_02_Emprestimo_Bancario.xlsx", sheet_name='BASE DE DADOS')
df.head()<jupyter_output><empty_output><jupyter_text>Definindo a função que será utilizada para calcular o iv e woe {% fn 1 %}.<jupyter_code>def iv_woe(data, target, bins=10, show_woe=False):
    
    #Empty Dataframe
    newDF,woeDF = pd.DataFrame(), pd.DataFrame()
    
    #Extract Column Names
    cols = data.columns
    
    #Run WOE and IV on all the independent variables
    for ivars in cols[~cols.isin([target])]:
        if (data[ivars].dtype.kind in 'bifc') and (len(np.unique(data[ivars]))>10):
            binned_x = pd.qcut(data[ivars], bins,  duplicates='drop')
            d0 = pd.DataFrame({'x': binned_x, 'y': data[target]})
        else:
            d0 = pd.DataFrame({'x': data[ivars], 'y': data[target]})
        d = d0.groupby("x", as_index=False).agg({"y": ["count", "sum"]})
        d.columns = ['Cutoff', 'N', 'Events']
        d['% of Events'] = np.maximum(d['Events'], 0.5) / d['Events'].sum()
        d['Non-Events'] = d['N'] - d['Events']
        d['% of Non-Events'] = np.maximum(d['Non-Events'], 0.5) / d['Non-Events'].sum()
        d['WoE'] = np.log(d['% of Events']/d['% of Non-Events'])
        d['IV'] = d['WoE'] * (d['% of Events'] - d['% of Non-Events'])
        d.insert(loc=0, column='Variable', value=ivars)
        print("Information value of " + ivars + " is " + str(round(d['IV'].sum(),6)))
        temp =pd.DataFrame({"Variable" : [ivars], "IV" : [d['IV'].sum()]}, columns = ["Variable", "IV"])
        newDF=pd.concat([newDF,temp], axis=0)
        woeDF=pd.concat([woeDF,d], axis=0)

        #Show WOE Table
        if show_woe == True:
            print(d)
    return newDF, woeDF<jupyter_output><empty_output><jupyter_text>Parametros necessários para chamar a função:- *data* - é o data frame aonde estão localizadas as variáveis dependentes e idenpendentes;- *target* - o nome da coluna onde está armazenado os dados da variável dependente;- *bins* - quantidade de divisões ou intervalo nas variáveis;- *show_woe* - quando verdadeiro (True), significa que a função irá imprimir a tabela de cálculo do WOE. Chamando a função para realizar o cálculo.<jupyter_code>iv, woe = iv_woe(data = df, target = 'classif')
print(iv)<jupyter_output>Variable        IV
0           idade  0.201024
0     experiencia  0.474158
0  tempo_endereco  0.279081
0           renda  0.093243
0    debito_renda  0.750072
0        cred_deb  0.261361
0  outros_debitos  0.114104
{'fraction_non_alphanumeric': 0.15389948006932408, 'fraction_numerical': 0.025649913344887348, 'mean_word_length': 3.488335925349922, 'pattern_counts': {'":': 1, '<': 16, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '161186', 'license': 'Apache-2.0', 'n_tokens_mistral': 1116, 'n_tokens_neox': 1065, 'n_words': 271, 'path': '_notebooks/2020-12-15-iv_woe.ipynb', 'repo_name': 'lucastiagooliveira/datascience', 'size': '2885'}