<jupyter_start><jupyter_text>Dia 1, primera parte: ML dia a dia<jupyter_code>import os
import subprocess
import sys

GIT_NAME='Intro_MLDL_19'
GIT_URL='https://<EMAIL>/riiaa/{}.git'.format(GIT_NAME)
IN_COLAB = 'google.colab' in sys.modules

def run_cmd(cmd):
    print('Output of "{}":'.format(cmd))
    print(subprocess.run(cmd,stdout=subprocess.PIPE, shell=True).stdout.decode('utf-8'))
        
if IN_COLAB:
    SRC_DIR='.'
    run_cmd('sudo apt-get install git-lfs')
    run_cmd('git lfs install')
    run_cmd('rm -rf sample_data')
    run_cmd('rm -rf {}'.format(GIT_NAME))
    run_cmd('git clone --verbose --progress {}'.format(GIT_URL))
    run_cmd('mv {}/* . '.format(GIT_NAME))
    run_cmd('rm -rf {}'.format(GIT_NAME))
else:
    SRC_DIR='..'
    
print('Using colab? {}, using root directory "{}"'.format(IN_COLAB,SRC_DIR))<jupyter_output><empty_output><jupyter_text>Librerias<jupyter_code>%matplotlib inline
%config InlineBackend.figure_format='retina'
import matplotlib
import matplotlib.pyplot as plt
import numpy as np
import scipy as sc
import pandas as pd
import sklearn
import matplotlib.pyplot as plt
import seaborn as sns
import os
from IPython.display import display
import sys
sys.path.append(os.path.join(SRC_DIR,'code'))
import higgs_score
data_path = os.path.join(SRC_DIR,'data')<jupyter_output><empty_output><jupyter_text>Algunas perspectivas sobre MLAI/ML/DL:![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/compare_AI_ML_DL.png?raw=true)De xkcd:![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/xkcd_ml.jpg?raw=true)Mi perspectiva:![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/ml_dl.png?raw=true)* <NAME> "Artificial Intelligence — The Revolution Hasn’t Happened Yet": https://medium.com/@mijordan3/artificial-intelligence-the-revolution-hasnt-happened-yet-5e1d5812e1e7* <NAME> "Living Together: Mind and Machine Intelligence https://arxiv.org/abs/1705.07996  Un paradigma para MLMachine learning (aprendizaje machin para la banda), trata de crear algoritmos $M$ que puedan "aprender", que significa esto?> A computer program is said to learn from experience $E$ with respect to some class of tasks $T$ and performance measure **P**, if its performance at tasks in $T$, as measured by $P$, improves with experience $E$.> > Machine Learning, <NAME>, 1997	Los ingredientes esenciales son:* $E$ (experiencia): De que vamos a aprender?* $T$ (tareas): Que queremos hacer?* $P$ (rendimiento): Como sabes que estamos mejorando?* $M$ (modelo, algoritmo): Como vamos a crear un sistema para aprender?  $E$ y $T$, vamos a conocer los datos!AKA exploratory data analysis (EDA)![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/atlas.png?raw=true)Mas informacion:http://opendata.cern.ch/record/328Usaremos pandas para manipular datos<jupyter_code>df=pd.read_csv(os.path.join(SRC_DIR,'data/HiggsBoson.csv'))
print(df.columns)
target_col = 'Label'
df[target_col] = df[target_col].replace(to_replace=['s','b'],value=[1,0])
df = df.drop('EventId',axis=1)
print(df.shape)
df.head(1)
features = ['DER_mass_MMC', 'DER_mass_transverse_met_lep']
for f in features:
    sns.distplot(df[f])
    plt.show()<jupyter_output><empty_output><jupyter_text>modelos discriminativos vs generativos![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/generative_descriminative.png?raw=true) Tareas, distribuciones y perdidas (loss functions, $P$)* Regression continua: Gaussiana (RMSE), t-student, gamma, beta, poisson, negative binomial, zero-inflated ..etc. * Regression ordinal: Ordinal (ordered logits,..muchos)* Classificacion binaria: Bernoulli (Binary cross entropy)* Classificacion multiclase: Multinomial (cross entropy)* Classificacion multilabel: Multiples bernoulli (Binary cross entropy)* y muchos mas  Visualiza nuestro "target" ($y$)<jupyter_code>sns.distplot(df[target_col])<jupyter_output><empty_output><jupyter_text>Busca relaciones visualesA "ojimetro" usando seaborn: https://seaborn.pydata.org/examples/index.html<jupyter_code>features = ['DER_mass_MMC', 'DER_mass_transverse_met_lep']
sns.pairplot(df[features+[target_col]])
plt.show()<jupyter_output><empty_output><jupyter_text>Pregunta: realmente necesitamos DL?![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/andrewng_dlml_data.png?raw=true)Siempre es importante tener un "baseline", modelos base para saber cual es límite inferior en prediccion de tus datos.  Definir performance $P$https://scikit-learn.org/stable/modules/model_evaluation.htmlHay que tener en cuenta nuestra metricas para optimizar (tipicamente loss functions) y las metrics que realmente nos interesan. En nuestro caso nos interesahttps://higgsml.lal.in2p3.fr/files/2014/04/documentation_v1.8.pdfApproximate median significance (AMS), es el numero estimado de sigmas que tu modelo puede predecir. Si llegas a **6** ya descubriste el boson de higgs!![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/sigma_gaussian.png?raw=true)<jupyter_code>import higgs_score<jupyter_output><empty_output><jupyter_text>Ojo!: Definir train/dev/test* https://scikit-learn.org/stable/modules/classes.htmlmodule-sklearn.model_selection* http://scikit.ml/stratification.html![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/data_splits.png?raw=true) design matrix ($X$ y $y$)<jupyter_code>features = ['DER_mass_MMC', 'DER_mass_transverse_met_lep',
       'DER_mass_vis', 'DER_pt_h', 'DER_deltaeta_jet_jet', 'DER_mass_jet_jet',
       'DER_prodeta_jet_jet', 'DER_deltar_tau_lep', 'DER_pt_tot', 'DER_sum_pt',
       'DER_pt_ratio_lep_tau', 'DER_met_phi_centrality',
       'DER_lep_eta_centrality', 'PRI_tau_pt', 'PRI_tau_eta', 'PRI_tau_phi',
       'PRI_lep_pt', 'PRI_lep_eta', 'PRI_lep_phi', 'PRI_met', 'PRI_met_phi',
       'PRI_met_sumet', 'PRI_jet_num', 'PRI_jet_leading_pt',
       'PRI_jet_leading_eta', 'PRI_jet_leading_phi', 'PRI_jet_subleading_pt',
       'PRI_jet_subleading_eta', 'PRI_jet_subleading_phi', 'PRI_jet_all_pt']

weights = df['Weight'].values
Y = df[target_col].values
X = df[features].values
from sklearn.model_selection import train_test_split
train_size = 0.3
X_train,X_test,Y_train,Y_test,w_train,w_test = train_test_split(X,Y,weights,train_size=train_size)
print(X_train.shape,Y_train.shape,w_train.shape)
print(X_test.shape,Y_test.shape,w_test.shape)<jupyter_output><empty_output><jupyter_text>ML: modelos lineales Regression logistica**Modelo:** $M_{W,b}(x) = g(W x + b) = g(\sum \theta_i x_i +b)$ con $g(z)=sigmoid(z)=\frac{1}{1+e^{-z}}$Nuestro model ($f(x)$) tiene la forma$$ y \approx f(x) = g(W\cdot x +b) $$donde $g(z)=sigmoid(z)=\frac{1}{1+e^{-z}}$, y para una coleccion de pares de inputs y targets $\{(x_i,y_i)\}$, queremos minimizar:$$ \sum_i^n \| y_i - f(x_i) \|^2$$<jupyter_code>from sklearn.linear_model import LogisticRegression
model = LogisticRegression(verbose=1)
model.fit(X_train,Y_train)<jupyter_output><empty_output><jupyter_text>Predecir con el modelo<jupyter_code>#Y_pred_train = model.predict(X_train) # predice la clase
#Y_pred_test = model.predict(X_test) # predice la clase
p_tol = 0.5
Y_pred_train = model.predict_proba(X_train)[:,1] > p_tol
Y_pred_test = model.predict_proba(X_test)[:,1] > p_tol
higgs_score.evaluate(Y_train, Y_pred_train, w_train, Y_test, Y_pred_test, w_test)<jupyter_output><empty_output><jupyter_text>Interpretar variables<jupyter_code>print('a = {}'.format(model.coef_))
print('b = {}'.format(model.intercept_))
sns.distplot(model.coef_,kde=False)
plt.show()
sns.barplot(x=features,y=model.coef_.ravel())
plt.xticks(rotation=90)
plt.show()<jupyter_output><empty_output><jupyter_text>Ojo: Preprocessing, standard scalinghttps://scikit-learn.org/stable/modules/classes.htmlmodule-sklearn.preprocessing<jupyter_code>from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
old_x = df['DER_mass_transverse_met_lep'].values.reshape(-1,1)
scaler.fit(old_x)
new_x = scaler.transform(old_x)
sns.distplot(old_x)
plt.show()
sns.distplot(new_x)
plt.show()<jupyter_output><empty_output><jupyter_text>Curva roc<jupyter_code>from sklearn.metrics import roc_curve, auc

fpr, tpr, thresholds = roc_curve(Y_test, Y_pred_test)
roc_auc = auc(fpr, tpr)
plt.plot(fpr, tpr, lw=1, alpha=0.3, label=str(model.__class__.__name__))
plt.plot(np.linspace(0,1,100),np.linspace(0,1,100), lw=2, alpha=0.3, label='Suerte')
plt.legend(loc='lower right')
plt.xlim([0,1])
plt.ylim([0,1])
plt.xlabel('Falsos Positivos')
plt.ylabel('Falsos Negativos')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Random Forest![](https://github.com/beangoben/deep_learning_escuela_verano_19/blob/master/media/dia2/rf.jpg?raw=true)<jupyter_code>from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
model.fit(X_train,Y_train)
#Y_pred_train = model.predict(X_train) # predice la clase
#Y_pred_test = model.predict(X_test) # predice la clase
p_tol = 0.5
Y_pred_train = model.predict_proba(X_train)[:,1] > p_tol
Y_pred_test = model.predict_proba(X_test)[:,1] > p_tol
higgs_score.evaluate(Y_train, Y_pred_train, w_train, Y_test, Y_pred_test, w_test)<jupyter_output><empty_output><jupyter_text>Feature importance<jupyter_code>sns.barplot(x=features,y=model.feature_importances_)
plt.xticks(rotation=90)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1403657118697812, 'fraction_numerical': 0.011943769157594335, 'mean_word_length': 7.395740905057675, 'pattern_counts': {'":': 2, '<': 61, '<?xml version=': 0, '>': 68, 'https://': 17, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '165353', 'license': 'MIT', 'n_tokens_mistral': 3727, 'n_tokens_neox': 3630, 'n_words': 615, 'path': 'notebooks/1_ML_dia_a_dia.ipynb', 'repo_name': 'mexicancoderalonso/ML-DL', 'size': '9500'}