<jupyter_start><jupyter_text>!Link to challenge! https://www.kaggle.com/t/b3ced76a60b94572a90740756f778fc8  MetricFor binary classification with a true label y $\in \{0,1\}$ and a probability estimate p = $\operatorname{Pr}(y = 1)$, the log loss per sample is the negative log-likelihood of the classifier given the true label:$$L_{\log}(y, p) = -\log \operatorname{Pr}(y|p) = -(y \log (p) + (1 - y) \log (1 - p))$$$This extends to the multiclass case as follows. Let the true labels for a set of samples be encoded as a 1-of-K binary indicator matrix Y, i.e., $y_{i,k} = 1$ if sample i has label k taken from a set of K labels. Let P be a matrix of probability estimates, with $p_{i,k} = \operatorname{Pr}(t_{i,k} = 1)$. Then the log loss of the whole set is$$L_{\log}(Y, P) = -\log \operatorname{Pr}(Y|P) = - \frac{1}{N} \sum_{i=0}^{N-1} \sum_{k=0}^{K-1} y_{i,k} \log p_{i,k}$$  Grading  Firstly, to get any mark, you must beat medium baseline scoreYour grade after challenge ends will be calculated as this:$$Grade = \frac{score - mid\_baseline\_score}{\1\_score - mid\_baseline\_score} * 10$$where score will be taken from private part results.  AboutIn this notebook we prepare a simple solution.<jupyter_code>import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.metrics import log_loss<jupyter_output><empty_output><jupyter_text>Read training and test files<jupyter_code>data = pd.read_csv('training.csv')
test = pd.read_csv('test.csv')
data.head()
test.head()<jupyter_output><empty_output><jupyter_text>Look at the labels set<jupyter_code>set(data.Label)<jupyter_output><empty_output><jupyter_text>Define training featuresExclude `Label` from the features set<jupyter_code>features = list(set(data.columns) - {'Label'})
features<jupyter_output><empty_output><jupyter_text>Divide training data into 2 parts<jupyter_code>training_data, validation_data = train_test_split(data, random_state=11, train_size=0.10)
len(training_data), len(validation_data)<jupyter_output><empty_output><jupyter_text>Simple logistic regression forest from `sklearn` trainingtrain multiclassification model<jupyter_code>from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import GridSearchCV
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

X_train = scaler.fit_transform(training_data[features])
%%time

clf = LogisticRegression(penalty='l2', n_jobs=-1, solver='saga', multi_class='multinomial', random_state=42)
param_grid = {'C': [0.1, 1]}

gscv = GridSearchCV(clf, param_grid, scoring='neg_log_loss', cv=3, n_jobs=-1, verbose=1)
gscv.fit(X_train, training_data.Label)
gscv.cv_results_<jupyter_output><empty_output><jupyter_text>Train best model:<jupyter_code>c = 1
clf = LogisticRegression(penalty='l2', C=c, n_jobs=-1, solver='saga', multi_class='multinomial')
clf.fit(X_train, training_data.Label)<jupyter_output>/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/sklearn/linear_model/sag.py:326: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge
  "the coef_ did not converge", ConvergenceWarning)<jupyter_text>Evaluate predictions on the validation sample<jupyter_code># predict each track
X_val = scaler.fit_transform(validation_data[features])
proba = clf.predict_proba(X_val)<jupyter_output><empty_output><jupyter_text>Log loss on the cross validation sample<jupyter_code>log_loss(validation_data.Label, proba)<jupyter_output><empty_output><jupyter_text>Prepare submission to kaggle<jupyter_code># predict test sample
X_test = scaler.fit_transform(test[features])
kaggle_proba = clf.predict_proba(X_test)
kaggle_ids = test.ID
from IPython.display import FileLink

def create_solution(ids, proba, names, filename='baseline.csv'):
    """saves predictions to file and provides a link for downloading """
    solution = pd.DataFrame({'ID': ids})
    
    for name in ['Ghost', 'Electron', 'Muon', 'Pion', 'Kaon', 'Proton']:
        solution[name] = proba[:, np.where(names == name)[0]]
    
    solution.to_csv('{}'.format(filename), index=False)
    return FileLink('{}'.format(filename))
    
create_solution(kaggle_ids, kaggle_proba, clf.classes_)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12476415094339623, 'fraction_numerical': 0.015330188679245283, 'mean_word_length': 6.682971014492754, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '190360', 'license': 'Apache-2.0', 'n_tokens_mistral': 1502, 'n_tokens_neox': 1433, 'n_words': 355, 'path': '03_lab/baseline.ipynb', 'repo_name': 'yandexdataschool/MLatImperial2018', 'size': '4240'}