<jupyter_start><jupyter_text>Rossmann:  all (Stage4 - publish)  Intro Keggle Kernel: https://www.kaggle.com/paso84/xgboost-in-python-with-rmspe  Usage **Input parameters**1. MODEL_TYPE: xgb or rf**Output**1. Publish data on mlflow server  Setup env  Set global variables<jupyter_code>!pwd
DATASETS_DIR = '../data'
MODELS_DIR = '../models'
METRICS_DIR = '../metrics'
NOTEBOOK_OUT_DIR = '../notebooks/outs'
# this cell is tagged `parameters`
MODEL_TYPE = 'rf'
MODEL_PKL = MODELS_DIR + '/model-' + MODEL_TYPE + '.pkl'
METRICS_OUT = METRICS_DIR + '/' + MODEL_TYPE + '.metrics'
STG_1_OUT_NBK = NOTEBOOK_OUT_DIR + '/stg_1_prepare_out.ipynb'
STG_2_OUT_NBK = NOTEBOOK_OUT_DIR + '/stg_2_train_' + MODEL_TYPE + '_out.ipynb'
STG_3_OUT_NBK = NOTEBOOK_OUT_DIR + '/stg_3_evaluate_'+ MODEL_TYPE + '_out.ipynb'
!cat {METRICS_OUT}<jupyter_output>rmspe(train): 0.026728006934463807
rmspe(valid): 0.06889040320666849<jupyter_text>Install required packages Se il notebook è eseguito su una macchina pulita installare i pacchetti necessari con i seguenti comandi ...<jupyter_code>#!curl https://raw.githubusercontent.com/andrea-gioia/boostrap.ai/master/???	 | bash
#!pip list<jupyter_output><empty_output><jupyter_text>Se il notebook è eseguito all'interno di un ambiente virtuale conda con tutti i pacchetti specificati nel file di requirements già installati fare solo un check eseguendo i seguenti comandi ...   Dump environment<jupyter_code>!python -V
!conda env list
conda env export
!pip list | grep mlflow
#!conda list
#!pip list<jupyter_output><empty_output><jupyter_text>Import required packages<jupyter_code>%load_ext autoreload
%autoreload 2

%matplotlib inline
from fastai.imports import *

import sys

import mlflow
import mlflow.sklearn<jupyter_output><empty_output><jupyter_text>Set random seed<jupyter_code># Set a seed value: 
seed_value= 42  


# Set `python` built-in pseudo-random generator at a fixed value: 
random.seed(seed_value) 

# Set `numpy` pseudo-random generator at a fixed value:
np.random.seed(seed_value) 

# Set `torch` pseudo-random generator at a fixed value:
torch.manual_seed(seed_value)
torch.backends.cudnn.deterministic = True 
torch.backends.cudnn.benchmark = False
    
if torch.cuda.is_available():
    torch.cuda.manual_seed(seed_value)
    torch.cuda.manual_seed_all(seed_value)<jupyter_output><empty_output><jupyter_text>Define shared functions<jupyter_code># nope<jupyter_output><empty_output><jupyter_text>Stage 3: publish  Prepare parameters<jupyter_code># Nothings to do ...<jupyter_output><empty_output><jupyter_text>Prepare metrics<jupyter_code>with open(METRICS_OUT) as f:
    content = f.readlines()
  
metrics = [];
for line in content:
    metrics.append( [line.split(':')[0].strip().replace("(", "_").replace(")", ""), float(line.split(':')[1].strip())] ) ;

for m in metrics:   
    print(m[0], m[1])<jupyter_output>rmspe_train 0.026728006934463807
rmspe_valid 0.06889040320666849<jupyter_text>Prepare artifacts<jupyter_code># Nothings to do ...
    
# Useful for multiple runs (only doing one run in this sample notebook)    
with mlflow.start_run():
    for m in metrics:   
        mlflow.log_metric(m[0], m[1])
    mlflow.log_param("model_type", MODEL_TYPE)
    mlflow.log_artifact(STG_1_OUT_NBK)
    mlflow.log_artifact(STG_2_OUT_NBK)
    mlflow.log_artifact(STG_3_OUT_NBK)
    mlflow.log_artifact(STG_1_OUT_NBK)
    mlflow.log_artifact(MODEL_PKL)
# FINE<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.115777194517352, 'fraction_numerical': 0.028871391076115485, 'mean_word_length': 5.686159844054581, 'pattern_counts': {'":': 0, '<': 35, '<?xml version=': 0, '>': 35, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '351208', 'license': 'MIT', 'n_tokens_mistral': 1397, 'n_tokens_neox': 1316, 'n_words': 264, 'path': 'misc/reproducibility/stg_4_publlish.ipynb', 'repo_name': 'Quantyca/demo-scene', 'size': '3429'}