<jupyter_start><jupyter_text>Virtual DriftDrift magnitude metrics       Computes drift magnitude metrics between base dataset t and dataset u.  Metrics:- TVD (Total Variation Distance)- Helinger- KL Divergence  **Steps**1. [Data exploration](Data-exploration)2. [Importing the function](Importing-the-function)3. [Running the function locally](Running-the-function-locally)  **Data exploration**<jupyter_code># Scikit-learn's wine dataset
from sklearn.datasets import load_wine

wine = load_wine()
print(wine["DESCR"])
wine_t_path = 'https://s3.wasabisys.com/iguazio/data/function-marketplace-data/virtual_drift/wine_t.pq'
wine_u_path = 'https://s3.wasabisys.com/iguazio/data/function-marketplace-data/virtual_drift/wine_u.pq'
wine_t=pd.read_parquet(wine_t_path)
wine_u=pd.read_parquet(wine_u_path)
print(f'wine_t and wine_u are generated from the wine dataset, where wine_t is the entire dataset while wine_u is a sample (50%) of the entire dataset. \n\
wine_t shape is {wine_t.shape[0]} and wine_u shape is {wine_u.shape[0]} \n\n')
wine_t.head()<jupyter_output>wine_t and wine_u are generated from the wine dataset, where wine_t is the entire dataset while wine_u is a sample (50%) of the entire dataset. 
wine_t shape is 178 and wine_u shape is 89<jupyter_text>**Importing the function**<jupyter_code>import mlrun

# Importing the function
mlrun.set_environment(project='function-marketplace')

fn = mlrun.import_function("hub://virtual_drift")
fn.apply(mlrun.auto_mount())<jupyter_output>> 2021-10-26 13:45:22,345 [info] created and saved project function-marketplace<jupyter_text>**Running the function locally**<jupyter_code>import os 

container = os.path.join('/',os.environ['V3IO_HOME'].split('/')[0])
user = os.environ["V3IO_USERNAME"]
rel_path = os.getcwd()[6:] + '/artifacts'
tsdb_path = os.path.join(user,rel_path) + "/output_tsdb"
virtual_drift_run=fn.run(params={'label_col': 'y',
                                 'results_tsdb_container': container[1:],
                                 'results_tsdb_table': tsdb_path},
                         inputs={'t': wine_t_path,
                                 'u': wine_u_path},
                         artifact_path=os.getcwd(),
                         local=True)
virtual_drift_run.artifact('class_u_pdf').show()
virtual_drift_run.artifact('class_t_pdf').show()
import v3io_frames as v3f
client = v3f.Client(os.environ["V3IO_FRAMESD"],container=container[1:])
client.read(backend='tsdb',table=tsdb_path)<jupyter_output>Warning - Server version '0.8.14' is different from client version '0.9.4'. Some operations may not work as expected.
{'fraction_non_alphanumeric': 0.12264513648596694, 'fraction_numerical': 0.019223375624759707, 'mean_word_length': 4.420833333333333, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 11, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '32424', 'license': 'Apache-2.0', 'n_tokens_mistral': 917, 'n_tokens_neox': 868, 'n_words': 190, 'path': 'virtual_drift/virtual_drift.ipynb', 'repo_name': 'yonishelach/functions', 'size': '2601'}