<jupyter_start><jupyter_text>ESML - `AutoMLFactory` and `ComputeFactory` PROJECT + DATA CONCEPTS + ENTERPRISE Datalake Design + DEV->PROD MLOps- `1)ESML Project`: The ONLY thing you need to remember is your `Project number` (and `BRONZE, SILVER, GOLD` concept )   -  ...`read earlier notebook ENTERPRISE Deployment of Models & Governance - MLOps  at scale- `3) DEV->TEST-PROD` (configs, compute, performance)    - ESML has config for 3 environemnts: Easy DEPLOY model across subscriptions and Azure ML Studio workspaces         - Save costs & time:             - `DEV` has cheaper compute performance for TRAIN and INFERENCE (batch, AKS)            - `DEV` has Quick-debug ML training (fast training...VS good scoring in TEST and PROD)        - How? ESML `AutoMLFactory` and `ComputeFactory`        - Where to config these?            - settings/dev_test_prod/`dev_test_prod_settings.json`            - settings/dev_test_prod/`train/*/automl/*`  Azure ML Studio Workspace- ESML will `Automap` and `Autoregister` Azure ML Datasets as: `IN, SILVER, BRONZE, GOLD`<jupyter_code>import repackage
repackage.add("../azure-enterprise-scale-ml/esml/common/")
from esml import ESMLDataset, ESMLProject

p = ESMLProject() # Will search in ROOT for your copied SETTINGS folder '../../../settings', you should copy template settings from '../settings'
p.active_model = 11
p.ws = p.get_workspace_from_config() #2) Load DEV or TEST or PROD Azure ML Studio workspace
p.inference_mode = False
datastore = p.init()<jupyter_output><empty_output><jupyter_text>ESML `GOLD` Dataset<jupyter_code>ds_01 = p.DatasetByName("ds01_diabetes")
print(ds_01.InData.name)
print(ds_01.Bronze.name)
print(ds_01.Silver.name)
#print(p.Gold.name)
df_01 = ds_01.Silver.to_pandas_dataframe() 

ds_02 = ds_01 = p.DatasetByName("ds02_other")
df_02 = ds_02.Silver.to_pandas_dataframe()
df_gold1_join = df_01.join(df_02) # left join -> NULL on df_02
print("Diabetes shape: ", df_01.shape)
print(df_gold1_join.shape)
ds_gold_v1 = p.save_gold(df_01)<jupyter_output><empty_output><jupyter_text>Look at `GOLD` vLatest<jupyter_code>import pandas as pd 
df = p.Gold.to_pandas_dataframe()
df.head()
train, validate, test = p.split_gold_3(0.6, "Y") # Also registers the datasets in AZURE as M03_GOLD_TRAIN | M03_GOLD_VALIDATE | M03_GOLD_TEST<jupyter_output><empty_output><jupyter_text>3) ESML TRAIN model -> See other notebook `esml_howto_2_train.ipynb`- `AutoMLFactory, ComputeFactory`- Get `Train COMPUTE` for `X` environment- Get `Train Hyperparameters` for `X` environment (less crossvalidations in DEV etc)  4a) ESML Scoring compare: Promote model or not? Register- `IF` newly trained model in `current` environment scores BETTER than existing model in `target` environment, then `new model` can be registered and promoted.-  `ValidationSet` comparison of offline/previous `AutoML run` for `DEV` environment- For `DEV`, `TEST` or `PROD` environment- Future roadmap: Also include `TestSet SCORING` comparison<jupyter_code>from baselayer_azure_ml import AutoMLFactory
p.dev_test_prod = "dev" # Current env, new unregistered model A to validate
target_env = "test" # Target env. Existing registered model B - Does Model A score better than Model B?

print("SCORING DRIFT: If new model scores better in DEV (new data, or new code), we can promote this to TEST & PROD \n")

promote, m1_name, r1_id, m2_name, r2_run_id = AutoMLFactory(p).compare_scoring_current_vs_new_model(target_env)

print("New Model: {} in environment {}".format(m1_name, p.dev_test_prod))
print("Existing Model: {} in environment {}".format(m2_name,target_env))

if (promote and p.dev_test_prod == target_env):# Can only register a model in same workspace (test->test) - need to retrain if going from dev->test
    AutoMLFactory(p).register_active_model(target_env)<jupyter_output>SCORING DRIFT: If new model scores better in DEV (new data, or new code), we can promote this to TEST & PROD 

Loading AutoML config settings from: dev
Compare model version in DEV with latest registered in TEST subscription/workspace

WARNING:root:The version of the SDK does not match the version the model was trained on.
WARNING:root:The consistency in the result may not be guaranteed.
WARNING:root:Package:azureml-automl-core, training version:1.30.0, current version:1.26.0
Package:azureml-automl-runtime, training version:1.30.0, current version:1.26.0
Package:azureml-core, training version:1.30.0, current version:1.26.0
Package:azureml-dataprep, training version:2.15.1, current version:2.13.2
Package:azureml-dataprep-native, training version:33.0.0, current version:32.0.0
Package:azureml-dataprep-rslex, training version:1.13.0, current version:1.11.2
Package:azureml-dataset-runtime, training version:1.30.0, current version:1.26.0
Package:azureml-defaults, training version:1.30.0, cu[...]<jupyter_text>START 2) TEST env - `register a model` starting "offline", not an active training run?  Alt 1) No ESMLProject dependency<jupyter_code>import repackage
repackage.add("../azure-enterprise-scale-ml/esml/common/")
from azureml.core import Workspace
from baselayer_azure_ml import AutoMLFactory

ws = p.get_workspace_from_config() # Simulate you init a Azure ML Workspace
AutoMLFactory().register_active_model_in_ws(ws,"dev") # Simulate you register a model in Workspace<jupyter_output>model.version 4
Model name AutoMLa4b60322a0 is registered.<jupyter_text>Alt 2) ESMLProject dependency: `ENVIRONMENT Self aware` and `config aware` - More `Future proof`: Features such as "able to register trained model in TARGET - from TEST to PROD without retraining"<jupyter_code>sys.path.append(os.path.abspath("../common/"))  # NOQA: E402
from esml import ESMLDataset, ESMLProject
from baselayer_azure_ml import AutoMLFactory
from azureml.core import Workspace

ws = p.get_workspace_from_config()

p = ESMLProject() # Makes it "environment aware (dev,test,prod)", and "configuration aware"
p.init(ws) 
p.dev_test_prod = "dev"
# ....train model....

model = AutoMLFactory(p).register_active_model(p.dev_test_prod)<jupyter_output>...
Using GEN2 as Datastore
ds01_diabetes
ds02_other

####### Automap & Autoregister - SUCCESS!
1) Auto mapped 2 ESML Dataset with registered Azure ML Datasets (potentially all 3: IN,BRONZE, SILVER) in Datastore project002lake 

Dataset 'ds01_diabetes' status:
 - IN_Folder_has_files
 - BRONZE_Folder_has_files
 - SILVER_Folder_has_files
Dataset 'ds02_other' status:
 - IN_Folder_has_files
 - BRONZE_Folder_has_files
 - SILVER_Folder_has_files

2) Registered each Dataset with suffixes (_IN_CSV, _BRONZE, _SILVER) 
 Tip: Use ESMLProject.Datasets list or .DatasetByName(myDatasetName) to read/write
#######
model.version 5
Model name AutoMLa4b60322a0 is registered.<jupyter_text>..Model compared, promoted, register - ready for deployment  4b) ESML Loadtesting performance- Using `GOLD_TEST` TestSet for AutoML to see which algorithm that is fastest, smallest size footprint- For `DEV`, `TEST` or `PROD` environment<jupyter_code>label = p.active_model["label"]
train, validate, test = p.split_gold_3() # Save as M03_GOLD_TRAIN | M03_GOLD_VALIDATE | M03_GOLD_TEST  # Alt: train_data, test_data = p.Gold.random_split(percentage=0.8, seed=223) 
test.head()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10101844205890449, 'fraction_numerical': 0.023671896504266446, 'mean_word_length': 5.179421768707483, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '37177', 'license': 'MIT', 'n_tokens_mistral': 2516, 'n_tokens_neox': 2421, 'n_words': 707, 'path': 'copy_my_subfolders_to_my_grandparent/notebook_demos/esml_regression_3_compare_and_register.ipynb', 'repo_name': 'jostrm/azure-enterprise-scale-ml', 'size': '7266'}