<jupyter_start><jupyter_text>Use Automated Machine LearningThere are many kinds of machine learning algorithm that you can use to train a model, and sometimes it's not easy to determine the most effective algorithm for your particular data and prediction requirements. Additionally, you can significantly affect the predictive performance of a model by preprocessing the training data, using techniques such as normalization, missing feature imputation, and others. In your quest to find the *best* model for your requirements, you may need to try many combinations of algorithms and preprocessing transformations; which takes a lot of time and compute resources.Azure Machine Learning enables you to automate the comparison of models trained using different algorithms and preprocessing options. You can use the visual interface in [Azure Machine Learning studio](https://ml/azure.com) or the SDK to leverage this capability. The SDK gives you greater control over the settings for the automated machine learning experiment, but the visual interface is easier to use. Before you startIn addition to the latest version of the **azureml-sdk** and **azureml-widgets** packages, you'll need the **azureml-train-automl** package to run the code in this notebook. Run the cell below to verify that it is installed.<jupyter_code>import azureml.train.automl.utilities as automl_utils<jupyter_output>Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception (azureml-dataprep-rslex 1.2.3 (c:\applications\anaconda\lib\site-packages), Requirement.parse('azureml-dataprep-rslex<1.19.0a,>=1.18.0dev0'), {'azureml-dataprep'}).<jupyter_text>Connect to your workspaceWith the required SDK packages installed, now you're ready to connect to your workspace.> **Note**: If you haven't already established an authenticated session with your Azure subscription, you'll be prompted to authenticate by clicking a link, entering an authentication code, and signing into Azure.<jupyter_code>import azureml.core
from azureml.core import Workspace

# Load the workspace from the saved config file
ws = Workspace.from_config()
print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))<jupyter_output>Ready to use Azure ML 1.33.0 to work with wsag<jupyter_text>Prepare dataYou don't need to create a training script for automated machine learning, but you do need to load the training data. In this case, you'll use a dataset containing details of diabetes patients, and then split this into two datasets: one for training, and another for model validation.<jupyter_code>from azureml.core import Dataset

default_ds = ws.get_default_datastore()

if 'diabetes dataset' not in ws.datasets:
    default_ds.upload_files(files=['data/diabetes.csv', 'data/diabetes2.csv'], # Upload the diabetes csv files in /data
                        target_path='diabetes-data/', # Put it in a folder path in the datastore
                        overwrite=True, # Replace existing files of the same name
                        show_progress=True)

    #Create a tabular dataset from the path on the datastore (this may take a short while)
    tab_data_set = Dataset.Tabular.from_delimited_files(path=(default_ds, 'diabetes-data/*.csv'))

    # Register the tabular dataset
    try:
        tab_data_set = tab_data_set.register(workspace=ws, 
                                name='diabetes dataset',
                                description='diabetes data',
                                tags = {'format':'CSV'},
                                create_new_version=True)
        print('Dataset registered.')
    except Exception as ex:
        print(ex)
else:
    print('Dataset already registered.')


# Split the dataset into training and validation subsets
diabetes_ds = ws.datasets.get("diabetes dataset")
train_ds, test_ds = diabetes_ds.random_split(percentage=0.7, seed=123)
print("Data ready!")<jupyter_output>Uploading an estimated of 2 files
Uploading data/diabetes2.csv
Uploaded data/diabetes2.csv, 1 files out of an estimated total of 2
Uploading data/diabetes.csv
Uploaded data/diabetes.csv, 2 files out of an estimated total of 2
Uploaded 2 files
Dataset registered.
Data ready!<jupyter_text>Prepare computeOne of the benefits of cloud compute is that it scales on-demand, enabling you to provision enough compute resources to process multiple child-runs of an automated machine learning experiment in parallel.Use the following code to specify an Azure Machine Learning compute cluster (it will be created if it doesn't already exist)..> **Important**: Change *your-compute-cluster* to the name of your compute cluster in the code below before running it! Cluster names must be globally unique names between 2 to 16 characters in length. Valid characters are letters, digits, and the - character.<jupyter_code>from azureml.core.compute import ComputeTarget, AmlCompute
from azureml.core.compute_target import ComputeTargetException

cluster_name = "agcluster"

try:
    # Check for existing compute target
    training_cluster = ComputeTarget(workspace=ws, name=cluster_name)
    print('Found existing cluster, use it.')
except ComputeTargetException:
    # If it doesn't already exist, create it
    try:
        compute_config = AmlCompute.provisioning_configuration(vm_size='STANDARD_DS11_V2', max_nodes=2)
        training_cluster = ComputeTarget.create(ws, cluster_name, compute_config)
        training_cluster.wait_for_completion(show_output=True)
    except Exception as ex:
        print(ex)<jupyter_output>InProgress.....
SucceededProvisioning operation finished, operation "Succeeded"
Succeeded
AmlCompute wait for completion finished

Minimum number of nodes requested have been provisioned<jupyter_text>> **Note**: Compute instances and clusters are based on standard Azure virtual machine images. For this exercise, the *Standard_DS11_v2* image is recommended to achieve the optimal balance of cost and performance. If your subscription has a quota that does not include this image, choose an alternative image; but bear in mind that a larger image may incur higher cost and a smaller image may not be sufficient to complete the tasks. Alternatively, ask your Azure administrator to extend your quota. Configure automated machine learningNow you're ready to configure the automated machine learning experiment.One of the most important configuration settings is the metric by which model performance should be evaluated. You can retrieve a list of the metrics that are calculated by automated machine learning for a particular type of model task (classification or regression) like this:<jupyter_code>import azureml.train.automl.utilities as automl_utils

for metric in automl_utils.get_primary_metrics('classification'):
    print(metric)<jupyter_output>AUC_weighted
average_precision_score_weighted
norm_macro_recall
accuracy
precision_score_weighted<jupyter_text>Having decided the metric you want to optimize (in this example, *AUC_weighted*), you can configure the automated machine learning run. To do this, you'll need an AutoML configuration that specifies the target metric as well as options like the data to use, how many combinations to try, and so on.> **Note**: In this example, you'll restrict the experiment to 4 iterations to reduce the amount of time taken. In reality, you'd likely try many more iterations.<jupyter_code>from azureml.train.automl import AutoMLConfig

automl_config = AutoMLConfig(name='Automated ML Experiment',
                             task='classification',
                             compute_target=training_cluster,
                             training_data = train_ds,
                             validation_data = test_ds,
                             label_column_name='Diabetic',
                             iterations=4,
                             primary_metric = 'AUC_weighted',
                             max_concurrent_iterations=2,
                             featurization='auto'
                             )

print("Ready for Auto ML run.")<jupyter_output>Ready for Auto ML run.<jupyter_text>Run an automated machine learning experimentOK, you're ready to go. Let's run the automated machine learning experiment.> **Note**: This may take some time!<jupyter_code>from azureml.core.experiment import Experiment
from azureml.widgets import RunDetails

print('Submitting Auto ML experiment...')
automl_experiment = Experiment(ws, 'mslearn-diabetes-automl-sdk')
automl_run = automl_experiment.submit(automl_config)
RunDetails(automl_run).show()
automl_run.wait_for_completion(show_output=True)<jupyter_output>Submitting Auto ML experiment...
Submitting remote run.<jupyter_text>View child run detailsWhen the experiment has completed, view the output in the widget, and click the run that produced the best result to see its details.Then click the link to view the experiment details in Azure Machine Learning studio and view the overall experiment details before viewing the details for the individual run that produced the best result. There's lots of information here about the performance of the model generated.You can also retrieve all of the child runs and view their metrics using the SDK:<jupyter_code>for run in automl_run.get_children():
    print('Run ID', run.id)
    for metric in run.get_metrics():
        print('\t', run.get_metrics(metric))<jupyter_output>Run ID AutoML_abfbb864-254a-49dd-ab01-9aae018ca8de_3
	 {'f1_score_macro': 0.400215895290784}
	 {'average_precision_score_micro': 0.899826619525177}
	 {'norm_macro_recall': 0.0}
	 {'recall_score_micro': 0.6672665916760405}
	 {'precision_score_micro': 0.6672665916760405}
	 {'precision_score_weighted': 0.4452447043669598}
	 {'AUC_macro': 0.9899685732874312}
	 {'weighted_accuracy': 0.8008628809959982}
	 {'f1_score_weighted': 0.534101392770513}
	 {'matthews_correlation': 0.0}
	 {'recall_score_weighted': 0.6672665916760405}
	 {'accuracy': 0.6672665916760405}
	 {'f1_score_micro': 0.6672665916760405}
	 {'balanced_accuracy': 0.5}
	 {'AUC_micro': 0.8848340864028666}
	 {'average_precision_score_weighted': 0.9905840031770856}
	 {'log_loss': 0.6230267775927263}
	 {'precision_score_macro': 0.33363329583802026}
	 {'average_precision_score_macro': 0.9884229823710426}
	 {'AUC_weighted': 0.9899685732874312}
	 {'recall_score_macro': 0.5}
	 {'confusion_matrix': 'aml://artifactId/ExperimentRun/dcid.AutoML_[...]<jupyter_text>Get the best runYou can retrieve the best-performing run, and view its details like this:<jupyter_code>!pip install azureml-train-automl-runtime
best_run, fitted_model = automl_run.get_output()
print(best_run)
print('\nBest Model Definition:')
print(fitted_model)

print('\nBest Run Metrics:')
best_run_metrics = best_run.get_metrics()
for metric_name in best_run_metrics:
    metric = best_run_metrics[metric_name]
    print(metric_name, metric)<jupyter_output>WARNING:root:The model you attempted to retrieve requires 'azureml-train-automl-runtime' to be installed at '==1.33.0'. Please install 'azureml-train-automl-runtime==1.33.0' (e.g. `pip install azureml-train-automl-runtime==1.33.0`) and then rerun the previous command.<jupyter_text>Finally, having found the best performing model, you can register it.<jupyter_code>from azureml.core import Model

# Register model
best_run.register_model(model_path='outputs/model.pkl', model_name='diabetes_model',
                        tags={'Training context':'Auto ML'},
                        properties={'AUC': best_run_metrics['AUC_weighted'], 'Accuracy': best_run_metrics['accuracy']})

# List registered models
for model in Model.list(ws):
    print(model.name, 'version:', model.version)
    for tag_name in model.tags:
        tag = model.tags[tag_name]
        print ('\t',tag_name, ':', tag)
    for prop_name in model.properties:
        prop = model.properties[prop_name]
        print ('\t',prop_name, ':', prop)
    print('\n')<jupyter_output>diabetes_model version: 1
	 Training context : Auto ML
	 AUC : 0.9904812577250306
	 Accuracy : 0.9520809898762654
{'fraction_non_alphanumeric': 0.07579017044514313, 'fraction_numerical': 0.03342710574218104, 'mean_word_length': 4.357712765957447, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 37, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '83552', 'license': 'MIT', 'n_tokens_mistral': 3666, 'n_tokens_neox': 3247, 'n_words': 1194, 'path': '12 - Use Automated Machine Learning.ipynb', 'repo_name': 'ambarishg/dp100', 'size': '12086'}