<jupyter_start><jupyter_text>Azure ML WorkspaceThis notebook will define and create a workspace, create an environment,and deploy the models to an endpoint for consumption.  Importing librariesAll utilised libraries are imported at the beginning of the notebook to avoid out of scope errors.<jupyter_code>from azureml.core import Workspace, Environment, Model
from azureml.core.authentication import InteractiveLoginAuthentication
from azureml.core.conda_dependencies import CondaDependencies
from azureml.core.model import InferenceConfig
from azureml.core.webservice import AciWebservice
import json
import pandas as pd
from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Initialise workspaceInitialise a workspace using the values from config.jsonIf this file is not found, an exception occurs.When an exception is detected, a workspace using the defined details is created.<jupyter_code># An authentication error started occurring with the Azure Portal, this seems to fix it
# Issue found at https://github.com/Azure/MachineLearningNotebooks/issues/131

# NOTE: To use this system, a tenant_id value must be provided. This is specific to the Azure account holder.
auth = InteractiveLoginAuthentication(tenant_id="")

try:
# Try to load workspace
    wspace = Workspace.from_config(auth=auth)
except Exception as ex:
# Create new workspace if existing is not found

# NOTE: To use this system, a subscription_id value must be provided. This is specific to the Azure account holder.
    wspace = Workspace.create(name="classification-workspace",
                              subscription_id="",
                              resource_group="classification-resources",
                              create_resource_group=True,
                              location="uksouth")

# Write workspace config to a local file
    wspace.write_config()

print("Workspace successfully loaded")<jupyter_output><empty_output><jupyter_text>Create an environmentCreate an environment for the model to run on the Azure servers.This environment will be defined using Pip package version dependencies and Conda dependencies.When registered, the details will be fetched and stored to the /.environ folder.The inference config defines the scoring script to be utilised.<jupyter_code># Load conda dependencies required for this environment.
# Register environment for the workspace
env = Environment("env")

conda_dep = CondaDependencies.create(pip_packages=["pandas == 1.1.3",
                                                   "azureml-core",
                                                   "azureml-defaults >= 1.0.45"],
                                     conda_packages=["scikit-learn == 0.23.2"])

env.python.conda_dependencies = conda_dep

env.register(workspace=wspace)

print("Environment Registered")

# Get the environment created from the workspace and create a file to verify it has been defined
my_env = Environment.get(workspace=wspace, name="env")
my_env.save_to_directory("./environ", overwrite=True)

print("Environment Saved")

inference_config = InferenceConfig(entry_script="score.py",
                                   environment=my_env)<jupyter_output><empty_output><jupyter_text>Register the modelsRegister the saved models produced within the creation notebook for use on Azure ML.This cell will load the models from their binary files (.pkl) and register them for use on Azure ML.The framework that each model utilises has also been defined here for each model.<jupyter_code>knn_model = Model.register(workspace=wspace,
                       model_name="knn",
                       model_path="models/knn.pkl",
                       model_framework="ScikitLearn",
                       description="Tuned KNN for Genre Classification")

svm_model = Model.register(workspace=wspace,
                       model_name="svm",
                       model_path="models/svm.pkl",
                       model_framework="ScikitLearn",
                       description="Tuned SVM for Genre Classification")<jupyter_output><empty_output><jupyter_text>Set up Azure Container InstanceDefine ACI Webservice.This is the hardware configuration that the endpoint's processing will be carried out on.<jupyter_code>aci_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)<jupyter_output><empty_output><jupyter_text>Deploy the modelsDeploy the models onto the defined ACI container using the environment, inference config, and workspace.The deployment of these models into the workspace will create a machine learning instance in which the ACI is deployed.<jupyter_code>service = Model.deploy(workspace=wspace,
                       name="music-genre-service",
                       models=[knn_model, svm_model],
                       inference_config=inference_config,
                       deployment_config=aci_config,
                       overwrite=True)

service.wait_for_deployment(show_output=True)
print(service.get_logs())

print(service.scoring_uri)<jupyter_output><empty_output><jupyter_text>Test the web serviceTest the web service has successfully deployed using test data.Testing data that was stored in a CSV format from the model creation processwill be sent to the webservice as a test set.<jupyter_code># Reading the test set from file
data_df = pd.read_csv("testing_data.csv")
data_json_str = data_df.to_json(orient="records")

response_pred, response_acc= service.run(json.dumps(data_json_str))

response_pred_df = pd.DataFrame(response_pred)
response_acc_df = pd.DataFrame(response_acc, index=["Classification Accuracy", "AUROC", "F1"])

print(response_pred_df)
print(response_acc_df)

# As plot_confusion_matrix requires the classifier to run, the matrix will be built and plotted separately here
knn_conf = confusion_matrix(y_true=response_pred_df["Genre"], y_pred=response_pred_df["KNN Prediction"])
knn_conf_mat = ConfusionMatrixDisplay(confusion_matrix=knn_conf,
                       display_labels=["Blues", "Classical", "Country", "Disco", "HipHop",
                                       "Jazz", "Metal", "Pop", "Reggae", "Rock"])
knn_conf_mat.plot(xticks_rotation="vertical")
plt.title("KNN Confusion Matrix")


svm_conf = confusion_matrix(y_true=response_pred_df["Genre"], y_pred=response_pred_df["SVM Prediction"])
svm_conf_mat = ConfusionMatrixDisplay(confusion_matrix=svm_conf,
                       display_labels=["Blues", "Classical", "Country", "Disco", "HipHop",
                                       "Jazz", "Metal", "Pop", "Reggae", "Rock"])
svm_conf_mat.plot(xticks_rotation="vertical")
plt.title("SVM Confusion Matrix")<jupyter_output><empty_output><jupyter_text>Delete ACIDelete the ACI instance to clean up (This will also delete the endpoint).<jupyter_code>service.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0772588389349629, 'fraction_numerical': 0.002473446820893351, 'mean_word_length': 3.806993006993007, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 34, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '516862', 'license': 'MIT', 'n_tokens_mistral': 1879, 'n_tokens_neox': 1811, 'n_words': 554, 'path': 'azure-ml-workspace.ipynb', 'repo_name': 'JackW-Dev/Music-Genre-Classification', 'size': '6873'}