<jupyter_start><jupyter_text>Deploy machine learning models to Azuredescription: (preview) deploy your machine learning or deep learning model as a web service in the Azure cloud.  Connect to your workspace<jupyter_code>from azureml.core import Workspace

ws = Workspace.from_config()
ws<jupyter_output><empty_output><jupyter_text>Register your modelA registered model is a logical container stored in the cloud, containing all files located at `model_path`, which is associated with a version number and other metadata.  Register a model from a local fileYou can register a model by providing the local path of the model. You can provide the path of either a folder or a single file on your local machine.<jupyter_code>import urllib.request
from azureml.core.model import Model

# Download model
urllib.request.urlretrieve("https://aka.ms/bidaf-9-model", "model.onnx")

# Register model
model = Model.register(ws, model_name="bidaf_onnx", model_path="./model.onnx")<jupyter_output><empty_output><jupyter_text>Define an inference configurationThe inference configuration below specifies that the machine learning deployment will use the file echo_score.py in the ./source_dir directory to process incoming requests and that it will use the Docker image with the Python packages specified in the project_environment environment.<jupyter_code>from azureml.core import Environment
from azureml.core.model import InferenceConfig

env = Environment(name="project_environment")
dummy_inference_config = InferenceConfig(
    environment=env,
    source_directory="./source_dir",
    entry_script="./echo_score.py",
)<jupyter_output><empty_output><jupyter_text>Define a deployment configuration<jupyter_code>from azureml.core.webservice import LocalWebservice

deployment_config = LocalWebservice.deploy_configuration(port=6789)<jupyter_output><empty_output><jupyter_text>Deploy your machine learning modelA deployment configuration specifies the amount of memory and cores to reserve for your webservice will require in order to run, as well as configuration details of the underlying webservice. For example, a deployment configuration lets you specify that your service needs 2 gigabytes of memory, 2 CPU cores, 1 GPU core, and that you want to enable autoscaling.The options available for a deployment configuration differ depending on the compute target you choose. In a local deployment, all you can specify is which port your webservice will be served on.<jupyter_code>service = Model.deploy(
    ws,
    "myservice",
    [model],
    dummy_inference_config,
    deployment_config,
    overwrite=True,
)
service.wait_for_deployment(show_output=True)
print(service.get_logs())<jupyter_output><empty_output><jupyter_text>Call into your model<jupyter_code>import requests
import json

uri = service.scoring_uri
requests.get("http://localhost:6789")
headers = {"Content-Type": "application/json"}
data = {
    "query": "What color is the fox",
    "context": "The quick brown fox jumped over the lazy dog.",
}
data = json.dumps(data)
response = requests.post(uri, data=data, headers=headers)
print(response.json())<jupyter_output><empty_output><jupyter_text>Define a new inference configurationNow that we've deployed with a dummy entry script, let's try using a real entry script. Notice the use of the AZUREML_MODEL_DIR environment variable to locate your registered model. Now that you've added some pip packages, you also need to update your inference configuration to add in those additional packages<jupyter_code>env = Environment(name="myenv")
python_packages = ["nltk", "numpy", "onnxruntime"]
for package in python_packages:
    env.python.conda_dependencies.add_pip_package(package)

inference_config = InferenceConfig(
    environment=env, source_directory="./source_dir", entry_script="./score.py"
)<jupyter_output><empty_output><jupyter_text>Deploy again and call your service<jupyter_code>service = Model.deploy(
    ws,
    "myservice",
    [model],
    inference_config,
    deployment_config,
    overwrite=True,
)
service.wait_for_deployment(show_output=True)
print(service.get_logs())<jupyter_output><empty_output><jupyter_text>Then ensure you can send a post request to the service:<jupyter_code>import requests

uri = service.scoring_uri

headers = {"Content-Type": "application/json"}
data = {
    "query": "What color is the fox",
    "context": "The quick brown fox jumped over the lazy dog.",
}
data = json.dumps(data)
response = requests.post(uri, data=data, headers=headers)
print(response.json())<jupyter_output><empty_output><jupyter_text>Re-deploy to cloudOnce you've confirmed your service works locally and chosen a remote compute target, you are ready to deploy to the cloud.Change your deploy configuration to correspond to the compute target you've chosen, in this case Azure Container Instances.<jupyter_code>from azureml.core.webservice import AciWebservice

deployment_config = AciWebservice.deploy_configuration(
    cpu_cores=0.5, memory_gb=1, auth_enabled=True
)<jupyter_output><empty_output><jupyter_text>Deploy your service again<jupyter_code>service = Model.deploy(
    ws,
    "myservice",
    [model],
    inference_config,
    deployment_config,
    overwrite=True,
)
service.wait_for_deployment(show_output=True)
print(service.get_logs())<jupyter_output><empty_output><jupyter_text>Call your remote webserviceWhen you deploy remotely, you may have key authentication enabled. The example below shows how to get your service key with Python in order to make an inference request.<jupyter_code>import requests
import json
from azureml.core import Webservice

service = Webservice(workspace=ws, name="myservice")
scoring_uri = service.scoring_uri

# If the service is authenticated, set the key or token
key, _ = service.get_keys()

# Set the appropriate headers
headers = {"Content-Type": "application/json"}
headers["Authorization"] = f"Bearer {key}"

# Make the request and display the response and logs
data = {
    "query": "What color is the fox",
    "context": "The quick brown fox jumped over the lazy dog.",
}
data = json.dumps(data)
resp = requests.post(scoring_uri, data=data, headers=headers)
print(resp.text)
print(service.get_logs())<jupyter_output><empty_output><jupyter_text>Delete resources<jupyter_code>service.delete()
model.delete()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08366281828246086, 'fraction_numerical': 0.0023722916337181716, 'mean_word_length': 6.2027334851936216, 'pattern_counts': {'":': 9, '<': 53, '<?xml version=': 0, '>': 53, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '73806', 'license': 'MIT', 'n_tokens_mistral': 1847, 'n_tokens_neox': 1798, 'n_words': 599, 'path': 'python-sdk/tutorials/deploy-local/1.deploy-local.ipynb', 'repo_name': 'daholste/azureml-examples', 'size': '6323'}