<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.Licensed under the MIT License.  Verify the NFS Setup in AMLArc  Prerequisites* [Setup Azure Arc-enabled Machine Learning Training and Inferencing on AKS on Azure Stack HCI](https://github.com/Azure/AML-Kubernetes/tree/master/docs/AKS-HCI/AML-ARC-Compute.md)* [Setup NFS Server on Azure Stack HCI and Use your Data and run managed Machine Learning Experiments On-Premises](https://github.com/Azure/AML-Kubernetes/tree/master/docs/AKS-HCI/Train-AzureArc.md)* (Optional) Upload some training data to NFS Server for verification  Initialize AzureML workspaceInitialize a [Workspace](https://docs.microsoft.com/azure/machine-learning/service/concept-azure-machine-learning-architectureworkspace) object from the existing workspace you created in the Prerequisites step. `Workspace.from_config()` creates a workspace object from the details stored in `config.json`. If you haven't done already please go to `config.json` file and fill in your workspace information.<jupyter_code>from azureml.core.workspace import Workspace,  ComputeTarget
from azureml.exceptions import ComputeTargetException

ws = Workspace.from_config()
print('Workspace name: ' + ws.name, 
      'Azure region: ' + ws.location, 
      'Subscription id: ' + ws.subscription_id, 
      'Resource group: ' + ws.resource_group, sep='\n')<jupyter_output><empty_output><jupyter_text>Setup compute targetIf you have done the compute target setup using [doc](https://github.com/Azure/AML-Kubernetes/tree/master/docs/AKS-HCI/AML-ARC-Compute.mdattach-your-azure-arc-enabled-cluster-to-your-azure-machine-learning-workspace-as-a-compute-target), you can replace the compute target name with `` or give via environment variable AML_COMPUTE_CLUSTER_NAME, and leave resource_id as it is. Below script will fetch the compute your created.Otherwise, provide both Arc K8S Resource Id and Compute Name to create a new compute target.* Find the Arc K8S Resource Id and replace the resource id below.  e.g. /subscriptions/``/resourceGroups/``/providers/Microsoft.Kubernetes/connectedClusters/``* Using 'kubectl create ns aml' to create a namespace in advance.* Replace the arc compute name `` or give via environment variable AML_COMPUTE_CLUSTER_NAME.<jupyter_code>from azureml.core.compute import KubernetesCompute
from azureml.core.compute import ComputeTarget
import os

ws = Workspace.from_config()

# choose a name for your Azure Arc-enabled Kubernetes compute
amlarc_compute_name = os.environ.get("AML_COMPUTE_CLUSTER_NAME", "<ArcComputeName>")

# resource ID for your Azure Arc-enabled Kubernetes cluster
resource_id = "/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.Kubernetes/connectedClusters/<clusterName>"

if amlarc_compute_name in ws.compute_targets:
   amlarc_compute = ws.compute_targets[amlarc_compute_name]
   if amlarc_compute and type(amlarc_compute) is KubernetesCompute:
      print("found compute target: " + amlarc_compute_name)
else:
   print("creating new compute target...")
   ns = "aml"
    
   instance_types = {
    "defaultInstanceType": {
      "nodeSelector": None,
      "resources": {
        "requests": {
          "cpu": "1",
          "memory": "4Gi",
          "nvidia.com/gpu": 0
        },
        "limits": {
          "cpu": "1",
          "memory": "4Gi",
          "nvidia.com/gpu": 0
        }
      }
    }
  }

   amlarc_attach_configuration = KubernetesCompute.attach_configuration(resource_id = resource_id, namespace = ns, default_instance_type="defaultInstanceType", instance_types = instance_types)
 
   amlarc_compute = ComputeTarget.attach(ws, amlarc_compute_name, amlarc_attach_configuration)

 
   amlarc_compute.wait_for_completion(show_output=True)
    
   # For a more detailed view of current KubernetesCompute status, use get_status()
   print(amlarc_compute.get_status().serialize())

print(f"compute target id in endpoint yaml: azureml:{amlarc_compute.name}, instance type name in endpoint yaml: {amlarc_compute.default_instance_type}")
from azureml.core.compute import KubernetesCompute

attach_name = amlarc_compute_name
arcK_target = KubernetesCompute(ws, attach_name)<jupyter_output><empty_output><jupyter_text>Configure the training job and submitThis experiment will list the contents of the NFS mounting point on training pods.  Create an experiement<jupyter_code>from azureml.core import Experiment

experiment_name = 'nfs-demo'

exp = Experiment(workspace=ws, name=experiment_name)<jupyter_output><empty_output><jupyter_text>Create an environment<jupyter_code># customized environment

from azureml.core.environment import Environment
from azureml.core.conda_dependencies import CondaDependencies
# to install required packages
env = Environment('tutorial-env')
cd = CondaDependencies.create(pip_packages=['azureml-dataset-runtime[pandas,fuse]', 'azureml-defaults'], conda_packages = ['scikit-learn==0.22.1'])

env.python.conda_dependencies = cd<jupyter_output><empty_output><jupyter_text>Configure the training job`` is the same as the mountPath defined in mount-config.yaml.<jupyter_code>from azureml.core import ScriptRunConfig

nfs_folder = "<MountPathOnTrainingPod>" # training data are saved to <mountPoint> (have to use / as the path separator)

args = ['--nfs-folder', nfs_folder]
script_folder =  "nfs_script"
src = ScriptRunConfig(source_directory=script_folder,
                      script='test.py', 
                      arguments=args,
                      compute_target=arcK_target,
                      environment=env)<jupyter_output><empty_output><jupyter_text>Submit the jobRun your experiment by submitting your ScriptRunConfig object. Note that this call is asynchronous.<jupyter_code>run = exp.submit(config=src)
run.wait_for_completion(show_output=True)  # specify True for a verbose log<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09164969450101833, 'fraction_numerical': 0.002036659877800407, 'mean_word_length': 4.958543983822042, 'pattern_counts': {'":': 11, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '219942', 'license': 'MIT', 'n_tokens_mistral': 1765, 'n_tokens_neox': 1695, 'n_words': 457, 'path': 'docs/AKS-HCI/nfs/Verify_NFS_Setup_in_AMLArc.ipynb', 'repo_name': 'snowei/AML-Kubernetes', 'size': '5892'}