<jupyter_start><jupyter_text>Basic operation and principle of DevCloudYou will learn the basic operation and the principle of DevCloud in this session.- How to run commands on **the development host**- How to run a commands (a job) on **the edge computing node**- How to specify the edge computing node to run a job DevCloud for Edge consists of 3 groups of servers and 1 storage1. Development servers2. Edge computing node servers3. Job queueing servers4. 50GB of cloud storageDevelopment server runs Jupyter notebook. It's a Xeon server and the machine your are operating now.  Edge computing node is a group of computers. The edge nodes are in wide variety of hardware configurations and you can pick one and run your job for function and performance testing. ---- 1. Running several commands on the development host (a Xeon server)The commands starting with `!` will be passed to the OS shell and get run. You can run any generic Linux commands such as `pwd` and `ls`.  The commands you run will be executed on a Xeon server which is running this Jupyter notebook.<jupyter_code>!pwd
!ls -l
!echo Hello! I like OpenVINO<jupyter_output><empty_output><jupyter_text>---- 2. Sending a simple job to an **edge computing node**Using a simple job script and learn the basic operation to run a job on the edge computing node.   Let's create a simple job script and run it on the edge computing node.  The contents of the cell starting with `%%writefile ` will be wrote out to a file. You can create any text file in this manner.  The job script is a Linux shell script. You can use any generic Linux shell script syntax in the script.<jupyter_code>%%writefile job.sh
echo -n 'start dir='
pwd
cd devcloud-workshop-en
pwd
ls -l
echo Hello! I like OpenVINO<jupyter_output><empty_output><jupyter_text>Let's check the contents of the script just created.<jupyter_code>!cat job.sh<jupyter_output><empty_output><jupyter_text>Now you have created a job file. Let's send the job to the queue server with the `qsub` command. The `job` will be sent to one of an empty edge node and get run.  The `qsub` command can take several option parameters but here we submit the job with no options.  After you submit a job, the `job ID` will be displayed. The first digits are the `job number` of the job you have just submitted. |Option|Description||:--|:--||`-l` resource_name[=val][,resource_val[=val]]|Limits the compute node to run the job (=specify the node)||`-o` file|STDOUT log file||`-e` file|STDERR log file||`-N` job_name|Job name||`-d` working_dir|Working directory||`-F` args_for_job|Parameters for the job script|<jupyter_code>!qsub job.sh<jupyter_output><empty_output><jupyter_text>The status of the job can be checked with the `qstat` command. You can specify the job name with `-N` option, otherwise the job script file name will be used as the job name.  Run this cell repeatedly until the job is completed (until the job disappears).<jupyter_code>!qstat<jupyter_output><empty_output><jupyter_text>Check if the job output files are created properly (`job.sh.o`).<jupyter_code>!ls -l job.sh.o*<jupyter_output><empty_output><jupyter_text>Check the contents of the job output file. The `job` part in the output file name will be changed every time you run the job. You need to modify the `job` portion accordingly to check the result. The `job` is displayed afte you run the `qsub` command.  Example of a job output file name => `job.sh.o22345`<jupyter_code>!cat job.sh.o<job#><jupyter_output><empty_output><jupyter_text>Now you have learnt the basic operation of DevCloud and how to submit a job to an edge computing node. Here's the summary of basic flow of operation.  1. Create a job script file to submit (a shell script)2. Submit the job script with `qsub` command3. Check the status of the job with `qstat` command (`Q`=`Queued`, `R`=`Running`, `E`=`Error`)4. Check the job output in `.o`  **Extra assignment** Modify `job.sh` and check what you can do with it.  Hints for developing a job- The working directory at the edge computing node side is your home directory (`~`, `$HOME`, `/home/u123456`)  - Be aware that it's not the same as where you submit the job in the development server- The same storage is visible from both development server side and the edge computing node side. You don't need to transfer the files to be used in your job- The contents of the storage is protected. It's not visible to the other users nor even Intel operators ---- 3. Run a job on a specific edge computing node  Display the list of available edge computing nodesFirst, let's list the available edge computing nodes. The `pbsnodes` command will list the nodes.- **Node name** - Each `properties=` represents the set of node names of a edge node - For instance, you can understand there's 13 nodes in the same configuration are available because `uniq` command consolidates and count the same outputs  >`  13      properties = idc002mx8,compnode,iei,tank-870,intel-core,i5-6500te,skylake,intel-hd-530,ram8gb,net1gbe,hddl-r,iei-mustang-v100-mx8` - You can specify the node with any one of comma-separated node names in the line. For instance, you can use any one of `idc002mx8`, `intel-core`, or `intel-hd-530` to specify this node. The job will be run on one of nodes which has the name specified. The node name `intel-core` is shared among multiple nodes and the job will be run on one of them. If you specify more specific (and narrower) name, you can control the node to run your job a more precisely but your job will have a higher chance to wait for a longer time.<jupyter_code>!pbsnodes | grep "properties =" | sort | uniq -c<jupyter_output><empty_output><jupyter_text>Submit a job to a specific edge computing node- `qsub` command - You can specify the edge compute node with `-l` (limit) option in `qsub` command. You can use the `-l` option like `-l nodes=1:node_name`. For example, you can specify `qsub -l nodes=1:skylake job.sh` when you want to run the job on a `skylake` node.  - We'll re-use the `job.sh` job script file which we have created in the previous section.- Log file name - Here, we'll auto generate the job file name using a Python snippet from the `job ID` (a list of string like `['27214.v-qsvr-1.devcloud-edge']`) returned from the `qsub` command- Job completion detection - You run `qstat` command multiple times manually to know the completion of a job in previous section. Here, we'll use a Python snippet to detect it automatically<jupyter_code># submit a job (specifying i5-6500te as the edge node to run it)
job_id=!qsub -l nodes=1:i5-6500te job.sh

# generate log file name from job_id
job_num = job_id[0].split('.')[0]
log_file='job.sh.o'+job_num
err_file='job.sh.e'+job_num
print('job_id={}, log_file={}'.format(job_id, log_file))

import time
def waitForJobCompletion(jobNumber):
    print('Waiting for job completion...', end='')
    running=True
    while running:
        time.sleep(1)
        running=False
        status_list=!qstat         # Check job status
        for status in status_list:
            if jobNumber in status:    # if job_num is found in the status list, the job is still running
                running = True
        print(status.split()[4], end='')
    print('...Job {} completed'.format(job_num))    

# wait for the job to complete
waitForJobCompletion(job_num)<jupyter_output><empty_output><jupyter_text>Check the contents of the log fileCheck the line including `Resources:` to confirm if your job run on the specified edge computing node.<jupyter_code># The log_file is set in the previous cell
import os
os.environ['log_file']=log_file

!cat $log_file<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08125081774172445, 'fraction_numerical': 0.010597932748920581, 'mean_word_length': 4.432835820895522, 'pattern_counts': {'":': 0, '<': 42, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '247638', 'license': 'Apache-2.0', 'n_tokens_mistral': 2261, 'n_tokens_neox': 2172, 'n_words': 1086, 'path': 'devcloud-basic.ipynb', 'repo_name': 'yas-sim/devcloud-workshop-en', 'size': '7643'}