<jupyter_start><jupyter_text>Using FABRIC NVMe DevicesYour compute nodes can include fast NVMe storage devices. These devices are made available as FABRIC components and can be added to your nodes like any other component.This example notebook will demonstrate how to reserve and use FABRIC NVMe storage.  Configure the Environment<jupyter_code>import os
from fabrictestbed.slice_manager import SliceManager, Status, SliceState
import json
ssh_key_file_priv=os.environ['HOME']+"/.ssh/id_rsa"
ssh_key_file_pub=os.environ['HOME']+"/.ssh/id_rsa.pub"

ssh_key_pub = None
with open (ssh_key_file_pub, "r") as myfile:
    ssh_key_pub=myfile.read()
    ssh_key_pub=ssh_key_pub.strip()
credmgr_host = os.environ['FABRIC_CREDMGR_HOST']
print(f"FABRIC Credential Manager   : {credmgr_host}")

orchestrator_host = os.environ['FABRIC_ORCHESTRATOR_HOST']
print(f"FABRIC Orchestrator         : {orchestrator_host}")<jupyter_output><empty_output><jupyter_text>Create Slice Manager ObjectUsers can request tokens with different Project and Scopes by altering `project_name` and `scope` parameters in the refresh call below.<jupyter_code>slice_manager = SliceManager(oc_host=orchestrator_host, 
                             cm_host=credmgr_host ,
                             project_name='all', 
                             scope='all')

# Initialize the slice manager
slice_manager.initialize()<jupyter_output><empty_output><jupyter_text>Create a NodeThe cell below creates a slice that contains a single node with a 1TB NVMe device.  Set the Slice Name and FABRIC Site<jupyter_code>from fabrictestbed.slice_editor import ComponentType

slice_name="MySlice"
site_name="RENC"
node_name='Node1'
image = 'default_centos_8'
image_type = 'qcow2'
cores = 2
ram = 16
disk = 100

nvme_component_type = ComponentType.NVME
nvme_model = 'P4510'
nvme_name = 'nvme1'
from fabrictestbed.slice_editor import ExperimentTopology, Capacities 
# Create topology
t = ExperimentTopology()

# Add node
n1 = t.add_node(name=node_name, site=site_name)

# Set capacities
cap = Capacities()
cap.set_fields(core=cores, ram=ram, disk=disk)

# Set Properties
n1.set_properties(capacities=cap, image_type=image_type, image_ref=image)

# Add the PCI NVMe device
n1.add_component(ctype=nvme_component_type, model=nvme_model, name=nvme_name)

# Generate Slice Graph
slice_graph = t.serialize()

# Request slice from Orchestrator
return_status, slice_reservations = slice_manager.create(slice_name=slice_name, 
                                            slice_graph=slice_graph, 
                                            ssh_key=ssh_key_pub)

if return_status == Status.OK:
    slice_id = slice_reservations[0].get_slice_id()
    print("Submitted slice creation request. Slice ID: {}".format(slice_id))
else:
    print(f"Failure: {slice_reservations}")<jupyter_output><empty_output><jupyter_text>Get the Slice<jupyter_code>import time
def wait_for_slice(slice,timeout=180,interval=10,progress=False):
    timeout_start = time.time()

    if progress: print("Waiting for slice .", end = '')
    while time.time() < timeout_start + timeout:
        return_status, slices = slice_manager.slices(excludes=[SliceState.Dead,SliceState.Closing])

        if return_status == Status.OK:
            slice = list(filter(lambda x: x.slice_name == slice_name, slices))[0]
            if slice.slice_state == "StableOK":
                if progress: print(" Slice state: {}".format(slice.slice_state))
                return slice
            if slice.slice_state == "Closing" or slice.slice_state == "Dead":
                if progress: print(" Slice state: {}".format(slice.slice_state))
                return slice    
        else:
            print(f"Failure: {slices}")
        
        if progress: print(".", end = '')
        time.sleep(interval)
    
    if time.time() >= timeout_start + timeout:
        if progress: print(" Timeout exceeded ({} sec). Slice: {} ({})".format(timeout,slice.slice_name,slice.slice_state))
        return slice    


return_status, slices = slice_manager.slices(excludes=[SliceState.Dead,SliceState.Closing])

if return_status == Status.OK:
    slice = list(filter(lambda x: x.slice_name == slice_name, slices))[0]
    slice = wait_for_slice(slice, progress=True)

print()
print("Slice Name : {}".format(slice.slice_name))
print("ID         : {}".format(slice.slice_id))
print("State      : {}".format(slice.slice_state))
print("Lease End  : {}".format(slice.lease_end))<jupyter_output><empty_output><jupyter_text>Get the NodeRetrieve the node information and save the management IP address.<jupyter_code>return_status, experiment_topology = slice_manager.get_slice_topology(slice_object=slice)

node = experiment_topology.nodes[node_name]

management_ip = str(node.get_property(pname='management_ip'))
print("Node Name        : {}".format(node.name))
print("Management IP    : {}".format(management_ip))
print()

nvme1 = node.components[nvme_name]
print("NVMe Name        : {}".format(nvme1.name))
print("Details          : {}".format(nvme1.details))
print("Disk (G)         : {}".format(nvme1.get_property(pname='capacity_allocations').disk))
print("Units            : {}".format(nvme1.get_property(pname='capacity_allocations').unit))
print("Model            : {}".format(nvme1.model))
print("Type             : {}".format(nvme1.get_property(pname='type')))<jupyter_output><empty_output><jupyter_text>Setup SSH Connection for CommandsSetup paramiko to send commands to the node using ssh.<jupyter_code>import paramiko

key = paramiko.RSAKey.from_private_key_file(ssh_key_file_priv)
client = paramiko.SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())

client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(management_ip,username='centos',pkey = key)<jupyter_output><empty_output><jupyter_text>Configure the NVMe PCI DeviceNVMe storage is provided as bare PCI block devices and will likely need to be partitioned and formated before use.Run the command lspci to see your NVMe PCI device. This is the raw NVMe PCI device that is not yet formated or mounted.<jupyter_code>stdin, stdout, stderr = client.exec_command('lspci | grep NVMe')

print (str(stdout.read(),'utf-8').replace('\\n','\n'))<jupyter_output><empty_output><jupyter_text>Run fdisk to see the NVMe block device. This device is not yet formated. We will partition and format this device later.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo fdisk -l /dev/nvme*')

print (str(stdout.read(),'utf-8').replace('\\n','\n'))<jupyter_output><empty_output><jupyter_text>Find the NVMe block device.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo fdisk -l /dev/nvme*')

print (str(stdout.read(),'utf-8').replace('\\n','\n'))<jupyter_output><empty_output><jupyter_text>There are no partitions or files systems on this device. First, let's partition the disk.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo parted /dev/nvme0n1 mklabel gpt')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>We can check that it has been created using the command below.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo parted /dev/nvme0n1 print')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Next, let's print the free space in the NVMe drive in megabytes.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo parted /dev/nvme0n1 print unit MB print free')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>We can see that the disk is 1GB. Next, let's create a partition that uses up this entire space.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo parted --align optimal /dev/nvme0n1 mkpart primary ext4 0% 1000205MB')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Let's examine the partitions one more time to see the partition we created.<jupyter_code>stdin, stdout, stderr = client.exec_command('lsblk /dev/nvme0n1')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Now, let's format the disk with a `ext4` file system.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo mkfs.ext4 /dev/nvme0n1p1')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Next, we need to mount the file system.<jupyter_code>stdin, stdout, stderr = client.exec_command('sudo mkdir /mnt/nvme_mount && sudo mount /dev/nvme0n1p1 /mnt/nvme_mount')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Confirm that the file system is mounted and the correct size.<jupyter_code>stdin, stdout, stderr = client.exec_command('df -h /mnt/nvme_mount')
print(stdout.read().decode("utf-8"))
print(stderr.read().decode("utf-8"))<jupyter_output><empty_output><jupyter_text>Now we can use the file system mounted at `/mnt/nvme_mount`  Cleanup Your Experiment<jupyter_code>return_status, result = slice_manager.delete(slice_object=slice)

print("Response Status {}".format(return_status))
print("Response received {}".format(result))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12480068034442436, 'fraction_numerical': 0.008504305304560434, 'mean_word_length': 5.20990099009901, 'pattern_counts': {'":': 2, '<': 74, '<?xml version=': 0, '>': 74, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '114950', 'license': 'MIT', 'n_tokens_mistral': 3150, 'n_tokens_neox': 3076, 'n_words': 663, 'path': 'fabric_examples/basic_examples/basic_nvme_devices.ipynb', 'repo_name': 'fabric-testbed/jupyternb-template', 'size': '9407'}