<jupyter_start><jupyter_text>Quick StartBelow is a simple demo of interaction with the environment of the VM scheduling scenario.<jupyter_code>from maro.simulator import Env
from maro.simulator.scenarios.vm_scheduling import AllocateAction, DecisionPayload

env = Env(scenario="vm_scheduling", topology="azure.2019.10k", start_tick=0, durations=8638, snapshot_resolution=1)

metrics: object = None
decision_payload: DecisionPayload = None
is_done: bool = False

while not is_done:
    action: AllocateAction = None
    metrics, decision_payload, is_done = env.step(action)

print(metrics)<jupyter_output>{'total_vm_requests': 10000, 'total_energy_consumption': 863800.0, 'successful_allocation': 0, 'successful_completion': 0, 'failed_allocation': 0, 'failed_completion': 0, 'total_latency': Latency(Agent=0, Resource=0), 'total_oversubscriptions': 0, 'total_overload_pms': 0, 'total_overload_vms': 0}<jupyter_text>Environment of the VM schedulingTo initialize an environment, you need to specify the values of several parameters:- **scenario**: The target scenario of this Env.  - `vm_scheduling` denotes for the virtual machine scheduling.- **topology**: The target topology of this Env. As shown below, you can get the predefined topology list by calling `get_topologies(scenario='vm_scheduling')`- **start_tick**: The start tick of this Env. In vm_scheduling scenario, 1 tick represents as 5 minutes in real-time.  - In the demo above, `start_tick=0` indicates a simulation start from the beginning of the given topology.- **durations**: The duration of this Env, in the unit of tick/5 minutes.  - In the demo above, `durations=8638` indicates a simulation length of roughly 30 days (30d * 24h * 60m / 5). It is also the maximum length of this topology.- **snapshot_resolution**: The time granularity of maintaining the snapshots of the environments, in the unit of tick/5 minutes.  - In the demo above, `snapshot_resolution=1` indicates that a snapshot will be created and saved every tick during the simulation.  You can get all available scenarios and topologies by calling:<jupyter_code>from maro.simulator.utils import get_scenarios, get_topologies
from pprint import pprint
from typing import List

scenarios: List[str] = get_scenarios()
topologies: List[str] = get_topologies(scenario='vm_scheduling')

pprint(f'The available scenarios in MARO:')
pprint(scenarios)

print()
pprint(f'The predefined topologies in VM Scheduling:')
pprint(topologies)<jupyter_output>'The available scenarios in MARO:'
['cim', 'citi_bike', 'vm_scheduling']

'The predefined topologies in VM Scheduling:'
['azure.2019.10k', 'azure.2019.336k']<jupyter_text>Once you created an instance of the environment, you can easily access the real-time information of this environment, like:<jupyter_code>from maro.backends.frame import SnapshotList
from maro.simulator import Env
from pprint import pprint


# Initialize an Env for vm_scheduling scenario.
env = Env(scenario="vm_scheduling", topology="azure.2019.10k", start_tick=0, durations=8638, snapshot_resolution=1)

# The current tick.
tick: int = env.tick
print(f"The current tick: {tick}.")

# The current frame index, which indicates the index of current frame in the snapshot-list.
frame_index: int = env.frame_index
print(f"The current frame index: {frame_index}.")

# The whole snapshot-list of the environment, snapshots are taken in the granularity of the given snapshot_resolution.
# The example of how to use the snapshot will be shown later.
snapshot_list: SnapshotList = env.snapshot_list
print(f"There will be {len(snapshot_list)} snapshots in total.")

# The summary information of the environment.
summary: dict = env.summary
print(f"\nEnv Summary:")
pprint(summary)

# The metrics of the environment
metrics: dict = env.metrics
print(f"\nEnv Metrics:")
pprint(metrics)<jupyter_output>The current tick: 0.
The current frame index: 0.
There will be 8638 snapshots in total.

Env Summary:
{'event_payload': {'PENDING_DECISION': ['frame_index',
                                        'valid_pms',
                                        'vm_id',
                                        'vm_cpu_cores_requirement',
                                        'vm_memory_requirement',
                                        'remaining_buffer_time'],
                   'REQUEST': ['vm_info', 'remaining_buffer_time']},
 'node_detail': {'pms': {'attributes': {'cpu_cores_allocated': {'slots': 1,
                                                                'type': 'i2'},
                                        'cpu_cores_capacity': {'slots': 1,
                                                               'type': 'i2'},
                                        'cpu_utilization': {'slots': 1,
                                                            'type': 'f'},
                   [...]<jupyter_text>Interaction with the environmentBefore starting interaction with the environment, we need to know **DecisionPayload** and **Action** first. DecisionPayloadOnce the environment need the agent's response to promote the simulation, it will throw an **PendingDecision** event with the **DecisionPayload**. In the scenario of vm_scheduling, the information of `DecisionPayload` is listed as below:- **valid_pms** (List[int]): The list of the PM ID that is considered as valid (its CPU and memory resource is enough for the incoming VM request).- **vm_id** (int): The VM ID of the incoming VM request(VM request that is waiting for the allocation).- **vm_cpu_cores_requirement** (int): The CPU cores that is requested by the incoming VM request.- **vm_memory_requirement** (int): The memory resource that is requested by the incoming VM request.- **remaining_buffer_time** (int): The remaining buffer time for the VM allocation. The VM request will be treated as failed when the `remaining_buffer_time` is spent. The initial buffer time budget can be set in the `config.yml`. ActionOnce get a **PendingDecision** event from the environment, the agent should respond with an `Action`. Valid `Action` includes:- **None**. It means do nothing but ignore this VM request.- **AllocateAction**. It includes:  - vm_id (int): The ID of the VM that is waiting for the allocation.  - pm_id (int): The ID of the PM where the VM is scheduled to allocate to.- **PostponeAction**. It includes:  - vm_id (int): The ID of the VM that is waiting for the allocation.  - postpone_step (int): The number of times that the allocation to be postponed. The unit is `DELAY_DURATION`. 1 means delay 1 `DELAY_DURATION`, which can be set in the `config.yml`.  Generate random actionsThe demo code in the Quick Start part has shown an interaction mode that doing nothing(responding with `None` action). Here we read the detailed information about the `DecisionPayload` and randomly choose an available PM.<jupyter_code>import random

from maro.simulator import Env
from maro.simulator.scenarios.vm_scheduling import AllocateAction, DecisionPayload, PostponeAction

# Initialize an Env for vm_scheduling scenario
env = Env(scenario="vm_scheduling", topology="azure.2019.10k", start_tick=0, durations=8638, snapshot_resolution=1)

metrics: object = None
decision_event: DecisionPayload = None
is_done: bool = False
action: AllocateAction = None
    
# Start the env with a None Action
metrics, decision_event, is_done = env.step(None)

while not is_done:
    valid_pm_num: int = len(decision_event.valid_pms)
    if valid_pm_num <= 0:
        # No valid PM now, postpone.
        action: PostponeAction = PostponeAction(
            vm_id=decision_event.vm_id,
            postpone_step=1
        )
    else:
        # Randomly choose an available PM.
        random_idx = random.randint(0, valid_pm_num - 1)
        pm_id = decision_event.valid_pms[random_idx]
        action: AllocateAction = AllocateAction(
            vm_id=decision_event.vm_id,
            pm_id=pm_id
        )
    metrics, decision_event, is_done = env.step(action)

print(f"[Random] Topology: azure.2019.10k. Total ticks: 8638. Start tick: 0")
print(metrics)<jupyter_output>[Random] Topology: azure.2019.10k. Total ticks: 8638. Start tick: 0
{'total_vm_requests': 10000, 'total_energy_consumption': 2419711.6239830256, 'successful_allocation': 9902, 'successful_completion': 9084, 'failed_allocation': 98, 'failed_completion': 0, 'total_latency': Latency(Agent=0, Resource=0), 'total_oversubscriptions': 34415, 'total_overload_pms': 0, 'total_overload_vms': 0}<jupyter_text>Get the environment observationYou can also implement other strategies or build models to take action. At this time, real-time information and historical records of the environment are very important for making good decisions. In this case, the the environment snapshot list is exactly what you need.The information in the snapshot list is indexed by 3 dimensions:- A frame index (list). (int / List[int]) Empty indicates for all time slides till now.- A PM id (list). (int / List[int]) Empty indicates for all PMs.- An Attribute name (list). (str / List[str]) You can get all available attributes in env.summary as shown before.The return value from the snapshot list is a `numpy.ndarray` with shape (`num_frame` * `num_pms` * `num_attribute`, ).More detailed introduction to the snapshot list is [here](https://maro.readthedocs.io/en/latest/key_components/data_model.htmladvanced-features).<jupyter_code>from pprint import pprint

from maro.simulator import Env

# Initialize an Env for vm_scheduling scenario
env = Env(scenario="vm_scheduling", topology="azure.2019.10k", start_tick=0, durations=8638, snapshot_resolution=1)

# To get the attribute list that can be accessed in snapshot_list
pprint(env.summary['node_detail'], depth=2)
print()
# The attribute list of stations
pprint(env.summary['node_detail']['pms'])
from pprint import pprint

from maro.simulator import Env
from maro.simulator.scenarios.vm_scheduling import AllocateAction, DecisionPayload, PostponeAction

env = Env(scenario="vm_scheduling", topology="azure.2019.10k", start_tick=0, durations=8638, snapshot_resolution=1)

metrics: object = None
decision_event: DecisionPayload = None
is_done: bool = False
action: AllocateAction = None
    
metrics, decision_event, is_done = env.step(None)

while not is_done:
    # This demo is used to show how to retrieve the information from the snapshot,
    # we terminate at 2000th tick and see the output of the environment.
    if env.frame_index >= 2000 and len(decision_event.valid_pms) > 0:
        # Get current state information of the first 10 valid PMs.
        valid_pm_info = env.snapshot_list["pms"][
            env.frame_index:decision_event.valid_pms[:10]:["cpu_cores_capacity", "cpu_cores_allocated"]
        ].reshape(-1, 2)
        # Calculate to get the remaining cpu cores.
        cpu_cores_remaining = valid_pm_info[:, 0] - valid_pm_info[:, 1]
        # Show current state information of the first 10 valid PMs.
        print("For the first 10 valid PMs:")
        print(f"cpu core capacity:  {valid_pm_info[:, 0]}")
        print(f"cpu core allocated: {valid_pm_info[:, 1]}")
        print(f"cpu core remaining: {cpu_cores_remaining}")
        
        # Get the historical cpu utilization of the first valid PM in the recent 10 ticks.
        past_10_frames = [x for x in range(env.frame_index - 10, env.frame_index)]
        cpu_utilization_series = env.snapshot_list["pms"][
            past_10_frames:decision_event.valid_pms[0]:"cpu_utilization"
        ]
        # Show the historical information of the first valid PM.
        print("For the first valid PM:")
        print(f"Recent cpu utilization series is: {cpu_utilization_series}")
        
        break

    valid_pm_num: int = len(decision_event.valid_pms)
    if valid_pm_num <= 0:
        # No valid PM now, postpone.
        action: PostponeAction = PostponeAction(
            vm_id=decision_event.vm_id,
            postpone_step=1
        )
    else:
        # Randomly choose an available PM.
        random_idx = random.randint(0, valid_pm_num - 1)
        pm_id = decision_event.valid_pms[random_idx]
        action: AllocateAction = AllocateAction(
            vm_id=decision_event.vm_id,
            pm_id=pm_id
        )
    metrics, decision_event, is_done = env.step(action)<jupyter_output>For the first 10 valid PMs:
cpu core capacity:  [32. 32. 32. 32. 32. 32. 32. 32. 32. 32.]
cpu core allocated: [20. 20. 18. 26. 16. 24. 26. 24. 28. 26.]
cpu core remaining: [12. 12. 14.  6. 16.  8.  6.  8.  4.  6.]
For the first valid PM:
Recent cpu utilization series is: [4.43 4.68 3.98 3.59 4.64 4.3  4.76 4.52 3.83 4.02]
{'fraction_non_alphanumeric': 0.09207079926978332, 'fraction_numerical': 0.025160727041828715, 'mean_word_length': 3.7050037341299475, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 17, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '32324', 'license': 'MIT', 'n_tokens_mistral': 3900, 'n_tokens_neox': 3561, 'n_words': 1317, 'path': 'notebooks/vm_scheduling/interact_with_environment.ipynb', 'repo_name': 'Meroy9819/maro', 'size': '12599'}