<jupyter_start><jupyter_text>Running the fleet of Virtual Wind Turbines and Edge Devices**SageMaker Studio Kernel**: Data ScienceAfter deploying the inference app on the 5 IoT Core devices(EC2 instances, in this case), its time to set up the virtual fleet of turbines and the simulator app along with it. In this exercise you will run a local application written in Python3 that simulates 5 Wind Turbines. This application will interact with IoT Core devices via IoT MQTT to exchange information.Here you'll be the **Wind Turbine Farm Operator**. It's possible to visualize the data flowing from the sensors to the ML Model and analyze the anomalies. Also, you'll be able to inject noise (pressing some buttons) in the data to simulate potential anomalies with the equipment.            STEP-BY-STEP        APPLICATION ARCHITECTURE                              The components of the Simulator are: - [Simulator](simulator/simulator.py): Program that launches the virtual wind turbines and the edge devices. It also maintains a simple dashboard to show the data which is being sent for anomaly detection to the detector. - [Turbine](simulator/turbine.py): Virtual Wind Turbine. It reads the raw data collected from the 3D Prited Mini Turbine and stream it as a circular buffer. It also has a graphical representation in **IPython Widgets** that is rendered by the Simulator/Dashboard. It contains all the business logic of maintaining the virtual turbine. - [MQTT Client](simulator/mqttclient.py): An abstraction layer for MQTT communication providing convenience methods to publish and subscribe to topics. First, let us restore variables stored in the previous exercise<jupyter_code>%store -r project_id
%store -r deployment_id<jupyter_output><empty_output><jupyter_text>Install some dependencies<jupyter_code>!pip install ipywidgets
!pip install awsiotsdk<jupyter_output><empty_output><jupyter_text>Download the sensors data (streamed by the virtual wind turbines) NOTE: if you ran the [Lab 01- Data-Visualization](../../01-Data-Visualization/01-Data-Visualization.ipynb), then the data is already downloaded in `../../data` location. If you have skipped the lab 01, download the data by running the next cell, OR skip over to the cell following the next cell.<jupyter_code>!mkdir -p data
!curl https://aws-ml-blog.s3.amazonaws.com/artifacts/monitor-manage-anomaly-detection-model-wind-turbine-fleet-sagemaker-neo/dataset_wind_turbine.csv.gz -o data/dataset_wind.csv.gz
project_name='<<PROJECT_NAME>>'

import boto3

sm_client = boto3.client('sagemaker')
project_id = sm_client.describe_project(ProjectName=project_name)['ProjectId']
bucket_name = 'sagemaker-wind-turbine-farm-%s' % project_id<jupyter_output><empty_output><jupyter_text>Simulating The Wind Turbine FarmNow its time to run our simulator and start playing with the turbines and with the anomalies > After clicking on **Start**, each turbine will start buffering some data. It takes a few seconds but after completing this process, the application runs in real-time    > Try to press some buttons while the simulation is running, to inject noise in the data and see some anomalies<jupyter_code>import os
import sys
sys.path.insert(0, os.path.join(os.path.realpath('../../'),'simulator'))
import turbine
import simulator
import mqttclient
from importlib import reload
import logging

logging.basicConfig(level=logging.INFO)

reload(mqttclient)
reload(turbine)
reload(simulator)

simulator = simulator.WindTurbineFarmSimulator(5)
simulator.start()
simulator.show()<jupyter_output><empty_output><jupyter_text>Once your experiment is finished, stop the simulator. This will in turn also stops/halts all the turbines.<jupyter_code>simulator.halt()<jupyter_output><empty_output><jupyter_text>> If you want to experiment with the deployment process, with the wind farm running, go back to [Exercise 2](02-Training-with-Pytorch.ipynbBuilding-the-Deployment-Package-SageMaker-Edge-Manager) (in the last cells), go back to lab02 notebook and deploy a new `version` of the model, create a new greengrass component version of the model and create a new deployment version deploying this new model version.  CleanupRun the next cell only if you already finished exploring/hacking the content of the workshop.  This code will delete all the resouces created so far, including the **SageMaker Project** you've created<jupyter_code>import boto3
import time
from shutil import rmtree

iot_client = boto3.client('iot')
sm_client = boto3.client('sagemaker')
s3_resource = boto3.resource('s3')
ggv2_client = boto3.resource('greengrassv2')

policy_name='WindTurbineFarmPolicy-%s' % project_id
thing_group_name='WindTurbineFarm-%s' % project_id
fleet_name='wind-turbine-farm-%s' % project_id

#Cancel the deployment of greengrass v2
ggv2_client.cancel_deployment(deploymentId=deployment_id)

#Delete the core devices:
for i in range(5):
    ggv2_client.delete_core_device(coreDeviceThingName='edge-device-'+str(i))

# delete the ggv2 components
account_id = boto3.client('sts').get_caller_identity().get('Account')
region = boto3.session.Session().region_name

ggv2_client.delete_component(arn='arn:aws:greengrass:{}:{}:components:aws.samples.windturbine.detector:versions:1.0.0'.format(region,account_id))
ggv2_client.delete_component(arn='arn:aws:greengrass:{}:{}:components:aws.samples.windturbine.model:versions:1.0.0'.format(region,account_id))



# Delete all files from the S3 Bucket
s3_resource.Bucket(bucket_name).objects.all().delete()


# now deregister the devices from the fleet
resp = sm_client.list_devices(DeviceFleetName=fleet_name)
devices = [d['DeviceName'] for d in resp['DeviceSummaries']]
if len(devices) > 0:
    sm_client.deregister_devices(DeviceFleetName=fleet_name, DeviceNames=devices)

# now deregister the devices from the fleet
for i,cert_arn in enumerate(iot_client.list_targets_for_policy(policyName=policy_name)['targets']):
    for t in iot_client.list_principal_things(principal=cert_arn)['things']:
        iot_client.detach_thing_principal(thingName=t, principal=cert_arn)
    iot_client.detach_policy(policyName=policy_name, target=cert_arn)
    certificateId = cert_arn.split('/')[-1]
    iot_client.update_certificate(certificateId=certificateId, newStatus='INACTIVE')
    iot_client.delete_certificate(certificateId=certificateId)

iot_client.delete_role_alias(roleAlias='SageMakerEdge-%s' % fleet_name)
iot_client.delete_thing_group(thingGroupName=thing_group_name)

if os.path.isdir('agent'): rmtree('agent')
sm_client.delete_project(ProjectName=project_name)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.086969743043941, 'fraction_numerical': 0.007754295271400334, 'mean_word_length': 6.126760563380282, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 31, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '130149', 'license': 'MIT-0', 'n_tokens_mistral': 2034, 'n_tokens_neox': 1935, 'n_words': 617, 'path': 'lab/04-Run-Fleet/greengrass-v2/04-run-fleet-ggv2.ipynb', 'repo_name': 'vikeshpandey/amazon-sagemaker-edge-manager-workshop', 'size': '6577'}