<jupyter_start><jupyter_text>Copyright (c) Microsoft Corporation. All rights reserved.Licensed under the MIT License.  Tutorial: Load demo data and enrich it with NOAA ISD Weather data.In this tutorial, you load the demo data (a parquet file in Azure Blob), check the data schema, enrich it with NOAA ISD Weather data.Prerequisites:> You must install the PyPI package on the cluster:> * azureml-opendatasetsLearn how to:> * Load the demo data from Azure Blob> * Check the demo data schema> * Initialize NoaaIsdWeather class to load weather data> * Enrich the demo data with weather data> * Display the joined result annd stats  Load demo parquet file from Azure Blob<jupyter_code>from azure.storage.blob import BlockBlobService
from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()

container_name = 'tutorials'
account_name = 'azureopendata<PASSWORD>'
relative_path = 'noaa_isd_weather/demo.parquet'
df = spark.read.parquet('wasbs://%s@%s.blob.core.windows.net/%s' % (
    container_name,
    account_name,
    relative_path))
df.count()<jupyter_output><empty_output><jupyter_text>Display the demo data<jupyter_code>display(df.limit(10))<jupyter_output><empty_output><jupyter_text>Initialize NoaaIsdWeather class, get the enricher from it and enrich demo data<jupyter_code>from azureml.opendatasets.accessories.location_data import LatLongColumn
from azureml.opendatasets.accessories.location_time_customer_data import LocationTimeCustomerData
from azureml.opendatasets import NoaaIsdWeather


_customer_data = LocationTimeCustomerData(df, LatLongColumn('lat', 'long'), 'datetime')
weather = NoaaIsdWeather(cols=["temperature", "windSpeed", "seaLvlPressure"])
weather_enricher = weather.get_enricher()
joined_data = weather_enricher.enrich_customer_data_with_agg(
  customer_data_object=_customer_data,
  location_match_granularity=5,
  time_round_granularity='day',
  agg='avg')<jupyter_output><empty_output><jupyter_text>Display the joined result<jupyter_code>display(joined_data.data.limit(10))<jupyter_output><empty_output><jupyter_text>Convert the joined spark dataframe to pandas dataframe<jupyter_code>joined_data_pandas = joined_data.data.toPandas()<jupyter_output><empty_output><jupyter_text>Check the stats of joined result<jupyter_code>print(joined_data_pandas.info())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08729139922978177, 'fraction_numerical': 0.002139495079161318, 'mean_word_length': 8.027027027027026, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '495394', 'license': 'MIT', 'n_tokens_mistral': 780, 'n_tokens_neox': 756, 'n_words': 177, 'path': 'tutorials/data-join/01-weather-join-in-spark.ipynb', 'repo_name': 'Azure/OpenDatasetsNotebooks', 'size': '2334'}