<jupyter_start><jupyter_text>USB Sensor API and Plots The following notebook provides some examples of using the API to access data from the Urban Science Building (USB), collected by the Urban Observatory (https://urbanobservatory.ac.uk/). Further guidance can be found at https://api.usb.urbanobservatory.ac.uk/<jupyter_code>## Import Modules
import matplotlib
import pandas as pd
import json
import urllib.request
import datetime as dt
import matplotlib.pyplot as plt
import matplotlib.dates as mdates

matplotlib.rcParams.update({
    'font.size': 16
})<jupyter_output><empty_output><jupyter_text>Use the API to call number of rooms on the _nth_ floor of USB, filter to see which are currently occupied.<jupyter_code>## Select Floor - Options include G, and 1-6
floor = '2'

## Create API Call Components. 
## 'meta:buildingFloor' used to define floor
## 'metric=occupied' selects on those with occupancy sensors
## 'pageSize' (Default 10, currently set to 100, may need to be more depending on chosen floor).
callBase = 'https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/entity/'
floorCall = callBase + '?meta:buildingFloor=' + floor + '&metric=occupied&pageSize=100'

## Call API - note the different steps used within the json.loads process.
usbRmsOnFlr = json.loads(
    urllib
        .request
        .urlopen(floorCall)
        .read()
        .decode('utf-8')
)['items']

## Print API Call - Click to see JSON Output
print(floorCall)
print('Sensors located in %u rooms.' % len(usbRmsOnFlr))
## List currently occupied rooms
## Includes check to discount rooms that don't have a 'latest' field
for rm in usbRmsOnFlr:
    if 'latest' in rm['feed'][0]['timeseries'][0]:
        if 'value' in rm['feed'][0]['timeseries'][0]['latest']:
            if rm['feed'][0]['timeseries'][0]['latest']['value'] == 1:
                if "Room" in rm["name"]:
                    print(rm["name"])<jupyter_output>Urban Sciences Building: Floor 2: Room 2.004
Urban Sciences Building: Floor 2: Room 2.005
Urban Sciences Building: Floor 2: Room 2.008
Urban Sciences Building: Floor 2: Room 2.010
Urban Sciences Building: Floor 2: Room 2.011
Urban Sciences Building: Floor 2: Room 2.012
Urban Sciences Building: Floor 2: Room 2.014
Urban Sciences Building: Floor 2: Room 2.015
Urban Sciences Building: Floor 2: Room 2.020
Urban Sciences Building: Floor 2: Room 2.021
Urban Sciences Building: Floor 2: Room 2.022 Zone 2
Urban Sciences Building: Floor 2: Room 2.037 Zone 1
Urban Sciences Building: Floor 2: Room 2.038 Zone 1
Urban Sciences Building: Floor 2: Room 2.038 Zone 2
Urban Sciences Building: Floor 2: Room 2.048 Zone 1
Urban Sciences Building: Floor 2: Room 2.048 Zone 2
Urban Sciences Building: Floor 2: Room 2.048 Zone 3
Urban Sciences Building: Floor 2: Room 2.048 Zone 4
Urban Sciences Building: Floor 2: Room 2.048 Zone 5
Urban Sciences Building: Floor 2: Room 2.058 Zone 1
Urban Sciences Building: Floor[...]<jupyter_text>List all measured variables in a specific room.<jupyter_code>room = 'room-5-009'

roomCall = callBase + room

usbRoomData = json.loads(
    urllib
        .request
        .urlopen(roomCall)
        .read()
        .decode('utf-8')
)['feed']

print(roomCall)
## List Measured Variables
for variable in usbRoomData:
    print(variable['metric'])<jupyter_output>Window Alarm 1
Room Brightness
Room Temperature
CO2
Heating Valve Position
Relative Humidity
Mode
Mode Input
Actual Heating Set point
Actual Cooling Set point
Cooling Valve Position
Light Power Level
Occupancy Sensor
Eco Cooling Set Point
Eco Heating Set Point Write
Eco Cooling Set Point Write
Eco Heating Set Point<jupyter_text>Getting Historic Data Two techniques for demonstrated building up an API call to achieve this. 1. Using links with an entity/variable API return. 2. Constructing an API call programmatically. Method 1. Using a call of a specific entity and variable to quickly find measurements for the previous 24 hours. The following call will return information on measuring CO2 in Room 1.043.<jupyter_code>apiCall = 'https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/entity?meta:roomNumber=1.043&metric=CO2'

rawData = json.loads(urllib.request.urlopen(apiCall).read().decode('utf-8'))

## Print JSON data
## The HTTPS of interest is located in items>feed>timeseries>links 
## Look for "rel" variables of "archives" and "archives.friendly"
print(json.dumps(rawData, indent=1))
## Loop through JSON data structure to location of historic time-series HTTPS link, and call API using this.
## Note - some rooms will have multiple feeds depending on the sensor deployment.
## ['links'][1]['href'] and ['links'][3]['href'] should produce the same results, albeit with a different HTTPS.
historicAPI = rawData['items'][0]['feed'][0]['timeseries'][0]['links'][3]['href']
historicData = json.loads(urllib.request.urlopen(historicAPI).read().decode('utf-8'))['historic']['values']
print(historicAPI)<jupyter_output>https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/timeseries/room-1-043/co2/raw/historic<jupyter_text>Convert to pandas dataframe, and plot.<jupyter_code>## Convert JSON to Pandas DataFrame, Keeping Time and Value
dfHist = pd.DataFrame.from_records(historicData, exclude=['duration'])
dfHist.index = pd.to_datetime(dfHist["time"])
dfHist = dfHist.drop(columns="time")
dfHist
## Plot Data
fig, ax = plt.subplots(figsize=(26,9)) 

axFmt = mdates.DateFormatter('%H:%M')
ax.xaxis.set_major_formatter(axFmt)

ax.set_xlabel('Time')
ax.set_ylabel('CO2', color='darkred')
ax.plot(dfHist["value"], color='darkred')
ax.tick_params(axis='y', labelcolor='darkred')

plt.show()<jupyter_output><empty_output><jupyter_text>Method 2. We will compare Temperature and CO2 measurements in a room over a 2 week period. Set analysis time limits<jupyter_code>## Recommended to not exceed 25 days to allow inter0perability of graphs
startDate = dt.datetime.strptime('2020-02-01T00:00:00+0000', "%Y-%m-%dT%H:%M:%S%z")
endDate   = dt.datetime.strptime('2020-02-14T23:59:59+0000', "%Y-%m-%dT%H:%M:%S%z")<jupyter_output><empty_output><jupyter_text>Construct API Call<jupyter_code>## Choose Room of Interest
## Note some rooms will return a "Forbidden" error - if you recieve this, choose another!
room = 'room-5-009'

## Set Variables of Interest
var1 = 'Room Temperature'
var2 = 'CO2'

## Update Call Base
callBase = 'https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/timeseries/'

callVar1 = callBase + room + '/' + str(var1).replace(' ', '%20') + '/raw/historic/?startTime=' +\
startDate.isoformat().replace('+00:00', 'Z') + '&endTime=' + endDate.isoformat().replace('+00:00', 'Z')
callVar2 = callBase + room + '/' + str(var2).replace(' ', '%20') + '/raw/historic/?startTime=' +\
startDate.isoformat().replace('+00:00', 'Z') + '&endTime=' + endDate.isoformat().replace('+00:00', 'Z')

usbVar1 = json.loads(urllib.request.urlopen(callVar1).read().decode('utf-8'))['historic']['values']
usbVar2 = json.loads(urllib.request.urlopen(callVar2).read().decode('utf-8'))['historic']['values']

print(callVar1)
print(callVar2)<jupyter_output>https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/timeseries/room-5-009/Room%20Temperature/raw/historic/?startTime=2020-02-01T00:00:00Z&endTime=2020-02-14T23:59:59Z
https://api.usb.urbanobservatory.ac.uk/api/v2/sensors/timeseries/room-5-009/CO2/raw/historic/?startTime=2020-02-01T00:00:00Z&endTime=2020-02-14T23:59:59Z<jupyter_text>Check number of readings, and convert to pandas dataframe.<jupyter_code>print(var1+' has '+str(len(usbVar1))+' readings, '+ var2+' has '+str(len(usbVar2))+' readings.')
## Convert JSON to Pandas DataFrame, Keeping Time (as Index) and Value
dfVar1 = pd.DataFrame.from_records(usbVar1, exclude=['duration'])
dfVar1.index = pd.to_datetime(dfVar1["time"])
dfVar1 = dfVar1.drop(columns="time")
dfVar2 = pd.DataFrame.from_records(usbVar2, exclude=['duration'])
dfVar2.index = pd.to_datetime(dfVar2["time"])
dfVar2 = dfVar2.drop(columns="time")<jupyter_output><empty_output><jupyter_text>Plot time-series comparing Temperature and CO2.<jupyter_code>## Plot Data
fig, ax1 = plt.subplots(figsize=(26,9)) 

plt.xlim(startDate,endDate)

ax1.set_xlabel('Date')
ax1.set_ylabel('Temperature (Celsius)', color='forestgreen')
ax1.plot(dfVar1["value"], color='forestgreen')
ax1.tick_params(axis='y', labelcolor='forestgreen')

ax2 = ax1.twinx()

ax2.set_ylabel('CO2', color='navy') 
ax2.plot(dfVar2["value"], color='navy')
ax2.tick_params(axis='y', labelcolor='navy')

fig.tight_layout()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13783751916047635, 'fraction_numerical': 0.042212003301497464, 'mean_word_length': 5.464939024390244, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 36, 'https://': 8, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '497529', 'license': 'MIT', 'n_tokens_mistral': 3068, 'n_tokens_neox': 2797, 'n_words': 782, 'path': 'building/USB_API_Call_Example.ipynb', 'repo_name': 'urbanobservatory/uo-api-examples', 'size': '8481'}