<jupyter_start><jupyter_text>**Importing Required Libraries**<jupyter_code>from tensorflow.keras.models import load_model
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import pickle
import os<jupyter_output><empty_output><jupyter_text>**Libraries required for connecting to IBM Db2 database on the cloud**<jupyter_code>!pip3 install --user lazy
!pip3 install --user jaydebeapi
!pip3 uninstall --yes ibmdbpy
!pip3 install ibmdbpy --user --ignore-installed --no-deps
!wget -O $HOME/.local/lib/python3.8/site-packages/ibmdbpy/db2jcc4.jar https://ibm.box.com/shared/static/lmhzyeslp1rqns04ue8dnhz2x7fb6nkc.zip<jupyter_output>Requirement already satisfied: lazy in /home/mohitgurav/.local/lib/python3.8/site-packages (1.4)
Requirement already satisfied: jaydebeapi in /home/mohitgurav/.local/lib/python3.8/site-packages (1.2.3)
Requirement already satisfied: JPype1; python_version > "2.7" and platform_python_implementation != "Jython" in /home/mohitgurav/.local/lib/python3.8/site-packages (from jaydebeapi) (0.6.3)
Found existing installation: ibmdbpy 0.1.6
Uninstalling ibmdbpy-0.1.6:
  Successfully uninstalled ibmdbpy-0.1.6
Collecting ibmdbpy
  Using cached ibmdbpy-0.1.6-py2.py3-none-any.whl (148 kB)
Installing collected packages: ibmdbpy
Successfully installed ibmdbpy-0.1.6
--2020-06-29 19:03:49--  https://ibm.box.com/shared/static/lmhzyeslp1rqns04ue8dnhz2x7fb6nkc.zip
Resolving ibm.box.com (ibm.box.com)... 192.168.3.11
Connecting to ibm.box.com (ibm.box.com)|192.168.3.11|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: /public/static/lmhzyeslp1rqns04ue8dnhz2x7fb6[...]<jupyter_text>**Setting bus limt**<jupyter_code>bus_limit=200<jupyter_output><empty_output><jupyter_text>**Loading various models for accurate and specialized time series prediction**1.   Static Model2.   Dynamic Model<jupyter_code>static_model=load_model("model24hrspredict.h5")
dynamic_model=load_model("model2hrspredict(6hrsinput).h5")
static_scaler=pickle.load(open("scalernew.pkl",'rb'))
dynamic_scaler=pickle.load(open("scaler6hrswala.pkl",'rb'))<jupyter_output>/home/mohitgurav/.local/lib/python3.8/site-packages/sklearn/base.py:329: UserWarning: Trying to unpickle estimator MinMaxScaler from version 0.22.2.post1 when using version 0.23.1. This might lead to breaking code or invalid results. Use at your own risk.
  warnings.warn(<jupyter_text>**Function to print timetable for the day**<jupyter_code>def schedule(prediction):
    sch=[]
    for i,j in enumerate(prediction):
        print(f"{int(round(j[0]/bus_limit))} AT {str(i).zfill(2)}:00")
        sch.append([int(round(j[0]/bus_limit)),i])
    return sch<jupyter_output><empty_output><jupyter_text>**Function to handle special events which might have an effect on public transport**<jupyter_code>def special_events(prediction):
    start=prediction['START']
    stop=prediction['STOP']
    magnitude=prediction['MAGNITUDE']
    prediction['TIMESERIES'][start:stop]=prediction['TIMESERIES'][start:stop]*(1.0+magnitude/100)
    return prediction<jupyter_output><empty_output><jupyter_text>**Function to predict static time table for transport mobiles****Inputs:***   Data of past 24 hours*   Dictionary for special timeseries**Output:***   Schedule for the day<jupyter_code>def static_time_table(data,special_timeseries=None):
    y_pred=static_model.predict(data)[0]
    if special_timeseries!=None:
        special_timeseries['TIMESERIES']=y_pred[0]
        y_pred=special_events(special_timeseries)['TIMESERIES']
    y_pred=static_scaler.inverse_transform(y_pred.reshape(-1,1))
    number=schedule(y_pred)
    return [number,y_pred]<jupyter_output><empty_output><jupyter_text>**Connecting to IBM Db2 database on the cloud**<jupyter_code>from db2 import Db2
db2 = Db2(table_name='INTERSTATE_TRAFFIC')
db2.last24hours()
db2.last6hours()
db2.ida_traffic_pd.head()<jupyter_output><empty_output><jupyter_text>**Data cleaning**<jupyter_code>db2.last6hours()
db2.ida_traffic_pd.head()<jupyter_output><empty_output><jupyter_text>**Data cleaning**<jupyter_code>data=pd.read_csv("../Dataset/Metro_Interstate_Traffic_Volume.csv")
data['date_time']=pd.to_datetime(data['date_time'],infer_datetime_format=True)
data=data.iloc[:,-2:]
data=data.set_index('date_time')<jupyter_output><empty_output><jupyter_text>**Noramlization**<jupyter_code>data['traffic_volume']=static_scaler.fit_transform(data['traffic_volume'].values.reshape(-1,1))<jupyter_output><empty_output><jupyter_text>**Timeseries Prediction**<jupyter_code>num,pred=static_time_table(data['traffic_volume'][-24:].values.reshape(1,-1,1))
plt.plot(pred,label='Prediction')
plt.xlabel("Hours")
plt.ylabel("Expected Crowd")
plt.legend()<jupyter_output>2 AT 00:00
1 AT 01:00
1 AT 02:00
3 AT 03:00
7 AT 04:00
14 AT 05:00
19 AT 06:00
22 AT 07:00
22 AT 08:00
22 AT 09:00
22 AT 10:00
22 AT 11:00
22 AT 12:00
23 AT 13:00
23 AT 14:00
24 AT 15:00
24 AT 16:00
22 AT 17:00
20 AT 18:00
17 AT 19:00
16 AT 20:00
14 AT 21:00
12 AT 22:00
9 AT 23:00<jupyter_text>**Checking the dynamic model**<jupyter_code>data['traffic_volume']=dynamic_scaler.fit_transform(data['traffic_volume'].values.reshape(-1,1))
y_pred=dynamic_model.predict(data['traffic_volume'][-6:].values.reshape(1,-1,1))<jupyter_output><empty_output><jupyter_text>**Converting the normalized data into original scale**<jupyter_code>y_pred=dynamic_scaler.inverse_transform(y_pred)<jupyter_output><empty_output><jupyter_text>**Plotting the results**<jupyter_code>l=list(dynamic_scaler.inverse_transform(data['traffic_volume'][-6:].values.reshape(-1,1)))+list(y_pred[0])<jupyter_output><empty_output><jupyter_text>**Adding legend and labels to graph**<jupyter_code>plt.plot(l,color='black',label="prediction")
plt.axvline(5,color='r',label="start")
plt.axvline(7,color='blue',label="stop")
plt.xlabel("Hours")
plt.ylabel("Expected Crowd")
plt.legend()<jupyter_output><empty_output><jupyter_text>**Function to predict dynamic changes in static timetable****Inputs:***   Data of past 6 hours**Output:***   Crowd prediction for next 2 hours<jupyter_code>def dynamic_changes(data,sch,start_time):
    pred=dynamic_model.predict(data)[0]
    pred=schedule(dynamic_scaler.inverse_transform(pred.reshape(-1,1)))
    #temp=sch[start_time:start_time+2]
    sch[start_time:start_time+2]=list(map(lambda x:x[0],pred))
    return sch
dynamic_changes(data['traffic_volume'][-6:].values.reshape(1,-1,1),list(map(lambda x:x[0],num)),0)<jupyter_output>3 AT 00:00
2 AT 01:00
{'fraction_non_alphanumeric': 0.14195046439628484, 'fraction_numerical': 0.05309597523219814, 'mean_word_length': 7.603195739014647, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 66, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 2, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '404120', 'license': 'MIT', 'n_tokens_mistral': 2620, 'n_tokens_neox': 2436, 'n_words': 394, 'path': 'Integration/Integration.ipynb', 'repo_name': 'khallnayak/SBSPS-Challenge-1029-AI-powered-Public-Transport-Management-System', 'size': '6466'}