<jupyter_start><jupyter_text>02 predictions for time intervalsRelating patient/attendance transactional dataframes to a daily or hourly dataframe requires a many-to-many link, which is not possible directly in ft. A linking dataframe can be used which does not contain unique keys, and can be joined on both sides with a many relationship.<jupyter_code>import pandas as pd
import numpy as np
import featuretools as ft
from create_data import make_attendances_dataframe
df = make_attendances_dataframe(15)<jupyter_output><empty_output><jupyter_text>create all dataframes we need<jupyter_code>from create_data import make_timeindex_dataframe, make_HourlyTimeAttenNum_dataframe
df_ActiveVisits = make_HourlyTimeAttenNum_dataframe(df,'arrival_datetime','departure_datetime')
df_ActiveVisits.head()
df_hours = make_timeindex_dataframe(df,'hour','h')
df_hours.head(3)
df_days = make_timeindex_dataframe(df,'day','D')
df_days.head(3)<jupyter_output><empty_output><jupyter_text>Make entitity sets - as before<jupyter_code>import featuretools.variable_types as vtypes
data_variable_types = {'atten_id': vtypes.Id,
                       'pat_id': vtypes.Id,
                       'arrival_datetime': vtypes.Datetime,
                      'time_in_department': vtypes.Numeric,
                       'departure_datetime': vtypes.Datetime,
                       'gender': vtypes.Boolean,
                      'ambulance_arrival': vtypes.Boolean}
es = ft.EntitySet('Hospital')
es = es.entity_from_dataframe(entity_id='attendances',
                               dataframe=df,
                               index='atten_id',
                               time_index='arrival_datetime',
                              secondary_time_index={'departure_datetime':['time_in_department']}, # dictionary here!
                               variable_types=data_variable_types)<jupyter_output><empty_output><jupyter_text>make entity with each attendance and hour it is active<jupyter_code>df_ActiveVisits.head(3)
# Make linking-es (active_visits)
es = es.entity_from_dataframe(entity_id='active_visits',
                               dataframe=df_ActiveVisits,
                              make_index=True,
                              index='index',
                               variable_types={'atten_id':vtypes.Id,
                                              'hour':vtypes.Datetime})<jupyter_output><empty_output><jupyter_text>make entity with hourly index<jupyter_code>df_hours.head(3)
# Make hours eset
es = es.entity_from_dataframe(entity_id='hours',
                               dataframe=df_hours,
                               index='hour',
                               variable_types={'hour':vtypes.Datetime})<jupyter_output><empty_output><jupyter_text>As we have made more entities with dataframes (and not normalised them from existing entities) we must explicitly tell the entity set the relationships:<jupyter_code># add es relationships
rel_Atten_ActiveVisits = ft.Relationship(es["attendances"]["atten_id"],
                                    es["active_visits"]["atten_id"])
rel_Hours_ActiveVisits = ft.Relationship(es["hours"]["hour"],
                                    es["active_visits"]["hour"])
es = es.add_relationships([rel_Atten_ActiveVisits,rel_Hours_ActiveVisits])<jupyter_output><empty_output><jupyter_text>creating features for individual hoursNow we have all our entities linked we can run DFS on the entity "hours". This will generate features like  "COUNT(active_visits)" -> in other words -> "Occupancy" for that particlar time of day.<jupyter_code>fm, features = ft.dfs(entityset=es,
                     target_entity='hours',
#                     agg_primitives=[],
#                     trans_primitives=[],
                     verbose=True,
                     max_depth = 5)
fm.head(5)
features
fm.dropna(axis=1).head()<jupyter_output><empty_output><jupyter_text>Further notes  adding "Intesting" variables<jupyter_code>df.head()
es["attendances"]["ambulance_arrival"].interesting_values = [True, False]
fm, features = ft.dfs(entityset=es,
                     target_entity='hours',
                     agg_primitives=['count','mean','percent_true'],
                     trans_primitives=[],
                      where_primitives=['count'],
                     verbose=True,
                     max_depth = 8)
fm.head(5)
ft.list_primitives()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09298642533936652, 'fraction_numerical': 0.0029411764705882353, 'mean_word_length': 2.7883461868037704, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '340439', 'license': 'MIT', 'n_tokens_mistral': 1299, 'n_tokens_neox': 1247, 'n_words': 244, 'path': '02PredictionsForTimeIntervals.ipynb', 'repo_name': 'keoghdata/featuretools_training', 'size': '4420'}