<jupyter_start><jupyter_text>we also refactored it for speed by replacing parse_dates.because parse_dates reads the string and tries to infer the structure, it can take forever. However if I tell explicitely what the structure is, then it's much faster. Lookup "strftime" formatbut basically in the package we changed it to data.index=pd.to_datetime(data.index,format="%m/%d/%Y %H:%M:%S %p")<jupyter_code>data.head()

weekly_data=data.resample("W").sum()

weekly_data.plot()


data.resample("W").sum().plot()

#you have data by the hour, so when you say you want it on a daily basis choose if you want a mean or a sum of what happens in a day.
daily_average=data.resample("D").mean() #.head()
daily_total=data.resample("D").sum() #.head()
daily_total_rolling_sum=data.resample("D").sum().rolling(365).sum() #take that daily total and establish a rolling sum 
daily_total_rolling_sum.plot()

#setting the y axis

ax=data.resample("D").sum().rolling(365).sum().plot() #notice you are setting this plot as a variable ax
ax.set_ylim(0,None); #saying that y should start at 0


print(data.index) #this gives whatever you choose to be your index col
print(data.index.time) #turns that string of time into datetime for the TIME OF DAY ONLY

print(data.groupby(data.index.time).mean().head())
#looking at it intra day
#groupby simply orders them by similar groups. in this case they're grouped based on the time of day
#so data.index.time gets rid of the days part, just keeps time of day. And you group by all those who have the same time of day.
#print(data.groupby(data.index.time).mean().head()) 

data.groupby(data.index.time).mean().plot();
#for pivot tables, think of excel pivot tables where you can sort entire table based on column keys
#in pivot_table mode, your columns setting is no longer "extra data", it's just another way of segmenting
#the data you selected under "values"

pivoted=data.pivot_table(values=["Total"], index=data.index.time)
print(pivoted.iloc[:5])

#you can select your aggregation funciton. By default it's always the mean, but here you set it to sum
#basically you have the sum of all the bike rides by each our of everyday.
pivoted=data.pivot_table(values=["Total"], index=data.index.time,aggfunc=[np.sum])
print(pivoted.iloc[:5])

#the columns give you an extra way to segment your data

pivoted=data.pivot_table("Total", index=data.index.time, columns=data.index.date)
print(pivoted.iloc[:5,:5])

#when you plot it, you can think as your index as what's on your X
pivoted.plot(legend=False,alpha=0.01) #takes a while because you have a line for each day of the day

#so you can see how this would have been different from simply plotting the original data
#where each day is one after the other. here each day is it's own entire line
#plot() will plot each column, here the number of columns is the number of days

#you'll notice that most of there's some noise in middle, but most of the time
# the traffic happens during commute times

#the following will bring up the """ whatever is here""" that's inside the function 
#
get_data?<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10145395799676898, 'fraction_numerical': 0.004846526655896607, 'mean_word_length': 5.082514734774067, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '218461', 'license': 'MIT', 'n_tokens_mistral': 966, 'n_tokens_neox': 928, 'n_words': 408, 'path': 'bike_ride_example.ipynb', 'repo_name': 'alexkeenan/jupyternotebookworkflow', 'size': '3095'}