<jupyter_start><jupyter_text>Time Series - Lecture 1: Introduction Agenda- Iintroduction to time series- Datetime library in Python- Wrangle data in Python- Visualize time series data What is / Why Time Series?- Data drawn from a distribution correlated over time. (You have implictly assumed that data was drawn from an independent distribution.)- As a data analyst, you WILL encounter time series data. - Examples:    - Economics/Finance: Stock, bond, other asset prices, government policies    - Science/AI: climate (change) data, speech recognition    - Business: user behavior data     For example,<jupyter_code>from PIL import Image
Image.open("images/sp500.png")<jupyter_output><empty_output><jupyter_text>In a time series, time is often the independent variable and the goal is to make a forecast for the future. From the historical data, we can find some **patterns or trend** which will help us understand the movement and forecast. Some concepts to help you recognize patterns:For example,<jupyter_code>from PIL import Image
Image.open("images/pattern.png")<jupyter_output><empty_output><jupyter_text>1. The monthly housing sales (top left) show seasonality within each year. 2. The US treasury bill contracts (top right) show results from the Chicago market for 100 consecutive trading days in 1981. Here there is no seasonality, but an obvious downward trend. Possibly, if we had a much longer series, we would see that this downward trend is actually part of a long cycle, but when viewed over only 100 days it appears to be a trend.What do you think about the othe two graphs?3. The Australian quarterly electricity production (bottom left).4. The daily change in the Google closing stock price (bottom right).  Time Series in PythonPython has built in tools for dealing with time data, and we will first familiarize ourselves. Data Structure- Many observations from same individual/sample over time (i.e., $Y_t$ where $t=1,...,T$)- Detailed time information (e.g., $X_t=$ 2020-09-23 3-PM)For example,<jupyter_code>import pandas as pd
pd.read_fwf("http://www.cpc.ncep.noaa.gov/products/precip/CWlink/daily_ao_index/monthly.ao.index.b50.current.ascii", parse_dates = [[0, 1]], infer_datetime_format = True, header = None,)<jupyter_output><empty_output><jupyter_text>Datetime in Pandas- Pandas has a build-in feature to handle the time data with ease- Built in data structures    - Time Stamp    - Time Delta    - Time Period- To handle these structures, need numpy package  Timestamp<jupyter_code>import pandas as pd
import numpy as np
from datetime import datetime
pd.date_range('2020 Sep 23', periods = 5, freq = 'S')
pd.Timestamp('2020-09-23')
pd.Timestamp('2020-09-23 09:00')<jupyter_output><empty_output><jupyter_text>Timedelta- Can take both positive and negative values<jupyter_code>pd.Timedelta('1 day')
pd.Timestamp('2016-01-01 10:10') + pd.Timedelta('15 ns')<jupyter_output><empty_output><jupyter_text>Timeperiod- Refer to specific length of time between a start and end time stamp- Fancier setting allowed such as business days<jupyter_code>pd.Period('2020-09-23')
pd.Period('2020-09-23')
pd.period_range('2020-09-23 09:00:00', freq = 'B', periods = 10)
temp = pd.Series(range(10), pd.date_range('2020-09-23 09:00', periods = 10, freq = 'H'))
temp<jupyter_output><empty_output><jupyter_text>Format- There exist a pre-existing datetime object: (year, month, day, hour, minute, second, microsecond, and tzinfo).- The current Gregorian calendar indefinitely extended in both directions.- Often you will the date will be read in a string form, then reformat with "strptime".<jupyter_code>datetime.fromisoformat('2019-12-04')
datetime.strptime("12/11/2018", "%d/%m/%Y")
datetime.fromisoformat('2019-12-04')
datetime.fromisoformat('2019-12-04').max<jupyter_output><empty_output><jupyter_text>Time Zone handling<jupyter_code>pd.Timestamp('9/1/2020 00:00',tz='Europe/London')
temp1 = temp.tz_localize(tz = 'Europe/London')
temp1
temp1.tz_convert('Asia/Tokyo')<jupyter_output><empty_output><jupyter_text>Resampling<jupyter_code>temp = pd.date_range('1/1/2011', periods=10, freq='H')
dates = pd.Series(np.random.randn(len(temp)), index=temp)
dates
dates.asfreq('45Min')
temp1 = pd.Series(np.random.randn(len(temp)), index=temp)
temp1.asfreq('45Min',method="ffill")
temp1 = pd.Series(np.random.randn(len(temp)), index=temp)
temp1.asfreq('45Min',method="bfill")<jupyter_output><empty_output><jupyter_text>Empirical Case: New Year's ResolutionIn this session, we're going to check out Google trends data of keywords 'diet', 'gym' and 'finance' to see how they vary over time. Could there be more searches for these terms in January when we're all trying to turn over a new leaf? We're not going to do much mathematics today but we'll source Our data, visualize it and learn about trends and seasonality in time series data. First if all, we will focus on a visual exploration of the dataset in question, and then we will predict its values sometime in the future.So the question remains: could there be more searches for these terms in January when we're all trying to turn over a new leaf? Let's find out by going [here](https://trends.google.com/trends/explore?date=all&q=diet,gym,finance) and checking out the data (inspired this [this fivethirtyeight piece](https://fivethirtyeight.com/features/how-fast-youll-abandon-your-new-years-resolutions/)).  Reading In DataLet's read in some data:<jupyter_code>df = pd.read_csv('data/multiTimeline.csv', skiprows=1)
df.head()<jupyter_output><empty_output><jupyter_text>Use the .info() method to check out your data types, number of rows and more:<jupyter_code>df.info()
df.columns = ['month', 'diet', 'gym', 'finance']
df.month = pd.to_datetime(df.month)
df.set_index('month', inplace=True)
df.head()<jupyter_output><empty_output><jupyter_text>GraphingVisualization is a powerful tool for an analyst to recognize any pattern in the data.<jupyter_code>df.plot(figsize=(9,9), linewidth=2, fontsize=15)
plt.xlabel('Year', fontsize=20)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10682837680677854, 'fraction_numerical': 0.03206512709752451, 'mean_word_length': 5.8099547511312215, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '445214', 'license': 'MIT', 'n_tokens_mistral': 1988, 'n_tokens_neox': 1825, 'n_words': 646, 'path': 'Time Series Analysis - Lecture 1.ipynb', 'repo_name': 'doublebyte1/Time-Series-Analysis2', 'size': '6019'}