<jupyter_start><jupyter_text>`Time Series Analysis`  Time series analysis is a statistical technique that deals with time series data, or trend analysis.  Time series data means that data is in a series of  particular time periods or intervals.  The data is considered in three types`1. Time series data` : A set of observations on the values that a variable takes at different times.`2. Cross-sectional data` : Data of one or more variables, collected at the same point in time.`3. Pooled data`: A combination of time series data and cross-sectional data.<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import warnings
warnings.simplefilter("ignore")
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Get data from this link [data](https://finance.yahoo.com/quote/%5EGSPC/history?ltr=1)<jupyter_code>df = pd.read_csv("GSPC.csv")
df.head()<jupyter_output><empty_output><jupyter_text>__Lets see dtypes of the Dataframe using__ `dataframe.dtypes`<jupyter_code>df.dtypes.to_frame()<jupyter_output><empty_output><jupyter_text>__1. lets convert Date column to datetime dtype__<jupyter_code>df["Date"] = pd.to_datetime(df["Date"])
plt.figure(figsize=(12, 8))
plt.plot_date(x = df.Date, y=df.Close,fmt="-")
plt.show()<jupyter_output><empty_output><jupyter_text>**The first obvious thing to note, aside from the two giant dips at the tail end corresponding to the market crashes in 2018-02,2018-4 etc, is that the data is clearly non-stationary. This makes sense for market data as it tends to go up in the long run more than it goes down. This is a problem for time series analysis though as non-stationary data is hard to reason about. The first thing we can try is a first difference of the series. In other words, subtract the previous value t-1 from the current value t to get the difference d(t).**<jupyter_code>df['1st_difference'] = df['Close'] - df['Close'].shift()  
plt.figure(figsize=(12, 8))
plt.plot_date(x = df.Date, y=df["1st_difference"],fmt="-")
plt.show()<jupyter_output><empty_output><jupyter_text>**The data no longer appears to be trending up over time and is instead centered around 0. There's another problem though. Look at the variance. It's very small early on and steadily increases over time. This is a sign that the data is not only non-stationary but also exponentially increasing.To deal with this, we'll apply a log transform to the original series.**<jupyter_code>df['Log_Transform'] = df['Close'].apply(lambda x: np.log(x))    
plt.figure(figsize=(12, 8))
plt.plot_date(x = df.Date, y=df.Log_Transform,fmt="-")
plt.show()<jupyter_output><empty_output><jupyter_text>__So that gives us the original closing price with a log transform applied to "flatten" the data from an exponential curve to a linear curve.__<jupyter_code>df['Logged_Difference'] = df['Log_Transform'] - df['Log_Transform'].shift()  
df.plot(x = "Date",y = "Logged_Difference", figsize=(16, 12))
plt.show()<jupyter_output><empty_output><jupyter_text>__Much better! We now have a stationary time series model of daily changes__ Now let's create some lag variables y(t-1), y(t-2) etc. and examine their relationship to y(t). We'll look at 1 and 2-day lags along with weekly and monthly lags to look for "seasonal" effects.<jupyter_code>df['Lag_1'] = df['Logged_Difference'].shift()  
df['Lag_2'] = df['Logged_Difference'].shift(2)  
df['Lag_5'] = df['Logged_Difference'].shift(5)  
df['Lag_30'] = df['Logged_Difference'].shift(30)
sns.jointplot('Logged_Difference', 'Lag_1', df, kind='reg', size=13);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1116369710467706, 'fraction_numerical': 0.013919821826280624, 'mean_word_length': 5.963178294573644, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '92277', 'license': 'MIT', 'n_tokens_mistral': 1184, 'n_tokens_neox': 1124, 'n_words': 399, 'path': 'DataScience365/Day-16/Time Series Analysis part-1.ipynb', 'repo_name': 'HiteshGorana/DataScience365', 'size': '3592'}