<jupyter_start><jupyter_text>Predicting Time Series Data> If you want to predict patterns from data over time, there are special considerations to take in how you choose and construct your model. This chapter covers how to gain insights into the data before fitting your model, as well as best-practices in using predictive modeling for time series data. This is the Summary of lecture "Machine Learning for Time Series Data in Python", via datacamp.- toc: true - badges: true- comments: true- author: <NAME>- categories: [Python, Datacamp, Time_Series_Analysis, Machine_Learning]- image: images/price_percentile.png<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

plt.rcParams['figure.figsize'] = (10, 5)
plt.style.use('fivethirtyeight')<jupyter_output><empty_output><jupyter_text>Predicting data over time- Correlation and regression    - Regression is similar to calculating correlation, with some key differences        - Regression: A process that results in a formal model of the data        - Correlation: A statistic that describes the data. Less information than regression model- Correlation between variables often changes over time    - Time series often have patterns that change over time    - Two timeseries that seem correlated at one moment may not remain so over time.- Scoring regression models    - Two most common methods:        - Correlation ($r$)        - Coefficient of Determination ($R^2$)            - The value of $R^2$ is bounded on the top by 1, and can be infinitely low            - Values closer to 1 mean the model does a better jot of predicting outputs \       $1 - \frac{\text{error}(model)}{\text{variance}(testdata)}$<jupyter_code>prices = pd.read_csv('./dataset/tsa_prices.csv', index_col='date', parse_dates=True)
prices.head()
# Plot the raw values over time
prices.plot();
# Scatterplot with one company per axis
prices.plot.scatter('EBAY', 'YHOO');
# Scatterplot with color relating to time
prices.plot.scatter('EBAY', 'YHOO', c=prices.index, cmap=plt.cm.viridis, colorbar=False);<jupyter_output><empty_output><jupyter_text>Fitting a simple regression modelNow we'll look at a larger number of companies. Recall that we have historical price values for many companies. Let's use data from several companies to predict the value of a test company. You'll attempt to predict the value of the Apple stock price using the values of NVidia, Ebay, and Yahoo. Each of these is stored as a column in the all_prices DataFrame. Below is a mapping from company name to column name:```ebay: "EBAY"nvidia: "NVDA"yahoo: "YHOO"apple: "AAPL"```We'll use these columns to define the input/output arrays in our model.<jupyter_code>all_prices = pd.read_csv('./dataset/all_prices.csv', index_col=0, parse_dates=True)
all_prices.head()
from sklearn.linear_model import Ridge
from sklearn.model_selection import cross_val_score

# Use stock symbols to extract training data
X = all_prices[['EBAY', 'NVDA', 'YHOO']]
y = all_prices[['AAPL']]

# Fit and score the model with cross-validation
scores = cross_val_score(Ridge(), X, y, cv=3)
print(scores)<jupyter_output>[-6.09050633 -0.3179172  -3.72957284]<jupyter_text>Visualizing predicted valuesWhen dealing with time series data, it's useful to visualize model predictions on top of the "actual" values that are used to test the model.In this exercise, after splitting the data (stored in the variables ```X``` and ```y```) into training and test sets, you'll build a model and then visualize the model's predictions on top of the testing data in order to estimate the model's performance.<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

# Split our data into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.8,
                                                   shuffle=False, random_state=1)

# Fit our model and generate predictions
model = Ridge()
model.fit(X_train, y_train)
predictions = model.predict(X_test)
score = r2_score(y_test, predictions)
print(score)
# Visualize our predictions along with the "true" values, and print the score
fig, ax = plt.subplots(figsize=(15, 5))
ax.plot(range(len(y_test)), y_test, color='k', lw=3);
ax.plot(range(len(predictions)), predictions, color='r', lw=2);<jupyter_output><empty_output><jupyter_text>Advanced time series prediction- Data is messy    - Real-world data is often messy    - The two most common problems are missing data and outliers    - This often happens because of human error, machine error malfunction, database failure, etc..    - Visualizing your raw data makes it easier to spot these problems- Interpolation: using time to fill in missing data    - A common way to deal with missing data is to interpolate missing values    - With timeseries data, you can use time to assist in interpolation.    - In this case, interpolation means using the known values on either side of a gap in the data to make assumptions about what's missing- Using a rolling window to transform data    - Another common use of rolling windows is to transform the data- Finding outliers in your data    - Outliers are datapoints that are significantly statistically different from the dataset.    - They can have negative effects on the predictive power of your model, biasing it away from its "true" value    - One solution is to remove or replace outliers with a more representative value> Note: Be very careful about doing this - often it is difficult to determine what is a legitimately extreme value vs an abberation.  Visualizing messy dataLet's take a look at a new dataset - this one is a bit less-clean than what you've seen before.As always, you'll first start by visualizing the raw data. Take a close look and try to find datapoints that could be problematic for fitting models.<jupyter_code>prices = pd.read_csv('./dataset/prices_null.csv', index_col=0, parse_dates=True)
# Visualize the dataset
prices.plot(legend=False);
plt.tight_layout();

# Count the missing values of each time series
missing_values = prices.isnull().sum()
print(missing_values)<jupyter_output>EBAY    273
NVDA    502
YHOO    232
dtype: int64<jupyter_text>Imputing missing valuesWhen you have missing data points, how can you fill them in?In this exercise, you'll practice using different interpolation methods to fill in some missing values, visualizing the result each time. But first, you will create the function (```interpolate_and_plot()```) you'll use to interpolate missing data points and plot them.<jupyter_code># Create a function we'll use to interpolate and plot
def interpolate_and_plot(prices, interpolation):
    
    # Create a boolean mask for missing values
    missing_values = prices.isna()
    
    # Interpolate the missing values
    prices_interp = prices.interpolate(interpolation)
    
    # Plot the results, highlighting the interpolated values in black
    fig, ax = plt.subplots(figsize=(10, 5))
    prices_interp.plot(color='k', alpha=0.6, ax=ax, legend=False);
    
    # Note plot the interpolated values on top in red
    prices_interp[missing_values].plot(ax=ax, color='r', lw=3, legend=False);
# Interpolate using the latest non-missing value
interpolation_type = 'zero'
interpolate_and_plot(prices, interpolation_type)
# Interpolate using the latest non-missing value
interpolation_type = 'linear'
interpolate_and_plot(prices, interpolation_type)
# Interpolate with a quadratic function
interpolation_type = 'quadratic'
interpolate_and_plot(prices, interpolation_type)<jupyter_output><empty_output><jupyter_text>Transforming raw dataIn the last chapter, you calculated the rolling mean. In this exercise, you will define a function that calculates the percent change of the latest data point from the mean of a window of previous data points. This function will help you calculate the percent change over a rolling window.This is a more stable kind of time series that is often useful in machine learning.<jupyter_code># Your custom function
def percent_change(series):
    # Collect all *but* the last value of this window, then the final value
    previous_values = series[:-1]
    last_value = series[-1]
    
    # Calculate the % difference between the last value and the mean of earlier values
    percent_change = (last_value - np.mean(previous_values)) / np.mean(previous_values)
    return percent_change

# Apply your custom function and plot
prices_perc = prices.rolling(20).apply(percent_change)
prices_perc.loc["2014":"2015"].plot();<jupyter_output><empty_output><jupyter_text>Handling outliersIn this exercise, you'll handle outliers - data points that are so different from the rest of your data, that you treat them differently from other "normal-looking" data points. You'll use the output from the previous exercise (percent change over time) to detect the outliers. First you will write a function that replaces outlier data points with the median value from the entire time series.<jupyter_code>def replace_outliers(series):
    # Calculate the absolute difference of each timepoint from the series mean
    absolute_differences_from_mean = np.abs(series - np.mean(series))
    
    # Calculate a mask for the difference that are > 3 standard deviations from zero
    this_mask = absolute_differences_from_mean > (np.std(series) * 3)
    
    # Replace these values with the median across the data
    series[this_mask] = np.nanmedian(series)
    return series

# Apply your preprocessing functino to the timeseries and plot the results
prices_perc = prices_perc.apply(replace_outliers)
prices_perc.loc["2014":"2015"].plot();<jupyter_output><empty_output><jupyter_text>Creating features over time- Calculating "date-based" features    - Thus far we've focused on calculating "statistical" features - these are features that correspond statistical properties of the data, like "mean" , "standard deviation", etc    - However, don't forget the timeseries data oftenhas more "human" features associated with it, like days of the week, holidays, etc.    - These features are often useful when dealing with timeseries data that spans multiple years (such as stock value over time)  Engineering multiple rolling features at onceNow that you've practiced some simple feature engineering, let's move on to something more complex. You'll calculate a collection of features for your time series data and visualize what they look like over time. This process resembles how many other time series models operate.<jupyter_code># Define a rolling window with Pandas, excluding the right-most datapoint of the window
prices_perc_rolling = prices_perc['EBAY'].rolling(20, min_periods=5, closed='right')

# Define the features you'll calculate for each window
features_to_calculate = [np.min, np.max, np.mean, np.std]

# Calculate these features for your rolling window object
features = prices_perc_rolling.aggregate(features_to_calculate)

# Plot the results
ax = features.loc[:"2011-01"].plot();
prices_perc['EBAY'].loc[:"2011-01"].plot(ax=ax, color='k', alpha=0.2, lw=3);
ax.legend(loc=(1.01, 0.6));<jupyter_output><empty_output><jupyter_text>Percentiles and partial functionsIn this exercise, you'll practice how to pre-choose arguments of a function so that you can pre-configure how it runs. You'll use this to calculate several percentiles of your data using the same ```percentile()``` function in numpy.<jupyter_code>from functools import partial
percentiles = [1, 10, 25, 50, 75, 90, 99]

# Use a list comprehension to create a partial function for each quantile
percentile_functions = [partial(np.percentile, q=percentile) for percentile in percentiles]

# Calculate each of these quantiles on the data using a rolling window
prices_perc_rolling = prices_perc['EBAY'].rolling(20, min_periods=5, closed='right')
features_percentiles = prices_perc_rolling.aggregate(percentile_functions)

# Plot a subset of the result
ax = features_percentiles.loc[:"2011-01"].plot(cmap=plt.cm.viridis);
ax.legend(percentiles, loc=(1.01, 0.5));
plt.savefig('../images/price_percentile.png')<jupyter_output><empty_output><jupyter_text>Using "date" informationIt's easy to think of timestamps as pure numbers, but don't forget they generally correspond to things that happen in the real world. That means there's often extra information encoded in the data such as "is it a weekday?" or "is it a holiday?". This information is often useful in predicting timeseries data.<jupyter_code># Extract date features from the data, add them as columns
prices_perc['day_of_week'] = prices_perc.index.dayofweek
prices_perc['week_of_year'] = prices_perc.index.weekofyear
prices_perc['month_of_year'] = prices_perc.index.month

# Print prices_perc
print(prices_perc)<jupyter_output>EBAY      NVDA      YHOO  day_of_week  week_of_year  \
date                                                                  
2010-01-04       NaN       NaN       NaN            0             1   
2010-01-05       NaN       NaN       NaN            1             1   
2010-01-06       NaN       NaN       NaN            2             1   
2010-01-07       NaN       NaN       NaN            3             1   
2010-01-08       NaN       NaN       NaN            4             1   
...              ...       ...       ...          ...           ...   
2016-12-23  0.024842  0.147720 -0.038006            4            51   
2016-12-27  0.038030  0.014446 -0.027985            1            52   
2016-12-28  0.026925  0.117737 -0.029311            2            52   
2016-12-29  0.021850  0.129669 -0.028632            3            52   
2016-12-30  0.007285  0.068559 -0.026603            4            52   

            month_of_year  
date                       
2010-01-04          [...]
{'fraction_non_alphanumeric': 0.07555814122609265, 'fraction_numerical': 0.025743582284924734, 'mean_word_length': 3.9290322580645163, 'pattern_counts': {'":': 2, '<': 43, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '19513', 'license': 'Apache-2.0', 'n_tokens_mistral': 4104, 'n_tokens_neox': 3765, 'n_words': 1557, 'path': '_notebooks/2020-06-18-01-Predicting-Time-Series-Data.ipynb', 'repo_name': 'AntonovMikhail/chans_jupyter', 'size': '13758'}