<jupyter_start><jupyter_text>Seasonal Decomposing* The analysis here is to compare residuals after removing the Season and the Trend with Additive or Multiplicative methods.  * Additive Method: `Y[t] = T[t] + S[t] + e[t]`  * Miltiplicative Method: `Y[t] = T[t] * S[t] * e[t]`* The purpose is to observe   * Which decomposing method could generate more random residuals?<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
from statsmodels.tsa.seasonal import seasonal_decompose
sample_train_df1 = pd.read_pickle('sample_train_df1.pkl')
sample_test_df1 = pd.read_pickle('sample_test_df1.pkl')
print(sample_train_df1.shape, sample_test_df1.shape)

sample_train_df2 = pd.read_pickle('sample_train_df2.pkl')
sample_test_df2 = pd.read_pickle('sample_test_df2.pkl')
print(sample_train_df2.shape, sample_test_df2.shape)

sample_train_df1.head()<jupyter_output>(100, 2) (14, 2)
(100, 2) (10, 2)<jupyter_text>Decomposing PlotDecompose trend, seasonality are separated out from data. Sample 1 Additive Decomposing<jupyter_code>col = sample_train_df1['client_count']

decomposition = seasonal_decompose(col, model='additive')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Additive Decomposing - Training Sample 1')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()
col = sample_test_df1['client_count']

decomposition = seasonal_decompose(col, model='additive')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Additive Decomposing - Testing Sample 1')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Multiplicative Decomposing<jupyter_code>col = sample_train_df1['client_count']

decomposition = seasonal_decompose(col, model='multiplicative')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Multiplicative Decomposing - Training Sample 1')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()
col = sample_test_df1['client_count']

decomposition = seasonal_decompose(col, model='multiplicative')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Multiplicative Decomposing - Testing Sample 1')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Note* It seems that, for Sample 1, multiplicative method is better since its residuasl look more random, the residuals from the additive method has a bit trend left.  Sample 2 Additive Decomposing<jupyter_code>col = sample_train_df2['client_count']

decomposition = seasonal_decompose(col, model='additive')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Additive Decomposing - Training Sample 2')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()
col = sample_test_df2['client_count']

decomposition = seasonal_decompose(col, model='additive', period=2)
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Additive Decomposing - Testing Sample 2')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>Multiplicative Decomposing<jupyter_code>col = sample_train_df2['client_count']

decomposition = seasonal_decompose(col, model='multiplicative')
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Multiplicative Decomposing - Training Sample 2')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()
col = sample_test_df2['client_count']

decomposition = seasonal_decompose(col, model='multiplicative', period=2)
trend = decomposition.trend
seasonal = decomposition.seasonal
residual = decomposition.resid

plt.figure(figsize=(10,10))
plt.subplot(411)
plt.plot(col, label='Original')
plt.title('Multiplicative Decomposing - TestingSample 2')
plt.legend(loc='best')
plt.subplot(412)
plt.plot(trend, label='Trend')
plt.legend(loc='best')
plt.subplot(413)
plt.plot(seasonal,label='Seasonality')
plt.legend(loc='best')
plt.subplot(414)
plt.plot(residual, label='Residuals')
plt.legend(loc='best')
plt.tight_layout()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14915999371957922, 'fraction_numerical': 0.027633851468048358, 'mean_word_length': 8.740061162079511, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '525510', 'license': 'MIT', 'n_tokens_mistral': 2584, 'n_tokens_neox': 2261, 'n_words': 391, 'path': 'Bank_Fantasy/Golden_Bridge/seasonal_decomposing.ipynb', 'repo_name': 'hanhanwu/Hanhan_Break_the_Limits', 'size': '6369'}