<jupyter_start><jupyter_text>[amitnikhade.com](http://amitnikhade.com)  Future sales Prediction using:* LSTM* Linear regression  Pipeline1. Problem Definition2. Analyzing data3. Preparing train and test purpose data4. creating model5. Evaluate model6. Improve performance7. Present performances8. Save results  Load Data<jupyter_code>test = pd.read_csv('../input/competitive-data-science-predict-future-sales/test.csv')
item_categories = pd.read_csv('../input/competitive-data-science-predict-future-sales/item_categories.csv')
items = pd.read_csv('../input/competitive-data-science-predict-future-sales/items.csv')
shops = pd.read_csv('../input/competitive-data-science-predict-future-sales/shops.csv')
sales = pd.read_csv('../input/competitive-data-science-predict-future-sales/sales_train.csv',  parse_dates=['date'], infer_datetime_format=True, dayfirst=True)<jupyter_output><empty_output><jupyter_text>Analyze data<jupyter_code>print(sales.head())
print('____________________________')
print(sales.info())
print('____________________________')
print(sales.describe())
print(test.head())
print('____________________________')
print(test.info())
print('____________________________')
print(test.describe())
print(items.head())
print('____________________________')
print(items.info())
print('____________________________')
print(items.describe())
print(shops.head())
print('____________________________')
print(shops.info())
print('____________________________')
print(shops.describe())
print(item_categories.head())
print('____________________________')
print(item_categories.info())
print('____________________________')
print(item_categories.describe())
print(shops.head())
print('____________________________')
print(shops.info())
print('____________________________')
print(shops.describe())<jupyter_output>shop_name  shop_id
0   !Якутск Орджоникидзе, 56 фран        0
1   !Якутск ТЦ "Центральный" фран        1
2                Адыгея ТЦ "Мега"        2
3  Балашиха ТРК "Октябрь-Киномир"        3
4        Волжский ТЦ "<NAME>"        4
____________________________
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 60 entries, 0 to 59
Data columns (total 2 columns):
 #   Column     Non-Null Count  Dtype 
---  ------     --------------  ----- 
 0   shop_name  60 non-null     object
 1   shop_id    60 non-null     int64 
dtypes: int64(1), object(1)
memory usage: 1.1+ KB
None
____________________________
         shop_id
count  60.000000
mean   29.500000
std    17.464249
min     0.000000
25%    14.750000
50%    29.500000
75%    44.250000
max    59.000000<jupyter_text>Dataset merging operation<jupyter_code>df_item=pd.merge(items,item_categories,on='item_category_id',how='inner')
sales_train=pd.merge(sales,shops,on='shop_id',how='inner')
sales=pd.merge(sales_train,df_item,on='item_id',how='inner')<jupyter_output><empty_output><jupyter_text>keeping only the data in train dataset, present in the test dataset.<jupyter_code>sales = sales[sales['shop_id'].isin(test['shop_id'].unique())]
sales = sales[sales['item_id'].isin(test['item_id'].unique())]
sales<jupyter_output><empty_output><jupyter_text>Outliers<jupyter_code>import seaborn as sns
sns.boxplot(x=sales.item_cnt_day)
import seaborn as sns
sns.boxplot(x=sales.item_price)<jupyter_output><empty_output><jupyter_text>Remove outliers<jupyter_code>sales = sales[(sales.item_price < 300000 )& (sales.item_cnt_day < 1000)]
# remove negative item price
sales = sales[sales.item_price > 0].reset_index(drop = True)<jupyter_output><empty_output><jupyter_text>Aggregation on monthly basis<jupyter_code>sales = sales.groupby(["date_block_num","shop_id","item_id"])[['date_block_num','date', 'shop_id', 'item_id', 'item_price', 'item_cnt_day']].agg({"date_block_num":'mean',"date":["min",'max'],"item_price":"mean","item_cnt_day":"sum"})

sales
sales = sales.item_cnt_day.apply(list).reset_index()
sales<jupyter_output><empty_output><jupyter_text>**merging test with sales data on item_id and shop_id**<jupyter_code>sales_data = pd.merge(test,sales,on = ['item_id','shop_id'],how = 'left')<jupyter_output><empty_output><jupyter_text>**fill Nan values with 0 and drop shop_id and item_id**<jupyter_code>sales_data.fillna(0,inplace = True)
sales_data.drop(['shop_id','item_id'],inplace = True, axis = 1)
sales_data<jupyter_output><empty_output><jupyter_text>**preparing for time series data format**<jupyter_code>sales_data = sales_data.pivot_table(index = 'ID', columns='date_block_num', values = 'sum', aggfunc='sum')
sales_data<jupyter_output><empty_output><jupyter_text>**Fill NaN values with zero**<jupyter_code>sales_data = sales_data.fillna(0)
sales_data.head(20)<jupyter_output><empty_output><jupyter_text>**dividing data to features and label like a supervised problem**<jupyter_code>X=sales_data[sales_data.columns[:-1]]
y=sales_data[sales_data.columns[-1]]<jupyter_output><empty_output><jupyter_text>**splitting to training and testing**<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, Y_train, Y_test = train_test_split( X, y, test_size=0.20, random_state=1)<jupyter_output><empty_output><jupyter_text>**Linear Regression**<jupyter_code>from sklearn.linear_model import LinearRegression
regressor = LinearRegression()
#fitting data
regressor.fit(X_train, Y_train)<jupyter_output><empty_output><jupyter_text>Performance<jupyter_code>from sklearn.metrics import mean_squared_error
print('Train set mse:', mean_squared_error(Y_train, regressor.predict(X_train)))
print('Test set mse:', mean_squared_error(Y_test, regressor.predict(X_test)))
print('Test set score:', regressor.score(X_train,Y_train))
#cross validation score
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import TimeSeriesSplit
ts_cross_val = TimeSeriesSplit(n_splits=5)
print('cross_val_score',cross_val_score(regressor, X, y, cv = ts_cross_val, scoring= "neg_mean_squared_error"))
submission = pd.DataFrame({'ID':X_test.index,'item_cnt_month':regressor.predict(X_test)})
submission.to_csv('submission.csv',index = False)<jupyter_output><empty_output><jupyter_text>LSTMLong short term memory cells for time series<jupyter_code>import keras
from keras.models import Sequential
from keras.layers import LSTM,Dense,Dropout
from keras.models import load_model, Model

# defining model 
model = Sequential()
model.add(LSTM(units = 128,return_sequences=True,input_shape = (33,1)))
model.add(Dropout(0.5))
model.add(LSTM(units = 64,return_sequences=False,activation='relu'))
# model.add(Dropout(0.5))
model.add(Dense(32,activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(16))
model.add(Dense(1))

# opt = keras.optimizers.Adam(learning_rate=0.0001)
model.compile(loss = 'mse',optimizer = 'Nadam', metrics = ['mean_squared_error'])
model.summary()
#Splitting data for LSTM's
from sklearn.model_selection import train_test_split
X_train = np.expand_dims(sales_data.values[:,:-1],axis = 2)
y_train = sales_data.values[:,-1:]
model.fit(X_train,y_train, batch_size = 1024,epochs = 10, validation_split=0.1)
# performance 
import matplotlib.pyplot as plt
plt.plot(model.history.history['loss'], label='Train loss')
plt.plot(model.history.history['val_loss'], label='Validation loss')
plt.legend(loc='best')
# plt.title
plt.title('Regular LSTM')
plt.xlabel('Epochs')
plt.ylabel('MSE')
train_pred = model.predict(X_train)<jupyter_output><empty_output><jupyter_text>Performance on train data<jupyter_code>print('Train rmse:', np.sqrt(mean_squared_error(y_train, train_pred)))
X_test = sales_data.loc[:,sales_data.columns!=sales_data.columns[0]]
y_test = sales_data[sales_data.columns[0]]
icm = model.predict(np.expand_dims(X_test,axis = 2))<jupyter_output><empty_output><jupyter_text>performance on test data<jupyter_code>print('test rmse:', np.sqrt(mean_squared_error(y_test, icm)))
submission = pd.DataFrame({'ID':X_test.index, 'item_cnt_month':icm.reshape(214200,)})
submission.to_csv('submission.csv',index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14280322093608455, 'fraction_numerical': 0.022269753397081028, 'mean_word_length': 8.15783410138249, 'pattern_counts': {'":': 4, '<': 76, '<?xml version=': 0, '>': 75, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '69347', 'license': 'Apache-2.0', 'n_tokens_mistral': 2946, 'n_tokens_neox': 2750, 'n_words': 432, 'path': 'future-sales-prediction/future-sales-prediction-lstm-linear-regression.ipynb', 'repo_name': 'AmitNikhade/Kaggle', 'size': '7952'}