<jupyter_start><jupyter_text>Kaggle Bike Sharing Demand DatasetModified 'count' to log1p(count) for trainingLog can be used when target represents a count (that is non-negative values)Model now predicts as log1p(count). We need to convert it back to actual count using expm1(predicted_target)Reference:https://www.kaggle.com/apapiu/predicting-bike-sharing-with-xgboost by <NAME>To download dataset, sign-in and download from this link:https://www.kaggle.com/c/bike-sharing-demand/data Input Features: ['season', 'holiday', 'workingday', 'weather', 'temp',       'atemp', 'humidity', 'windspeed', 'year', 'month', 'day', 'dayofweek','hour']Target Feature: [log1p('count')]Objective: You are provided hourly rental data spanning two years. For this competition, the training set is comprised of the first 19 days of each month, while the test set is the 20th to the end of the month. You must predict the total count of bikes rented during each hour covered by the test set, using only information available prior to the rental period (Ref: Kaggle.com)<jupyter_code># Example
# Converts to log1p(count)
# Print original count back using expm1
print('Test log and exp')
test_count = 100
print('original value', test_count)
x = np.log1p(test_count) # log (x+1)
print('log1p', x)
print('expm1', np.expm1(x)) # exp(x) - 1
columns = ['count', 'season', 'holiday', 'workingday', 'weather', 'temp',
       'atemp', 'humidity', 'windspeed', 'year', 'month', 'day', 'dayofweek','hour']
df = pd.read_csv('train.csv', parse_dates=['datetime'],index_col=0)
df_test = pd.read_csv('test.csv', parse_dates=['datetime'],index_col=0)
# We need to convert datetime to numeric for training.
# Let's extract key features into separate numeric columns
def add_features(df):
    df['year'] = df.index.year
    df['month'] = df.index.month
    df['day'] = df.index.day
    df['dayofweek'] = df.index.dayofweek
    df['hour'] = df.index.hour
add_features(df)
add_features(df_test)
plt.plot(df['2011']['count'],label='2011')
plt.plot(df['2012']['count'],label='2012')
plt.xticks(fontsize=14, rotation=45)
plt.xlabel('Date')
plt.ylabel('Rental Count')
plt.title('2011 and 2012 Rentals (Year to Year)')
plt.legend()
plt.show()
plt.plot(df['2011']['count'].map(np.log1p),label='2011')
plt.plot(df['2012']['count'].map(np.log1p),label='2012')
plt.xticks(fontsize=14, rotation=45)
plt.xlabel('Date')
plt.ylabel('Log(Rental Count)')
plt.title('2011 and 2012 Rentals (Year to Year)')
plt.legend()
plt.show()
plt.boxplot([df['count']], labels=['count'])
plt.title('Box Plot - Count')
plt.ylabel('Target')
plt.grid(True)
# Let's see how the data distribution changes with log1p
# Evenly distributed
plt.boxplot([df['count'].map(np.log1p)], labels=['log1p(count)'])
plt.title('Box Plot - log1p(Count)')
plt.ylabel('Target')
plt.grid(True)
df["count"] = df["count"].map(np.log1p)
df.head()
df_test.head()
df.dtypes
# Save all data
df.to_csv('bike_all.csv',index=True,index_label='datetime',columns=columns)<jupyter_output><empty_output><jupyter_text>Training and Validation Set Target Variable as first column followed by input features Training, Validation files do not have a column header<jupyter_code># Training = 70% of the data
# Validation = 30% of the data
# Randomize the datset
np.random.seed(5)
l = list(df.index)
np.random.shuffle(l)
df = df.loc[l]
rows = df.shape[0]
train = int(.7 * rows)
test = rows-train
rows, train, test
columns
# Write Training Set
df.iloc[:train].to_csv('bike_train.csv'
                          ,index=False,header=False
                          ,columns=columns)
# Write Validation Set
df.iloc[train:].to_csv('bike_validation.csv'
                          ,index=False,header=False
                          ,columns=columns)
# Test Data has only input features
df_test.to_csv('bike_test.csv',index=True,index_label='datetime')
print(','.join(columns))
# Write Column List
with open('bike_train_column_list.txt','w') as f:
    f.write(','.join(columns))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15500376222723852, 'fraction_numerical': 0.022824178580386257, 'mean_word_length': 5.280314960629921, 'pattern_counts': {'":': 0, '<': 10, '<?xml version=': 0, '>': 10, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '107961', 'license': 'Apache-2.0', 'n_tokens_mistral': 1428, 'n_tokens_neox': 1320, 'n_words': 354, 'path': 'DeepLearning/BikeSharingRegression/bikerental_data_preparation_rev3.ipynb', 'repo_name': 'jaypeeml/AWSSagemaker', 'size': '3996'}