<jupyter_start><jupyter_text>(Ford GoBike System Data Exploration) by (<NAME>) Preliminary WranglingGoBike Data is a dataset for bike trips happened on Febraury 2018, Each trip is anonymized and includes:-{ Trip Duration (seconds), Start Time and Date, End Time and Date, Start Station ID, Start Station Name, Start Station LatitudeStart Station Longitude, End Station ID, End Station Name, End Station Latitude, End Station Longitude, Bike ID andUser Type (Subscriber or Customer – “Subscriber” = Member or “Customer” = Casual) }.data source: https://www.lyft.com/bikes/bay-wheels/system-data<jupyter_code># import all packages and set plots to be embedded inline
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sb
import calendar

%matplotlib inline
df= pd.read_csv('Ford GoBike.csv')
df.head()<jupyter_output><empty_output><jupyter_text>Data Assessing> In this section, we will check the data, if it is clean or need more pre-processing steps.<jupyter_code># get information about the data
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 106718 entries, 0 to 106717
Data columns (total 14 columns):
duration_sec               106718 non-null int64
start_time                 106718 non-null object
end_time                   106718 non-null object
start_station_id           106718 non-null int64
start_station_name         106718 non-null object
start_station_latitude     106718 non-null float64
start_station_longitude    106718 non-null float64
end_station_id             106718 non-null int64
end_station_name           106718 non-null object
end_station_latitude       106718 non-null float64
end_station_longitude      106718 non-null float64
bike_id                    106718 non-null int64
user_type                  106718 non-null object
bike_share_for_all_trip    106718 non-null object
dtypes: float64(4), int64(4), object(6)
memory usage: 11.4+ MB<jupyter_text>from above info we need to convert start_time, end_time from object to pandas Datetime and user_type from object to category.<jupyter_code>df[['start_time','end_time']]= df[['start_time','end_time']].apply(pd.to_datetime, format='%Y-%m-%d %H:%M:%S.%f')
df['user_type']= df['user_type'].astype('category')
# check if the data have missing values
nan_counts = df.isna().sum()
base_color = sb.color_palette()[0]
sb.barplot(x= nan_counts.index.values, y= nan_counts, color = base_color)
plt.xticks(rotation=90);
nan_counts<jupyter_output><empty_output><jupyter_text>that is good data has no missing values.<jupyter_code>#let`s check for duplicates
df.duplicated().sum()<jupyter_output><empty_output><jupyter_text>that is good data has no duplicated rows.<jupyter_code>df.shape<jupyter_output><empty_output><jupyter_text>What is the structure of your dataset?> the dataset contains 106718 rows and 14 columns What is/are the main feature(s) of interest in your dataset?> I`m interested in duration to see the average duration of biking, user_type (customer, subscriber) and start station to see the most visited stations. What features in the dataset do you think will help support your investigation into your feature(s) of interest?> I think start time will be used to analyse time based events, user_type, duration_sec and station names but longitude and latitude out of my analysis.    Univariate Exploration> In this section, investigate distributions of individual variables. Ifyou see unusual points or outliers, take a deeper look to clean things upand prepare yourself to look at relationships between variables.<jupyter_code># let`s explore the distribution of duration varible using histogram
plt.figure(figsize=[8,6])
bins= 10** np.arange(np.log10(df.duration_sec.min()), np.log10(df.duration_sec.max())+0.02, 0.02)
plt.hist(data=df, x='duration_sec',bins=bins)
plt.xscale('log')
plt.xticks([50,100,200,300,500, 1e3, 2e3, 5e3, 1e4, 2e4], [50,100,200,300,500, '1k', '2k', '5k', '10k', '20k'])
plt.xlabel('duration (seconds)')
plt.ylabel('frequency');<jupyter_output><empty_output><jupyter_text>it seems to be normally distributed, the most common values almost in the range of 200:2K seconds.<jupyter_code># let`s explore the distribution of user_type varible using bar chart
base_color= sb.color_palette()[0]
sb.countplot(data=df, x='user_type',color=base_color);<jupyter_output><empty_output><jupyter_text>we noticed that the Subscriber type is the most common type.  let`s get some insights from start time<jupyter_code># we will extract month, dayofweek, hour from the start_time
df['start_month']= df.start_time.dt.strftime('%b')
df['start_day_week']= df.start_time.dt.strftime('%a')
df['start_hour']= df.start_time.dt.strftime('%H')
df.head()<jupyter_output><empty_output><jupyter_text>refrence https://docs.python.org/2/library/datetime.htmlstrftime-strptime-behavior<jupyter_code>fig, ax = plt.subplots(nrows=3, figsize = [8,8])

default_color = sb.color_palette()[0]
sb.countplot(data = df, x = 'start_month', color = default_color, order=df.start_month.value_counts().index, ax = ax[0])
sb.countplot(data = df, x = 'start_day_week', color = default_color, order=df.start_day_week.value_counts().index, ax = ax[1])
sb.countplot(data = df, x = 'start_hour', color = default_color, order=df.start_hour.value_counts().index, ax = ax[2]);<jupyter_output><empty_output><jupyter_text>from above figures we can see that:-1- all data happened at February month.2- Tuesday, Thursday, Wednesday almost equal and have the most bikers.3- 8 AM, 5 PM is the most hours have bikers.<jupyter_code># the most 10 start station
df.start_station_name.value_counts(sort=True, ascending=False)[:10].plot(kind='bar');<jupyter_output><empty_output><jupyter_text>the most 10 start station with bikers.  Bivariate Exploration> In this section, investigate relationships between pairs of variables in yourdata. Make sure the variables that you cover here have been introduced in somefashion in the previous section (univariate exploration).<jupyter_code># let`s explore the relationship between duarion and user_type using boxplot 
plt.figure(figsize=[8,6])
sb.boxplot(data=df, x='user_type', y='duration_sec', color=base_color)
plt.yscale('log')
plt.yticks([50,100,200,500, 1e3, 2e3, 5e3, 1e4, 2e4,4e4,8e4], [50,100,200,500, '1k', '2k', '5k', '10k', '20k','40k','80k']);<jupyter_output><empty_output><jupyter_text>- although the number of subscriber is greater than number of customer, the average customer duration is higher than subscriber.<jupyter_code>sb.barplot(data = df, x = 'start_day_week', y = 'duration_sec', palette='Blues',order=df.start_day_week.value_counts().index)
plt.ylabel('Avg. Duration (seconds)');<jupyter_output><empty_output><jupyter_text>- Saturday and Sunday have the most average duration.<jupyter_code># let`s explore the relation between user_type and day of week using 
ax = sb.countplot(data = df, x = 'start_day_week', hue = 'user_type')
ax.legend(loc = 9, framealpha = 0.2, title = 'user_type');<jupyter_output><empty_output><jupyter_text>- Saturday has the most bikers in customer type- Tuesday, Wedensday, Thursday has the most bikers in subscriber type<jupyter_code># let`s explore the relation between user_type and hour of day
ax = sb.countplot(data = df, x = 'start_hour', hue = 'user_type')
ax.legend(loc = 2, framealpha = 0.2, title = 'user_type');<jupyter_output><empty_output><jupyter_text>- 12 PM, 3 PM, 4 PM, 5 PM has the most bikers in customer type.- 8 AM and 5 PM has the most bikers in subscriber type.<jupyter_code>#let`s explore the relationship between day of week and hour of day
plt.figure(figsize=[15,6])
sb.countplot(data = df, x = 'start_hour', hue = 'start_day_week', palette = 'Blues');<jupyter_output><empty_output><jupyter_text>- Wedensday and Thursday 8 AM have the most bikers.  Multivariate Exploration> Create plots of three or more variables to investigate your data evenfurther. Make sure that your investigations are justified, and follow fromyour work in the previous sections.<jupyter_code>plt.figure(figsize=[8,6])
sb.heatmap(df.corr(), annot = True, fmt = '.2f', cmap = 'vlag_r', center = 0);<jupyter_output><empty_output><jupyter_text>- there are linear (positive, negative) relationship between some features like (start_station_longitude, end_station_station_longitude) and (start_station_id, end_station_longitude) but have no meaning<jupyter_code>plt.figure(figsize=[14,6])
sb.pointplot(data = df, x = 'start_hour', y = 'duration_sec', hue = 'start_day_week',
             dodge = 0.5, linestyles = "",palette='Blues')
plt.yscale('log')
plt.yticks([50,100,200,500, 1e3, 2e3, 5e3, 1e4, 2e4,4e4,8e4], [50,100,200,500, '1k', '2k', '5k', '10k', '20k','40k','80k']);<jupyter_output><empty_output><jupyter_text>- Friday and Thursday have the most bikers than other days of week.- Wedensday 3 AM has the most duration.<jupyter_code>g = sb.FacetGrid(data = df, col = 'start_day_week',col_wrap=2, height = 4)
g.map(sb.boxplot, 'user_type', 'duration_sec',order=df.user_type.value_counts().index)
plt.yscale('log')
plt.yticks([50,100,200,500, 1e3, 2e3, 5e3, 1e4, 2e4,4e4,8e4], [50,100,200,500, '1k', '2k', '5k', '10k', '20k','40k','80k']);<jupyter_output><empty_output><jupyter_text>- during all days of week the median (duration) of customer type is higher than the median (duration) of subscriber type<jupyter_code>plt.figure(figsize=[10,6])
ax = sb.barplot(data = df, x = 'start_hour', y = 'duration_sec', hue = 'user_type')
ax.legend(loc = 1, framealpha = 1, title = 'user_type')
plt.yscale('log')
plt.yticks([50,100,200,500, 1e3, 2e3, 5e3, 1e4, 2e4,4e4,8e4], [50,100,200,500, '1k', '2k', '5k', '10k', '20k','40k','80k']);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12050936949994824, 'fraction_numerical': 0.04627808261724816, 'mean_word_length': 6.03056768558952, 'pattern_counts': {'":': 0, '<': 78, '<?xml version=': 0, '>': 85, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '283963', 'license': 'Apache-2.0', 'n_tokens_mistral': 3571, 'n_tokens_neox': 3233, 'n_words': 956, 'path': 'notebook/Data Exploration.ipynb', 'repo_name': 'Aboalarbe/Communicate-Data-Findings-GoBike-System-Data', 'size': '9674'}