<jupyter_start><jupyter_text>Exploring FordGoBikes Dataset Table of Contents1. Introduction2. Explanatory Data Analysis4. Conclusions     1. Introduction  Dataset Overview: FordGoBikes dataset includes a lot of features as it has more than 180,000 records of bikes trips and their information about individual rides made in a bike-sharing system covering the greater San Francisco Bay area in 2017, including stations coordinates, user types, birth year, gender, which can be easly interpret some insights.  Questions:    Which age subscribe the service often?    Which is the Busiest days in the week?    What is the Peak hours in the day?    Which Gender using the service the most?    What is the Average Number of Trips per Day?    What is the Average Number of Trips per Age?    Which Shares the bike the most?    Which are the most used Stations?    How long Trips take each Day?    What is Number of Trips and Distance per Age?<jupyter_code># Importing libraries
import datetime
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
DF = pd.read_csv('201902-fordgobike-tripdata.csv', parse_dates=['start_time','end_time'])<jupyter_output><empty_output><jupyter_text>Data WranglingReplacing Null Values and Dropping the rest.<jupyter_code>DF["member_birth_year"].fillna(int(DF["member_birth_year"].mean()), inplace = True)
DF["member_gender"].fillna("Other", inplace = True)
DF.dropna(inplace=True)<jupyter_output><empty_output><jupyter_text>Feature EngineeringAdding additional columns for [Age, Trip_distance, Day_time, Day_of_the_week, biker_speed].<jupyter_code># Calculating the Age
DF = DF.astype({"member_birth_year": int})
DF['Age'] = 2021 - DF['member_birth_year']
#Calculating the distance between the destinations
DF['Trip_distance'] = np.sqrt(((DF["start_station_latitude"] - DF["end_station_latitude"])**2) + ((DF["start_station_longitude"] - DF["end_station_longitude"])**2))
#Calculating the Day time(hour) of the trip
DF['Day_time'] = DF.start_time.dt.hour
#Calculating the Day of the week of the trip
DF["Day_week"] = DF['start_time'].dt.day_name()
#Calculating the speed of the bike driver (distance/time)
DF['Biker_speed'] = DF['Trip_distance']/DF['duration_sec']
#Dropping DF['Age'] > 80 years (203 rows)
DF.drop(DF[DF['Age'] > 80].index, inplace = True)<jupyter_output><empty_output><jupyter_text>Function for Sorting the DataFrame & Plotting Bar Chart<jupyter_code>def Sorting(df,column,order):
    """ Description of function:  
    Parameters:
    df (Dataframe): Dataframe as input
    column (Str):   String representation of the column to sort with
    order (bool):   True for ascending, False for descending order
    
    Returns:
    Dataframe: Dataframe orderd by column values"""
    df = df.sort_values(by=[column], ascending=order)
    return df

def Plot(title, xlabel, ylabel, x, y, c, s=4):
    """ Description of function:  
    Parameters:
    title (Dataframe): Dataframe as input
    xlabel (Str):      String representation of the column to sort with
    ylabel (bool):     True for ascending, False for descending order
    x (list):          list or array of values
    y (list):          list or array of values
    c (Str):           String representation of color name
    s (int):           Integer value of figure hight
    
    Returns:
    Dataframe: Dataframe orderd by column values"""
    plt.figure(figsize=(16, s))
    plt.title(title, fontsize=16)
    plt.bar(x, y, color=c, alpha=0.7)
    plt.xlabel(xlabel); plt.ylabel(ylabel); plt.xticks(rotation=90, size=8)
    plt.show()<jupyter_output><empty_output><jupyter_text>2. Explanatory Data Analysis  Which age subscribe the service often:<jupyter_code>plt.figure(figsize=(16, 4))
subscriber = DF[DF['user_type'] == 'Subscriber']
by_subscriber = subscriber.groupby('Age')[['user_type']].count()
sns.barplot(x=by_subscriber.index, y= by_subscriber["user_type"]);
plt.title('Ages using the service', fontsize=16); plt.xlabel('Age'); plt.ylabel('Subscribers');<jupyter_output><empty_output><jupyter_text>Most subscribers lie between 26 to 37 years old.  Which is the Busiest days in the week:<jupyter_code>DW = DF['Day_week'].value_counts()/len(DF)
Plot('Busiest Days', 'Days of Week','Frequency', DW.index, DW.values, 'midnightblue', 4)<jupyter_output><empty_output><jupyter_text>Thursday is the busiest while Saturday as a weekend is the less busy.  What is the Peak hours in the day:<jupyter_code>DT = DF['Day_time'].value_counts()/len(DF)
Plot('Busiest Hours', 'Hours of the Day', 'Frequency', DT.index, DT.values, 'midnightblue', 4)<jupyter_output><empty_output><jupyter_text>Daily peak starts from 8 at morning and begin to decrease from afternoon, and then start to rise again from 3 and reaches the highst point at 5 PM.It's look like we have two peaks at 8AM and 5PM which makes sense, Going to work vs. Leaving.  Which Gender using the service the most:<jupyter_code>Gender_Count = round(DF['member_gender'].value_counts()/len(DF)*100)
fig, axes = plt.subplots(1, 2,figsize=(16, 4))
Gender_Count.plot(kind='pie', ax=axes[0])

prev_count = 0
for cat in range(Gender_Count.shape[0]):
    blocks = np.arange(prev_count, prev_count + Gender_Count[cat]); x = blocks % 10; y = blocks // 10 
    plt.bar(x = x, height = 0.8, width = 0.8, bottom = y); prev_count += Gender_Count[cat]
plt.title('Gender using the service', fontsize=16); plt.xlabel('Gender'); plt.ylabel('Frequency');
plt.legend(Gender_Count.index, bbox_to_anchor = (1, 0.5), loc = 6)
plt.axis('off')
plt.axis('square')
plt.show()<jupyter_output><empty_output><jupyter_text>Male is a dominant in this job, while about 7% didn't specify.  What is the Average Number of Trips per Day:<jupyter_code># ploting the column DF[Day_week] frequency
by_day = DF.groupby('Day_week').count()
fig, axes = plt.subplots(1, 1,figsize=(16, 4))
plt.xlabel('Day_week')
plt.ylabel('Trips')
plt.title('Weekly peak')
plt.plot(DF['Day_week'].unique().tolist(), DF['Day_week'].value_counts().values, c='seagreen');<jupyter_output><empty_output><jupyter_text>Thursday and Wednesday were the highest peak of the trips, While Friday and Saturday were the lowest as they are the weekends in US.  What is the Average Number of Trips per Age:<jupyter_code>Plot('Average Number of Trips per Age', 'Age', 'Trips', DF['Age'].unique()[:80].tolist(), DF['Age'].value_counts(), 'seagreen');<jupyter_output><empty_output><jupyter_text>Most of the users are between ages 20-50 with an average of 37 is where the most of the users arround.  Which Shares the bike the most:<jupyter_code># Bike_share for gender
plt.figure(figsize=(16, 4))
sns.countplot(y=DF['member_gender'], hue = DF['bike_share_for_all_trip'], color='seagreen', palette=['seagreen','midnightblue']);
plt.title('Gender Bike Shares', fontsize=16); plt.xlabel('Count'); plt.ylabel('Gender');<jupyter_output><empty_output><jupyter_text>Males portion in data is the highest while sharing is higher compared to males.  Which are the most used Stations:<jupyter_code>Data = DF.groupby(['start_station_latitude', 'start_station_longitude']).count().sort_values('Age')
x = np.array(Data.index.values.tolist())[:,0]
y = np.array(Data.index.values.tolist())[:,1]
h = np.array(Data.Age.values.tolist())
plt.figure(figsize=(16, 6))
sns.scatterplot(x=x, y=y, hue=h, size=h, sizes=(20, 200))
plt.title('Most used Stations', fontsize=16); plt.xlabel('Latitude'); plt.ylabel('Longtude');
plt.show()<jupyter_output><empty_output><jupyter_text>There are three spots which rides happens frequently with most frequency in the bottom spot.  How long Trips take each Day:<jupyter_code>plt.figure(figsize=(16, 6))
sns.barplot(x = DF['Day_week'], y = DF['duration_sec'], hue=DF['member_gender']);
plt.title('Trips Duration', fontsize=16); plt.xlabel('Day of the Week'); plt.ylabel('Duration per Second');<jupyter_output><empty_output><jupyter_text>Weekends is where the longest trips happens, while there is a little diffrence between durations of Males and females, females is tend for shorter trips.  What is Number of Trips and Distance per Age:<jupyter_code>x = DF['Age'].value_counts().index
y = DF['Age'].value_counts()
h = (DF.groupby('Age')['Trip_distance'].mean()*1000000).astype(int)
plt.figure(figsize=(16, 6))
sns.scatterplot(x=x, y=y, hue=h, size=h, sizes=(20, 200))
plt.title('Number of Trips and Distance per Age', fontsize=16); plt.xlabel('Age'); plt.ylabel('Number of Trips');
plt.show()<jupyter_output><empty_output><jupyter_text>Both Most Number of Trips and Trip Durations happens between ages 25 and 45, while after 45 number of trips tend to decrease as well as the trip distances.     4. Conclusion First, the data had a lot of Nulls which made it impossible to drop them all, so i had to replace the Null values in member_birth_year column with average birth year, also with member_gender we will replace them with 'Other' keyword, as there is a lot of records have much information so not be dropped, those two columns which contained most of the Nulls, the dataset contained no duplicated rows, small subset of the data contained invalid values, which had to be handled either by removing them or putting default value, as the age it had some records of ages more than 140, a little bit of Feature Engineering had to be done at first to create four columns Age, Trip_distance, Day_time, Day_week, Biker_speed to generate further more insights.- First thing i created visualizations to see the distrbution of ages as i saw the minimum value in that column is 1878 which translated to 143 years old, which seems there is outliers here, so i dropped ages more than 80 (203 rows).- Second i made a correlation matrix too see which columns has the highest correlation. helped in creating further more insights.- Thrid i created some insights to display some answers to the questions stated before, first one was two insights for Highest Peaks based on [Day_week, Day_time] which showed that Thursday and Wednesday were the highest peak of the trips, While Friday and Saturday were the lowest as they are the weekends in US. and Daily peak starts from 9 at morning and begin to decrease from afternoon, and then start to rise again from 3 and reaches the highst point at 5 PM.- Fourth insight made to see Which age uses the service often which showed that Most of the users are between ages 20-50 with an average of 37 is where the most of the users arround.- Fifth insight was made to illustrate the Stations scatter distribution which showed there are three spots which rides happens frequently with most frequency in the bottom spot.- Sixth insight was about Average Trip duration for each day which showed that Weekends is where the longest trips happens.- Seventh insight was about Gender bike share, as stated that Males portion in data is the highest while sharing is higher compared to males.- The Eighth insight here showed Subscribers regarding Age where Most subscribers lie between 26 to 37 years old.- in the Ninth insight we showed the Average Speed per Gender and as usual Males has the highest driving speed.  LimitationsFirst, the data had a lot of Nulls which made it impossible to drop them all, and made it harder to do the Data Preparation, as member_gender and member_birth_year had about 8265, and some columns had about 197 like end_station_id, but on the other side there were a critical issues that needs to be discussed:- Missing valuesI had to replace the Null values in member_birth_year column with average birth year, also with member_gender we will replace them with 'Other' keyword, as there is a lot of records have much information so not be dropped.- Invalid valuesSome of the ages had invalid values as some ages were over 100, so i had to drop them.Overall the data did answer most of the questions, but for the most accurate anwer it needs to be enlarged and enhanced.  Thanks a lot...<jupyter_code>!jupyter nbconvert Explanatory FordGoBikes.ipynb --to slides --post serve --template output_toggle<jupyter_output>This application is used to convert notebook files (*.ipynb) to various other
formats.

WARNING: THE COMMANDLINE INTERFACE MAY CHANGE IN FUTURE RELEASES.

Options

-------



Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.


--debug

    set log level to logging.DEBUG (maximize logging output)

--generate-config

    generate default config file

-y

    Answer yes to any questions instead of prompting.

--execute

    Execute the notebook prior to export.

--allow-errors

    Continue notebook execution even if one of the cells throws an error and include the error message in the cell output (the default behaviour is to abort conversion). This flag is only relevant if '--execute' was specified, too.

--stdin

    read a single notebook file from stdin. Write the resulting notebook with default basename 'notebook.*'

--stdout

    Write notebook ou[...]
{'fraction_non_alphanumeric': 0.09572833435771358, 'fraction_numerical': 0.014059618930547019, 'mean_word_length': 5.142991977347806, 'pattern_counts': {'":': 1, '<': 60, '<?xml version=': 0, '>': 62, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '354750', 'license': 'MIT', 'n_tokens_mistral': 4071, 'n_tokens_neox': 3762, 'n_words': 1561, 'path': 'Explanatory FordGoBikes.ipynb', 'repo_name': 'khaledxmust/Exploring-FordGoBikes', 'size': '13016'}