<jupyter_start><jupyter_text>Project Overview I will be working as a data scientist working for an electric car-sharing service company. I have been tasked to process stations data to understand electric car usage over time by solving for the following research question;Research Question;1. Identify the most popular hour of the day for picking up a shared electric car (Bluecar) in the city of Paris over the month of April 2018.2. What is the most popular hour for returning cars?3. What station is the most popular?    a) Overall?    b) At the most popular picking hour?  4. What postal code is the most popular for picking up Blue cars? Does the most popular station belong to that postal code?    a) Overall?    b) At the most popular picking hour?5. Do the results change if you consider Utilib and Utilib 1.4 instead of Blue cars?   1.0 Importing our Libraries<jupyter_code># Importing the libraries we will need for this project

# Importing the pandas library

import pandas as pd

# Importing the numpy library

import numpy as np

# importing the datetime library

from datetime import datetime<jupyter_output><empty_output><jupyter_text>1.1 Reading the Dataset from our CSV file Dataset link for download or access: [Link](https://bit.ly/autolib_dataset)<jupyter_code># We will first import our dataset that we will use for the examples. We will do this by first
# then storing the url location of our dataset to the variable url
url = 'http://bit.ly/autolib_dataset'

# We will read the dataset from above url and store the dataframe in the variable df
autolib = pd.read_csv(url)
autolib<jupyter_output><empty_output><jupyter_text>1.2 Previewing our Dataset<jupyter_code># Let's preview the first 10 rows of our data
# 
autolib.head(10)<jupyter_output><empty_output><jupyter_text>1.3 Accessing Information about our Dataset<jupyter_code># Getting to know more about the dataset by accessing its information
# 
autolib.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 5000 entries, 0 to 4999
Data columns (total 25 columns):
 #   Column               Non-Null Count  Dtype 
---  ------               --------------  ----- 
 0   Address              5000 non-null   object
 1   Cars                 5000 non-null   int64 
 2   Bluecar counter      5000 non-null   int64 
 3   Utilib counter       5000 non-null   int64 
 4   Utilib 1.4 counter   5000 non-null   int64 
 5   Charge Slots         5000 non-null   int64 
 6   Charging Status      5000 non-null   object
 7   City                 5000 non-null   object
 8   Displayed comment    111 non-null    object
 9   ID                   5000 non-null   object
 10  Kind                 5000 non-null   object
 11  Geo point            5000 non-null   object
 12  Postal code          5000 non-null   int64 
 13  Public name          5000 non-null   object
 14  Rental status        5000 non-null   object
 15  Scheduled at         47 non-null     object
 16  Slots [...]<jupyter_text>1.4 Cleaning our Dataset Let us perform data cleaning procedures below providing a documentation for our actions and reasons  1.41) Validity<jupyter_code># Procedure 1: Irrelevant Data Observation
# Data Cleaning Action: Dropping columns
# Explanation: No question to be answered require the columns dropped columns.
#
autolib.drop(['Displayed comment', 'Geo point', 'Charge Slots', 'Charging Status', 'Scheduled at', 'Station type' , 'Subscription status'], axis = 1, inplace = True)
autolib
# Procedure 2: Outliers
# Data Cleaning Action: Identifying the outliers
# Explanation: we identify the ouliers and decide whether to keep or drop them  .
# Calculating our first, third quantiles and then later our IQR
#

Q1 = autolib.quantile(0.25)
Q3 = autolib.quantile(0.75)
IQR = Q3 - Q1
IQR

# The rule of thumb is that anything not in the range of 
# (Q1 - 1.5 IQR) and (Q3 + 1.5 IQR) is an outlier, and can be removed.
#

(autolib < (Q1 - 1.5 * IQR)) |(autolib > (Q3 + 1.5 * IQR))

# Uncommenting the line of code below is how we removing outliers based on the IQR range and stores the result in the data frame 'electric_out'
# but for now we are going to keep them

# autolib_out = autolib[~((autolib < (Q1 - 1.5 * IQR)) |(autolib > (Q3 + 1.5 * IQR))).any(axis=1)]
# autolib_out<jupyter_output><empty_output><jupyter_text>1.42) Accuracy<jupyter_code># Procedure 1: In-record & cross-datasets errors
# Data Cleaning Action: No action taken
# Explanation: the dataset was accurate no fixing needed.<jupyter_output><empty_output><jupyter_text>1.43) Completeness<jupyter_code># Procedure 1: missing values
# Data Cleaning Action: checking for missing values
# Explanation: we need to check if we have missing values 
#
autolib.isnull().sum()

# no missing values<jupyter_output><empty_output><jupyter_text>1.44) Consitency<jupyter_code># Procedure 1: duplicates
# Data Cleaning Action: checking for dupicates in our dataset
# Explanation: Duplicates are data points that are repeated in your dataset. 
# These should be simply removed.
#
autolib[autolib.duplicated()]

# Identifying which observations are duplicates
autolib2 = autolib.drop_duplicates()

# previwing for the non duplicates

autolib2.head()<jupyter_output><empty_output><jupyter_text>1.45) Uniformity<jupyter_code># Procedure 1: Standardization
# Data Cleaning Action: Fixing messy column names
# Explanation: We have column names which are in uppercase, with spaces, 
# and whitespace all around we remove this to make the column names standard
#

autolib2.columns = autolib2.columns.str.strip().str.lower().str.replace(' ', '_').str.replace('(', '').str.replace(')', '')
autolib2.head()
# Procedure 1:  Date Type Conversion
# Data Cleaning Action: Combining the columns year,day and time into one column and changing the column to datatime datatype
# Explanation: We have change the columns year,month,day and time to there respective data type


autolib2['DateTime']=pd.to_datetime(autolib2[['year','month','day', 'hour', 'minute']])
autolib2.head()
autolib2.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 5000 entries, 0 to 4999
Data columns (total 19 columns):
 #   Column              Non-Null Count  Dtype         
---  ------              --------------  -----         
 0   address             5000 non-null   object        
 1   cars                5000 non-null   int64         
 2   bluecar_counter     5000 non-null   int64         
 3   utilib_counter      5000 non-null   int64         
 4   utilib_1.4_counter  5000 non-null   int64         
 5   city                5000 non-null   object        
 6   id                  5000 non-null   object        
 7   kind                5000 non-null   object        
 8   postal_code         5000 non-null   int64         
 9   public_name         5000 non-null   object        
 10  rental_status       5000 non-null   object        
 11  slots               5000 non-null   int64         
 12  status              5000 non-null   object        
 13  year                5000 non-null   int64       [...]<jupyter_text>1.5 Answering Research Questions  Bluecar Counter<jupyter_code># To compute usage, we will need to understand that we have to join successive (in time) measures/counters for a given station, 
# as the difference will tell whether a car was picked up, returned, or nothing happened.
# We create a new column where we find this difference in the Bluecar Counter
#

autolib2['bluecar_diff'] = autolib2['bluecar_counter'].diff()
autolib2.head(10)
# Question 1
# Identify the most popular hour of the day for picking up a shared electric car (Bluecar) in the city of Paris over the month of April 2018

# We specify that city == Paris

# positive value mean a car was returned

# Negative value means a car was taken

# Zero shows that there was no change

bluecar_paris = autolib2[autolib2['city'] =='Paris'] 

bluecar_paris.head()
# To identify the most popular hour of the day for picking up 
#
bluecar_paris[bluecar_paris['bluecar_diff'] < 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour of the day for picking up a shared electric car (Bluecar) in the city of Paris is 2100hr<jupyter_code># Question 2
# What is the most popular hour for returning cars?
#
bluecar_paris[bluecar_paris['bluecar_diff'] > 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour of the day for returning a shared electric car (Bluecar) in the city of Paris is 0300hr<jupyter_code># Question 3a:
# What station is the most popular overall?
# 

bp = bluecar_paris[(bluecar_paris['kind'] == 'STATION') & (bluecar_paris['status'] == 'ok')]
bp.groupby('public_name')['public_name'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The station that is the most popular overall is Paris/Porte de Montrouge/8<jupyter_code># Question 3b:
# What station is the most popular At the most popular picking hour?
# 

bp = bluecar_paris[(bluecar_paris['kind'] == 'STATION') & (bluecar_paris['status'] == 'ok') & (bluecar_paris['hour'] == 21)]
bp.groupby('public_name')['public_name'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The station that is the most popular at the most popular picking hour ie 2100hr is Paris/Tronchet/19<jupyter_code># Question 4a:
# What postal code is the most popular for picking up Blue cars Overall?
# 

bp = bluecar_paris[(bluecar_paris['kind'] == 'STATION') & (bluecar_paris['status'] == 'ok')]
bp.groupby('postal_code')['postal_code'].count().sort_values(ascending= False)

# Does the most popular station belong to that postal code?
# The postal code with 75015 is Paris/Suffren/2 and the most popular station is Paris/Porte de Montrouge/8 
# therfore the most popular station does not belong to that postal code<jupyter_output><empty_output><jupyter_text>The postal code that is the most popular for picking up Blue cars Overall 75015<jupyter_code># Question 4b:
# What postal code is the most popular for picking up Blue cars at the most popular picking hour?
# 

bp = bluecar_paris[(bluecar_paris['kind'] == 'STATION') & (bluecar_paris['status'] == 'ok') & (bluecar_paris['hour'] == 21)]
bp.groupby('postal_code')['postal_code'].count().sort_values(ascending= False)

# Does the most popular station belong to that postal code?
# The postal code with 75014 is Paris/Raymond Losserand/145 and the most popular station at the most popular picking hour 
# is Paris/Tronchet/19  therfore the most popular station does not belong to that postal code<jupyter_output><empty_output><jupyter_text>The postal code that is the most popular for picking up Blue cars at the most popular picking hour ie 2100hr is 75014  Utilib Counter<jupyter_code># We create a new column where we find this difference in the Bluecar Counter
#

autolib2['utilib_diff'] = autolib2['utilib_counter'].diff()
autolib2.head(10)
# Question 1
# Identify the most popular hour of the day for picking up a shared electric car (Utilib) in the city of Paris over the month of April 2018
#
Utilib_paris = autolib2[autolib2['city'] =='Paris'] 

Utilib_paris.head()
Utilib_paris[Utilib_paris['utilib_diff'] < 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour of the day for picking up a shared electric car (Utilib) in the city of Paris over the month of April 2018 is 1900hr<jupyter_code># Question 2
# What is the most popular hour for returning cars?
#
Utilib_paris[Utilib_paris['utilib_diff'] > 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour for returning these cars is 0600hr<jupyter_code># Question 3a:
# What station is the most popular overall?
# 

Utilib_paris.groupby('public_name')['utilib_diff'].count().sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>The station is the most popular overall is Paris/Porte de Montrouge/8<jupyter_code># Question 3b:
# What station is the most popular At the most popular picking hour?
# 
Utilib_paris[(Utilib_paris.hour == 19)]['public_name'].value_counts()<jupyter_output><empty_output><jupyter_text>The station that is the most popular at the most popular picking hour ie 1900hr is Paris/Perle/18<jupyter_code># Question 4a:
# What postal code is the most popular for picking up Utilib cars Overall?
# 


uc = Utilib_paris[(Utilib_paris.city == 'Paris') & (Utilib_paris.utilib_diff <= 0)]
uc.groupby('postal_code')['hour'].count().sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>The postal code that is the most popular for picking up Utilib cars Overall is 75015<jupyter_code># Question 4b:
# What postal code is the most popular for picking up Utilib cars at the most popular picking hour?
# 
uc = Utilib_paris[(Utilib_paris['kind'] == 'STATION') & (Utilib_paris['status'] == 'ok') & (Utilib_paris['hour'] == 19)] 
uc.groupby('postal_code')['postal_code'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The postal code that is the most popular for picking up Utilib cars at the most popular picking hour ie 1900hr is 75020  Utilib 1.4 Counter<jupyter_code># Creating a new column where we find the difference in the number of Utilib 1.4 counter


#bluecar_paris.drop(["utilib_two_diff", "utilib_1.4_diff"], axis = 1, inplace = True) 

autolib2['utilib_two_diff'] = autolib2['utilib_1.4_counter'].diff()
autolib2.head(10)
# bluecar_paris['utilib_two_diff'] = bluecar_paris['utilib_1.4_counter'].diff()

# Question 1
# Identify the most popular hour of the day for picking up a 
# shared electric car (Utilib 1.4 counter) in the city of Paris over the month of April 2018
#
Utilib2_paris = autolib2[autolib2['city'] =='Paris'] 

Utilib2_paris.head()
Utilib2_paris[Utilib2_paris['utilib_two_diff'] < 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour of the day for picking up a shared electric car (Utilib 1.4) in the city of Paris over the month of April 2018 is 0300hr<jupyter_code># Question 2
# What is the most popular hour for returning cars?
#
Utilib2_paris[Utilib2_paris['utilib_two_diff'] > 0].groupby('hour')['hour'].count().sort_values(ascending= False)<jupyter_output><empty_output><jupyter_text>The most popular hour for returning cars is 0900hr<jupyter_code># Question 3a:
# What station is the most popular overall?
# 

uc2 = Utilib2_paris[(Utilib2_paris['kind'] == 'STATION') & (Utilib2_paris['status'] == 'ok')]
uc2.groupby('public_name')['utilib_two_diff'].count().sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>The station that is the most popular overall is Paris/Porte de Montrouge/8<jupyter_code># Question 3b:
# What station is the most popular At the most popular picking hour?
# 
uc2 = Utilib2_paris[(Utilib2_paris['kind'] == 'STATION') & (Utilib2_paris['status'] == 'ok')  & (Utilib2_paris['hour'] == 3)]
uc2.groupby('public_name')['utilib_two_diff'].count().sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>The station that is the most popular at the most popular picking hour ie 0300hr is Paris/Mortier/15<jupyter_code># Question 4a:
# What postal code is the most popular for picking up Utilib cars Overall?
# 

uc = Utilib2_paris[(Utilib2_paris.city == 'Paris') & (Utilib2_paris.utilib_two_diff <= 0)]
uc.groupby('postal_code')['utilib_two_diff'].count().sort_values(ascending=False)<jupyter_output><empty_output><jupyter_text>The postal code that is the most popular for picking up Utilib cars Overall is 75015<jupyter_code># Question 4b:
# What postal code is the most popular for picking up Utilib cars at the most popular picking hour?
# 
uc = Utilib2_paris[(Utilib2_paris['kind'] == 'STATION') & (Utilib2_paris['status'] == 'ok') & (Utilib2_paris['hour'] == 3)] 
uc.groupby('postal_code')['utilib_two_diff'].count().sort_values(ascending=False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09669781931464175, 'fraction_numerical': 0.03190031152647975, 'mean_word_length': 4.424467725582967, 'pattern_counts': {'":': 0, '<': 116, '<?xml version=': 0, '>': 114, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '27', 'dir': 'jupyter-structured-clean-dedup', 'id': '339136', 'license': 'MIT', 'n_tokens_mistral': 5444, 'n_tokens_neox': 4984, 'n_words': 1730, 'path': 'electric_car_sharing_python_notebook.ipynb', 'repo_name': 'Jasminewakini/WK-5-Electric-Car-Sharing-Project', 'size': '16050'}