<jupyter_start><jupyter_text>Indipendent project week 4  Loading our libraries<jupyter_code># import pandas
import pandas as pd


# import numpy
import numpy as np

from datetime import datetime
# Load our dataset

Electric_cars = pd.read_csv("http://bit.ly/autolib_dataset")
Electric_cars.head()
# Let's preview the first 10 rows of our data
# 
Electric_cars.head(10)
# Getting to know more about the dataset by accessing its information
#
Electric_cars.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 5000 entries, 0 to 4999
Data columns (total 25 columns):
 #   Column               Non-Null Count  Dtype 
---  ------               --------------  ----- 
 0   Address              5000 non-null   object
 1   Cars                 5000 non-null   int64 
 2   Bluecar counter      5000 non-null   int64 
 3   Utilib counter       5000 non-null   int64 
 4   Utilib 1.4 counter   5000 non-null   int64 
 5   Charge Slots         5000 non-null   int64 
 6   Charging Status      5000 non-null   object
 7   City                 5000 non-null   object
 8   Displayed comment    111 non-null    object
 9   ID                   5000 non-null   object
 10  Kind                 5000 non-null   object
 11  Geo point            5000 non-null   object
 12  Postal code          5000 non-null   int64 
 13  Public name          5000 non-null   object
 14  Rental status        5000 non-null   object
 15  Scheduled at         47 non-null     object
 16  Slots [...]<jupyter_text>Cleaning our dataset  Checking for missing values<jupyter_code># Creating a function to check
def missing_values(data):

  # identify the missing values from the Electric_cars dataset
  #
  miss = data.isnull().sum().sort_values(ascending = False)
  # the percentage of missing values
  percentage = (data.isnull().sum() / len(data)).sort_values(ascending = False)
  #storing  total missing values in percentage in dataframe
  #
  missing_data = pd.DataFrame({'Total' : miss, 'Percentage' : percentage})
  #if the percentage is 0, indicates no missing values hence removed
  #
  missing_data.drop(missing_data[missing_data['Percentage'] == 0].index, inplace = True)

  return missing_data

missing_values(Electric_cars)

Electric_cars.columns
# Creating a function to drop unnecessary columns
#
def drop_columns(data, cols):
  #drop columns not required
  data.drop(cols, axis = 1, inplace = True)

  return data

#columns to be dropped
columns = ['Cars', 'Displayed comment', 'Scheduled at', 'Charge Slots', 'Charging Status', 'Geo point', 'ID']
Electric_cars = drop_columns(Electric_cars, columns)<jupyter_output><empty_output><jupyter_text>**Conclusion***   The column car was dropped since it contains similar values to those in bluecar counter. This is described in the documentation.*   The columns Displayed comment and Scheduled had too many missing values*   The column Geo point is used to provide location which is also provided in the column city*  The columns Charge Slots and Charging Status are not required in the study*   The column id is similar to address column therefore unnessecary<jupyter_code># preview the altered table contents

Electric_cars.head()
# Checking for outliers
# check to see if any values appear True if so they are outliers
#
Q1 = Electric_cars.quantile(0.25)
Q3 = Electric_cars.quantile(0.75)

IQR = Q3 - Q1


(Electric_cars < (Q1 - 1.5 * IQR)) |(Electric_cars > (Q3 + 1.5 * IQR))

# If we were to keep our complete rows; meaning we drop any record with a missing value, then
#
Electric_cars = Electric_cars.dropna()

Electric_cars.shape<jupyter_output><empty_output><jupyter_text>**Conclusion**We did not drop any columns since there were no more missing values<jupyter_code># Checking for completeness of data
# Explanation: checking if any columns have missing values
#
Electric_cars.isnull().any()
# Procedure 1: Duplicates
# Data Cleaning Action: No action taken
# Explanation: There were no duplicate instances
#
Electric_cars.duplicated()
# Checking the names of columns
#

Electric_cars.columns
# change column names to be in lower case 
Electric_cars.columns = Electric_cars.columns.str.lower()

Electric_cars.columns
# Combine day,month and year. also add hour and minute
pd.to_datetime((Electric_cars.year*10000+Electric_cars.month*100+Electric_cars.day).apply(str),format='%Y%m%d')
Electric_cars["Datetime"] = pd.to_datetime(Electric_cars[['year', 'month', 'day', 'hour', 'minute']])
Electric_cars.head()
# Drop any other unnecessary colums
# Creating a function to drop unnecessary columns
#
def drop_columns(data, cols):
  #drop columns not required
  data.drop(cols, axis = 1, inplace = True)

  return data

#columns to be dropped
columns = ['year', 'month', 'day', 'minute']
Electric_cars = drop_columns(Electric_cars, columns)<jupyter_output><empty_output><jupyter_text>ConclusionWe dropped the year, month, day, minute columns since we used them to create Datetime column to summarize them<jupyter_code># Checking Datetime format
#

Electric_cars['Datetime'].max()
# sort the data into ascending Datetime to enable there to be an order
#

from datetime import datetime

Electric_data = Electric_cars.sort_values(by='Datetime')<jupyter_output><empty_output><jupyter_text>Sorted the Datetime column to clearly indicate dates that follow each other<jupyter_code># Creating a column for bluecar usage
Electric_data['bluecar_diff'] = Electric_cars['bluecar counter'].diff()

# Creating a column for utilib usage
Electric_data['utilib_diff'] = Electric_cars['utilib counter'].diff()

# Creating a column for utilib 1.4 usage
Electric_data['utilib_two_diff'] = Electric_cars['utilib 1.4 counter'].diff()

# preview the table
Electric_data.head()<jupyter_output><empty_output><jupyter_text>Adding a difference column for the three car counters (bluecar counter, utilib counter and utilib 1.4 counter)+ve number means that a car was returned -ve number means that a car was picked 0 means that nothing happened, i.e no car was picked nor returned  Analysis<jupyter_code># Identify the most popular hour of the day for picking up a shared electric car (Bluecar) in the city of Paris over the month of April 2018
# We selcet city == 'Paris' 

paris_data = Electric_data[Electric_data['city'] =='Paris'] 

paris_data.head()
# What is the most popular hour for picking cars
# this lists the hour for picking cars
#

paris_data[(paris_data.bluecar_diff < 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1);
# What is the most popular hour for returning cars?
#this lists the hours for returning the cars 

paris_data[(paris_data.bluecar_diff > 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1);<jupyter_output><empty_output><jupyter_text><jupyter_code># What station is the most popular overall?
# 

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok')].groupby('public name')['public name'].count().sort_values(ascending= False).head(1);


# What station is the most popular at the most popular picking hour?
#

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok') & (paris_data['hour'] == 21)].groupby('public name')['public name'].count().sort_values(ascending= False).head(1);

# What postal code is the most popular for picking up Blue cars overall? Does the most popular station belong to that postal code?

# First find the postal code of the most popular station overall
#
paris_data[paris_data['bluecar_diff'] < 0].groupby('postal code').count()[['bluecar_diff']].sort_values(by = 'bluecar_diff', ascending = False).head(1);


# then cross check the postal code of the  most popular station
#

paris_data[paris_data['public name'] == 'Paris/Porte de Montrouge/8'][['public name', 'postal code']].head(1)<jupyter_output><empty_output><jupyter_text>**Conclusion**1.  The postal codes are not the same<jupyter_code># What postal code is the most popular for picking up Blue cars at the most popular picking hour? 
# Does the most popular station belong to that postal code?
# 
# First find the postal code of the most popular station at most popular picking hour

paris_data[(paris_data.bluecar_diff < 0) & (paris_data.hour == 21)].groupby('postal code').count()[['bluecar_diff']].sort_values(by = 'bluecar_diff', ascending = False).head(1)

#paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok') & (paris_data['hour'] == 21)].groupby('postal code')['postal code'].count().sort_values(ascending= False).head(1)

# Then cross check the name of the most popular station

paris_data[paris_data['public name'] == 'Paris/Porte de Montrouge/8'][['public name', 'postal code']].head(1)<jupyter_output><empty_output><jupyter_text>**Conclusion***   The postal codes do not match  Using Utilib counter for analysis<jupyter_code># Identify the most popular hour of the day for picking up a shared electric car (Utilib) in the city of Paris over the month of April 2018
#

paris_data[(paris_data.utilib_diff < 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1)
# What is the most popular hour for returning cars?

#Electric_paris[Electric_paris['Utilib_Diff'] > 0].groupby('hour')['hour'].count().sort_values(ascending= False)
paris_data[(paris_data.utilib_diff > 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1)
# What is the most popular station overall?
#

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok')].groupby('public name')['public name'].count().sort_values(ascending= False).head(1)

# What station is the most popular? At the most popular picking hour?
#

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok') & (paris_data['hour'] == 19)].groupby('public name')['public name'].count().sort_values(ascending= False).head(1)<jupyter_output><empty_output><jupyter_text>The stations match<jupyter_code># What postal code is the most popular for picking up Blue cars overall? Does the most popular station belong to that postal code?

# First find the postal code of the most popular station overall
#
paris_data[paris_data['utilib_diff'] < 0].groupby('postal code').count()[['utilib_diff']].sort_values(by = 'utilib_diff', ascending = False).head(1)


# then cross check the postal code of the  most popular station
#

paris_data[paris_data['public name'] == 'Paris/Porte de Montrouge/8'][['public name', 'postal code']].head(1)<jupyter_output><empty_output><jupyter_text>The postal codes do not match<jupyter_code># What postal code is the most popular for picking up Blue cars at the most popular picking hour? 
# Does the most popular station belong to that postal code?
# 
# First find the postal code of the most popular station at most popular picking hour

paris_data[(paris_data.utilib_diff < 0) & (paris_data.hour == 19)].groupby('postal code').count()[['utilib_diff']].sort_values(by = 'utilib_diff', ascending = False).head(1)

# then cross check the postal code of the  most popular station
#

paris_data[paris_data['public name'] == 'Paris/Porte de Montrouge/8'][['public name', 'postal code']].head(1)<jupyter_output><empty_output><jupyter_text>ConclusionThe postal codes do not match  Using Utilib 1.4 counter for analysis<jupyter_code># Identify the most popular hour of the day for picking up a shared electric car (Utilib 1.4 Counter) in the city of Paris over the month of April 2018
#

paris_data[(paris_data.utilib_two_diff < 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1)

# What is the most popular hour for returning cars?
#

paris_data[(paris_data.utilib_two_diff > 0)].groupby('hour')['hour'].count().sort_values(ascending = False).head(1)
# What is the most popular station overall?
#

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok')].groupby('public name')['public name'].count().sort_values(ascending= False).head(1)

# What station is the most popular? At the most popular picking hour?
#

paris_data[(paris_data['kind'] == 'STATION') & (paris_data['status'] == 'ok') & (paris_data['hour'] == 3)].groupby('public name')['public name'].count().sort_values(ascending= False).head(1)


# What postal code is the most popular for picking up Blue cars overall? Does the most popular station belong to that postal code?

# First find the postal code of the most popular station overall
#
paris_data[paris_data['utilib_two_diff'] < 0].groupby('postal code').count()[['utilib_two_diff']].sort_values(by = 'utilib_two_diff', ascending = False).head(1)


# then cross check the postal code of the  most popular station
#

paris_data[paris_data['public name'] == 'Paris/Porte de Montrouge/8'][['public name', 'postal code']].head(1)<jupyter_output><empty_output><jupyter_text>ConclusionThey do not match hence the most popular station does not belong to that postal code<jupyter_code># What postal code is the most popular for picking up Blue cars at the most popular picking hour? 
# Does the most popular station belong to that postal code?
# 
# First find the postal code of the most popular station at most popular picking hour

paris_data[(paris_data.utilib_two_diff < 0) & (paris_data.hour == 3)].groupby('postal code').count()[['utilib_two_diff']].sort_values(by = 'utilib_two_diff', ascending = False).head(1)

# then cross check the postal code of the  most popular station
#

paris_data[paris_data['public name'] == 'Paris/Quatre septembre/10'][['public name', 'postal code']].head(1)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11482223706672605, 'fraction_numerical': 0.016700066800267203, 'mean_word_length': 5.1357012750455375, 'pattern_counts': {'":': 0, '<': 67, '<?xml version=': 0, '>': 62, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '402890', 'license': 'MIT', 'n_tokens_mistral': 4451, 'n_tokens_neox': 4106, 'n_words': 1354, 'path': 'Moringa_Data_Science_Prep_W4_Independent_Project_2022_02_Colleta_Khaemba_Python.ipynb', 'repo_name': 'Nabwile/Python-project-on-Electric-car-usage-in-Paris', 'size': '13473'}