<jupyter_start><jupyter_text>Importing libraries and Exploring the Dataset.<jupyter_code>import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats
sns.set() #setting the default seaborn style for our plots
%matplotlib inline
#importing the dataset
uber_df=pd.read_csv("Uber Request Data.csv")
#checking the head of the data
uber_df.head()
#Info about the data shape of data, type of individual columns

uber_df.info()
uber_df.shape<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 6745 entries, 0 to 6744
Data columns (total 6 columns):
 #   Column             Non-Null Count  Dtype  
---  ------             --------------  -----  
 0   Request id         6745 non-null   int64  
 1   Pickup point       6745 non-null   object 
 2   Driver id          4095 non-null   float64
 3   Status             6745 non-null   object 
 4   Request timestamp  6745 non-null   object 
 5   Drop timestamp     2831 non-null   object 
dtypes: float64(1), int64(1), object(4)
memory usage: 316.3+ KB<jupyter_text>Expected output:1-The data should consist of (6745) data record with (6) attributes. (1) integer64 type, (1) float type , (4) object .2- there are missing data in ( Driver id  )  and (Drop timestamp)  Task 2: Checking missing values .<jupyter_code># check the missing data.
uber_df.isna().apply(pd.value_counts)<jupyter_output><empty_output><jupyter_text>Expected output:1- there are (2650) missing data record at Driver id.2- there are (3914) missing data record at Drop timestamp.  analyze the data.<jupyter_code>#Convert Request_timestamp & Drop_timestamp to uniform datetime format
uber_df["Request_timestamp"] = uber_df["Request timestamp"].apply(lambda x : pd.to_datetime(x))
uber_df["Drop_timestamp"] = uber_df["Drop timestamp"].apply(lambda x : pd.to_datetime(x))

uber_df.info()
uber_df['Status'].value_counts()
#Check if the Driver id is null only for 'No Cars Available' Status
uber_df[(uber_df.Status == 'No Cars Available') & (uber_df["Driver id"].isnull())].shape
#Check if drop timestamp is null only for 'No Cars Available' & 'Cancelled'
uber_df[((uber_df.Status == 'No Cars Available') | (uber_df.Status == 'Cancelled'))
        & (uber_df["Drop timestamp"].isnull())].shape<jupyter_output><empty_output><jupyter_text>Expected output:(Driver id) and (Drop timestamp)  are  lower importanet feateares for machine learning model<jupyter_code>#Extract the hour from requested timestamp
uber_df["Request hour"] = uber_df["Request_timestamp"].dt.hour
uber_df
uber_df["Pickup point"].value_counts()<jupyter_output><empty_output><jupyter_text>Creating visual methods to analyze the data.<jupyter_code># remove Request timestamp & Drop timestamp & Request_timestamp & Drop_timestamp because :
                                                                                         # the missing data 
                                                                                         # we allready extract Request hour

uber_df.drop('Request timestamp',axis=1,inplace=True)
uber_df.drop('Drop timestamp',axis=1,inplace=True)
uber_df.drop('Request_timestamp',axis=1,inplace=True)
uber_df.drop('Drop_timestamp',axis=1,inplace=True)
uber_df.head()
plt.hist(uber_df["Request hour"],bins=24)
plt.xlabel("Request hour")
plt.ylabel("No. of Requests")
plt.show()
#Demand is more during evening & morning hours
#divide the time of the day into five categories
def  period(x):
    if x < 5:
        return "Early Morning"
    elif 5 <= x < 10:
        return "Morning"
    elif 10 <= x < 17:
        return "Day Time"
    elif 17 <= x < 22:
        return "Evening"
    else:
        return "Late Night"
uber_df['Time'] = uber_df['Request hour'].apply(lambda x: period(x))
uber_df['Time'].value_counts().plot.bar()
plt.show()
#Maximum demand during 'Evening' hours

uber_df.head()
uber_df["Pickup point"].value_counts().plot.pie(autopct='%1.0f%%')
plt.show()

uber_df["Status"].value_counts().plot.pie(autopct='%1.0f%%')
plt.show()<jupyter_output><empty_output><jupyter_text>Task 5: make logistic regression model<jupyter_code>def Availability(x):
    if  x== 'No Cars Available':
        return 0 
    else:
        return 1
uber_df['Availability']=uber_df['Status'].apply(lambda x : Availability(x) )
uber_df['Availability'].head()
dummy=pd.get_dummies(uber_df['Pickup point'])
dummy
uber_df=pd.concat([uber_df,dummy],axis=1)
uber_df
uber_df['Availability'].value_counts()
uber_df.drop('Request id',axis=1,inplace=True)
uber_df.drop('Pickup point',axis=1,inplace=True)
uber_df.drop('Driver id',axis=1,inplace=True)
uber_df.drop('Status',axis=1,inplace=True)
uber_df.drop('Time',axis=1,inplace=True)

uber_df.head()
X=uber_df.drop(columns=['Availability'])
Y=uber_df['Availability']
print(X.shape)
print(Y.shape)
X=np.array(X)
Y=np.array(Y)
Y=Y.reshape(-1,1)
Y.shape
# Splitting the dataset into the Training set and Test set
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, Y, test_size = 0.25, random_state = 41)
# Feature Scaling
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
# Fitting Logistic Regression to the Training set
from sklearn.linear_model import LogisticRegression
classifier = LogisticRegression(random_state = 0)
classifier.fit(X_train, y_train)
# Predicting the Test set results
y_pred = classifier.predict(X_test)

# Making the Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
cm
from sklearn.metrics import accuracy_score
acc=accuracy_score(y_test, y_pred)
acc
print('Accuracy is :',acc,'%')<jupyter_output>Accuracy is : 0.7077652637818613 %
{'fraction_non_alphanumeric': 0.11917454858125537, 'fraction_numerical': 0.02304385210662081, 'mean_word_length': 4.311415525114155, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '608528', 'license': 'Apache-2.0', 'n_tokens_mistral': 1986, 'n_tokens_neox': 1875, 'n_words': 502, 'path': 'Uber_Request_availability_model.ipynb', 'repo_name': 'GunH-colab/UCS-21', 'size': '5815'}