<jupyter_start><jupyter_text>**Connect With Me in Linkedin  :-** https://www.linkedin.com/in/dheerajkumar1997/  Dates and TimesA special type of categorical variable are those that instead of taking traditional labels, like color (blue, red), or city (London, Manchester), take dates as values. For example, date of birth ('29-08-1987', '12-01-2012'), or time of application ('2016-Dec', '2013-March').Datetime variables can contain dates only, or time only, or date and time.Typically, we would never work with a date variable as a categorical variable, for a variety of reasons:- Date variables usually contain a huge number of individual categories, which will expand the feature space dramatically- Date variables allow us to capture much more information from the dataset if preprocessed in the right wayIn addition, often, date variables will contain dates that were not present in the dataset that we used to train the machine learning algorithm. In fact, will contain dates placed in the future respect to the dates present in the dataset we used to train. Therefore, the machine learning model will not know what to do with them, because it never saw them while being trained. **I will cover different was of pre-processing/engineering date variables in the section "Engineering date variables".**============================================================================= Real Life example: Peer to peer lending (Finance) Lending Club**Lending Club** is a peer-to-peer Lending company based in the US. They match people looking to invest money with people looking to borrow money. When investors invest their money through Lending Club, this money is passed onto borrowers, and when borrowers pay their loans back, the capital plus the interest passes on back to the investors. It is a win for everybody as they can get typically lower loan rates and higher investor returns.If you want to learn more about Lending Club follow this [link](https://www.lendingclub.com/).The Lending Club dataset contains complete loan data for all loans issued through 2007-2015, including the current loan status (Current, Late, Fully Paid, etc.) and latest payment information. Features include credit scores, number of finance inquiries, address including zip codes and state, and collections among others. Collections indicates whether the customer has missed one or more payments and the team is trying to recover their money.The file is a matrix of about 890 thousand observations and 75 variables. More detail on this dataset can be found in [Kaggle's website](https://www.kaggle.com/wendykan/lending-club-loan-data)Let's go ahead and have a look at the variables!<jupyter_code>import pandas as pd
import numpy as np

import matplotlib.pyplot as plt
%matplotlib inline
# let's load the Lending Club dataset with a few selected columns

use_cols = ['loan_amnt', 'grade', 'purpose', 'issue_d', 'last_pymnt_d']

data = pd.read_csv('loan.csv', usecols=use_cols)

data.head()
# let's inspect at the pandas type of object used to store the information

data.dtypes
# now let's parse the dates, currently coded as strings, into datetime format
# this will allow us to make some analysis afterwards

data['issue_dt'] = pd.to_datetime(data.issue_d)
data['last_pymnt_dt'] = pd.to_datetime(data.last_pymnt_d)

data[['issue_d', 'issue_dt', 'last_pymnt_d', 'last_pymnt_dt']].head()
# let's see how much money Lending Club has disbursed
# (i.e., lent) over the years to the different risk
# markets (grade variable)

fig = data.groupby(['issue_dt', 'grade'])['loan_amnt'].sum().unstack().plot(
    figsize=(14, 8), linewidth=2)

fig.set_title('Disbursed amount in time')
fig.set_ylabel('Disbursed Amount (US Dollars)')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09344042838018742, 'fraction_numerical': 0.012048192771084338, 'mean_word_length': 5.3108108108108105, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 3, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '128321', 'license': 'Apache-2.0', 'n_tokens_mistral': 1013, 'n_tokens_neox': 946, 'n_words': 494, 'path': 'Feature Engineering Techniques/Dates.ipynb', 'repo_name': 'DheerajKumar97/Feature-Engineering-Techniques---Python--ML', 'size': '3735'}