<jupyter_start><jupyter_text>What exactly is a variable?A variable is any characteristic, number, or quantity that can be measured or counted. They are called "variables" because the value they take may vary, and it usually does. The following are examples of variables:-Age (21, 35, 62,...)-Gender (male, female)-Income (GBP 20000, GBP 35000, GBP 45000,...)-House price (GBP 350000, GBP 570000,...)-Country of birth (China, Russia, Costa Rica, etc.)-Eye colour (brown, green, blue,...)-Vehicle make (Ford, Volkswagen,...)Most variables in a data set can be classified into one of two major types:- **Numerical variables** - **Categorical variables**=================================================================================== Categorical VariablesThe values of a categorical variable are selected from a group of **categories**, also called **labels**. Examples are gender (male or female) and marital status (single, married, divorced, or widowed). Other examples of categorical variables include:- Intended use of loan (debt-consolidation, car purchase, wedding expenses, etc.)- Mobile network provider (Vodafone, Orange,...)- PostcodeCategorical variables can be further categorised into:- **Ordinal Variables**- **Nominal variables** Ordinal variablesOrdinal variables are categorical variables whose categories can be meaningfully ordered. For example:- Student's grade on an exam (A, B, C, or Fail).- Days of the week, where Monday is 1 and Sunday is 7.- Educational level, with the categories of elementary school, high school, college graduate, and PhD ranked from 1 to 4.   Nominal variablesThere is no inherent order in the labels for nominal variables. Country of birth, for example, has nominal values of Argentina, England, Germany, and so on. Nominal variables include the following:- Car color (blue, grey, silver,...)- Vehicle make (Citroen, Peugeot,...)- City (Manchester, London, Chester,...)In principle, there is nothing that indicates an intrinsic order to the labels, and in fact, they are all equal.**To be considered:**Sometimes categorical variables are coded as numbers (e.g., gender may be coded as 0 for males and 1 for females). The variable is still categorical, despite the use of numbers.In a similar way, individuals in a survey may be coded with a number that uniquely identifies them (for example, to avoid storing personal information). This number is really a label, and the variable is then categorical. The number has no importance other than to allow the observation to be uniquely identified.When working with a dataset in a commercial setting, it's ideal if the data comes with a dictionary that specifies whether the numbers in the variables are to be considered categories or numerical. If the numbers are categories, the dictionary will explain what each variable's value signifies.============================================================================= In this demo: Peer-to-peer lending (Finance)In this demo, we will use a toy data set that simulates data from a peer-to-peer finance company.To obtain the data, go to the lecture **Download datasets** in **Section 1**. This is a toy dataset that I created and comes within the Datasets.zip file that you can find in this lecture.<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt
# Let's load the dataset.

# Variable definitions:
#----------------------
# loan_purpose: intended use of the loan
# market: the risk market assigned to the borrower (based in their financial situation)
# householder: whether the borrower owns or rents their property

data = pd.read_csv('../loan.csv')

data.head()
# Let's inspect the variable householder,
# which indicates whether the borrowers own their home,
# or if they are renting, among other things.

data['householder'].unique()
# Let's make a bar plot with the number of loans
# disbursed in each category of home ownership.

# The code below counts the number of observations (borrowers)
# in each category and then makes a bar plot.

fig = data['householder'].value_counts().plot.bar()
fig.set_title('Householder')
fig.set_ylabel('Number of customers')<jupyter_output><empty_output><jupyter_text>The majority of the borrowers either own their house with a mortgage or rent their property. A few borrowers own their homes completely.<jupyter_code>data['householder'].value_counts()
# The "loan_purpose" variable is another categorical variable
# that indicates how the borrowers intend to use the
# money they are borrowing. For example, to improve their
# house, or to cancel previous debt.

data['loan_purpose'].unique()<jupyter_output><empty_output><jupyter_text>Debt consolidation means that the borrower will use the loan to cancel previous debts; car purchase means that the borrower will use the money to buy a car; and so on. It gives an idea of the intended use of the loan.<jupyter_code># Let's make a bar plot with the number of borrowers
# in each category.

# The code below counts the number of observations (borrowers)
# per category and then makes a plot.

fig = data['loan_purpose'].value_counts().plot.bar()
fig.set_title('Loan Purpose')
fig.set_ylabel('Number of customers')<jupyter_output><empty_output><jupyter_text>The majority of debtors plan to use the money to "consolidate debt." This is a common occurrence. The borrowers want to consolidate all of their debts from various financial products into a single debt, which will be the new loan they will receive from the peer-to-peer organization. This loan will normally provide a benefit to the borrower, such as reduced interest rates or a longer repayment time than a credit card, for example.<jupyter_code># Let's look at one additional categorical variable:
# "market", which represents the risk market or risk band
# assigned to the borrower.

data['market'].unique()
# Let's make a bar plot with the number of borrowers
# per category.

fig = data['market'].value_counts().plot.bar()
fig.set_title('Status of the Loan')
fig.set_ylabel('Number of customers')<jupyter_output><empty_output><jupyter_text>Most customers are assigned to markets B and C. A and B are lower risk customers, and E is the highest risk customer. The higher the risk, the more likely the customer is to default; thus, the finance companies charge higher interest rates on those loans.<jupyter_code># Finally, let's look at a variable that is numerical,
# but its numbers have no real meaning.

data['customer_id'].head()<jupyter_output><empty_output><jupyter_text>Each id corresponds to a single consumer. This number is assigned to identify the client in the event that it is needed, while guaranteeing confidentiality and data protection.<jupyter_code># The variable has as many different id values as customers:
# in this case 10000.

len(data['customer_id'].unique())<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09991210079109288, 'fraction_numerical': 0.006592440668033987, 'mean_word_length': 5.1504504504504505, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '352796', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1786, 'n_tokens_neox': 1656, 'n_words': 909, 'path': 'Section-02-Types-of-Variables/02.2-Categorical-Variables.ipynb', 'repo_name': 'Golufu/feature-engineering-for-machine-learning', 'size': '6826'}