<jupyter_start><jupyter_text>Introduction to machine learningThe following is used to show some simple examples of how to use machine learning. The notebook is structured as follows:* import used libraries* generation of a data set used* visualising the data* Linar regression* Creating probabilities* Classification* Further reading  Import packages<jupyter_code>import numpy as np
import pandas as pd
import random
import math
import matplotlib<jupyter_output><empty_output><jupyter_text>Create dataWe want to show how to linear regression works, to do that we create some data based on a function price``` price = size * 43.000 kr + 1000.000 kr```adding noise to the square meter price.<jupyter_code>square_meter_prize = 43000
bias = 1000000
def generate_data_point(id, square_meter_prize=43000, bias = 1000000):
    noise_size = 10000
    noise = int(random.random() * noise_size - noise_size/2)
    size = random.choice([x for x in range(50,100)])
    
    return { "id": id, "size": size, "price": size * (square_meter_prize + noise) + bias}
def generate_house_data(number = 20):
    return [generate_data_point(i) for i in range(number)]
house_data = generate_house_data()
house_data
df = pd.DataFrame(house_data)
df[['size', 'price']].head(10).sort_values('size',ascending=True)
house_plot = df.plot.scatter(x='size', y='price', title="house prices pr square meter")<jupyter_output><empty_output><jupyter_text>Linear RegressionNow imagine you want to use this data to be able to predict new prices. We will want to make a model that can predict what we know already. Meaning that if we feed the size part of a data point from above into our model it will predict the price.<jupyter_code>from sklearn.linear_model import LinearRegression

X = pd.DataFrame(df['size'])
y = pd.DataFrame(df['price'])
reg = LinearRegression().fit(X, y)
print(f"price = {reg.coef_[0][0]}X + {reg.intercept_[0]}")


def predict(x):
    return reg.coef_[0][0] * x + reg.intercept_[0]

predict(55)

house_plot = df.plot.scatter(x='size', y='price', title="house prices pr square meter",)
house_plot.plot([55, 95], [predict(55), predict(95)], 'k-', lw=2)<jupyter_output><empty_output><jupyter_text>ClassificationInstead of wanting to predict the prize, we now want to know if the bank will allow you to buy the house or not. In other words we want to classify each house into either bank_approves or not. Again I do some simplyfications and say that the bank will approve if the price is less than or equal to 3.5 times your salary (plus minus some noise to make it more interesting)<jupyter_code>salary = 1500000
max_price = df['price'].max()
max_price<jupyter_output><empty_output><jupyter_text>The following method is used to calculate if the bank approves a house, based on the price.<jupyter_code>def bank_approves(salary: float, price: float):
    return 1.0 if salary * 3.25 >= price + (1e6 - 1000000 * random.random()) else 0.

df['bank_approves'] = df['price'].apply(lambda x: bank_approves(salary, x))
df.head(20)<jupyter_output><empty_output><jupyter_text>Let's visualise the data again, now with a coloting that shows if the bank approves or not. An orange dot significates that the bank approve, while blue it doesn't.<jupyter_code>import matplotlib.pyplot as plt

groups = df.groupby('bank_approves')
# Plot
fig, ax = plt.subplots()
ax.margins(0.05) # Optional, just adds 5% padding to the autoscaling
for name, group in groups:
    ax.plot(group['size'], group.price, marker='o', linestyle='', ms=12, label=name)
ax.legend()

plt.show()<jupyter_output><empty_output><jupyter_text>We cannot use linear regression as we did above, since there is not a linear relationship between size and if the bank approves. The problem is that its not that the probability decreases linearly. There are a lot of house the bank will approve for sure... and then a gray area followed by houses that it wont approve. A solution could be to use Logistic Regression, which we will try in the following. To make it easier for the model we will normalise the price (normalise means that you will scale the values down such that they will be on average zero and distributed around zero.)<jupyter_code>df['normalized_x'] = (df['price']-df['price'].mean())/df['price'].std()<jupyter_output><empty_output><jupyter_text>Let's just have another look at the data:<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>Implementing LogisticRegression is very similar to the linear regression above.<jupyter_code>from sklearn.linear_model import LogisticRegression

X = pd.DataFrame(df['size'])
y = pd.DataFrame(df['bank_approves'])
reg = LogisticRegression().fit(X, y)<jupyter_output>/Users/kifal2/Dropbox/git/ml-presentation/venv/lib/python3.8/site-packages/sklearn/utils/validation.py:63: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  return f(*args, **kwargs)<jupyter_text>The predict method og the reg object, predicts if the house will be approved by the bank or not (1.0 means approve, 0.0 mean not approved)<jupyter_code>reg.predict([[77.]])<jupyter_output><empty_output><jupyter_text>But how is it that it always produces `1.0` or `0.0`? Actually the model produces a probability, which is then meassured against a cut line. To see this let's do a plot of the probabilities (using `predict_proba`)<jupyter_code>df['can_buy'] = df['size'].apply(lambda x: reg.predict_proba([[x]])[0][1])
df[['price', 'size', 'bank_approves']].head(10)
df.plot.scatter(x='size', y='can_buy')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10658810926620246, 'fraction_numerical': 0.021424745581146223, 'mean_word_length': 5.806804374240583, 'pattern_counts': {'":': 3, '<': 44, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '51251', 'license': 'MIT', 'n_tokens_mistral': 1841, 'n_tokens_neox': 1718, 'n_words': 630, 'path': 'Machine Learning Presentation.ipynb', 'repo_name': 'kimfalk/ml-presentation', 'size': '5601'}