<jupyter_start><jupyter_text>Exploring Various Machine Learning Principles*<NAME>*This video focuses on concepts common throughout machine learning. Types of Learning**Supervised learning** is machine learning for labeled data. We use data with labels or a target variable to train an algorithm and apply the algorithm to predict labels for unlabeled data. Algorithms such as support vector machines, decision trees, etc. engage in supervised learning. The rest of the section will focus on supervised learning.**Unsupervised learning** is machine learning using unlabeled data. There is no labeled data in our training set yet the algorithm must still predict what it will be for test data. Clustering methods, such as $k$-means, are considered unsupervised learning.**Semisupervised learning** is a mixture of supervised and unsupervised learning, where some of the training data has labels and some does not. Labels and Target VariablesWe use an algorithm to approximate a function that determines the value of a target variable. Our **target variable** is the variable we are trying to predict. It could be binary (did a website visitor buy a product or not?), categorical (what color shoe did a website visitor purchase?), or continuous (how much money did a visitor spend on a website?).Our training data includes not only the target variable for observations but also **features**, other variables that could be used for prediction. For the examples above, features could include whether a visitor has an account with the website or not, whether the visitor has been to the website before or not, or how long the visitor's session has lasted. Demographic variables such as sex, age, geographic location and others could be useful features too.Example: Below is the `iris` dataset with labels corresponding to the species of flowers. Sepal length, sepal width, petal length, and petal width are features, while the species is the target.<jupyter_code>from sklearn.datasets import load_iris
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
iris_obj = load_iris()
print(iris_obj.DESCR)
iris_obj.data
iris_obj.feature_names
iris_obj.target
iris_obj.target_names
# Randomly choose a subset of the iris data
train = np.random.choice(np.arange(len(iris_obj.data)), size=100, replace=False)
train = np.sort(train)
train
# All remaining data will be treated as "unlabeled"
test = np.setdiff1d(np.arange(len(iris_obj.data)), train)
test
# Plot sepal length vs sepal width, labelling species
marker_map = {0: 'o', 1: 's', 2: '^'}
color_map = {0: 'blue', 1: 'green', 2: 'red'}
var1, var2 = 0, 1    # Sepal length and sepal width variables
for length, width, species in zip(iris_obj.data[train, var1], iris_obj.data[train, var2], iris_obj.target[train]):
    plt.scatter(x=length, y=width, marker=marker_map[species], c=color_map[species])
plt.xlabel(iris_obj.feature_names[var1])
plt.ylabel(iris_obj.feature_names[var2])
plt.show()
# This plot includes unlabeled data

# Plot sepal length vs sepal width, labelling species
marker_map = {0: 'o', 1: 's', 2: '^'}
color_map = {0: 'blue', 1: 'green', 2: 'red'}
var1, var2 = 0, 1    # Sepal length and sepal width variables
for length, width, species in zip(iris_obj.data[train, var1], iris_obj.data[train, var2], iris_obj.target[train]):
    plt.scatter(x=length, y=width, marker=marker_map[species], c=color_map[species])
# Plot unlabeled data too
for length, width in zip(iris_obj.data[test, var1], iris_obj.data[test, var2]):
    plt.scatter(x=length, y=width, marker='o', c="black")
plt.xlabel(iris_obj.feature_names[var1])
plt.ylabel(iris_obj.feature_names[var2])
plt.show()
# This plot uses different variables

# Plot petal length vs petal width, labelling species
marker_map = {0: 'o', 1: 's', 2: '^'}
color_map = {0: 'blue', 1: 'green', 2: 'red'}
var1, var2 = 2, 3    # Sepal length and sepal width variables
for length, width, species in zip(iris_obj.data[train, var1], iris_obj.data[train, var2], iris_obj.target[train]):
    plt.scatter(x=length, y=width, marker=marker_map[species], c=color_map[species])
# Plot unlabeled data too
for length, width in zip(iris_obj.data[test, var1], iris_obj.data[test, var2]):
    plt.scatter(x=length, y=width, marker='o', c="black")
plt.xlabel(iris_obj.feature_names[var1])
plt.ylabel(iris_obj.feature_names[var2])
plt.show()<jupyter_output><empty_output><jupyter_text>Supervised Learning Goal**Error** signifies the degree to which our algorithm does not correctly predict the target variable. For classification problems it's how much our algorithm chooses the wrong label, and for regression problems it may be how distant predictions are from the correct value. An ideal algorithm is one that has small error.In addition to error we may have a **loss function** which quantifies how wrong the algorithm was. This may be different than merely a count of errors or the distance from the predicted value to the actual value. Training algorithms involves choosing an algorithm that minimizes its loss function on the training data. We hope that the model will **generalize** to unseen data and that error on future data is small as well.Let's load in the `titanic` dataset, which includes details about the passengers aboard the *Titanic* and who survived the disaster.<jupyter_code>import pandas as pd
from pandas import DataFrame
titanic = pd.read_csv("titanic.csv")
titanic.head()<jupyter_output><empty_output><jupyter_text>Here, the variable `Survived` will be the target variable. Our first algorithm will do the following: *Predict the most common outcome of the target variable.*<jupyter_code>titanic.Survived.value_counts()<jupyter_output><empty_output><jupyter_text>In this case, out algorithm does the following: *Always predict 0 (did not survive).* This is a dumb algorithm, yet it should be taken seriously; if a more sophisticated algorithm does no better than the dumb one, the sophisticated algorithm is not worth the effort.Here, the loss function is the number of incorrect predictions. If we had no other features, the loss function is minimized by our algorithm. Here I compute the algorithm's error (incorrect predictions).<jupyter_code>predicted = pd.Series([0] * len(titanic))
(titanic.Survived - predicted).abs().sum()    # Error count (trivial here)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08725257323832146, 'fraction_numerical': 0.008234362628661916, 'mean_word_length': 5.6694825765575505, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '597698', 'license': 'MIT', 'n_tokens_mistral': 1861, 'n_tokens_neox': 1783, 'n_words': 788, 'path': 'Chapter02/MachineLearningPrinciples.ipynb', 'repo_name': 'PacktPublishing/Training-Systems-using-Python-Statistical-Modeling', 'size': '6322'}