<jupyter_start><jupyter_text>Quadratic Discriminant AnalysisIn this exercise, you will implement a QDA on your own. The QDA computes $p(\omega|x)=\frac{p(x|\omega)*p(\omega)}{p(x)}$, and assumes that the likelihood $p(x|\omega)$ follows a normal distribution (conditional on each class $\omega$).   Exercise 1A fisher asks you for help in classifying fish. Recently, he catched the following fish:| Length (m)    | Type          | | ------------- |-------------  || 1.3           | sea bass       || 0.7           | salmon       || 0.62           | salmon      || 0.9           | salmon       || 0.91          | sea bass       || 0.31          | herring       || 0.26           | herring       |* Calculate the class priors $p(\omega)$ for each class of fish* Calculate the parameters $\mu$ and $\sigma^2$ for the likelihoods $p(x|\omega)$. * The fisher catches a new fish with length $x = 0.82 m$. Calculate the posteriori probability $p(\omega|x)$ for each class. Which class is most likely? Priors: * $p(sea bass) = 2/7$* $p(salmon) = 3/7$* $p(herring) = 2/7$Likelihoods:* $\mu_{sb} = 1.105 \sigma^2_{sb} = 0.076$* $\mu_{sa} = 0.74 \sigma^2_{sa} = 0.021$* $\mu_{h} = 0.285 \sigma^2_{h} = 0.00125$A posteriori (without normalizing constant):* $p(sea bass | x) = 0.0046 * 2/7 =  0.0013$* $p(salmon | x) = 0.134 * 3/7 =  0.0038$* $p(herring | x) = 0 * 2/7 =  0$Thus, the fish is most likely a salmon.  Exercise Implement a function `priors(classes)` that calculates a prior $p(x)$ for each class, given a vector of class labels.The input should be an array of classes, e.g. `np.array(["stand","sit","sit","stand"])`. The output should be a pandas data frame with columns `class` and `prior`.<jupyter_code>from scipy.io import arff
import scipy.stats
import pandas as pd
import numpy as np

def priors(classes):
    unique, counts = np.unique(classes, return_counts=True)
    counts = counts/counts.sum()
    df = pd.DataFrame(np.array([unique,counts]).transpose(),columns=["class","prior"])
    return(df)
    
pp = priors(np.array(["stand","sit","sit","sit","stand"]))


data = arff.loadarff('features1.arff')
df = pd.DataFrame(data[0])

dat = df.loc[:, ["AccX_mean","class"]]
dat.columns = ["x","class"]

prior = priors(dat["class"])
prior<jupyter_output><empty_output><jupyter_text>ExerciseImplement a function `likelihood(data)`, that takes a data frame consisting of a column $omega$ and a column $x$, and estimates the parameters of the likelihood $p(x|\omega)$ for each class $\omega$, i.e. that returns a mean and a variance for each class. Thus, the output should be a data frame with columns `omega`, `mean` and `variance`. Plot the likelihoods for each class.<jupyter_code>from scipy.io import arff
import scipy.stats
import pandas as pd
import numpy as np

def likelihood(data):
    uc = np.unique(data["class"])
    def getMV(c):
        dsub = data.loc[data["class"]==c,"x"]
        return([dsub.mean(),dsub.var()])
    mvs = list(map(getMV, uc))
    r = pd.DataFrame(mvs,columns = ["mean","variance"])
    r["class"] = uc
    return r
    
data = arff.loadarff('features1.arff')
df = pd.DataFrame(data[0])

dat = df.loc[:, ["AccX_mean","class"]]
dat.columns = ["x","class"]
lik = likelihood(dat)
lik<jupyter_output><empty_output><jupyter_text>ExerciseImplement a function `myqda(newdat,lik,priors)`, that returns the most likely class for a new observation `newdat`. Test your implementation on the dataset `features1.arff`. Train your QDA (i.e. compute priors and likelihood parameters), and then classify the data. How good is your classification?<jupyter_code>from scipy.io import arff
import scipy.stats

def mylda(newdat,lik,prior):
    def getClass(d):
        def getProb(c,d):
            pr = prior.loc[prior["class"]==c,"prior"].values[0]
            m = lik.loc[lik["class"] == c,"mean"].values[0]
            v = lik.loc[lik["class"] == c,"variance"].values[0]
            li = scipy.stats.norm(m, v).pdf(d)
            return(li*pr)
        probs = np.array([getProb(c,d) for c in prior["class"].values])
        return prior.loc[np.argmax(probs),"class"]
    newclasses = np.array([getClass(dat) for dat in newdat])
    return newclasses


data = arff.loadarff('features1.arff')
df = pd.DataFrame(data[0])

dat = df.loc[:, ["AccX_mean","class"]]
dat.columns = ["x","class"]

lik = likelihood(dat)
prior = priors(dat["class"])

nc = mylda(dat["x"][1:100],lik,prior)
print(sum(nc == dat["class"][1:100])/100)<jupyter_output>0.47
{'fraction_non_alphanumeric': 0.16610549943883277, 'fraction_numerical': 0.024242424242424242, 'mean_word_length': 3.632016632016632, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '300822', 'license': 'MIT', 'n_tokens_mistral': 1590, 'n_tokens_neox': 1496, 'n_words': 437, 'path': 'english_exercises_from_WS19/03-LDA.ipynb', 'repo_name': 'stefanluedtke/AI-II-Exercises', 'size': '4455'}