<jupyter_start><jupyter_text>Expectation Maximisation$\newcommand{\x}{\mathbf{x}}\newcommand{\y}{\mathbf{y}}\newcommand{\z}{\mathbf{z}}\newcommand{\class}{\mathcal{C}}\newcommand{\mean}{\boldsymbol{\mu}}\newcommand{\cov}{\boldsymbol{\Sigma}}\newcommand{\icov}{\boldsymbol{\Sigma}^{-1}}\newcommand{\normdist}{\mathcal{N}}\newcommand{\tp}{\top}\newcommand{\params}{\mathbf{\theta}}$| | ||--|--|| **Names** | *<NAME>, <NAME>* || **Group** | *AML 25* |In this lab we will build a probabilistic classifier and estimate its parameters. In order to illustrate how the complexity of learning changes when we do not observe all the variables, we will start with the model depicted in figure 1. We will then switch to the graphical model depicted in figure 2 and compare the twomodels’ performance.Read through the complete exercise handout before starting your implementation: the hints at the end, and the instructions for the subsequent exercises will help you for the current one Gaussian Distribution  **Figure 1**: Bayesian classifier with Normally distributed observations. The class labels C are discrete variables in one-of-K format. The observations x are multivariate and continuous, and their distribution is described with a Gaussian probability density function. In this model, we want to find the posterior probability$p(\class_k|\x_n)$ for all data points $\x_n$. We get this from Bayes' rule$$  p(\class_k|\x_n) = \frac{ p(\x_n|\class_k) p(\class_k) }{    \sum_j p(\x_n|\class_j) p(\class_j) },$$where we can see from the graph that the data elements areconditionally independent given the class and we model theclass-conditional probability with a normal distribution$$  p(\x_n|\class_k) = \normdist(\x_n;\mean_k,\cov_k)\\  = \frac{1}{(2\pi)^{d/2}|\cov|^{1/2}} \exp -\frac{1}{2}    (\x_n-\mean_k)^\tp \icov_k (\x_n-\mean_k)$$Load the data in `target.mat`, which consistsof two classes `A` and `B`. Split the data in train and testsets and plot the training data, in order to get a feel for it. Just by looking at the data, do you think the model will perform well?**Question 1 [10 marks]**Do the training, which consists of finding the prior probabilities for each class, $p(\class_k)$, and the parameters of the class-conditional distributions, the means $\mean_k$ and covariance matrices $\cov_k$. Now use Bayes' rule to compute the most likely class label for each data point in the test set.  Include the resulting confusion matrix and error rate inyou report.<jupyter_code>%matplotlib inline
import numpy as np
import scipy.stats as st
import scipy.ndimage as ndim
import matplotlib.pyplot as plt
import operator
import pylab
import time
from IPython import display

from matplotlib.patches import Ellipse

pylab.rcParams['figure.figsize'] = (10.0, 8.0)


def normPDF(data,mu,sigma):
    """My implementation of the Gaussian distribution. Returns the density, given mean mu and covariance sigma,
    for every data point (row) of data"""
    N,nDim = data.shape
    res = np.zeros(N)
    
    icov = np.linalg.inv(sigma)
    covdet = np.linalg.det(sigma)
    denom = np.sqrt((2*np.pi)**nDim * covdet)
    
    for i in range(N):
        diff = data[i,:]-mu
        res[i] = np.exp(-.5 * np.dot(diff, np.dot(icov,diff.T))) / denom
    
    return res


# Load in the data
D = np.load("target.npz")
data = D['data']
labels = D['labels']
A=data[labels==0,:]
B=data[labels==1,:]

nA = A.shape[0]
nB = B.shape[0]
N = nA + nB

data = np.vstack((A,B))
plt.scatter(A[:,0],A[:,1])
plt.scatter(B[:,0],B[:,1],20,'red')

labels = np.concatenate((np.zeros(nA),np.ones(nB)))

perm = np.random.permutation(N)
datap = data[perm,:]
labelp = labels[perm]

trainData = datap[:4*N/5,:]
trainLab  = labelp[:4*N/5]
testData  = datap[4*N/5:,:]
testLab   = labelp[4*N/5:]
# [ANSQ1] Do not modify this line
# Answer Q1 is below<jupyter_output><empty_output><jupyter_text>SHORT DISCUSSION FOR Q1From the picture we can see assume that this model will not perform well as data points have the shape of concetric circles and the data points of one class aren't placed together in the space but separated by the data points of another class and it's hard to imagine such Gaussian distributions which can handle this problem.<jupyter_code>from sklearn.metrics import accuracy_score, confusion_matrix
def norm_param(arr):
    return np.mean(arr, axis = 0), np.cov(np.transpose(arr))


def posterior(data, mu, sigma):
    p_x_0 = normPDF(data, mu[0], sigma[0])
    p_x_1 = normPDF(data, mu[1], sigma[1])
    return p_x_0 * priors[0] / (p_x_0 * priors[0] + p_x_1 * priors[1]) 


def classify(data, mu, sigma):
    posterior_c_0 = posterior(data, mu, sigma)
    predictions = [0 if prob > 0.5 else 1 for prob in posterior_c_0]
    return predictions
classes = 2
priors = np.zeros(classes)
mu = np.zeros((classes, classes))
sigma = np.zeros((2, classes, classes))

for i in range(classes):
    class_data = trainData[trainLab == i]
    priors[i] = float(len(class_data)) / len(trainLab)
    mu[i], sigma[i] = norm_param(class_data)
    
y_pred = classify(testData, mu, sigma)
# visualisation of trained models

fig = plt.gcf()
plt.clf()        
ax = fig.add_subplot(111, aspect='equal')
plt.scatter(data[:,0], data[:,1])
for m in range(classes):
    [val,vect] = np.linalg.eig(sigma[m])
    e = Ellipse(xy=mu[m], width=2*np.sqrt(val[0]), height=2*np.sqrt(val[1]), angle=180*np.arctan2(vect[0,0],vect[1,0])/np.pi)
    e.set_facecolor([1,0,0])
    ax.add_artist(e)
plt.show()
acc = accuracy_score(testLab, y_pred)
conf_matrix = confusion_matrix(testLab, y_pred)
print "error_rate = ", 1 - acc
print "conf_matrix = \n", conf_matrix<jupyter_output>error_rate =  0.44625
conf_matrix = 
[[244 157]
 [200 199]]<jupyter_text>SHORT DISCUSSIONAs we assumed, the accuracy of the proposed model is low due to the fact that one Gaussian distribution per class can't represent the actual distribution of the datapoints of the class well.  Mixture of Gaussians **Figure 2**: Graphical model of a Bayesian classifier with observations described by a Mixture of Gaussians(MOG). Each class has a number of mixture elements, which are each modelling part of the data. Whichdatapoints belong to which mixture element is controlled by the latent variable $\z$.In this model, the class-conditional distributions $p(\x|\class_k)$ aremodelled by a **mixture** of Gaussians rather than a single Gaussian, so that the total data-likelihood\begin{align}  p(\x_n,\class_k) &= p(\class_k) \sum_j p(\x_n|z_j,\class_k)p(z_j|\class_k),\end{align}where\begin{align}  p(\x_n|z_j,\class_k) = \normdist(\x_n;\mean_{jk},\cov_{jk}).\end{align}Since the latent variables controlling the mixture components are not observed, we need to optimise the log-likelihood with the EMalgorithm. You are provided with some code to help you pull this off:<jupyter_code># [ANSQ2] [ANSQ3] Do not modify this line

def invalid(sigma):
    """Is this a valid covariance matrix?"""
    if np.isfinite(sigma).all():
        return np.linalg.cond(sigma)>1e3
    return True

class MoG:
    def __init__(self,data,nMix=2):
        self.pi = np.ones(nMix) / nMix
        self.nPoints, self.nDim = data.shape
        self.nMix = nMix
        self.data = data
        mu = np.mean(data)
        self.mu = []
        self.sigma = []
        self.resp = np.zeros((self.pi.shape[0], self.data.shape[0]))
        for i in range(nMix):
            self.mu.append(mu + .1 * i * np.random.randn(self.nDim))
            self.sigma.append(np.eye(self.nDim))
            
    def em(self,maxIter=100,minInc=1e-5):
        llhist = np.zeros(maxIter)
        for iteration in range(maxIter):
            resp, ll = self.expectation()
            llhist[iteration]=ll
            if iteration>0:
                delta = llhist[iteration] - llhist[iteration-1]
                print "Iteration",iteration,":",delta
#                 print llhist[:iteration+1],llhist[iteration], llhist[iteration-1],delta
                if delta < minInc:
                    if delta < 0.:
                        print "WARNING: log-likelihood decreased in iteration",iteration
                    llhist = llhist[:iteration]
                    break
            self.maximisation(resp)
            self.plot()
        return llhist
            
    def expectation(self):
        """Compute p(z|x,\theta)"""
        # Answer to Q2
        for i in range(self.nMix):
            self.resp[i] = self.pi[i] * normPDF(self.data, self.mu[i], self.sigma[i])

        self.resp = self.resp / sum(self.resp)  
        return self.resp, sum(np.log(self.likelihood(self.data)))
    
    def maximisation(self,resp):
        # Answer to Q3
        count = np.sum(resp, axis = 1)
        self.pi = count / self.nPoints
            
        for i in range(self.nMix):
            self.mu[i] = np.dot(self.data.T, resp[i]) / count[i]
    
        for i in range(self.nMix):
            sigma_new = np.dot((self.data - self.mu[i]).T * resp[i], self.data - self.mu[i]) / count[i]
            
        # if values are too big then reinitialising
            if invalid(sigma_new):
                print "INVALID"
                mean = np.mean(self.data) + .1*np.random.randn(self.nDim)
                self.sigma[i] = np.eye(self.nDim)
            else:
                self.sigma[i] = sigma_new
                
            
    def plot(self):
        fig = plt.gcf()
        display.display(fig)
        display.clear_output(wait=True)
        plt.clf()        
        ax = fig.add_subplot(111, aspect='equal')
        plt.scatter(self.data[:,0],self.data[:,1])
        for m in range(self.nMix):
            [val,vect] = np.linalg.eig(self.sigma[m])
            e = Ellipse(xy=self.mu[m], width=2*np.sqrt(val[0]), height=2*np.sqrt(val[1]), angle=180*np.arctan2(vect[0,0],vect[1,0])/np.pi)
                    
            e.set_facecolor([1,0,0])
            ax.add_artist(e)
        plt.show()
        time.sleep(.1)
        
    def likelihood(self,data):
        """Compute the marginal likelihood of the data given the mixture model, p(x|\theta), for
        all elements in `data`"""
        p = np.zeros(data.shape[0])
        for m in range(self.nMix):
            p += self.pi[m] * normPDF(data,self.mu[m],self.sigma[m])
        return p<jupyter_output><empty_output><jupyter_text>SHORT DISCUSSION We noticed that the final position of the ellipses really depends on the initialisation.  Multiple runs may result in different position: rotation approximately 90 degrees. It can be explained by the symmetry of training data points or by the fact that this data can't be approximated well by this number of components in Gaussian mixture.However, the relative position of three (there actually three of them, one is just covered by another) ellipses stays approximately the same from run to run.<jupyter_code>mog = MoG(A,3)
llhist = mog.em()
plt.figure()
plt.plot(llhist)<jupyter_output><empty_output><jupyter_text>The code above requires a number of methods in order to work,implementing the intialisation, E-step and M-step of the EMalgorithm. Those methods are: * `(resp, ll) = expectation()` which implements the E-step  of the algorithm. It uses the data, and the current parameter values. The function does not change  the parameters but computes `resp`, a matrix that contains the  probability of each mixture component given the data  $p(\z|\x,\params)$, and `ll`, the log-likelihood of the data set  under the mixture model. This is **Question 2 [10 marks]*** `maximisation(resp)` wich uses the data and takes as inputs the corresponding responsibilities. The method updates the class by storing the updated parameter values. This is **Question 3 [10 marks]**Include comments in your answer, explaininghow you compute the responsibilities $p(\z|\x,\params)$ and what theupdates are that maximise the complete expected log-likelihood.**Question 4 [10 marks]**Use the same data in `target.npz`, and optimise the model parameters for the classifier withmixture-of-Gaussians distributions. Use the same train and test setsas for exercise1 and compare the performance of bothclassifiers. Discuss the differences you found.*Optional Extra:* The contour plots of the distributions do notreflect the priors ($p(\z|\class_l)$) and therefore do not give you anexact idea of the probability density under the model. In order tovisualise that, you could plot the probability density under bothmodels in three dimensions. Hints1. Include a check in your M-step to avoid singularities. You can  check this by computing `c = np.linalg.cond(SIGMA)`: if c becomes larger  than a given threshold (say, $10^{10}$), do not perform the  parameter update and reset the mean and covariance. The provided function,  `invalid(SIGMA)`, does that check for you.1. The responsibilities $p(\z|\x,\params)$ are stored in the matrix  `Q` in the E-step of the algorithm. You know the size of this  matrix in advance: if you create a matrix of zeros of that size and  fill in the values, your code will be much more efficient than if  your grow the matrix as you compute the values.<jupyter_code># [ANSQ4] Do not modify this line
# Answer to Q4

def training_mixture(data, k):
    mog = MoG(data, k)
    llhist = mog.em()
    return mog

k0 = 9
mog_0 = training_mixture(trainData[trainLab == 0], k0)
k1 = 9
mog_1 = training_mixture(trainData[trainLab == 1], k1)
def posterior(data):
    p_x_0 = mog_0.likelihood(data)
    p_x_1 = mog_1.likelihood(data)
    return p_x_0 * priors[0] / (p_x_0 * priors[0] + p_x_1 * priors[1]) 


def classify(data):
    posterior_c_0 = posterior(data)
    predictions = [0 if prob > 0.5 else 1 for prob in posterior_c_0]
    return predictions
y_pred = classify(testData)
acc = accuracy_score(testLab, y_pred)
conf_matrix = confusion_matrix(testLab, y_pred)
print "error_rate on the test set = ", 1 - acc
print "conf_matrix = \n", conf_matrix<jupyter_output>error_rate on the test set =  0.02875
conf_matrix = 
[[393   8]
 [ 15 384]]<jupyter_text>The mixture model allows us to represent the distribution of our data points much more accurate than a single Gaussian distribution per class. However, the training step now takes much longer to compute.  Discussion of the Implementation **Question 5 [8 marks]**Discuss the following with your teammate and include your thoughts inyour report:1. What is a sensible way to initialise the parameters of your  model?1. How many components should you try? Having plotted the data, can  you guess what number of components should be optimal? How does your  classification accuracy evolve as the number of components change?1. Do the results depend on the initialisation?1. Why is hint 1 important? [ANSQ5] Do not modify this lineAnswer to Q51) As the EM algorithm usually takes more iterations to converge than the K-means algorithm and each cycle requires significantly more computation, it is common to find initialization for a Gaussian mixture model throught running K-means algorithm. The covariance matrices can be initialized to the sample covariances of the clusters, the same goes for the means - the means of the clusters. The mixing coefficients can be set to the fractions of data points assigned to respective clusters.2) * The optimal choice of k should strike a balance between maximum compression of the data using a single cluster, and maximum accuracy by assigning each data point to its own cluster. * Yes, the plot may give us some hints on the number components: we can see how many relatively close groups of points there are and therefore, choose that number of groups as number of components. But we need to limit ourselves: as we grow the number of components, we need more data to estimate the parameters well and avoid overfitting. * When we increase the number of components, the classification accuracy increases as the number of components grows up. But at some point, increasing the number of components doesn't result in any sufficient benefit in accuracy on the test set and we start overfitting on the train set: the accuracy on train set continues growing (we can come to the cricial case of number of components = number of points) but accuracy on test set starts decreasing.3) Yes, the results depend on the intitialisation: in general, there multiple local maxima of log likelihood function and the EM algorithm is not guaranteed to find the largest of them.4) It is important because we want our algorithm to converge as fast as possible, and EM algorithm can take multiple iterations to converge, that's why we try to choose sensible parameters not very far from actual values.  Log-ProbabilitiesAs we have seen in the class, one problem that occurs often when doing probabilistic modelling, is that since probabilities lay in the range $[0,\dots,1]$, the joint probability of multiple variables becomes very small very fast. This is a problem, because computers can only represent numbers with a finite precision and too small a probability will result in anunderflow, hence introducing numerical errors in an otherwise mathematically correct algorithm.The solution that is typically used is then to use the logarithm ofprobabilities, rather than the probabilities themselves. The productsof probabilities then become sums of log-probabilities:$$  \ln[p(a)p(b)] = \ln p(a) + \ln p(b)$$However a solution must be found when sums are involved. How do wecompute$$  \ln[p(a) + p(b)]$$when $\ln p(a)$ and $\ln p(b)$ are known? The solution is not todo the following:$$  \ln[p(a) + p(b)] = \ln[ \exp (\ln p(a)) + \exp(\ln p(b)) ].$$As an example to illustrate this, try to compute $\ln[p(a)+p(b)]$ when $\lnp(a) = -1000$ and $\ln p(b) = -1001$. (Really, do try the aboveapproach before reading on. The correct solution is $-999.69$.) We've seen that we can solve this problem as follows:$$  \ln[p(a)+p(b)] = \ln[ \exp( \ln p(a)) + \exp( \ln p(b))] \\  = \ln\left[ \exp(\ln p(a)) \left( 1 + \frac{\exp( \ln p(b))}{\exp( \ln      p(a))}\right)\right] \\  = \ln p(a) + \ln[1+\exp( \ln p(b) - \ln p(a) ) ],$$which is now easy to compute, because either the difference between$\ln p(a)$ and $\ln p(b)$ is small (and then the exponent can becomputed without numerical problems) or it is large (and then theexponent of the difference goes to zero, so that the $\ln(1+0)\rightarrow \ln(1) = 0$. You do still need to pay attention that youtake the largest value out of the $\ln$, so that the difference goesto large negative values rather than large positive values. The above discussion is pretty important: as we start using largerdata sets, the numerical issues become dominant.  In effect, youshould never use probabilities in your code and always work withlog-probabilities. The products then become sums, and the sums becomethe function explained above. This has the added advantage that $\ln$and $\exp$ are pretty expensive operations. If you use distributionsfrom the exponential family, such as the Gaussian, computing thelog-probability is computationally slightly cheaper than computing theprobability. A final benefit is that sometimes the code is easier todebug, because it is clear what errors are due to implementation bugsand what errors are due to the limited precision of the machine. Have a look at the lecture notes for more information on this topic.** Question 6 [10 marks] **You may have noticed such numerical problems in the previous exercise,especially when using larger numbers of mixture components. In thisexercise, we will change the E-step of the previous exercise andcompute everything with log-probabilities.1. First implement a function called  `logsumexp` that computes  the sum of two log-probabilities as described above. Make sure that  this function takes the largest value out of the logarithm. Write the  function so that both scalars and matrices can be used as arguments.1. Discuss how we can use this function to compute sums of more  than two probabilities using log-probabilities.1. Check your implementation by computing $\ln[p(a)+p(b)]$ when  $\ln p(a)=-1000$ and $\ln p(b)=-1001$.1. Now convert the E-step of the previous exercise. Implement the  function `logNormPDF` which is functionally equivalent  to `normPDF` but never computes the logarithm or exponent and is therefore numerically stable:$$    \normdist(\x_n;\mean,\cov)    = \frac{1}{(2\pi)^{d/2}|\cov|^{1/2}} \exp -\frac{1}{2}     (\x-\mean)^\tp \icov (\x-\mean)\\    \ln \normdist(\x_n;\mean,\cov)    = -\frac{1}{2} \left( d \ln 2\pi + \ln |\cov| + (\x-\mean)^\tp    \icov (\x-\mean) \right)$$Notice, however, that only the function `expectation` is changed,and that it must still return $p(\z|\x,\params)$, not $\lnp(\z|\x,\params)$. In the adapted function, you compute $\lnp(\z|\x,\params)$, but you need to exponentiate it before returningthe value. However now the computation does not suffer from theunderflows and is therefore more precise.<jupyter_code># [ANSQ6] Do not modify this line
# Your answer to Question 6

def logsumexp(pa_log, pb_log):
    """Implementation of the logarithm of sum of two log-probabilities. 
    Returns the sum of two log-probabilities"""
    if pb_log > pa_log:
        temp = pb_log
        pb_log = pa_log
        pa_log = temp
    return pa_log + np.log(1 + np.exp(pb_log - pa_log))

vect_logsumexp = np.vectorize(logsumexp)<jupyter_output><empty_output><jupyter_text>If one wants to apply logsumexp function to a non-scalar, then it's possible to use vectorized version of this function which is available under the name "vect_logsumexp".<jupyter_code>vect_logsumexp([-1000, -2], [-1001, -3])<jupyter_output><empty_output><jupyter_text>2) To compute the sum of more than two probabilites using log-probabilities and the function defined above, we can apply the function logsumexp recursively: first compute the sum of the first two log-probabilities, and after that, add all the rest elements, one by one, to the sum.Given the list $p = [p_0, p_1, p_2, ..., p_{n-1}]$ of $n$ log-probabiltites:1. s = p_02. for i in range(1, n):    * s = logsumexp(s, p_i)    3) We check the correctness of implementation of `logsumexp` function:<jupyter_code>logsumexp(-1000, -1001)<jupyter_output><empty_output><jupyter_text>4) To implement the more efficient and stable version of the E-step of the EM algorithm, we make use of the following formula:$$ \gamma(z_{nk}) = \frac{\pi_k \normdist(\x_n;\mean,\cov)}{\sum_{j=1}^K \pi_j \normdist(\x_j;\mean,\cov)} $$$$ \ln \gamma(z_{nk}) = \ln\pi_k + \ln\normdist(\x_n;\mean,\cov) - \ln\sum_{j=1}^K \pi_j \normdist(\x_j;\mean,\cov) $$First, we implement lognormPDF:$$    \ln \normdist(\x_n;\mean,\cov)    = -\frac{1}{2} \left( d \ln 2\pi + \ln |\cov| + (\x-\mean)^\tp    \icov (\x-\mean) \right)$$After that, we inherit the class MoG_log from class MoG and change the implementation of functions `expectation` and `likelihood` using new function `lognormPDF`.<jupyter_code>def lognormPDF(data,mu,sigma):
    """Implementation of the logarithm of Gaussian distribution. Returns the logarithm of density, 
    given mean mu and covariance sigma, for every data point (row) of data"""
    N,nDim = data.shape
    res = np.zeros(N)
    
    icov = np.linalg.inv(sigma)
    covdet = np.linalg.det(sigma)
    denom = np.sqrt((2*np.pi)**nDim * covdet)
    
    for i in range(N):
        diff = data[i,:]-mu
        res[i] = -.5 * (nDim * np.log(2 * np.pi) + np.log(covdet)+ np.dot(diff, np.dot(icov,diff.T)))
    
    return res<jupyter_output><empty_output><jupyter_text>Test correctness of the implementation<jupyter_code>np.exp(lognormPDF(data, mu[0], sigma[0]))
normPDF(data, mu[0], sigma[0])
class MoG_log(MoG):
    def expectation(self):
        """Compute p(z|x,\theta)"""
        # Answer to Q6.4
        for i in range(self.nMix):
            self.resp[i] = np.log(self.pi[i]) + lognormPDF(self.data, self.mu[i], self.sigma[i])

        sumexp = self.resp[0]
        for i in range(1, self.nMix): 
            sumexp = vect_logsumexp(sumexp, self.resp[i])
        
        self.resp -= sumexp
        return np.exp(self.resp), sum(self.likelihood(self.data))
        
    def likelihood(self,data):
        """Compute the marginal likelihood of the data given the mixture model, p(x|\theta), for
        all elements in `data`"""
        p = np.log(self.pi[0]) + lognormPDF(self.data, self.mu[0], self.sigma[0])
        for m in range(1, self.nMix):
            p = vect_logsumexp(p, np.log(self.pi[m]) + lognormPDF(self.data, self.mu[m], self.sigma[m]))
        return p
mog = MoG_log(A, 3)
llhist = mog.em()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10962261068452867, 'fraction_numerical': 0.013233131841202417, 'mean_word_length': 3.697812739831159, 'pattern_counts': {'":': 1, '<': 43, '<?xml version=': 0, '>': 46, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '279737', 'license': 'MIT', 'n_tokens_mistral': 7601, 'n_tokens_neox': 7286, 'n_words': 2852, 'path': 'labs/lab2/lab2_em.ipynb', 'repo_name': 'melaanya/twente-aml', 'size': '24504'}