<jupyter_start><jupyter_text>INFSCI 2915 Foundations- Machine Learning - Spring 2018 Assignment 3Issued Monday, 3/26/2018; Due Monday, 11:59pm, 4/09/2018Total points: 100<jupyter_code># Type in your information in the double quotes
firstName = ""
lastName = ""
pittID = ""<jupyter_output><empty_output><jupyter_text>Problem 1.Linear Discriminant Analysis (LDA)and Quadratic Discriminant Analysis(QDA)      [30 points]  Writing a code is not required for this problem.  Problem 1-1  Assume we have K classes to be classified with one feature $(x)$. The prior probability ofclass $k$ is $𝜋_{k} = 𝑃(𝑌 = 𝑘)$. Assume that the feature in class k has Gaussian distribution ofmean $μ_{k}$ and variance $σ^2 (𝒩(μ,𝜎^{2}))$.The variance is the same for all classes. Prove that the Bayes’ classifier (that chooses class k with largest $𝑃(𝑌 = 𝑘|𝑥))$ is equivalent to assigning an observation to the class for which the discriminant function $𝛿_{k}(x)$ ismaximized, where \begin{array} \\𝛿_{k}(x) = x\frac{\mu _{k}}{\sigma ^{2}}- \frac{\mu_{2}^{k}}{2\sigma ^{2}}+ log(\pi _{k})\end{array} What is the name of this classifier?<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 1-2  Extend **Problem 1-1** to include **p** features. With features from each class drawn from aGaussian distribution with mean vector $μ_{k}$ and covariance matrix $Σ_{k}$ (which is nowdifferent for each class). Find the discriminant function that maximizes **𝑃(𝑌 = 𝑘|𝑥)**. Isthe relationship with the feature vector **x** linear? What is this classifier?<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 1-3  - Explain Bias-variance trade-off in choosing between LDA and QDA<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 2. Support Vector Machines  [20 points] Writing a code is not required for this problem.  Problem 2-1 Answer the following quesionts  - Describe limitations of using Maximal Margin Classifier - What is the difference between Support Vector Classifier and Maximal Margin Classifier- Explain Bias-variance trade-off that occurs when we choose large and small values for the Slack Variables<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 2-2How does the Radial Basis Function Kernel in SVM measure the similarity between a test point and a training example? Discuss the impact of choosing a large RBF parameter **𝛾**on the learning algorithm.<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 3 Classification Performance Evaluation and Cross validation [30 points]   Problem 3-1 In a fraud detection system, a QDA classifier’s confusion matrix is found to be: |        |Predicted Class - Not fraud| Predicted Class - fraud||:--:|:-------------------------------:||Actual class – Not fraud|1000|20||Actual class – fraud|30|5| - Evaluate the overall error rate and the accuracy- Evaluate the precision and the recall - Is dataset balanced?- Comment on the results.<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 3-2 Assuming we want to lower the misclassification of fraud. How can you modify the classifier to do better classification using information from confusion matrix?<jupyter_code>#Write your narrative answer here (no code is needed for this part)<jupyter_output><empty_output><jupyter_text>Problem 3-3 Cross validation, SVM  In this exercise, we will use SVM  for breast cancer classification (malignant or benign). Use a code below to download the dataset:<jupyter_code>from sklearn.datasets import load_breast_cancer
dataset = load_breast_cancer()
print(dataset.keys())<jupyter_output>dict_keys(['data', 'target', 'target_names', 'DESCR', 'feature_names'])<jupyter_text>Follow steps to answer questions.- Scale the features with MinMaxScaler- Split breast cancer dataset into two, test dataset and train dataset- Find best SVM classifier model. Try values of C=[0.001, 0.1, 1, 10, 1000] and Gamma = [0.001, 0.1, 1, 10, 1000]. - Use 3-fold cross validation to find the best parameters (using all possible combinations of these values for C and gamma).- Report your parameters- Report accuracy, confusion matrix, precision, and recall (use Test dataset, SVM classifier model with best parameters)- Comment your results<jupyter_code>#Write your answer here<jupyter_output><empty_output><jupyter_text>Problem 3-4 Cross validation, Logistic regression - Repeat **Problem 3-3** but instead of SVM find best Logistic regression classifier model. Try values of C= [0.001,0.01,1,10,100, 1000] and Penalty = ["l1","l2"]<jupyter_code>#Write your answer here<jupyter_output><empty_output><jupyter_text>Problem 4  Neural Networks [20 points] In this exercise we will clasify Iris species (Setosa, Versicolor, Virginica) using Neural Networks.Use a code below to download the dataset:<jupyter_code>from sklearn.datasets import load_iris
dataset=load_iris()
print(dataset.keys())<jupyter_output>dict_keys(['data', 'target', 'target_names', 'DESCR', 'feature_names'])<jupyter_text>Follow steps to answer questions.- Scale the feautures with MinMaxScaler, then use sklearn MLPClassifier. - Build a model that has two hidden layers, the first layer has 10 neurons and second layer has 5 neurons. - Use 'relu' activation function, and set the regularization parameter alpha=0.5. - Set the random_state=0 for splitting and building all models.  Problem 4-1 - Use gradient descent to solve the optimization  problem (i.e. get the weights), and choose random_state=0 (which corresponds to a particular initializationo of weight values). - Report accuracy, confusion matrix, precision, and recall<jupyter_code>#Write your answer here<jupyter_output><empty_output><jupyter_text>Problem 4-2 - Repeat **Problem 4-1** but with a model that use random_state=10 to initialize the weights. - Report accuracy, confusion matrix, precision, and recall<jupyter_code>#Write your answer here<jupyter_output><empty_output><jupyter_text>Problem 4-3 - Repeat **Problem 4-2** but with model that use L-BFGS (a numerical quasi-Newton method) instead of stochastic gradient descent to find the weights.- Report accuracy, confusion matrix, precision, and recall- Comment on results<jupyter_code>#Write your answer here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09056489474481297, 'fraction_numerical': 0.021808268968650613, 'mean_word_length': 5.670707070707071, 'pattern_counts': {'":': 0, '<': 59, '<?xml version=': 0, '>': 59, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '64390', 'license': 'Apache-2.0', 'n_tokens_mistral': 2045, 'n_tokens_neox': 1878, 'n_words': 786, 'path': 'assignment3/Assignment 3.ipynb', 'repo_name': 'BeMoreMature/ml_course', 'size': '6603'}