<jupyter_start><jupyter_text>Stats216v: Statistical LearningStanford UniversitySummer 2017<NAME> (<EMAIL>)  10. Unsupervised Learning  10.1.R1You are analyzing a dataset where each observation is an age, height, length, and width of a particular turtle. You want to know if the data can be well described by fewer than four dimensions (maybe for plotting), so you decide to do Principal Component Analysis. Which of the following is most likely to be the loadings of the first Principal Component?1. (1, 1, 1, 1)2. (.5, .5, .5, .5)3. (.71, -.71, 0, 0)4. (1, -1, -1, -1) Gyu-Ho's Answer: 2.We know that options 1 and 4 cannot be right because the sum of the squared loadings exceeds 1. The second option is most likely correct because we expect all four variables to be positively correlated with each-other.Note that it is fairly common for the loadings of the first principal component to all have the same sign. In such a case, the principal component is often referred to as a size component.  10.2.R1Suppose we a data set where each data point represents a single student's scores on a math test, a physics test, a reading comprehension test, and a vocabulary test.We find the first two principal components, which capture 90% of the variability in the data, and interpret their loadings. We conclude that the first principal component represents overall academic ability, and the second represents a contrast between quantitative ability and verbal ability.What loadings would be consistent with that interpretation? Choose all that apply.1. (0.5, 0.5, 0.5, 0.5) and (0.71, 0.71, 0, 0)2. (0.5, 0.5, 0.5, 0.5) and (0, 0, -0.71, -0.71)3. (0.5, 0.5, 0.5, 0.5) and (0.5, 0.5, -0.5, -0.5)4. (0.5, 0.5, 0.5, 0.5) and (-0.5, -0.5, 0.5, 0.5)5. (0.71, 0.71, 0, 0) and (0, 0, 0.71, 0.71)6. (0.71, 0, -0.71, 0) and (0, 0.71, 0, -0.71) Gyu-Ho's Answer: 3, 4.For the first two choices, the two loading vectors are not orthogonal. For the fifth and sixth choices, the first set of loadings only has to do with two specific tests. For the third and fourth pairs of loadings, the first component is proportional to average score, and the second component measures the difference between the first pair of scores and the second pair of scores.  10.3.R1True or False: If we use k-means clustering, will we get the same cluster assignments for each point, whether or not we standardize the variables. Gyu-Ho's Answer: False.The points are assigned to centroids using Euclidean distance. If we change the scaling of one variable, e.g. by dividing it by 10, then that variable will matter less in determining Euclidean distance.  10.4.R1True or False: If we cut the dendrogram at a lower point, we will tend to get more clusters (and cannot get fewer clusters). Gyu-Ho's Answer: True.After cutting the dendrogram at threshold t, we keep all the joins with linkage distance less than t and discard the joins with larger linkage distance. Thus, decreasing the threshold gives us fewer joins, and thus more clusters. If, in decreasing the threshold, we don't cross a junction of the dendrogram, the number of clusters will remain the same.  10.5.R1In the heat map for breast cancer data, which of the following depended on the output of hierarchical clustering?1. The ordering of the rows2. The ordering of the columns3. The coloring of the cells as red or green Gyu-Ho's Answer: 1, 2.The dendrograms obtained from hierarchical clustering were used to order the rows and columns. The coloring of the cells was based on gene expression, but without the hierarchical clustering step, the heat map would not have looked like anything meaningful.<jupyter_code>LoadLibraries = function() {
    library(MASS)
    install.packages("ISLR")
    library(ISLR)
    print("Libraries have been loaded!")
}

LoadLibraries()<jupyter_output>Updating HTML index of packages in '.Library'
Making 'packages.html' ... done<jupyter_text>10.R.1Suppose we want to fit a linear regression, but the number of variables is much larger than the number of observations. In some cases, we may improve the fit by reducing the dimension of the features before.In this problem, we use a data set with n = 300 and p = 200, so we have more observations than variables, but not by much. Load the data `x`, `y`, `x.test`, and `y.test` from `10.R.RData`.First, concatenate `x` and `x.test` using the rbind functions and perform a principal components analysis on the concatenated data frame (use the `"scale=TRUE"` option). To within 10% relative error, what proportion of the variance is explained by the first five principal components?<jupyter_code>load("10.R.RData", verbose=T)
# 0.34986<jupyter_output>Loading objects:
  x
  y
  x.test
  y.test<jupyter_text>10.R.2The previous answer suggests that a relatively small number of "latent variables" account for a substantial fraction of the features' variability. We might believe that these latent variables are more important than linear combinations of the features that have low variance.We can try forgetting about the raw features and using the first five principal components (computed on `rbind(x,x.test)`) instead as low-dimensional derived features. What is the mean-squared test error if we regress y on the first five principal components, and use the resulting model to predict `y.test`?<jupyter_code># 0.9923<jupyter_output><empty_output><jupyter_text>10.R.3Now, try an OLS linear regression of y on the matrix `x`. What is the mean squared predition error if we use the fitted model to predict `y.test` from `x.test`?<jupyter_code># 3.657<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.074206847105216, 'fraction_numerical': 0.03656569277648324, 'mean_word_length': 4.386100386100386, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '313013', 'license': 'MIT', 'n_tokens_mistral': 1713, 'n_tokens_neox': 1524, 'n_words': 847, 'path': 'notebooks/stats216v/chapter-10-unsupervised.ipynb', 'repo_name': 'gyuho/dplearn', 'size': '5592'}