<jupyter_start><jupyter_text>Introduction to Machine Learning and TensorflowNote: This doesn't actually go into tensorflow yet. Sorry about thatWelcome! This interactive machine learning and tensorflow tutorial is meant to combine existing tutorials to go through some basics of using this graphical library for the purposes of machine learning. The material for this tutorial comes largely from the google machine learning tutorial:https://developers.google.com/machine-learning/crash-course/For helpful tidbits about Jupyter notebooks themselves check out:https://www.youtube.com/watch?v=HW29067qVWkFor helpful latex symbol references see: http://cs.brown.edu/about/system/managed/latex/doc/symbols.pdf https://www.sharelatex.com/learn/Subscripts_and_superscriptsOperators_using_subscripts_and_superscripts  Authors:  <NAME> (As you update this please add your name :D )  Table of Contents What is Machine LearningFramework - missing supervised vs unsupervised section Starting Simple - Linear RegressionModels and LabelsLossReducing Loss   What is Machine Learning? The framework We spend lots of time trying to come up with answers to a whole slew of questions, often coming up with a solution then seeing if we can find a better one. What if we could write a code that would do that for us? We can! And that's what machine learning boils down to. In this tutorial we will use tensorflow, as well as some other code libraries to create iterative codes that find good solutions to problems (including, hopefully, some hard ones). We'll talk about what makes a solution good and what makes a problem hard. There are different types of problems that can be tackled, often that require different methods and arise from differences in available data. Here, at least at first, we'll focus on something called "Supervised Machine Learning." This means for every data point I have (or example) I not only have the features associated with that example, but also a label. I will then build a model trained on several of these examples that, when trained, can take in a new feature set and give me a reasonable guess about what the label for that might be.Model - defines relationship between feature & estimate of labelExample - instance of data, both features and labels$$(\vec{x}, y)$$Feature - a bit of data about the object, combined into a feature vector$$\vec{x} = x_1 + x_2 + ... + x_n$$Label - The true value for the informaiton about the object that one is going to model$$y$$Say I want to predict height, so I get a bunch of data from a big group of people. I record the height of both their parents, their weight, and whether or not they can dunk on a regulation height basketball hoop. These are all features that may be useful for my model. I also record their true height. That is the label in this case. Since we're predicting how tall they will be, which is predicting a continuous value, this is a regression problem. Machine learning can also be used to solve classification problems (for example are they taller or shorter than 6 feet).Regression - predict continous valuesClassification - predict category memborship   Starting simple - Linear Regression Using a model to generate labelsYou might remember linear regression from stats. Linear regression is a way to estimate the value of a label from features. Each feature is multiplied by a corresponding weight (or slope term as described in many statistics textbooks). There is an additional term corresponding to a bias term (or an intercept term as it is often described in statistics. Our model thus will look something like the following:$$\hat{y} = \vec{w}•\vec{x}+b$$Say we have some data, namely a bunch of examples containing a single feature for each example and a corresponding label. Data such as the data below, where we want to estimate a measure of Income for a given area via median house value for that same area.<jupyter_code>%matplotlib widget
%matplotlib inline

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np

california_housing_dataframe = pd.read_csv("https://storage.googleapis.com/mledu-datasets/california_housing_train.csv", sep=",")
california_housing_dataframe = california_housing_dataframe.reindex(
    np.random.permutation(california_housing_dataframe.index))
california_housing_dataframe["median_house_value"] /= 100000.0

x = california_housing_dataframe.median_house_value
y = california_housing_dataframe.median_income
#z = california_housing_dataframe.population

num_examples = 100

plt.scatter(x[0:(num_examples-1)], y[0:(num_examples-1)])
plt.xlabel('median house value') #confused about the units here
plt.ylabel('Income')
#plt.show()<jupyter_output><empty_output><jupyter_text>We see here a scatter plot showing a bunch of examples. We could make a guess ourselves at what the intercept and slope value should be. Once we make a guess for each of those parameters, for any feature value we see our model offers a guess about what the label value should be. This is our model, in this case taking the form of a line. We can see our linear model in the scatter plot below. Feel free to change the $w_1$ and b values around in the code below to generate different models.$$\hat{y} = w_1•x_1+b$$parameters - the free variables in our model we change to make the model as accurate as possible... in this example case our $w_1$ and b terms<jupyter_code>w_1 = 1
b = 0


regressX_0 = min(x)
regressX_1 = max(x)

regressY_0 = w_1*regressX_0 + b
regressY_1 = w_1*regressX_1 + b

plt.plot([regressX_0, regressX_1], [regressY_0, regressY_1], c='r')
plt.scatter(x[0:(num_examples-1)], y[0:(num_examples-1)])
plt.xlabel('median house value') #confused about the units here, I basically normalized em beteen 1 and 10 but I don't know what they were originally
plt.ylabel('Income')
plt.show()<jupyter_output><empty_output><jupyter_text>Loss - Measuring how good our guesses areOur label guesses ($\hat{y}$) now fall on that line. With a given x value, or feature, we can make a guess about our label value. A new question arises... how good are our guesses? In order to measure this we need to compare our estimated label value ($\hat{y}$) with the true label value (y). Functions that make this comparison are called loss functions in machine learning.Loss - A comparison between a label value predicted by a model and the true label valueYou may recall from statistics that when preforming a linear regression we would try to minimize something called the Mean Square Error$$MSE = \frac {\sum_{(\vec{x},y)\in D} (y-\hat{y}(x))^2} {N}$$N = the number of data points D = the set of data pointsThis is a way of measuring how far the predicted label value $(\hat{y})$ for each feature set $(\vec{x})$ differs from the true label (y). This is a type of loss measurement! In Machine Learning we'll use a similar measurement of loss called L2 loss, which is the same aside from the division by the number of data points.$$L2 Loss = \sum_{(x,y)\in D} (y-\hat{y}(\vec{x}))^2$$Below you can manipulate the weight and bias values again, and see how this L2 loss changes as you do so. Since loss represents the difference between a true label value and the estimated one, the smaller that loss the more accurate the model.Later we'll talk about different measurements of loss, which ones are good for which situatios, and more.<jupyter_code>w_1 = 1.5 ; b = 0.5

regressX_0 = min(x); regressX_1 = max(x)
regressY_0 = w_1*regressX_0 + b; regressY_1 = w_1*regressX_1 + b

plt.plot([regressX_0, regressX_1], [regressY_0, regressY_1], c='r')
plt.scatter(x[0:(num_examples-1)], y[0:(num_examples-1)])
plt.xlabel('median house value'); plt.ylabel('Income')
plt.show()

y_est = np.zeros([num_examples,1]); L2Loss = 0

for i in range(0,(num_examples)):

    y_est[i] = w_1*x[i] + b
    L2Loss = L2Loss + ((y[i])-(y_est[i]))**2

print("L2Loss = %s" %L2Loss)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07266041085467918, 'fraction_numerical': 0.010651787978696424, 'mean_word_length': 4.970476911430734, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 14, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '612287', 'license': 'MIT', 'n_tokens_mistral': 2254, 'n_tokens_neox': 2131, 'n_words': 1077, 'path': 'deep learning/TensorFlow_examples/Intro to Machine Learning and Tensorflow 0.1.ipynb', 'repo_name': 'jtbuch/applied-statistics-reading-group', 'size': '7898'}