<jupyter_start><jupyter_text>Use regression to predict financial data for Enron employees and associates. Once you know some financial data about an employee, like their salary, __what would you predict for the size of their bonus?__  - Target : Bonus  - Input  : Salary<jupyter_code>## list the features you want to look at--first item 
## in the list will be the "target" feature
features_list = ["bonus", "salary"]
data = featureFormat( dictionary, features_list, remove_any_zeroes=True)
target, features = targetFeatureSplit( data )
### training-testing split needed in regression, just like classification
#from sklearn.model_selection import train_test_split
feature_train, feature_test, target_train, target_test = train_test_split(features, target, test_size=0.5, random_state=42)
train_color = "b"
#test_color = "b"<jupyter_output><empty_output><jupyter_text>You will be fitting your regression using the blue (training) points only. (You may have noticed that instead of the standard 10%, we’ve put 50% of our data into the test set--that’s because in Part 5, we will switch the training and testing datasets around and splitting the data evenly makes this more straightforward.)<jupyter_code>## Your regression goes here!. Please name it reg, so that the plotting code below picks it up and 
## plots it correctly. Don't forget to change the test_color above from "b" to "r" to differentiate training points from test points.

test_color = "r"
from sklearn.linear_model import LinearRegression
reg = LinearRegression()
reg.fit(feature_train, target_train)<jupyter_output><empty_output><jupyter_text>Extracting Slope and Intercept<jupyter_code>print ('Coefficients : ', reg.coef_)
print ('Intercept : ', reg.intercept_)<jupyter_output>Intercept :  662220.4176015807<jupyter_text>Regression Score: Training DataImagine you were a less savvy machine learner, and didn’t know to test on a holdout test set. Instead, you tested on the same data that you used to train, by comparing the regression predictions to the target values (i.e. bonuses) in the training data. What score do you find? You may not have an intuition yet for what a “good” score is; this score isn’t very good (but it could be a lot worse).<jupyter_code>print ('R-squared value on training data : ', reg.score(feature_train, target_train))<jupyter_output>R-squared value on training data :  0.04164573226747781<jupyter_text>Regression Score: Test DataNow compute the score for your regression on the test data, like you know you should. What’s that score on the testing data?<jupyter_code>print ('R-squared value on testing data : ', reg.score(feature_test, target_test))
reg.predict(feature_test)
for feature, target in zip(feature_test, target_test):
    plt.scatter( feature, target, color=test_color) 

for feature, target in zip(feature_train, target_train):
    plt.scatter( feature, target, color=train_color )
plt.xlabel('salary')
plt.ylabel('bonus');

### labels for the legend
plt.scatter(feature_test[0], target_test[0], color=test_color, label="test")
plt.scatter(feature_test[0], target_test[0], color=train_color, label="train")

try:
    plt.plot( feature_test, reg.predict(feature_test) )
except NameError:
    pass

plt.xlabel(features_list[1])
plt.ylabel(features_list[0])
plt.legend()
plt.show()
### labels for the legend
plt.scatter(feature_test, target_test, color=test_color, label="test")
plt.scatter(feature_test[0], target_test[0], color=train_color, label="train")

try:
    plt.plot( feature_test, reg.predict(feature_test) )
except NameError:
    pass

plt.xlabel(features_list[1])
plt.ylabel(features_list[0])
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>You might notice a few data points that fall outside the main trend, someone who gets a high salary (over a million dollars!) but a relatively small bonus. This is an example of an outlier. As things stand right now, this point falls into the test set (and probably hurting the score on our test data as a result).  Let’s add a little hack to see what happens if it falls in the training set instead. Add these two lines near the bottom of ```finance_regression.py```, i.e., right before ```plt.xlabel(features_list[1])```:  > reg.fit(feature_test, target_test)  plt.plot(feature_train, reg.predict(feature_train), color="b")   Sneak Peek: Outliers Break Regressions<jupyter_code>### labels for the legend
plt.scatter(feature_test[0], target_test[0], color=test_color, label="test")
plt.scatter(feature_test[0], target_test[0], color=train_color, label="train")

try:
    plt.plot( feature_test, reg.predict(feature_test) )
except NameError:
    pass

reg.fit(feature_test, target_test)
plt.plot(feature_train, reg.predict(feature_train), color="purple")                   #  bepaalt regression line obv trainingdata
plt.xlabel(features_list[1])
plt.ylabel(features_list[0])
plt.legend()
plt.show()
print ('Coefficients : ', reg.coef_)<jupyter_output>Coefficients :  [2.36761212]<jupyter_text>Now we’ll be drawing two regression lines, one fit on the test data (with outlier) and one fit on the training data (no outlier, purple line). Look at the plot now--big difference, huh? That single outlier is driving most of the difference.  What’s the slope of the new regression line?<jupyter_code>reg.coef_<jupyter_output><empty_output><jupyter_text>The slope is about 2.27 after removing the outlier, which is a big difference from what we had before (about 5.4).  A small number of outliers makes a big difference! From Python 3.3 forward, a change to the order in which dictionary keys are processed was made such that the orders are randomized each time the code is run. This will cause some compatibility problems with the graders and project code, which were run under Python 2.7. To correct for this, add the following argument to the __featureFormat__ call on line 26 of __finance_regression.py__> sort_keys = '../tools/python2_lesson06_keys.pkl'This will open up a file in the tools folder with the Python 2 key order.  Regressing Bonus Against LTI(long term incentive)There are lots of finance features available, some of which might be more powerful than others in terms of predicting a person’s bonus. For example, suppose you thought about the data a bit and guess that the “long_term_incentive” feature, which is supposed to reward employees for contributing to the long-term health of the company, might be more closely related to a person’s bonus than their salary is.A way to confirm that you’re right in this hypothesis is to regress the bonus against the long term incentive, and see if the regression score is significantly higher than regressing the bonus against the salary. Perform the regression of bonus against long term incentive--what’s the score on the test data?<jupyter_code>### list the features you want to look at--first item in the 
### list will be the "target" feature
features_list = ["bonus", 'long_term_incentive']
data = featureFormat( dictionary, features_list, remove_any_zeroes=True)
target, features = targetFeatureSplit( data )
feature_train, feature_test, target_train, target_test = train_test_split(features, target, test_size=0.5, random_state=42)
train_color = "b"

test_color = "r"
reg2 = LinearRegression()
reg2.fit(feature_train, target_train)<jupyter_output><empty_output><jupyter_text>Extracting Slope and Intercept<jupyter_code>print ('Coefficients : ', reg2.coef_)
print ('Intercept : ', reg2.intercept_)<jupyter_output>Intercept :  1044304.0637582359<jupyter_text>Regression Score (on test data)<jupyter_code>print ('R-squared value on testing data : ', reg2.score(feature_test, target_test))<jupyter_output>R-squared value on testing data :  0.10891175736247383<jupyter_text>We have a better score when using long-term incentive to predict someone's bonus (as compared to salary), which translates to a better fit.<jupyter_code>for feature, target in zip(feature_test, target_test):
    plt.scatter( feature, target, color=test_color ) 

for feature, target in zip(feature_train, target_train):
    plt.scatter( feature, target, color=train_color )
    
plt.xlabel('Long term incentive')
plt.ylabel('bonus');
### labels for the legend
plt.scatter(feature_test, target_test, color=test_color, label="test")
plt.scatter(feature_test[0], target_test[0], color=train_color, label="train")

try:
    plt.plot( feature_test, reg2.predict(feature_test) )
except NameError:
    pass

plt.xlabel(features_list[1])
plt.ylabel(features_list[0])
plt.legend()
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08914956011730206, 'fraction_numerical': 0.01560117302052786, 'mean_word_length': 5.89248181083266, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '414268', 'license': 'CNRI-Python', 'n_tokens_mistral': 2675, 'n_tokens_neox': 2517, 'n_words': 945, 'path': '7_finance_regression_jn.ipynb', 'repo_name': 'dirkkadijk/ENRON-data_machine-learning-project', 'size': '8525'}