<jupyter_start><jupyter_text>Comparision of Machine Learning Methods vs Rule Based* Traditionally, Educational Institutions use rule based models to generate risk score which then informs resource allocation. For example, *[Hiller et al, 1999](https://www.ets.org/Media/Research/pdf/RD_Connections18.pdf)** Instead, we'll build a simple model using basic ML techniques and demonstrate why the risk scores generated are better<jupyter_code>## Imports
import pandas as pd 
import seaborn as sns
sns.set(color_codes=True)
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Setup* First, we need to generate simulated data and read it into a data frame<jupyter_code># Gen Data
%run sim.py
stud_df.gpa = pd.to_numeric(stud_df.gpa)
stud_df.honors = pd.to_numeric(stud_df.honors)
stud_df.psat = pd.to_numeric(stud_df.psat)<jupyter_output><empty_output><jupyter_text>Determine if the student undermatched-or-was-properly matched<jupyter_code>avg_gpas = stud_df.groupby('college').gpa.mean()
def isUndermatched(student):
    if student.gpa >= (avg_gpas[student.college] + .50):
        return True
    else:
        return False
stud_df['undermatch_status'] = stud_df.apply(isUndermatched, axis =1 )
#stud_df.groupby('race').undermatch_status.value_counts()<jupyter_output><empty_output><jupyter_text>Rule Based Model * Simple GPA and PSAT rule<jupyter_code>msk = np.random.rand(len(stud_df)) < 0.8
train = stud_df[msk]
test = stud_df[~msk]
print("Training Set Length: ", len(train))
print("Testing Set Length: ", len(test))<jupyter_output>Training Set Length:  3151
Testing Set Length:  849<jupyter_text>The Rules* We have 3 observed variables - GPA, PSAT, race* Predict which college based on those observed variables. * Rules based on *[Hoxby, et al 2013](http://www.brookings.edu/~/media/projects/bpea/spring-2013/2013a_hoxby.pdf)*<jupyter_code>stud_df.psat.hist()
def rule_based_model(student_r): 
    """returns a college for each student passed"""
    risk_score = 0
    if student_r.race == 'aa':
        risk_score += 1
    if student_r.race == 'latino':
        risk_score += .5 
    if student_r.psat >= 170 and student_r.honors <= 3:
        risk_score += 1
    return risk_score
test['risk_score'] = test.apply(rule_based_model, axis = 1)<jupyter_output>/Users/hunterowens/anaconda3/lib/python3.5/site-packages/ipykernel/__main__.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  if __name__ == '__main__':<jupyter_text>Machine Learning Model* Simple Logisitic Regression<jupyter_code>from sklearn import linear_model
feature_cols = ['psat', 'gpa', 'honors']
X = train[feature_cols]
y = train['undermatch_status']

# instantiate, fit
lm = linear_model.LogisticRegression()
lm.fit(X, y)
# The coefficients
print('Coefficients: \n', lm.coef_)
# The mean square error
print("Residual sum of squares: %.2f"
      % np.mean((lm.predict(test[feature_cols]) - test['undermatch_status']) ** 2))
# Explained variance score: 1 is perfect prediction
lm.predict(train[feature_cols])
sns.lmplot(x='psat', y='undermatch_status', data=test, logistic=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11632100991884581, 'fraction_numerical': 0.014126840997896002, 'mean_word_length': 5.629482071713148, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '132445', 'license': 'CC0-1.0', 'n_tokens_mistral': 1192, 'n_tokens_neox': 1122, 'n_words': 279, 'path': 'EduMLvsRuleBased.ipynb', 'repo_name': 'hunterowens/machine-learning-in-edu', 'size': '3327'}