<jupyter_start><jupyter_text>Lambda School Data Science*Unit 2, Sprint 3, Module 1*--- Define ML problemsYou will use your portfolio project dataset for all assignments this sprint. AssignmentComplete these tasks for your project, and document your decisions.- [ ] Choose your target. Which column in your tabular dataset will you predict?- [ ] Choose which observations you will use to train, validate, and test your model. And which observations, if any, to exclude.- [ ] Determine whether your problem is regression or classification.- [ ] Choose your evaluation metric.- [ ] Begin to clean and explore your data.- [ ] Begin to choose which features, if any, to exclude. Would some features "leak" information from the future?<jupyter_code>dataurl = 'https://raw.githubusercontent.com/Nckflannery/Unit2-Project/master/Data/NFL%20Combine%20Data%20-%20Combine%20Results.csv'
import pandas as pd
df = pd.read_csv(dataurl)
df.head()
# Let's rename the columns to make our lives easier later
df.columns
df.columns = ['Year', 'Name', 'YearName', 'College', 'POS', 'Height',
       'Weight', 'Hand Size', 'Arm Length', 'Wonderlic',
       '40 Yard', 'Bench Press', 'Vert Leap', 'Broad Jump',
       'Shuttle', '3Cone', '60Yd Shuttle', 'Pick', 'Round']
df.head()<jupyter_output><empty_output><jupyter_text>Choose a target<jupyter_code># Our target will be 'Round'. Given someone's ht, wt, combine stats, and college, can we predict where in the draft they will be picked?
df['Round'].describe()
df['Round'].value_counts()<jupyter_output><empty_output><jupyter_text>Train, test, val split<jupyter_code># Let's use the 2015 year as our test data
# and random split the rest into train/val
split = 2015
train = df[df['Year'] < split]
test = df[df['Year'] == split]
train.shape, test.shape, df.shape
from sklearn.model_selection import train_test_split

my_train, my_val = train_test_split(train, random_state=42, test_size=.2)
my_train.shape, my_val.shape, train.shape
target = 'Round'
exclude = ['Year', 'Name', 'YearName', 'Pick', 'Round']
X_train = my_train.drop(exclude, axis=1)
y_train = my_train[target]
X_val = my_val.drop(exclude, axis=1)
y_val = my_val[target]
X_train.shape, y_train.shape, X_val.shape, y_val.shape<jupyter_output><empty_output><jupyter_text>Determine problem type and choose evaluation metric  This will be a regression problem.<jupyter_code># Since there are multiple modes, we will use the mean as the baseline
x = df.copy()
baseline = x['Round'].mean().astype(int)
x['base_pred'] = [baseline] * len(x['Round'])
x.head()
from sklearn.metrics import accuracy_score
base_pred = x['base_pred']
y_true = x['Round']
accuracy_score(base_pred, y_true)
# We need to deal with NaN's first
# Pick goes up to 336. Since a higher pick is a worse outcome, we will replace the NaNs with HIGH values
x['Round'] = x['Round'].fillna(20)
x['Round'].describe()
# We now see that our mode is overwhelmingly undrafted! (20)
baseline = x['Round'].mode()[0].astype(int)
x['base_pred'] = [baseline] * len(x['Round'])
base_pred = x['base_pred']
x.head()
accuracy_score(y_true, base_pred)
import seaborn as sns
sns.distplot(x['Round']);<jupyter_output><empty_output><jupyter_text>Explore and clean data<jupyter_code># First we will make a copy of df to work with, and later create a function to do everything we explore here
x = df.copy()
x.head()
x['POS'].unique()
import matplotlib.pyplot as plt

# with plt.style.context('fivethirtyeight'):
fig, ax = plt.subplots(figsize=(8,8))
sns.scatterplot(data=x[x['POS'] == 'RB'], x='40 Yard', y='Round', label='Running Backs')
sns.scatterplot(data=x[x['POS'] == 'C'], x='40 Yard', y='Round', label='Centers')
ax.set_ylim(0,13)
ax.set_xlim(4, 6)
ax.axhline(0, lw=.5, color='black')
ax.legend(loc=1)
ax.set_title('Round Drafted vs. 40 Yard Dash Speed', size=25);
plotter = pd.DataFrame((x[x['Pick'] == 1]).POS.value_counts())
with plt.style.context('default'):
    fig, ax = plt.subplots(figsize=(8,8))
    plotter.plot.bar(ax=ax)
    ax.set_title('Number 1 Picks by Position')
plotter1 = pd.DataFrame((x[x['Round'] == 1]).POS.value_counts())
with plt.style.context('fivethirtyeight'):
    fig, ax = plt.subplots(figsize=(8,8))
    plotter1.plot.bar(ax=ax)
    ax.set_title('1st Round Picks by Position')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14563334113790682, 'fraction_numerical': 0.015921329899321, 'mean_word_length': 5.72755905511811, 'pattern_counts': {'":': 0, '<': 22, '<?xml version=': 0, '>': 21, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '272255', 'license': 'MIT', 'n_tokens_mistral': 1511, 'n_tokens_neox': 1438, 'n_words': 433, 'path': 'module1/assignment_applied_modeling_1.ipynb', 'repo_name': 'Nckflannery/DS-Unit-2-Applied-Modeling', 'size': '4271'}