<jupyter_start><jupyter_text>Importar librerías<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re<jupyter_output><empty_output><jupyter_text>Carga dataframe<jupyter_code>df = pd.read_csv('./Data/data_nba.csv', index_col=0, low_memory=False)
df.reset_index(drop=True, inplace=True)
df.shape<jupyter_output><empty_output><jupyter_text>Transformación columnas<jupyter_code>df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 548064 entries, 0 to 72223
Data columns (total 32 columns):
 #   Column      Non-Null Count   Dtype  
---  ------      --------------   -----  
 0   date        548064 non-null  object 
 1   team        548064 non-null  object 
 2   player      548064 non-null  object 
 3   role        548064 non-null  object 
 4   MP          548064 non-null  object 
 5   FG          548064 non-null  int64  
 6   FGA         548064 non-null  int64  
 7   FG_PCT      519079 non-null  float64
 8   FG3         548064 non-null  int64  
 9   FG3A        548064 non-null  int64  
 10  FG3_PCT     323121 non-null  float64
 11  FT          548064 non-null  int64  
 12  FTA         548064 non-null  int64  
 13  FT_PCT      320142 non-null  float64
 14  ORB         548064 non-null  int64  
 15  DRB         548064 non-null  int64  
 16  TRB         548064 non-null  int64  
 17  AST         548064 non-null  int64  
 18  STL         548064 non-null  int64  
 19  BLK[...]<jupyter_text>Date<jupyter_code>df['date'] = pd.to_datetime(df['date'])<jupyter_output><empty_output><jupyter_text>Height<jupyter_code>df['height'].value_counts()
def feet_inch_to_cm(text):
    """Returns a number in cm if given a string with the format X-Y
    where X are the feet and Y the inches"""
    try:
        return int(re.findall('(\d*)-', text)[0]) * 30.48 + int(re.findall('-(\d*)', text)[0]) * 2.54
    except:
        pass
df['height'] = df['height'].apply(lambda x: feet_inch_to_cm(x))<jupyter_output><empty_output><jupyter_text>Weight<jupyter_code>def pound_to_kg(text):
    """Returns a number in kg if given a string with the format Xlb
    where X are the pounds"""
    try:
        return round(int(re.findall('(\d*)lb', text)[0]) * 0.4535924, 2)
    except:
        pass
df['weight'] = df['weight'].apply(lambda x: pound_to_kg(x))<jupyter_output><empty_output><jupyter_text>Birth date La cambiamos a datetime para poder crear la variable 'age' a partir de ella:<jupyter_code>df['birthDate'] = pd.to_datetime(df['birthDate'])<jupyter_output><empty_output><jupyter_text>Age<jupyter_code>df.isna().sum()<jupyter_output><empty_output><jupyter_text>Eliminamos los 7633 valores sin información sobre el jugador<jupyter_code>df.dropna(subset=['salary'], inplace=True)<jupyter_output><empty_output><jupyter_text>Creamos nueva columna 'age'<jupyter_code>df['age'] = df['date'] - df['birthDate']<jupyter_output><empty_output><jupyter_text>Le aplicamos una lambda para que muestre el valor en años<jupyter_code>df['age'] = df['age'].apply(lambda x: int(np.floor(x.days/365)))<jupyter_output><empty_output><jupyter_text>Hacemos value_counts() para ver si hay outliers<jupyter_code>df['age'].sort_values().value_counts()<jupyter_output><empty_output><jupyter_text>Del notebook anterior, recordamos que en algunos casos al hacer scrapping no reconoció bien el jugador y por eso aparecía el salario mínimo, que eran 4.737$.Eliminamos estas observaciones.<jupyter_code>df[(df['age'] > 42) & (df['salary'] == 4737)].head(3)
index_names = df[(df['age'] > 42) & (df['salary'] == 4737)].index
df.drop(index_names, inplace=True)
df[(df['age'] > 42) & (df['salary'] == 4737)]
# Reseteamos el índice de nuevo

df.reset_index(drop=True, inplace=True)<jupyter_output><empty_output><jupyter_text>Eliminamos birth date Como ya tenemos la edad, podemos eliminar la variable con la fecha de nacimiento 'birthDate':<jupyter_code>df.drop('birthDate', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Name Borramos la columna ya que no aporta valor al tener la info del nombre en 'player'<jupyter_code>df.drop('name', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Position<jupyter_code>df['position'].value_counts()<jupyter_output><empty_output><jupyter_text>Creamos 5 columnas con las 5 posiciones que nos muestre si el jugador puede jugar en cada una de las posiciones con un boolean:- Point Guard- Center- Power Forward- Shooting Guard- Small Forward<jupyter_code>df['pos_PG'] = df['position'].apply(lambda x: 0 if x.find('Point Guard') == -1 else 1)
df['pos_C'] = df['position'].apply(lambda x: 0 if x.find('Center') == -1 else 1)
df['pos_PF'] = df['position'].apply(lambda x: 0 if x.find('Power Forward') == -1 else 1)
df['pos_SG'] = df['position'].apply(lambda x: 0 if x.find('Shooting Guard') == -1 else 1)
df['pos_SF'] = df['position'].apply(lambda x: 0 if x.find('Small Forward') == -1 else 1)<jupyter_output><empty_output><jupyter_text>Hacemos una comprobación al azar<jupyter_code>df.sample(15)[['position', 'pos_PG', 'pos_C', 'pos_PF', 'pos_SG', 'pos_SF']]<jupyter_output><empty_output><jupyter_text>Podemos borrar 'position'<jupyter_code>df.drop('position', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Plus_minus Tenemos 61 entradas que son intervenciones para las cuales no nos dice si el balance ha sido positivo o negativo<jupyter_code>df.isna().sum()
df[df['PLUS_MINUS'].isna()]<jupyter_output><empty_output><jupyter_text>Procedemos a eliminarlas<jupyter_code>df.dropna(subset=['PLUS_MINUS'], inplace=True)
# Reseteamos el índice de nuevo

df.reset_index(drop=True, inplace=True)<jupyter_output><empty_output><jupyter_text>Role En la variable 'role' sólo hay 2 posibles valores: 'Starter' y 'Reserve'<jupyter_code>df['role'].value_counts()
df['role'] = df['role'].apply(lambda x: 1 if x=='Starter' else 0)
df['role'].value_counts()<jupyter_output><empty_output><jupyter_text>Rebounds La variable TRB (total rebounds) es la suma de ORB (offensive rebounds) + DRB (deffensive rebounds).Al ser redundante, la podemos eliminar.<jupyter_code>df.drop('TRB', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Shoots La variable 'shoots' admite 3 resultados<jupyter_code>df['shoots'].value_counts()<jupyter_output><empty_output><jupyter_text>Haremos una columna booleana para saber si son zurdos y otra si son diestros. Aquí no haremos one-hot encoding porque queremos marcar a los ambidiestros con un 1 en ambas columnas, así que procederemos de forma similar a como lo hemos hecho con la variable 'position'.<jupyter_code>df['hand_right'] = df['shoots'].apply(lambda x: 0 if x.find('Right') == -1 else 1)
df['hand_left'] = df['shoots'].apply(lambda x: 0 if x.find('Left') == -1 else 1)<jupyter_output><empty_output><jupyter_text>Comprobamos al azar 3 ambidiestros, 3 zurdos y 3 diestros:<jupyter_code>df[df['shoots'] == 'Left Right'][['shoots', 'hand_right', 'hand_left']].sample(3)
df[df['shoots'] == 'Left'][['shoots', 'hand_right', 'hand_left']].sample(3)
df[df['shoots'] == 'Right'][['shoots', 'hand_right', 'hand_left']].sample(3)<jupyter_output><empty_output><jupyter_text>Vemos que es correcto, ya podemos eliminar la columna 'shoots'<jupyter_code>df.drop('shoots', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Minutes played La variable MP (minutes played) está como objeto, debemos pasarla a numérica<jupyter_code>df['MP'].apply(lambda x: 0 if x.find(':') == -1 else 1).value_counts()<jupyter_output><empty_output><jupyter_text>Le pasamos esta función para eliminar los ':' y luego convertimos todos los valores<jupyter_code>def seconds_to_min(text):
    """Returns a number in minutes if given a string with the format XX:YY
    where XX are the minutes and YY the seconds"""
    if text.find(':') != -1:
        return int(re.findall('(\d*):', text)[0]) + int(re.findall(':(\d*)', text)[0])/60
    else:
        return text
df['MP'] = df['MP'].apply(lambda x: seconds_to_min(x)).astype('float')<jupyter_output><empty_output><jupyter_text>Actual dataset Actualmente tenemos un dataset con las siguientes variables:- date: fecha de la intervención- team: equipo de la intervención- player: jugador- role: '0' si es suplente, '1' si es titular- MP: minutes played- FG: field goals- FGA: field goals attempt- FG_PCT: field goals %- FG3: 3 points- FG3A: 3 points attempts- FG3_PCT: 3 points %- FT: free throws- FTA: free throws attempts- FT_PCT: free throws %- ORB: offensive rebounds- DRB: deffensive rebounds- AST: assists- STL: steals- BLK: blocks- TOV: turnovers- PF: personal fouls- PTS: total points- PLUS_MINUS: balance de puntos durante la intervención- season_end: temporada- salary: salario- height: altura- weight: peso- age: edad- pos_PG: posición PG- pos_C: posición C- pos_PF: posición PF- pos_SG: posición SG- pos_SF: posición SF- hand_right: diestro- hand_left: zurdo<jupyter_code>df.info()
df.sample(10)
# Exportamos el dataset para sobreescribir el archivo que teníamos

df.to_csv('./Data/data_nba.csv')
plt.rcParams['figure.figsize'] = (20,15)
sns.heatmap(df.corr(), annot=True, cmap='coolwarm', center=0)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13630880579010857, 'fraction_numerical': 0.033665972146068646, 'mean_word_length': 5.350974930362117, 'pattern_counts': {'":': 0, '<': 121, '<?xml version=': 0, '>': 124, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '30', 'dir': 'jupyter-structured-clean-dedup', 'id': '444408', 'license': 'Apache-2.0', 'n_tokens_mistral': 3582, 'n_tokens_neox': 3331, 'n_words': 817, 'path': 'NBA 02 data cleansing - EDA.ipynb', 'repo_name': 'borjatv/NBA-project', 'size': '9119'}