<jupyter_start><jupyter_text>Escalamiento de datos numéricos  Transformaciones lineales<jupyter_code>import timeit
import numpy as np
import pandas as pd 
import matplotlib.pyplot as plt
from sklearn import datasets, linear_model

X, y = datasets.load_diabetes(return_X_y=True)
raw = X[:, None, 2]<jupyter_output><empty_output><jupyter_text>Detalles del dataset: https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_diabetes.html<jupyter_code># escalamiento max-min
max_raw = max(raw)
min_raw = min(raw)
scaled = (2*raw - max_raw -min_raw)/(max_raw - min_raw)

# normalización Z-score

avg = np.average(raw)
std = np.std(raw)
z_scaled = (raw - avg)/std

fig, axs = plt.subplots(3, 1, sharex=True, tight_layout=True)

axs[0].hist(raw)
axs[1].hist(scaled)
axs[2].hist(z_scaled)
# modelos para entrenamiento
def train_raw():
    linear_model.LinearRegression().fit(raw, y)

def train_scaled():
    linear_model.LinearRegression().fit(scaled, y)

def train_z_scaled():
    linear_model.LinearRegression().fit(z_scaled, y)
raw_time = timeit.timeit(train_raw, number = 100)
scaled_time = timeit.timeit(train_raw, number = 100)
z_scaled_time = timeit.timeit(train_raw, number = 100)
print('trainning time for raw data : {} '.format(raw_time))
print('trainning time for scaled data : {}'.format(scaled_time))
print('trainning time for z_scaled data : {}'.format(z_scaled_time))<jupyter_output>trainning time for raw data : 0.0699994839997089 
trainning time for scaled data : 0.023044682999170618
trainning time for z_scaled data : 0.019402459998673294<jupyter_text>* max-min scaling: mejor para datos uniformemente distribuidos* z-score scaling: mejor para datos distribuidos "normalmente" (forma de campana de gauss) Utilidades de Scikit Learn : https://scikit-learn.org/stable/modules/preprocessing.html  Transformaciones no lineales<jupyter_code>df = pd.read_csv('cars.csv')
df.price_usd.hist()<jupyter_output><empty_output><jupyter_text>Transformación con $$tanh(x)$$<jupyter_code>p = 10000
df.price_usd.apply(lambda x: np.tanh(x/p)).hist()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12080536912751678, 'fraction_numerical': 0.0364333652924257, 'mean_word_length': 6.616788321167883, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '577799', 'license': 'MIT', 'n_tokens_mistral': 832, 'n_tokens_neox': 773, 'n_words': 160, 'path': 'estadisticas-descriptivas/clases_notebooks/[clase-15]Procesamiento-datos-numericos.ipynb', 'repo_name': 'francomanca93/matematicas-data-science', 'size': '2086'}