<jupyter_start><jupyter_text><NAME><jupyter_code>df = pd.read_hdf('data/car.h5')
df.shape
df.columns<jupyter_output><empty_output><jupyter_text>Dummy Modelfirst model without f.e.<jupyter_code>df.select_dtypes(np.number).columns
feats = ['car_id']
X = df[feats].values
y = df['price_value'].values

model = DummyRegressor()
model.fit(X,y)
y_pred = model.predict(X)

mae(y, y_pred)


[x for x in df.columns if 'price' in x]
df['price_currency'].value_counts()<jupyter_output><empty_output><jupyter_text>Nomalize PLN and EUR<jupyter_code>df['price_currency'].value_counts(normalize=True) * 100<jupyter_output><empty_output><jupyter_text>PLN    99.80844EUR     0.19156Only 0,2% data in  database in col [price_currency] use EUR   > *Indented block*<jupyter_code>df['price_currency'].value_counts()
df = df[df[price_currency']].value_counts()<jupyter_output><empty_output><jupyter_text>without normalize - del EUR because only 0,2% od data<jupyter_code>df = df[ df['price_currency'] != 'EUR' ]
df.shape
df['price_currency'].value_counts()<jupyter_output><empty_output><jupyter_text>Features<jupyter_code>df.head()
for feat in df.columns:
  print(feat)
df['param_color'].factorize()  # change categorical data to numeric data - fuction factorize()<jupyter_output><empty_output><jupyter_text>Explore-  id arrays for id[0] - for  name of colors.<jupyter_code>df['param_color'].factorize()[0]<jupyter_output><empty_output><jupyter_text>* in this place - could add variable to pandas series and looking for sample()  or values()<jupyter_code>SUFFIX_CAT = '__cat'

for feat in df.columns:
  if isinstance(df[feat][0], list): continue
  
  factorized_values = df[feat].factorize()[0]
  if SUFFIX_CAT in feat:                
    # if value is exist not add new parameter - add value to sum of param
    df[feat] = factorized_values
  else:
    df[feat + SUFFIX_CAT] = factorized_values 
    # if value is not exist then add new cat param __a , __b etc<jupyter_output><empty_output><jupyter_text>how many feats in values .After that - not looking at price name param value because - need to predict model for price.<jupyter_code>cat_feats = [x for x in df.columns if SUFFIX_CAT in x ]
cat_feats
len(cat_feats)
cat_feats = [x for x in df.columns if SUFFIX_CAT in x ]
cat_feats = [x for x in cat_feats if 'price' not in x]    # minus 3 feats with name 'price'
len(cat_feats)<jupyter_output><empty_output><jupyter_text>To predict without analize - all 151 feats<jupyter_code>X = df[cat_feats].values
y = df['price_value'].values

model = DecisionTreeRegressor(max_depth=5)
scores = cross_val_score(model, X, y, cv=3, scoring='neg_mean_absolute_error')
# scores in minues value - minimalize to 0 
## better to predict
np.mean(scores)<jupyter_output><empty_output><jupyter_text>-19566.588937368324 much better score than before DUMMY_MODEL = 39465.934630440985 Below - next model m in pragmatic ways (not so fast) shows automaticly most important feats on heat_map table<jupyter_code>m = DecisionTreeRegressor(max_depth=5)
m.fit(X,y)

imp = PermutationImportance(m,random_state=0).fit(X,y)
eli5.show_weights(imp, feature_names=cat_feats)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11555555555555555, 'fraction_numerical': 0.021904761904761906, 'mean_word_length': 6.2105263157894735, 'pattern_counts': {'":': 0, '<': 46, '<?xml version=': 0, '>': 47, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '405707', 'license': 'MIT', 'n_tokens_mistral': 1216, 'n_tokens_neox': 1146, 'n_words': 266, 'path': 'day3_simple_model.ipynb', 'repo_name': 'miklaszewskigrzegorz/dw_matrix_car', 'size': '3162'}