<jupyter_start><jupyter_text>todas as promoçoes acontecem depois de 6/30, ou seja temos APENAS informaçoes de promoçoes que vao acontecer no periodo que temos q prever as vendas<jupyter_code>#adicionando coluna recommendedRetailPrice a order

RetailPrice = item[['itemID','recommendedRetailPrice']]
order = pd.merge(order, RetailPrice, on='itemID')

print(len(order[order.salesPrice == order.recommendedRetailPrice]))
print(len(order[order.salesPrice != order.recommendedRetailPrice]))<jupyter_output>8321
2173634<jupyter_text>grande parte das vendas n~ao utiliza preço recomendado pelo retail  **adicionando a feature: moda de cada produto(itemID)**<jupyter_code>def get_mode_sales_price(id):
  sales_id_certo = order[order['itemID'] == id]
  mode = sales_id_certo['salesPrice'].mode()

  #a moda pode ter mais de um valor, 
  #para torna-la um unico foi escolhido arbitrariamente a mediana
  return mode.median()
#adicionando a moda de cada produto no item
item['modeSalesPrice'] =item['itemID'].map(get_mode_sales_price )

#colocando valor da moda no order
modeSalesPrice = item[['itemID','modeSalesPrice']]
order = pd.merge(order, modeSalesPrice, on='itemID')

print(order['modeSalesPrice'].describe().round(1))
print(order['modeSalesPrice'].describe().round(1))
print(order['modeSalesPrice'].isna().sum() )
print(len(order[order['salesPrice'] == order['modeSalesPrice']]))
print(len(order[order['salesPrice'] != order['modeSalesPrice']]))<jupyter_output>1737973
443982<jupyter_text>80% das vendas seguem o valor da moda<jupyter_code>print(len(order[order['salesPrice'] < order['modeSalesPrice']]))<jupyter_output>255515<jupyter_text>11% das vendas tem valor abaixo da moda, poderiam ser considerados promoçoes (?);<jupyter_code>print(len(order[order['salesPrice'] > order['modeSalesPrice']]))<jupyter_output>188467<jupyter_text>8% restantes que tem valor acima da moda, o que acho bizarro  **adicionando a feature: diferença da moda com salesPrice**<jupyter_code>order['difModa'] = order['salesPrice'] - order['modeSalesPrice']

print(order['difModa'].describe().round(1))
print(order['difModa'].isna().sum() )<jupyter_output>count    2181955.0
mean          -0.2
std            9.3
min        -5141.5
25%            0.0
50%            0.0
75%            0.0
max          972.5
Name: difModa, dtype: float64
0
{'fraction_non_alphanumeric': 0.1331028522039758, 'fraction_numerical': 0.035868625756266204, 'mean_word_length': 6.325949367088608, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '508097', 'license': 'MIT', 'n_tokens_mistral': 889, 'n_tokens_neox': 819, 'n_words': 176, 'path': 'sasaki/01-featureModa.ipynb', 'repo_name': 'joaopedromattos/DMC2020', 'size': '2314'}