<jupyter_start><jupyter_text>用 KNN 的方式做推薦 台新分析開發環境案客戶資料https://drive.google.com/drive/u/2/folders/1AAXWPcB9xiDe_qX87ce5XL-utkASWxRe請先連到這個分享資料夾，並且按下右鍵，加入你的 Google 雲端硬碟<jupyter_code># 這個是授權存取你的 Google 雲端硬碟
from google.colab import drive
drive.mount('/content/drive')
# 確認真的連上了資料
!ls drive/My\ Drive/台新分析開發環境案客戶資料
# 正式開始
import pandas as pd
import os.path
import numpy as np
DATA_DIR='drive/My Drive/台新分析開發環境案客戶資料'
sr_1_fn = os.path.join(DATA_DIR, 'Copy of sr_1.csv')
with open(sr_1_fn, encoding='cp950', errors='replace') as f:
    df = pd.read_csv(f, nrows=1000000, usecols=['CUST_NO', 'TXN_DESC'])
df.head()
df.TXN_DESC.nunique()
TXN = df.TXN_DESC.str.extract(r'([^\s０－９（５１２）]+)')
df['TXN_DESC'] = TXN
df.TXN_DESC.nunique()
# 看一下使用頻率
txn_counts = df.TXN_DESC.value_counts()
txn_counts.quantile(np.arange(.5,1,.01))
# 只留下交易一定次數以上的商店
good_txn = txn_counts[txn_counts>50]
#df = df.loc[df.TXN_DESC.isin(good_txn.index)]
df.drop(df.loc[~df.TXN_DESC.isin(good_txn.index)].index, inplace=True)
df.TXN_DESC.nunique()
# 看看用戶交易次數
cust_counts = df.CUST_NO.value_counts()
cust_counts.quantile(np.arange(0,1,.01))
good_cust = cust_counts[cust_counts>15]
#df = df.loc[df.CUST_NO.isin(good_cust.index)]
df.drop(df.loc[~df.CUST_NO.isin(good_cust.index)].index, inplace=True)
print(df.TXN_DESC.nunique())
print(df.CUST_NO.nunique())
# pivot dataframe, 變成我們要的矩陣
df = df.pivot_table(index='TXN_DESC', columns='CUST_NO',  aggfunc=len, fill_value=0 )
df
# 用來方便印出使用者資料
def print_txn(n):
    print(df.iloc[n].name)
for i in range(5):
  print_txn(i)
  print()
# sklearn NN
from sklearn.neighbors import NearestNeighbors
model_knn = NearestNeighbors(metric='correlation', algorithm='brute')
model_knn.fit(df)
def find_related(name):
    distances, indices = model_knn.kneighbors(df.loc[[name]], n_neighbors=6)
    for i, d in zip(indices[0][1:], distances[0][1:]):
        print(df.iloc[i].name, d)
find_related('台中新光影城')
find_related('友邦人壽')
find_related('AGODA')
find_related('一卡通自動加值')
find_related('中友百貨公司')<jupyter_output>中友百貨股份有限公司 0.7391644363048884
中友百貨宮廷餐廳 0.7592232470301343
Ｈ＆Ｍ台中中友店 0.8487060299229123
兆豐產物保險股份有限公司 0.8538372870809443
台灣優衣庫有限公司台中崇德路 0.9067876844284717
{'fraction_non_alphanumeric': 0.15281003251277286, 'fraction_numerical': 0.05945192754296331, 'mean_word_length': 6.977777777777778, 'pattern_counts': {'":': 0, '<': 4, '<?xml version=': 0, '>': 6, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '147510', 'license': 'Apache-2.0', 'n_tokens_mistral': 1230, 'n_tokens_neox': 1229, 'n_words': 135, 'path': 'CF/knn_txn.ipynb', 'repo_name': 'tjwei/taishin20190827', 'size': '2153'}