<jupyter_start><jupyter_text>실험일지 01-2- 모델적용    - Logistic Regression- max_features = 140000 / grid : 0.568353(rank:1) / kaggle : 0.52893- max_features = 120000 / grid : 0.567046(rank:5) / kaggle : 0.52674- max_features = 100000 / grid : 0.567885(rank:3) / kaggle : 0.52899- skfold = 5, mf 120000이 1등 - 이를 토대로 skfold = 10으로 유지<jupyter_code>import warnings
warnings.filterwarnings(action='ignore')
def kaggle_format(df):
    df['label'][df['label'] == 'none'] = 0
    df['label'][df['label'] == 'offensive'] = 1
    df['label'][df['label'] == 'hate'] = 2
    return df<jupyter_output><empty_output><jupyter_text>1. 데이터 읽기<jupyter_code>df = pd.read_csv('total_20210121.csv')
df = df[['title', 'comments', 'hate']]
df.columns = ['title', 'comments', 'label']
# df['label'][df['label'] == 'none'] = 0 
# df['label'][df['label'] == 'offensive'] = 1
# df['label'][df['label'] == 'hate'] = 2
df.head()<jupyter_output><empty_output><jupyter_text>2. Logistic Regression 모델 적용- tfidf, lgs - GridSearchCV- 데이터가 독립적이고 동일한 분포를 가진 경우    - KFold, RepeatedKFold, LeaveOneOut(LOO), LeavePOutLeaveOneOut(LPO)- 동일한 분포가 아닌 경우    - StratifiedKFold, RepeatedStratifiedKFold, StratifiedShuffleSplit<jupyter_code>from sklearn.model_selection import KFold, StratifiedKFold, GridSearchCV, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
kf = KFold(n_splits=10)
skfold = StratifiedKFold(n_splits=10)
mf_grid = {
    'tfidf__max_features': [15000, 100000, 110000, 120000, 130000, 140000, 150000],
}
pipeline = Pipeline([('tfidf', TfidfVectorizer(min_df=0.0, analyzer = 'char', 
                                               ngram_range = (1,3), sublinear_tf=True)),
                     ('lgs', LogisticRegression(multi_class='multinomial', class_weight = 'balanced', 
                                                random_state=10))])

grid_f1 = GridSearchCV(pipeline, param_grid=mf_grid, n_jobs=5, cv=skfold, scoring='f1_macro')
grid_f1.fit(X, y)
# skfold n_split = 10 일때, 'tfidf__max_features': 140000 best
f1_score_df = pd.DataFrame(grid_f1.cv_results_)
f1_score_df[['params', 'rank_test_score', 'mean_test_score', 'std_test_score']]
# skfold n_split = 5 일때, 'tfidf__max_features': 120000 best<jupyter_output><empty_output><jupyter_text>3. dev로 평가- skfold = 5 일때, max_features = 120000 가장좋음- skfold = 10 일때, max_features = 140000 가장좋음- f1 score로 따로 확인해 봤을때는 140000 이 더 좋음- 결과에 의해서는 10개로 쪼갰을때가 조금더 정확함<jupyter_code>dev_df = pd.read_csv('./korean-hate-speech-master/labeled/dev.tsv', sep='\t')

dev_df.head()
dev_df = dev_df[['comments', 'hate']]
dev_df.columns = ['comments', 'label']
dev_df.tail()
vec_tf = TfidfVectorizer(min_df = 0.0, analyzer = 'char', sublinear_tf=True, 
    ngram_range = (1,3), max_features=140000)

vec_tf.fit(X)
X_vec = vec_tf.transform(X)
X_tf = vec_tf.transform(dev_df['comments'])
y_tf = dev_df['label']
print(X_tf.shape)
X_tf.toarray()
lgs = LogisticRegression(multi_class='multinomial', class_weight = 'balanced', 
                                                random_state=10)
lgs.fit(X_vec, y)
pred = lgs.predict(X_tf)
from sklearn.metrics import f1_score

f1_score(y_tf, pred, average='macro')
from sklearn.metrics import f1_score

f1_score(y_tf, pred, average='macro')<jupyter_output><empty_output><jupyter_text>4. test 검증<jupyter_code>test_df = pd.read_csv('./korean-hate-speech-master/test.no_label.tsv', sep='\t')

X_test = vec_tf.transform(test_df['comments'])
pred = lgs.predict(X_test)
test_df['label'] = pred
df = kaggle_format(test_df)
df.to_csv('140000.csv', index=False)<jupyter_output><empty_output><jupyter_text>실험 01-03- skfold n_split = 10 인 상태로- max_features = 140000 ~ 150000사이에 최적의 값 확인- max_features = 140000 / grid : 0.568353 / kaggle : 0.52893  1. 데이터 다시 읽기<jupyter_code>df = pd.read_csv('total_20210121.csv')
df = df[['title', 'comments', 'hate']]
df.columns = ['title', 'comments', 'label']

X, y = df['comments'], df['label']<jupyter_output><empty_output><jupyter_text>2. GridSearchCV 최적값 확인- 1000단위로 확인- 그다음 100단위로 확인<jupyter_code>from sklearn.model_selection import KFold, StratifiedKFold, GridSearchCV, cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import Pipeline
# 적용
skfold = StratifiedKFold(n_splits=10)

mf_grid = {
    'tfidf__max_features': list(range(140000, 150001, 1000)),
}
pipeline = Pipeline([('tfidf', TfidfVectorizer(min_df=0.0, analyzer = 'char', 
                                               ngram_range = (1,3), sublinear_tf=True)),
                     ('lgs', LogisticRegression(multi_class='multinomial', class_weight = 'balanced', 
                                                random_state=10))])

grid_f1 = GridSearchCV(pipeline, param_grid=mf_grid, n_jobs=5, cv=skfold, scoring='f1_macro')
grid_f1.fit(X, y)
# 확인
f1_score_df = pd.DataFrame(grid_f1.cv_results_)
f1_score_df[['params', 'rank_test_score', 'mean_test_score', 'std_test_score']]
# 이 결과는 포기하고,, ngram 변화로 확인해야 할듯<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1457579412947326, 'fraction_numerical': 0.059308403699236026, 'mean_word_length': 4.552455357142857, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '104693', 'license': 'MIT', 'n_tokens_mistral': 2255, 'n_tokens_neox': 2138, 'n_words': 355, 'path': 'code/gijoong/02_tfidf_lg_try02.ipynb', 'repo_name': 'kkobooc/NLP_KoreanHateSpeech', 'size': '4974'}