<jupyter_start><jupyter_text>6조

- 팀명: SYH : super youth hero
- 주제: 기계 시설물 분야 AI 학습용 데이터를 활용하여 기계 시설물의 고장 여부 예측 모델 개발
- 여수진,성은지,홍승표,문지수
- [참고](https://www.aidatahackathon.com/)

 절차
1. 문제 분석  
2. 데이터 정제
3. 모델 생성
4. 모델 평가 - k겹 교차 검증이나 등등..
5. 모델 튜닝 - 그리드 탐색이나 랜덤 탐색, 앙상블 등
6. 최상의 모델과 오차 분석
7. 테스트 세트로 평가하기<jupyter_code># libarary import
import os
import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow import keras
import sklearn as skl

# seed setting
np.random.seed(42)
tf.random.set_seed(42)

BASE_DIR = "/content/drive/MyDrive/"
DATA_DIR = '/content/drive/MyDrive/Colab Notebooks/daejeon_data'
from google.colab import drive
drive.mount('/content/drive')<jupyter_output>Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).<jupyter_text>데이터 형태
csv 파일  
,(콤마)로 분류된 시계열 데이터  
+ current Header  
  - 파일명, 에러 아이디, RPM, 정격전류,
  - 사용전류, Sample Rate 값, R전류의 RMS (RRMS),
  - R 전류의 MEAN (RMEAN), R 전류의 MAX (RMAX),
  - S 전류의 RMS (SRMS), S 전류의 MEAN (SMEAN),
  - S 전류의 MAX (SMAX), T 전류의 RMS (TRMS),
  - T 전류의 MEAN (TMEAN), T 전류의 MAX (TMAX)
+ kimm Header
  - 파일명, Sample Rate 값, Label, Label_No
+ vibration Header
  - 파일명, Sample Rate, Sample Rate, Sample Rate

 데이터 구조

op-csv : 모터 정상 가동 중 측정한 데이터  
no-csv : 모터가 해당 고장 레이블인 상태에 측정한 데이터

current(전류 데이터) - __11, 15, 18.5, 2.2, 3.7, 5.5, 7.5__ (단위: kw)  
kimm(유선진동 데이터) - __11, 15, 18.5, 2.2, 3.7, 5.5, 7.5__ (단위: kw)  
vibration(무선진동 데이터) - __11, 15, 18.5, 2.2, 3.7, 5.5, 7.5__ (단위: kw)  

|레이블|
|---|
|정상|
|회전체 불평형(unbalance)|
|베어링 불량(bearing)|
|벨트 느슨함(belt)|
|축정렬 불량(misalignment)|


전류의 경우 \[r,s,t전류 값] target(y)는 \[레이블종류] 특성 \[kw]  
유선진동의 경우 \[] target(y)는 \[레이블종류] 특성 \[kw]  
무선진동의 경우 \[] target(y)는 \[레이블종류] 특성 \[kw]  

target을 원-핫-벡터로
  1. 문제 분석

지도 학습 문제, 회귀 문제,   
다중 분류 ( 소프트맥스 회귀? ) (예측 결과는 레이블 하나에 해당하도록 나와야 함)  
다층 퍼셉트론 이용(예정)
시계열 데이터 -> RNN ?
  2. 데이터 정제 및 분석<jupyter_code># !unzip -uq "/content/drive/MyDrive/경진대회_data.zip" -d "압축 푼 파일 저장할 경로"<jupyter_output><empty_output><jupyter_text>,로 구분된 csv 파일 형태로 데이터 가져오기<jupyter_code>%cd '/content/drive/MyDrive'
os.getcwd()
import pickle
classSet = ("bearing", "belt", "misalignment", "unbalance")

def DatadirLoad(dirname):
    csv_op_dict = {classSet[0]:[], classSet[1]:[], classSet[2]:[],classSet[3]:[]}
    csv_no_dict = {classSet[0]:[], classSet[1]:[], classSet[2]:[],classSet[3]:[]}

    Save_op_Name, Save_no_Name = 'Data_op_dir.pickle','Data_no_dir.pickle'
    Save_op_path = os.path.join(BASE_DIR,Save_op_Name)
    Save_no_path = os.path.join(BASE_DIR,Save_no_Name)

    if os.path.isdir(Save_op_path):
        with open(Save_op_path, 'rb') as fr:
            csv_op_dict = picle.load(fr)
    if os.path.isdir(Save_no_path):
        with open(Save_no_path, 'rb') as fr:
            csv_no_dict = picle.load(fr)
        return csv_op_dict,csv_no_dict

    dirpath = os.path.join(DATA_DIR,dirname)
    if os.path.isdir(dirpath):
        for d in os.listdir(dirpath):
            d_path = dirpath+'/'+d
            if os.path.isdir(d_path):
                Error_Class_Lable,kw_Name,*_ = d.split('_')
                print(Error_Class_Lable,kw_Name)
                data_dir_list = [ x for x in os.listdir(d_path) if len(x)<2]
                data_dir_list.sort()
                #print(data_dir_list)
                if len(data_dir_list) > 0:
                    for d2 in data_dir_list:
                        d2_path = os.path.join(d_path,d2)

                        # op-csv file # 정상 작동
                        d2_op_path = os.path.join(d2_path,'4.op-csv')
                        d2_op_data = [x for x in os.listdir(d2_op_path) if x.endswith('.csv')]
                        op_csv_path = os.path.join(d2_op_path,d2_op_data[0])
                        csv_op_dict[Error_Class_Lable].append(op_csv_path)
                        with open(Save_op_path,'wb') as fw:
                            pickle.dump(csv_op_dict,fw)
                        # no-csv file # 고장 발생
                        d2_no_path = os.path.join(d2_path,'5.no-csv')
                        d2_no_data = [x for x in os.listdir(d2_no_path) if x.endswith('.csv')]
                        no_csv_path = os.path.join(d2_no_path,d2_no_data[0])
                        csv_no_dict[Error_Class_Lable].append(no_csv_path)
                        with open(Save_no_path,'wb') as fw:
                            pickle.dump(csv_no_dict,fw)

    return csv_op_dict, csv_no_dict
class_name = {'bearing':0, 'belt':1, 'misalignment':2, 'normal':3, 'unbalance':4}
def read_csv_dir(path, max_row_size):
    t_pd = None
    update_row_size = max_row_size
    y = None
    if os.path.isfile(path):
        data_Name = os.path.basename(path)
        dir_name,Ekw,dir_num,opno  = os.path.dirname(path).split('/')[6:]
        Error_Class_Lable,kw_Name,*_ = Ekw.split('_')      
        opno = 0 if opno[0] == '4' else 1
        # 정상 : False(0), 고장 : True(1)

        if dir_name[0] == 'c': #current(전류)
            t_pd = pd.read_csv(path, header=None,encoding='utf-8-sig',skiprows=range(15))
            t_pd.drop(3, axis=1, inplace=True)
            t_pd['Motor'] = float(kw_Name[:-2])
            # t_pd['target'] = Error_Class_Lable if opno == 1 else 'normal'
            t_pd['target'] = class_name[Error_Class_Lable] if opno == 1 else class_name['normal']
            y = t_pd['target']
            t_pd.drop(['target'], axis=1, inplace=True)

        elif dir_name[0] == 'k': # kimm(유선진동)
            t_pd = pd.read_csv(path, header=None,encoding='utf-8-sig',skiprows=range(4))
            t_pd.drop(1, axis=1, inplace=True)
            t_pd['Motor'] = float(kw_Name[:-2])
            t_pd['target'] = class_name[Error_Class_Lable] if opno == 1 else class_name['normal']
            y = t_pd['target']
            t_pd.drop(['target'], axis=1, inplace=True)

        elif dir_name[0] == 'v': #vibration(무선진동)
            t_pd = pd.read_csv(path, header=None,encoding='utf-8-sig',skiprows=range(4))
            t_pd.drop(1, axis=1, inplace=True)
            t_pd['Motor'] = float(kw_Name[:-2])
            t_pd['target'] = class_name[Error_Class_Lable] if opno == 1 else class_name['normal']
            y = t_pd['target']
            t_pd.drop(['target'], axis=1, inplace=True)
            if (t_pd.values.shape[0]) > max_row_size:
                update_row_size = t_pd.values.shape[0]
            elif (t_pd.values.shape[0]) < max_row_size:
                update_row_size = 6000
                tmp = np.zeros((int(update_row_size - t_pd.values.shape[0]),int(t_pd.values.shape[1])))
                y_tmp = np.zeros(y.values.shape)
                new_t = np.vstack([t_pd.values,tmp])
                new_y = np.append(y,y_tmp,axis=0)
                return new_t.tolist(), update_row_size, new_y.tolist()

        else: 
            print('error')
            print('dir_category : {}, lable : {}, op: {}'.format(dir_name, Error_Class_Lable,opno))
            return 0
    return t_pd.values.tolist(), update_row_size, y.values.tolist()
    
# dataset 생성
def makeDataset(op_dict, no_dict, category_num, shape_op = 0):
    """
    input: 정상data의 path list, 오류 data의 path list
    output: numpy의 ndarray 타입 형태로 dataset 반환
    category_num:
    0은 current 데이터
    1은 유선 진동 데이터 
    2는 무선 진동 데이터 
    shape_op :
     0이면 (38200, 60 , 5(특성 수)) 형태, 
     1이면 (38200(data개수 * row/60한 값), 5(특성 수), 60)
    """
    tmp = []
    max_row_size = 0
    y_tmp = []
    Y = None

    category_name = None
    if category_num == 0:
        category_name = 'current'
    elif category_num == 1:
        category_name = 'kimm'
    else:category_name = 'vibration'

    save_x_Name, save_y_Name = category_name+'_x.pickle',category_name+'_y.pickle'
    save_x_path = os.path.join(BASE_DIR,save_x_Name)
    save_y_path = os.path.join(BASE_DIR,save_y_Name)

    # if os.path.isdir(load_x_path):
    #     with open(load_x_path, 'rb') as fr:
    #         X = picle.load(fr)
    # if os.path.isdir(load_y_path):
    #     with open(load_y_path, 'rb') as fr:
    #         y = picle.load(fr)
    #     return X,y 

    for k,v in op_dict.items():
        for p in v:
            data, max, y = read_csv_dir(p, max_row_size)
            tmp.append(data)
            y_tmp.append(y)
            max_row_size = max
    for k,v in no_dict.items():
        for p in v:
            data, max, y = read_csv_dir(p, max_row_size)
            tmp.append(data)
            y_tmp.append(y)
            max_row_size = max
    
    if shape_op == 0:
        dataset = np.stack(tmp)
        feature_num = 4 if category_num == 0 else 2
        Y = np.stack(y_tmp)
        shape = (-1,60,feature_num) 
        y_shape = (-1,60)

    else:
        dataset = np.stack(tmp,axis=2)
        feature_num = 4 if category_num == 0 else 2
        dataset = dataset.reshape((-1,feature_num,60)).copy()
        Y = np.stack(y_tmp)
        Y = Y.reshape((-1,60)).copy()
        shape = (-1,feature_num,60)
        y_shape = (-1,60)
    with open(save_x_path,'wb') as fw:
        pickle.dump(dataset.reshape(shape),fw)
    with open(save_y_path,'wb') as fw:
        pickle.dump(Y.reshape(y_shape),fw)

    return  dataset.reshape(shape), Y.reshape(y_shape)
def load_dataset(category_num=0):
    """
    0: current dataset
    1: kimm dataset
    2: vibration dataset
    """
    category_name = None
    if category_num == 0:
        category_name = 'current'
    elif category_num == 1:
        category_name = 'kimm'
    else:category_name = 'vibration'

    load_x_Name, load_y_Name = category_name+'_x.pickle',category_name+'_y.pickle'
    load_x_path = os.path.join(BASE_DIR,load_x_Name) #ex. /content/drive/MyDrive/current_x.pickle
    load_y_path = os.path.join(BASE_DIR,load_y_Name) 

    if os.path.isfile(load_x_path):
        print('exist')
        with open(load_x_path, 'rb') as fr:
            X = pickle.load(fr)
        if os.path.isfile(load_y_path):
            print('exist')
            with open(load_y_path, 'rb') as fr:
                y = pickle.load(fr)
            return X,y
    else:
        print('not exist')
        if category_num == 0:
            current_op_csv,current_no_csv = DatadirLoad("current(전류)")
            current_x , current_y = makeDataset(current_op_csv,current_no_csv,0,shape_op=0)
            return current_x,current_y
        elif category_num == 1:
            kimm_op_csv, kimm_no_csv = DatadirLoad("kimm(유선진동)")
            kimm_x, kimm_y = makeDataset(kimm_op_csv,kimm_no_csv, 1)
            return kimm_x,kimm_y
        else:
            vibration_op_csv, vibration_no_csv = DatadirLoad("vibration(무선진동)")
            vibration_x, vibration_y = makeDataset(vibration_op_csv, vibration_no_csv,2)
            return vibration_x, vibration_y

# current_op_csv,current_no_csv = DatadirLoad("current(전류)")
# kimm_op_csv, kimm_no_csv = DatadirLoad("kimm(유선진동)")
# vibration_op_csv, vibration_no_csv = DatadirLoad("vibration(무선진동)")
current_x,current_y = load_dataset(0) # current
kimm_x,kimm_y = load_dataset(1) # kimm
vibration_x, vibration_y = load_dataset(2) # vibr
pred_name = ['bearing', 'belt', 'misalignment','normal', 'unbalance']
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>전류 데이터<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(current_x,current_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)
X_train.shape, y_train.shape<jupyter_output><empty_output><jupyter_text>일반적인 다층 퍼셉트론(MLP)<jupyter_code>model = keras.models.Sequential()
model.add(keras.layers.Flatten(input_shape=[60, 4]))
model.add(keras.layers.Dense(300, activation="relu"))
model.add(keras.layers.Dense(100, activation="relu"))
model.add(keras.layers.Dense(5, activation="softmax")) 
model.summary()
model.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
history = model.fit(X_train,y_train,epochs=30,validation_data=(X_valid,y_valid))
X_new = X_train[:5]
y_proba = model.predict(X_new)
y_proba.round(2)
y_pred = model.predict_classes(X_new)
y_pred
y_train[:5]<jupyter_output><empty_output><jupyter_text>CNN<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(current_x,current_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)
X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
X_train.shape
X_train[0]
# 각 특성 값의 범위를 0~1로 조정
X_train[:,:,3:] = X_train[:,:,3:]/100
X_valid[:,:,3:] = X_valid[:,:,3:]/100
X_test[:,:,3:] = X_test[:,:,3:]/100
from functools import partial

cnn_model = keras.models.Sequential([
    keras.layers.Conv2D(64,7,activation="relu",padding="same",input_shape=(60,4,1)),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(2),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(2),
    keras.layers.Flatten(),
    keras.layers.Dense(128,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(64,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(units=5, activation='softmax')
])
cnn_model.summary()
cnn_model.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
history = cnn_model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
717/717 [==============================] - 15s 9ms/step - loss: 0.7528 - accuracy: 0.6626 - val_loss: 0.0140 - val_accuracy: 0.9971
Epoch 2/10
717/717 [==============================] - 6s 9ms/step - loss: 3.3331 - accuracy: 0.9797 - val_loss: 0.0103 - val_accuracy: 0.9971
Epoch 3/10
717/717 [==============================] - 6s 9ms/step - loss: 0.0288 - accuracy: 0.9921 - val_loss: 0.0086 - val_accuracy: 0.9971
Epoch 4/10
717/717 [==============================] - 6s 9ms/step - loss: 0.1789 - accuracy: 0.9927 - val_loss: 0.0116 - val_accuracy: 0.9971
Epoch 5/10
717/717 [==============================] - 6s 9ms/step - loss: 0.0637 - accuracy: 0.9943 - val_loss: 0.0086 - val_accuracy: 0.9971
Epoch 6/10
717/717 [==============================] - 6s 9ms/step - loss: 0.0378 - accuracy: 0.9919 - val_loss: 0.0094 - val_accuracy: 0.9971
Epoch 7/10
717/717 [==============================] - 6s 9ms/step - loss: 0.0107 - accuracy: 0.9969 - val_loss: 0.0086 - val_accuracy: 0.9971
Epoch[...]<jupyter_text>이전 모델보다 훈련, 검증 세트 모두 높은 정확도를 보임<jupyter_code># score = model.evaluate(X_test, y_test)
X_new = X_train[:10] # pretend we have new images
y_pred = cnn_model.predict(X_new)
y_pred = cnn_model.predict_classes(X_new)
y_pred
y_train[:10]<jupyter_output><empty_output><jupyter_text>유선진동<jupyter_code>pred_name = ['bearing', 'belt', 'misalignment','normal', 'unbalance']
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(kimm_x,kimm_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)<jupyter_output><empty_output><jupyter_text>일반적인 다층 퍼셉트론(MLP)<jupyter_code>model2 = keras.models.Sequential()
model2.add(keras.layers.Flatten(input_shape=[60, 2]))
model2.add(keras.layers.Dense(300, activation="relu"))
model2.add(keras.layers.Dense(100, activation="relu"))
model2.add(keras.layers.Dense(5, activation="softmax")) 
model2.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
history2 = model2.fit(X_train,y_train,epochs=30,validation_data=(X_valid,y_valid))<jupyter_output>Epoch 1/30
1440/1440 [==============================] - 3s 2ms/step - loss: 2.1606 - accuracy: 0.4907 - val_loss: 1.3745 - val_accuracy: 0.5014
Epoch 2/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3761 - accuracy: 0.4982 - val_loss: 1.3709 - val_accuracy: 0.5014
Epoch 3/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3733 - accuracy: 0.4999 - val_loss: 1.3721 - val_accuracy: 0.5014
Epoch 4/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3722 - accuracy: 0.4998 - val_loss: 1.3707 - val_accuracy: 0.5014
Epoch 5/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3702 - accuracy: 0.5013 - val_loss: 1.3697 - val_accuracy: 0.5014
Epoch 6/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3708 - accuracy: 0.5009 - val_loss: 1.3711 - val_accuracy: 0.5014
Epoch 7/30
1440/1440 [==============================] - 3s 2ms/step - loss: 1.3680 - accuracy: 0.5015 - val_loss: 1.3692 - val_accuracy:[...]<jupyter_text>CNN<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(kimm_x,kimm_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)
X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
X_train.shape
# 각 특성 값의 범위를 0~1로 조정
X_train[:,:,1:] = X_train[:,:,1:]/100
X_valid[:,:,1:] = X_valid[:,:,1:]/100
X_test[:,:,1:] = X_test[:,:,1:]/100
from functools import partial

cnn_model2 = keras.models.Sequential([
    keras.layers.Conv2D(64,7,activation="relu",padding="same",input_shape=(60,2,1)),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(2),
    keras.layers.Flatten(),
    keras.layers.Dense(128,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(64,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(units=5, activation='softmax')
])
cnn_model2.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
history = cnn_model2.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
1440/1440 [==============================] - 23s 10ms/step - loss: 1.4028 - accuracy: 0.4969 - val_loss: 1.3660 - val_accuracy: 0.5014
Epoch 2/10
1440/1440 [==============================] - 15s 10ms/step - loss: 1.3725 - accuracy: 0.4983 - val_loss: 1.3935 - val_accuracy: 0.5014
Epoch 3/10
1440/1440 [==============================] - 15s 10ms/step - loss: 1.3710 - accuracy: 0.5003 - val_loss: 1.3687 - val_accuracy: 0.5014
Epoch 4/10
1440/1440 [==============================] - 15s 10ms/step - loss: 1.3672 - accuracy: 0.4998 - val_loss: 1.3724 - val_accuracy: 0.5014
Epoch 5/10
1440/1440 [==============================] - 15s 10ms/step - loss: 1.3581 - accuracy: 0.5014 - val_loss: 1.3605 - val_accuracy: 0.5014
Epoch 6/10
1440/1440 [==============================] - 15s 11ms/step - loss: 1.3581 - accuracy: 0.5009 - val_loss: 1.3572 - val_accuracy: 0.5014
Epoch 7/10
1440/1440 [==============================] - 15s 11ms/step - loss: 1.3518 - accuracy: 0.5016 - val_loss: 1.3506 -[...]<jupyter_text>전류 데이터와 달리 유선 진동 데이터는 CNN 을 적용해도 훈련,검증 세트의 정확도가 50%를 웃도는 결과를 나타냄<jupyter_code>X_new = X_train[:10]
y_pred = cnn_model2.predict_classes(X_new)
y_pred
y_train[:10]<jupyter_output><empty_output><jupyter_text>CNN + 배치정규화<jupyter_code>cnn_model2_1 = keras.models.Sequential([
    keras.layers.Conv2D(64,7,padding="same",input_shape=(60,2,1),use_bias=False),
    keras.layers.BatchNormalization(),
    keras.layers.Activation("relu"),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(128,3,padding="same",use_bias=False),
    keras.layers.BatchNormalization(),
    keras.layers.Activation("relu"),
    keras.layers.Conv2D(128,3,padding="same",use_bias=False),
    keras.layers.BatchNormalization(),
    keras.layers.Activation("relu"),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(256,3,padding="same",use_bias=False),
    keras.layers.BatchNormalization(),
    keras.layers.Activation("relu"),
    keras.layers.Conv2D(256,3,padding="same",use_bias=False),
    keras.layers.BatchNormalization(),
    keras.layers.Activation("relu"),
    keras.layers.MaxPool2D(2),
    keras.layers.Flatten(),
    keras.layers.Dense(128,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(64,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(units=5, activation='softmax')
])
cnn_model2_1.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
cnn_model2_1.fit(X_train,y_train,epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
1440/1440 [==============================] - 18s 11ms/step - loss: 1.5930 - accuracy: 0.4689 - val_loss: 1.3677 - val_accuracy: 0.5014
Epoch 2/10
1440/1440 [==============================] - 17s 12ms/step - loss: 1.3811 - accuracy: 0.4980 - val_loss: 1.3622 - val_accuracy: 0.5014
Epoch 3/10
1440/1440 [==============================] - 16s 11ms/step - loss: 1.3850 - accuracy: 0.5001 - val_loss: 1.3632 - val_accuracy: 0.5014
Epoch 4/10
1440/1440 [==============================] - 17s 11ms/step - loss: 1.3733 - accuracy: 0.4997 - val_loss: 1.3611 - val_accuracy: 0.5014
Epoch 5/10
1440/1440 [==============================] - 17s 11ms/step - loss: 1.3794 - accuracy: 0.5014 - val_loss: 1.3701 - val_accuracy: 0.5014
Epoch 6/10
1440/1440 [==============================] - 17s 12ms/step - loss: 1.3715 - accuracy: 0.5009 - val_loss: 1.3654 - val_accuracy: 0.5014
Epoch 7/10
1440/1440 [==============================] - 17s 12ms/step - loss: 1.3682 - accuracy: 0.5014 - val_loss: 1.3583 -[...]<jupyter_text>기본 CNN에 배치정규화를 추가해도 이전보다 좋은 성능이 나오지 않음, 오히려 기존보다 미미하게 나쁜 결과를 나타냄  ResNet-34<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(kimm_x,kimm_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)
X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
X_train.shape
# 각 특성 값의 범위를 0~1로 조정
X_train[:,:,1:] = X_train[:,:,1:]/100
X_valid[:,:,1:] = X_valid[:,:,1:]/100
X_test[:,:,1:] = X_test[:,:,1:]/100
# 잔차유닛
class ResidualUnit(keras.layers.Layer):
    def __init__(self, filters, strides=1, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.activation = keras.activations.get(activation)
        self.main_layers = [
            keras.layers.Conv2D(filters,3,strides=strides, padding="same", use_bias=False),
            keras.layers.BatchNormalization(),
            self.activation,
            keras.layers.Conv2D(filters, 3,strides=1,padding="same",use_bias=False),
            keras.layers.BatchNormalization()]
        self.skip_layers=[]
        if strides > 1:
            self.skip_layers = [
                keras.layers.Conv2D(filters,1,strides=strides,padding="same",use_bias=False),
                keras.layers.BatchNormalization()]
    def call(self,inputs):
        Z = inputs
        for layer in self.main_layers:
            Z = layer(Z)
        skip_Z = inputs
        for layer in self.skip_layers:
            skip_Z = layer(skip_Z)
        return self.activation(Z+skip_Z)
kimm_resnet_model = keras.models.Sequential()
kimm_resnet_model.add(keras.layers.Conv2D(64,7,strides=2,input_shape=[60,2,1],padding="same",use_bias=False))
kimm_resnet_model.add(keras.layers.BatchNormalization())
kimm_resnet_model.add(keras.layers.Activation("relu"))
kimm_resnet_model.add(keras.layers.MaxPool2D(pool_size=3, strides=2,padding="same"))
prev_filters = 64
for filters in [64]*3+[128]*4+[256]*6+[512]*3:
    strides = 1 if filters == prev_filters else 2
    kimm_resnet_model.add(ResidualUnit(filters,strides=strides))
    prev_filters=filters
kimm_resnet_model.add(keras.layers.GlobalAvgPool2D())
kimm_resnet_model.add(keras.layers.Flatten())
kimm_resnet_model.add(keras.layers.Dense(5,activation="softmax"))
kimm_resnet_model.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
kimm_resnet_model.fit(X_train,y_train,epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
1440/1440 [==============================] - 79s 50ms/step - loss: 1.3172 - accuracy: 0.5261 - val_loss: 1.5119 - val_accuracy: 0.5590
Epoch 2/10
1440/1440 [==============================] - 71s 49ms/step - loss: 0.8087 - accuracy: 0.6680 - val_loss: 1.1126 - val_accuracy: 0.6762
Epoch 3/10
1440/1440 [==============================] - 70s 49ms/step - loss: 0.6505 - accuracy: 0.7254 - val_loss: 1.8109 - val_accuracy: 0.6389
Epoch 4/10
1440/1440 [==============================] - 71s 49ms/step - loss: 0.5762 - accuracy: 0.7594 - val_loss: 1.7035 - val_accuracy: 0.5993
Epoch 5/10
1440/1440 [==============================] - 70s 49ms/step - loss: 0.5170 - accuracy: 0.7823 - val_loss: 2.1009 - val_accuracy: 0.5453
Epoch 6/10
1440/1440 [==============================] - 71s 49ms/step - loss: 0.4907 - accuracy: 0.7886 - val_loss: 1.7658 - val_accuracy: 0.6054
Epoch 7/10
1440/1440 [==============================] - 70s 49ms/step - loss: 0.4207 - accuracy: 0.8139 - val_loss: 1.9326 -[...]<jupyter_text>CNN 보다 훈련, 검증 세트 둘 다 정확도는 높아졌으나, __검증 세트의 정확도__가 __훈련세트의 정확도__보다 __낮게__ 나타남<jupyter_code><jupyter_output><empty_output><jupyter_text>무선진동<jupyter_code>vibration_x, vibration_y = makeDataset(vibration_op_csv, vibration_no_csv,1)
vibration_x.shape, vibration_y.shape
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(vibration_x,vibration_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)<jupyter_output><empty_output><jupyter_text>일반적인 다층 퍼셉트론(MLP)<jupyter_code>model3 = keras.models.Sequential()
model3.add(keras.layers.Flatten(input_shape=[60, 2]))
model3.add(keras.layers.Dense(300, activation="relu"))
model3.add(keras.layers.Dense(100, activation="relu"))
model3.add(keras.layers.Dense(5, activation="softmax")) 
model3.compile(loss="sparse_categorical_crossentropy",
              optimizer="sgd",
              metrics=["accuracy"])
history3 = model3.fit(X_train,y_train,epochs=30,validation_data=(X_valid,y_valid))<jupyter_output>Epoch 1/30
720/720 [==============================] - 2s 2ms/step - loss: 2.2386 - accuracy: 0.4752 - val_loss: 1.4057 - val_accuracy: 0.4931
Epoch 2/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3804 - accuracy: 0.4993 - val_loss: 1.3744 - val_accuracy: 0.4931
Epoch 3/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3727 - accuracy: 0.4973 - val_loss: 1.3731 - val_accuracy: 0.4931
Epoch 4/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3593 - accuracy: 0.5050 - val_loss: 1.3776 - val_accuracy: 0.4931
Epoch 5/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3770 - accuracy: 0.4916 - val_loss: 1.3714 - val_accuracy: 0.4931
Epoch 6/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3672 - accuracy: 0.4973 - val_loss: 1.3735 - val_accuracy: 0.4931
Epoch 7/30
720/720 [==============================] - 1s 2ms/step - loss: 1.3643 - accuracy: 0.5010 - val_loss: 1.3734 - val_accuracy: 0.4931
Epoch [...]<jupyter_text>CNN<jupyter_code>X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
X_train.shape
# 각 특성 값의 범위를 0~1로 조정
X_train[:,:,1:] = X_train[:,:,1:]/100
X_valid[:,:,1:] = X_valid[:,:,1:]/100
X_test[:,:,1:] = X_test[:,:,1:]/100
from functools import partial

cnn_model3 = keras.models.Sequential([
    keras.layers.Conv2D(64,7,activation="relu",padding="same",input_shape=(60,2,1)),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.Conv2D(128,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(1),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.Conv2D(256,3,activation="relu",padding="same"),
    keras.layers.MaxPool2D(2),
    keras.layers.Flatten(),
    keras.layers.Dense(128,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(64,activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(units=5, activation='softmax')
])
cnn_model3.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
history = cnn_model3.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
720/720 [==============================] - 9s 11ms/step - loss: 0.8908 - accuracy: 0.5913 - val_loss: 0.7197 - val_accuracy: 0.6658
Epoch 2/10
720/720 [==============================] - 8s 11ms/step - loss: 0.7547 - accuracy: 0.6448 - val_loss: 0.6401 - val_accuracy: 0.6612
Epoch 3/10
720/720 [==============================] - 8s 11ms/step - loss: 0.6536 - accuracy: 0.6729 - val_loss: 0.7226 - val_accuracy: 0.6277
Epoch 4/10
720/720 [==============================] - 8s 11ms/step - loss: 0.6964 - accuracy: 0.6664 - val_loss: 0.6289 - val_accuracy: 0.6767
Epoch 5/10
720/720 [==============================] - 8s 11ms/step - loss: 0.6844 - accuracy: 0.6731 - val_loss: 0.6231 - val_accuracy: 0.6760
Epoch 6/10
720/720 [==============================] - 8s 11ms/step - loss: 0.6196 - accuracy: 0.6862 - val_loss: 0.6346 - val_accuracy: 0.6741
Epoch 7/10
720/720 [==============================] - 8s 11ms/step - loss: 0.6722 - accuracy: 0.6626 - val_loss: 0.6402 - val_accuracy: 0.6759[...]<jupyter_text>Resnet34<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(vibration_x,vibration_y[:,0],train_size = 0.6, random_state=42)
X_test, X_valid, y_test, y_valid = train_test_split(X_test, y_test, test_size = 0.5, random_state=42)
X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
X_train[:,:,1:] = X_train[:,:,1:]/100
X_valid[:,:,1:] = X_valid[:,:,1:]/100
X_test[:,:,1:] = X_test[:,:,1:]/100
i# 잔차유닛
class ResidualUnit(keras.layers.Layer):
    def __init__(self, filters, strides=1, activation="relu", **kwargs):
        super().__init__(**kwargs)
        self.activation = keras.activations.get(activation)
        self.main_layers = [
            keras.layers.Conv2D(filters,3,strides=strides, padding="same", use_bias=False),
            keras.layers.BatchNormalization(),
            self.activation,
            keras.layers.Conv2D(filters, 3,strides=1,padding="same",use_bias=False),
            keras.layers.BatchNormalization()]
        self.skip_layers=[]
        if strides > 1:
            self.skip_layers = [
                keras.layers.Conv2D(filters,1,strides=strides,padding="same",use_bias=False),
                keras.layers.BatchNormalization()]
    def call(self,inputs):
        Z = inputs
        for layer in self.main_layers:
            Z = layer(Z)
        skip_Z = inputs
        for layer in self.skip_layers:
            skip_Z = layer(skip_Z)
        return self.activation(Z+skip_Z)
vibration_resnet_model = keras.models.Sequential()
vibration_resnet_model.add(keras.layers.Conv2D(64,7,strides=2,input_shape=[60,2,1],padding="same",use_bias=False))
vibration_resnet_model.add(keras.layers.BatchNormalization())
vibration_resnet_model.add(keras.layers.Activation("relu"))
vibration_resnet_model.add(keras.layers.MaxPool2D(pool_size=3, strides=2,padding="same"))
prev_filters = 64
for filters in [64]*3+[128]*4+[256]*6+[512]*3:
    strides = 1 if filters == prev_filters else 2
    vibration_resnet_model.add(ResidualUnit(filters,strides=strides))
    prev_filters=filters
vibration_resnet_model.add(keras.layers.GlobalAvgPool2D())
vibration_resnet_model.add(keras.layers.Flatten())
vibration_resnet_model.add(keras.layers.Dense(5,activation="softmax"))
vibration_resnet_model.compile(loss="sparse_categorical_crossentropy", optimizer="nadam", metrics=["accuracy"])
vibration_resnet_model.fit(X_train,y_train,epochs=10, validation_data=(X_valid, y_valid))<jupyter_output>Epoch 1/10
720/720 [==============================] - 42s 48ms/step - loss: 0.9188 - accuracy: 0.6118 - val_loss: 0.6542 - val_accuracy: 0.6827
Epoch 2/10
720/720 [==============================] - 34s 48ms/step - loss: 0.6833 - accuracy: 0.6611 - val_loss: 0.6438 - val_accuracy: 0.6842
Epoch 3/10
720/720 [==============================] - 35s 48ms/step - loss: 0.6477 - accuracy: 0.6728 - val_loss: 0.6062 - val_accuracy: 0.6914
Epoch 4/10
720/720 [==============================] - 35s 48ms/step - loss: 0.6372 - accuracy: 0.6807 - val_loss: 0.6239 - val_accuracy: 0.6852
Epoch 5/10
720/720 [==============================] - 35s 49ms/step - loss: 0.6384 - accuracy: 0.6780 - val_loss: 110.8870 - val_accuracy: 0.5863
Epoch 6/10
720/720 [==============================] - 35s 49ms/step - loss: 0.7073 - accuracy: 0.6556 - val_loss: 0.7104 - val_accuracy: 0.6279
Epoch 7/10
720/720 [==============================] - 35s 49ms/step - loss: 0.6300 - accuracy: 0.6801 - val_loss: 0.6071 - val_accurac[...]
{'fraction_non_alphanumeric': 0.19228481351496043, 'fraction_numerical': 0.07554577360061865, 'mean_word_length': 3.5362924986508366, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 69, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '363117', 'license': 'MIT', 'n_tokens_mistral': 15341, 'n_tokens_neox': 14109, 'n_words': 2244, 'path': 'project2/project2.ipynb', 'repo_name': 'eunnjji/MultiCampus-EmpAcademy', 'size': '33622'}