<jupyter_start><jupyter_text>CNN을 이용한 표지판 분류  -  Step1 : 도로교통공단 교통안전표지 일람표에서 표지판 이미지 추출  - 출처 : https://www.koroad.or.kr/kp_web/safeDataView.do?board_code=DTBBS_030&board_num=100162-  Step 2 : Image Augementation 진행  - 실제 표지판 분류에 활용될 영상을 고려햐여, 회전, 휘어짐 정도, 명도, 각도 등을 고려-  Step 3 : 모델링  - keras로 CNN 구현-  Step 4 : 검증  - 결과 눈으로 직접 확인  image augementation- 각 표지판 이미지를 1000장씩 augementation 진행- 영상에서 표지판이 인식되는 이미지의 모양을 고려하여, 이미지 왜곡, 회전등의 값을 설정<jupyter_code>import os
import glob
import numpy as np

path = './traffin_sign_png/'
full_names = os.listdir(path)
labels = sorted([each.split('.')[0] for each in full_names])
# example of brighting image augmentation
from tqdm.notebook import tqdm
from numpy import expand_dims
from keras.preprocessing.image import load_img
from keras.preprocessing.image import img_to_array
from keras.preprocessing.image import ImageDataGenerator

import cv2

os.mkdir('./traffic_image')

range_ = tqdm(labels)
for dir_num in range_:
    # 이미지 로드
    img = load_img("./traffin_sign_png/{}.png".format(dir_num))
    # Numpy array 로 변환
    data = img_to_array(img)
    # expand dimension to one sample
    samples = expand_dims(data, 0)
    # image data augmentation generator 생성
    datagen = ImageDataGenerator(
        brightness_range=[0.2, 2.0],
        zoom_range=[0.3, 1],
        rotation_range=20,
        height_shift_range=0.2,
        width_shift_range=0.2)
    # prepare iterator
    it = datagen.flow(samples, batch_size=1)
    os.mkdir('./traffic_image/{}'.format(dir_num))

    for i in range(1000):
        batch = it.next()
        image = batch[0].astype("uint8")
        # rgb 변환
        b, g, r = cv2.split(image)
        img_astro3_rgb = cv2.merge([r, g, b])
        cv2.imwrite("./traffic_image/{}/{}_{}.png".format(dir_num,
                                                          dir_num, i), img_astro3_rgb)<jupyter_output><empty_output><jupyter_text>X, Y 설정, Train, Test split<jupyter_code>from PIL import Image
import os
import glob
import numpy as np
from sklearn.model_selection import train_test_split

caltech_dir = "./traffic_image/"
categories = labels
nb_classes = len(labels)

image_w = 64
image_h = 64

X = []
y = []

for idx, cat in enumerate(categories):

    #   one-hot 돌리기.
    label = [0 for i in range(nb_classes)]
    label[idx] = 1

    image_dir = caltech_dir + "/" + str(cat)
    files = glob.glob(image_dir+"/*.png")
    print(cat, " 파일 길이 : ", len(files))
    # 이미지 파일을 64 x 64 로 줄이고, 벡터화 시켜 X에 저장, one-hot-encoding된 라벨도 저장
    for i, f in enumerate(files):
        img = Image.open(f)
        img = img.convert("RGB")
        img = img.resize((image_w, image_h))
        data = np.asarray(img)

        X.append(data)
        y.append(label)

X = np.array(X)
y = np.array(y)
X_train, X_test, y_train, y_test = train_test_split(X, y)
xy = (X_train, X_test, y_train, y_test)
X_train.shape
y_train_test = y_train.reshape(-1,1)
y_train_test.shape
y_train.shape<jupyter_output><empty_output><jupyter_text>npy파일로 저장- 행렬값 저장<jupyter_code>import pickle 
pickle.dump(xy, open("./model/multi_image_data.npy", 'wb'), protocol=4)
X_train, X_test, y_train, y_test = np.load('./model/multi_image_data.npy',allow_pickle=True)
X_train.shape
y_train.shape<jupyter_output><empty_output><jupyter_text>픽셀값 정규화- 픽셀 정보는 0~255 값만 가지므로 일반적으로 바로 255로 나누는 방식으로 정규화 진행<jupyter_code># 일반화
X_train = X_train.astype(float) / 255
X_test = X_test.astype(float) / 255<jupyter_output><empty_output><jupyter_text>모델링<jupyter_code>import os
import glob
import numpy as np
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D, Dense, Flatten, Dropout
from keras.callbacks import EarlyStopping, ModelCheckpoint
from keras.layers import BatchNormalization
import matplotlib.pyplot as plt
import keras.backend.tensorflow_backend as K

nb_classes = len(labels)

with K.tf_ops.device('/device:GPU:0'):
    model = Sequential()

    model.add(Conv2D(32, (3, 3), padding="same",
                     input_shape=X_train.shape[1:], activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Conv2D(64, (3, 3), padding="same", activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Conv2D(128, (3, 3), padding="same", activation='relu'))
    model.add(MaxPooling2D(pool_size=(2, 2)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(256, activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(nb_classes, activation='softmax'))

    # 학습을 돌리는 방법을 정의 : cost function을 설정하고, 어떻게 최적화 할건지 방법을 정하고
    model.compile(loss='categorical_crossentropy',
                  optimizer='adam', metrics=['accuracy'])
    model_dir = './model'

    if not os.path.exists(model_dir):
        os.mkdir(model_dir)

    model_path = model_dir + '/multi_img_classification.model'
    checkpoint = ModelCheckpoint(
        filepath=model_path, monitor='val_loss', verbose=1, save_best_only=True)
    early_stopping = EarlyStopping(monitor='val_loss', patience=6)
model.summary()
history = model.fit(X_train, y_train, batch_size=32, epochs=50, validation_split=0.2, callbacks=[checkpoint, early_stopping])<jupyter_output>Train on 40800 samples, validate on 10200 samples
Epoch 1/50
40800/40800 [==============================] - 229s 6ms/step - loss: 2.7341 - accuracy: 0.2342 - val_loss: 1.3379 - val_accuracy: 0.6121

Epoch 00001: val_loss improved from inf to 1.33788, saving model to ./model/multi_img_classification.model
Epoch 2/50
40800/40800 [==============================] - 201s 5ms/step - loss: 1.3139 - accuracy: 0.5773 - val_loss: 0.6551 - val_accuracy: 0.8278

Epoch 00002: val_loss improved from 1.33788 to 0.65508, saving model to ./model/multi_img_classification.model
Epoch 3/50
40800/40800 [==============================] - 183s 4ms/step - loss: 0.8574 - accuracy: 0.7219 - val_loss: 0.4560 - val_accuracy: 0.8602

Epoch 00003: val_loss improved from 0.65508 to 0.45595, saving model to ./model/multi_img_classification.model
Epoch 4/50
40800/40800 [==============================] - 176s 4ms/step - loss: 0.6784 - accuracy: 0.7828 - val_loss: 0.3402 - val_accuracy: 0.9067

Epoch 00004: val_loss imp[...]<jupyter_text>accuracy, loss 값 확인<jupyter_code>plot_target = ['loss', 'val_loss', 'accuracy', 'val_accuracy']
for each in plot_target:
    plt.plot(history.history[each], label=each)
    
plt.legend()
plt.show()
model.evaluate(X_test, y_test)<jupyter_output>17000/17000 [==============================] - 19s 1ms/step<jupyter_text>틀린애들 눈으로 확인<jupyter_code>from keras.models import load_model
model = load_model('model/multi_img_classification.model')
y_test[0]
import numpy as np

predicted_result = model.predict(X_test)
predicted_labels = np.argmax(predicted_result, axis=1)
predicted_labels[:10]<jupyter_output><empty_output><jupyter_text>- test 데이터에서 라벨 추출<jupyter_code>y_labels = []
for vector in y_test:
    for idx, i in enumerate(vector):
        if i != 0:
            y_labels.append(idx)
y_labels = np.array(y_labels)
y_labels<jupyter_output><empty_output><jupyter_text>- 예측한 라벨과 실제 라벨을 비교해 잘못 예측된 결과 추출<jupyter_code>wrong_result = []

for n in range(0, len(y_test)):
    if predicted_labels[n] != y_labels[n]:
        wrong_result.append(n)
        
len(wrong_result)<jupyter_output><empty_output><jupyter_text>- 잘못 예측된 결과를 랜덤하게 선택<jupyter_code>import random

samples = random.choices(population=wrong_result, k=4)<jupyter_output><empty_output><jupyter_text>직접 눈으로 확인<jupyter_code>label_to_str = ["+자형교차로","T자형교차로","Y자형교차로","ㅏ자형교차로","ㅓ자형교차로","우선도로","우합류도로","좌합류도로","회전형교차로","철길건널목","우로굽은도로","좌로굽은도로","우좌로이중굽은도로","좌우로이중굽은도로","2방향통행","오르막경사","내리막경사","도로폭이좁아짐","우측차로없어짐","좌측차로없어짐","우측방통행","양측방통행","중앙분리대시작","중앙분리대끝남","신호기","미끄러운도로","강변도로","노면고르지못함","과속방지턱","낙석도로","횡단보도","어린이보호","자전거","도로공사중","비행기","횡풍","터널","교량","야생동물보호","위험","상습정체구간","통행금지","자동차통행금지","화물자동차통행금지","승합자동차통행금지","이륜자동차및원동기장치자전거통행금지","자동차, 이륜자동차빛원동기장치자전거통행금지","경운기, 트렉터및 손수레통행금지","자전거통행금지","진입금지","직진금지","우회전금지","좌회전금지","유턴금지","앞지르기금지","정차,주차금지","주차금지","차중량제한","차높이제한","차폭제한","차간거리확보","최고속도제한","최저속도제한","서행","일시정지","양보","보행자보행금지","위험물적재차량 통행금지"]

plt.figure(figsize=(14,12))

for idx, n in enumerate(samples):
    plt.subplot(4, 2, idx+1)
    plt.imshow(X_test[n].reshape(64,64,3), cmap='Greys', interpolation='nearest')
    plt.title('Label : ' + label_to_str[y_labels[n]] + ',  Predict : ' + label_to_str[predicted_labels[n]])
    plt.axis('off')

plt.show()<jupyter_output><empty_output><jupyter_text>실제 촬영이미지로 테스트<jupyter_code>from keras.models import load_model
model = load_model('model/multi_img_classification.model')
from PIL import Image
# 이미지 파일을 64 x 64 로 줄이고, 벡터화 시켜 X에 저장
image_w = 64
image_h = 64
X = []
img = Image.open('test2.jpeg')
img = img.convert("RGB")
img_resized = img.resize((image_w, image_h))
data = np.asarray(img_resized)
X.append(data)
X = np.array(X)
X = X.astype(float) / 255
img
result = model.predict(X)
label_to_str[np.argmax(result, axis=1)[0]]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1564133941484036, 'fraction_numerical': 0.04194014907108688, 'mean_word_length': 4.608234560199626, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 47, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '620213', 'license': 'MIT', 'n_tokens_mistral': 4254, 'n_tokens_neox': 4617, 'n_words': 686, 'path': 'trafic_sign.ipynb', 'repo_name': 'HowardHowonYu/traffic-sign-recognition', 'size': '8989'}