<jupyter_start><jupyter_text>try and Exception<jupyter_code>print("test")
import logging

try:
    1/0
except BaseException:
    logging.exception("An exception was thrown!")<jupyter_output>ERROR:root:An exception was thrown!
Traceback (most recent call last):
  File "<ipython-input-2-61a7b8231f1e>", line 4, in <module>
    1/0
ZeroDivisionError: division by zero<jupyter_text>에러  Error(s) in loading state_dict for DataParallel- https://www.programmersought.com/article/43823425009/- https://titanwolf.org/Network/Articles/Article?AID=e869e61c-1fcf-409d-91ed-5753fdbdfbdbgsc.tab=0RuntimeError: Error(s) in loading state_dict for DataParallel:	Missing key(s) in state_dict: "module.conv1.weight", "module.conv1.bias", "module.conv2.weight", "module.conv2.bias", "module.fc1.weight", "module.fc1.bias", "module.fc2.weight", "module.fc2.bias", "module.fc3.weight", "module.fc3.bias". 	Unexpected key(s) in state_dict: "conv1.weight", "conv1.bias", "conv2.weight", "conv2.bias", "fc1.weight", "fc1.bias", "fc2.weight", "fc2.bias", "fc3.weight", "fc3.bias".<jupyter_code>Net = get_model_network()
net = Net()
state_dict = net.state_dict()
for key, value in state_dict.items():
    print(key)<jupyter_output>conv1.weight
conv1.bias
conv2.weight
conv2.bias
fc1.weight
fc1.bias
fc2.weight
fc2.bias
fc3.weight
fc3.bias<jupyter_text>아래 코드- 아래는 저자가 워크샵을 작성하기 위해서 테스트로 작성한 코드이기에 무시 하십시오  데이터 변환 예시- 0.3444 같은 float 은 numpy.tobytes() 변환시에 4개의 byte 로 표현됨.- 보통 이미지는 0 ~ 255 의 Unsigned Integerr (uint8) 로서 1개의 byte로 표현됨.- 보통 이미지파일(에: A.jpg)에서 데이타를 읽으면 uint8 로 읽은 다음에 0~255 를 스케일링하여 float32 로 표현을 함.<jupyter_code>dataiter = iter(test_loader)
images, labels = dataiter.next()
image = images[0:1]
print("image content: ", image[0][0][0][0:5])
print("images.shape: ", image.shape)
print("type image: ", type(image))
image_arr = image.numpy()
print("image arr .shape: ", image_arr.shape)
print("type image arr: ", type(image_arr))
img_fla = image_arr.flatten()
print("img_fla type: ", type(img_fla))
print("img_fla shape: ", img_fla.shape)
print("img_fla content: ", np.array(img_fla))
img_fla_bytes = img_fla.tobytes()

print("img_fla length: ", len(img_fla_bytes))
input_data = img_fla_bytes
if isinstance(input_data, (np.ndarray)):
    np_array = input_data # 로컬에서 테스트시에 numpy.ndarray 로 제공 되는 경우
elif isinstance(input_data, (bytes)):
    np_array = np.frombuffer(input_data, dtype='uint8')
else: # 토치 서브를 통해서 numpy.ndarray가 제공되면, IOByte( 확인 필요) 로 제공되기에 디코딩이 필요함.
    np_array = decoder.decode(input_data, content_type)<jupyter_output>image content:  tensor([0.2392, 0.2471, 0.2941, 0.3020, 0.2549])
images.shape:  torch.Size([1, 3, 32, 32])
type image:  <class 'torch.Tensor'>
image arr .shape:  (1, 3, 32, 32)
type image arr:  <class 'numpy.ndarray'>
img_fla type:  <class 'numpy.ndarray'>
img_fla shape:  (3072,)
img_fla content:  [ 0.23921573  0.24705887  0.2941177  ... -0.02745098  0.01176476
 -0.1372549 ]
img_fla length:  12288<jupyter_text>Type 체크<jupyter_code>import numpy as np
print(type(image_arr))
if isinstance(image_arr, (bytearray, np.generic)):
    print("ndarray")
else:
    print("else")
import time
time.sleep(3)
print("hello")<jupyter_output>hello<jupyter_text>모델 로딩 에러아래는 훈련시 파이토치 버전과 노트북에서 사용한 파이토치 버전이 달라서 에러 발생함.```RuntimeError: version_ <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/serialize/inline_container.cc:132, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 2. Your PyTorch installation may be too old. (init at /opt/conda/conda-bld/pytorch_1579022034529/work/caffe2/serialize/inline_container.cc:132)frame 0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x47 (0x7f7250faa627 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/lib/libc10.so)frame 1: caffe2::serialize::PyTorchStreamReader::init() + 0x1f5b (0x7f725acf29ab in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/lib/libtorch.so)frame 2: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::string const&) + 0x64 (0x7f725acf3bc4 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/lib/libtorch.so)frame 3:  + 0x6d1326 (0x7f728873d326 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/lib/libtorch_python.so)frame 4:  + 0x28c076 (0x7f72882f8076 in /home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/lib/libtorch_python.so)frame 5: _PyCFunction_FastCallDict + 0x281 (0x5654c11d7d21 in /home/ec2-user/anaconda3/envs/pytorch_p36/bin/python)```  리파지토리 검색<jupyter_code>! aws ecr describe-repositories --repository-names pytorch-training --registry-id 763104351884
! aws ecr list-images --repository-name pytorch-training --registry-id 763104351884 --max-items 100 | grep 1.4<jupyter_output>"imageTag": "1.3.1-gpu-py36-cu101-ubuntu16.04-v2.1-2019-12-11-02-41-47"
            "imageDigest": "sha256:ee07b739ab5da8403c3fabc11c4fd6c7c105df68a81efaa919d8721e75e8d4b3",
            "imageTag": "1.4.0-cpu-py27-ubuntu16.04-v3.2"
            "imageDigest": "sha256:ee07b739ab5da8403c3fabc11c4fd6c7c105df68a81efaa919d8721e75e8d4b3",
            "imageTag": "1.4-cpu-py2"
            "imageDigest": "sha256:ee07b739ab5da8403c3fabc11c4fd6c7c105df68a81efaa919d8721e75e8d4b3",
            "imageTag": "1.4.0-cpu-py27-ubuntu16.04-v3.2-2020-04-04-00-46-20"
            "imageDigest": "sha256:ee07b739ab5da8403c3fabc11c4fd6c7c105df68a81efaa919d8721e75e8d4b3",
            "imageTag": "1.4.0-cpu-py27-ubuntu16.04"
            "imageDigest": "sha256:ee07b739ab5da8403c3fabc11c4fd6c7c105df68a81efaa919d8721e75e8d4b3",
            "imageTag": "1.4.0-cpu-py2"
            "imageTag": "1.4.0-cpu-py36-ubuntu16.04-v3.1"
            "imageTag": "1.4.0-cpu-py36-ubuntu16.04-v3.1-2020-03-19-21-16-17"
   [...]
{'fraction_non_alphanumeric': 0.14049727520435967, 'fraction_numerical': 0.11852861035422343, 'mean_word_length': 4.962436548223351, 'pattern_counts': {'":': 13, '<': 22, '<?xml version=': 0, '>': 21, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '399656', 'license': 'Apache-2.0', 'n_tokens_mistral': 2920, 'n_tokens_neox': 2790, 'n_words': 418, 'path': 'cifar10/code/Z.TEST.ipynb', 'repo_name': 'gonsoomoon-ml/SageMaker-PyTorch-Step-By-Step', 'size': '5872'}