<jupyter_start><jupyter_text>1. pad_sequence- max seq_len에 맞춰서 padding- link: https://stackoverflow.com/questions/51030782/why-do-we-pack-the-sequences-in-pytorch param QA- batch_fifst: https://discuss.pytorch.org/t/could-someone-explain-batch-first-true-in-lstm/15402    - `False`인 경우        - input: seq_len, batch_size, features        - output: seq_len, batch_size, features    - `True`인 경우        - input: batch_size, seq_len, features        - output: batch_size, seq_len, features 2. pack_padded_sequence- pad=0을 계산하지 않기 위한 작업- return:    - elems: 데이터를 flatten한 값    - 각 time step에서 훈련할 데이터 갯수(pad 데이터 제외) - 참고:https://github.com/HarshTrivedi/packing-unpacking-pytorch-minimal-tutorial<jupyter_code>import torch
# pad_sequence
a = [torch.tensor([1,2,3]), torch.tensor([3, 4])]
b = torch.nn.utils.rnn.pad_sequence(a, batch_first=True)
b
# pack_padded_sequence
seq_len = [len(t) for t in a]
packed_input = torch.nn.utils.rnn.pack_padded_sequence(b, batch_first=True,\
                                        lengths=seq_len)
packed_input.data
# 마지막에 train set batch size 1인 이유는, 
# b에서 볼 수 있듯, 데이터가 3, pad=0 이므로 
# 0 제외한 갯수 
packed_input.batch_sizes<jupyter_output><empty_output><jupyter_text>3. contiguous, repea, copy_- link: https://discuss.pytorch.org/t/contigious-vs-non-contigious-tensor/30107/2- link2: https://stackoverflow.com/questions/48915810/pytorch-contiguous- 관련 에러: view operation runtime error    - view는 discontiguous에서는 작동 안됨    - ` invalid argument 2: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Call .contiguous() before .view().`- stride: `Stride is the jump necessary to go from one element to the next one in the specified dimension`    - 다음 row로 가려면; 다음 column으로 가려면 메모리 몇개를 skip해야 하는가<jupyter_code># row: 0 -> 3가려면 3개 메모리 skip
# col: 0 -> 1가려면 1개 메모리 skip
# 즉 메모리가 0,1,2,3,4,5,..이렇게 저장중
x = torch.arange(12).view(4,3)
x, x.stride()<jupyter_output><empty_output><jupyter_text>- view 에러 케이스    - transpose 후 메모리는 contiguous하지 않음    - 그상태로 flatten 시도하면 에러<jupyter_code>y = x.t()

print(y.is_contiguous()) # not contiguous
print(y.stride()) # stride 바뀜!!

# error
try:
    y.view(-1)
except RuntimeError as e:
    print(e)

# solution
y = y.contiguous()
print(y)
print(y.stride())
print(y.view(-1))<jupyter_output>False
(1, 3)
view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
tensor([[ 0,  3,  6,  9],
        [ 1,  4,  7, 10],
        [ 2,  5,  8, 11]])
(4, 1)
tensor([ 0,  3,  6,  9,  1,  4,  7, 10,  2,  5,  8, 11])
{'fraction_non_alphanumeric': 0.1286527514231499, 'fraction_numerical': 0.0349146110056926, 'mean_word_length': 3.6083916083916083, 'pattern_counts': {'":': 0, '<': 12, '<?xml version=': 0, '>': 14, 'https://': 5, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '93954', 'license': 'MIT', 'n_tokens_mistral': 1169, 'n_tokens_neox': 1189, 'n_words': 273, 'path': '07.DeepLearning/pytorch/pad_sequence_contiguous.ipynb', 'repo_name': 'HenryPaik1/Study', 'size': '2635'}