CUDA out of memory
{'login': 'li1117heex', 'id': 47059217, 'node_id': 'MDQ6VXNlcjQ3MDU5MjE3', 'avatar_url': 'https://avatars.githubusercontent.com/u/47059217?v=4', 'gravatar_id': '', 'url': 'https://api.github.com/users/li1117heex', 'html_url': 'https://github.com/li1117heex', 'followers_url': 'https://api.github.com/users/li1117heex/followers', 'following_url': 'https://api.github.com/users/li1117heex/following{/other_user}', 'gists_url': 'https://api.github.com/users/li1117heex/gists{/gist_id}', 'starred_url': 'https://api.github.com/users/li1117heex/starred{/owner}{/repo}', 'subscriptions_url': 'https://api.github.com/users/li1117heex/subscriptions', 'organizations_url': 'https://api.github.com/users/li1117heex/orgs', 'repos_url': 'https://api.github.com/users/li1117heex/repos', 'events_url': 'https://api.github.com/users/li1117heex/events{/privacy}', 'received_events_url': 'https://api.github.com/users/li1117heex/received_events', 'type': 'User', 'site_admin': False}
[]
[]
["Could you provide more details ? What's the code you ran ?"
 '```python\r\ntokenizer = FunnelTokenizer.from_pretrained(\'funnel-transformer/small\')\r\n\r\ndef tokenize(batch):\r\n    return tokenizer(batch[\'text\'], padding=\'max_length\', truncation=True,max_length=512)\r\n\r\ndataset = load_dataset("bookcorpus",split=\'train[:1000]\').shuffle()\r\ndataset = dataset.map(tokenize, batched=True, batch_size=512)\r\n\r\n# dataset = LineByLineTextDataset(\r\n#     tokenizer=tokenizer,\r\n#     file_path="./wiki1000.txt",\r\n#     block_size=128\r\n# )\r\n\r\ndata_collator = DataCollatorForLanguageModeling(\r\n    tokenizer=tokenizer, mlm=True, mlm_probability=0.15\r\n)\r\n\r\nconfig=FunnelConfig(\r\n    return_dict=True\r\n)\r\n\r\nmodel= FunnelForMaskedLM(config=config)\r\n\r\ntraining_args = TrainingArguments(\r\n    output_dir="./checkpoints",\r\n    overwrite_output_dir=True,\r\n    do_train=True,\r\n    num_train_epochs=1,\r\n    per_device_train_batch_size=16,\r\n    per_device_eval_batch_size=16,\r\n    save_steps=10000,\r\n    logging_dir=\'./ptlogs\'\r\n)\r\n\r\ntrainer = Trainer(\r\n    model=model,\r\n    args=training_args,\r\n    data_collator=data_collator,\r\n    train_dataset=dataset,\r\n)\r\ntrainer.train()\r\n```'
 '`RuntimeError: CUDA out of memory. Tried to allocate 954.00 MiB (GPU 0; 15.90 GiB total capacity; 14.35 GiB already allocated; 753.75 MiB free; 14.39 GiB reserved in total by PyTorch)\r\nException raised from malloc at /pytorch/c10/cuda/CUDACachingAllocator.cpp:272 (most recent call first):`\r\n\r\npart of error output'
 'from funnel model to bert model : error still happened\r\n\r\nfrom your dataset to LineByLineTextDataset : error disapeared'
 'notice i just loaded 1000 rows of data'
 'the error happens when executing loss.backward()'
 "Since you're using a data collator you don't need to tokenizer the dataset using `map`. Could you try not to use `map` and only the data collator instead ? The data collator is supposed to pad to the longest sequence in each batch afaik, instead of padding to 512.\r\n\r\nAlso cc @sgugger "
 'Closing this one.\r\nFeel free to re-open if you have other questions about this issue']
In your dataset ,cuda run out of memory as long as the trainer begins:
however, without changing any other element/parameter,just switch dataset to `LineByLineTextDataset`,everything becames OK.

{'url': 'https://api.github.com/repos/huggingface/datasets/issues/757/reactions', 'total_count': 0, '+1': 0, '-1': 0, 'laugh': 0, 'hooray': 0, 'confused': 0, 'heart': 0, 'rocket': 0, 'eyes': 0}