CUDA out of memory
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 8.00 GiB total capacity; 7.07 GiB already allocated; 0 bytes free; 7.31 GiB reserved in total by PyTorch)
not sure how this is going down. Rtx3070 8gb. I'm too noob to know what to do next
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 8.00 GiB total capacity; 7.07 GiB already allocated; 0 bytes free; 7.31 GiB reserved in total by PyTorch)
not sure how this is going down. Rtx3070 8gb. I'm too noob to know what to do next
Use "--gpu-memory 7" in bat starting file. Idk why but setting limit to 1 gb less than your gpu has cures the problem
ok nmv it did not help
just switched the error to KeyError: 'model.layers.39.self_attn.q_proj.wf1'
:-}