Thank you!

by CulturedMan - opened Dec 27, 2023

Discussion

CulturedMan

Dec 27, 2023

•

edited Dec 28, 2023

Thanks for the 3.0bpw! I am currently stuck with 12gb vram, and I'm able to run this with 6144 context / alpha 1.75 with minimal spillover to the CPU.

LoneStriker

Owner Dec 28, 2023

Thanks for the feedback. Picking quant sizes can be tricky with all of the different model sizes (from "interesting" merges) as well as GPU VRAM sizes.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment