mosaicml

/

mpt-7b

like 1.16k

Follow

Mosaic ML, Inc. 354

Yes, you can run on CPU with the standard model.generate() instructions as long as you have enough RAM to hold the model weights. This is ~30GB if using FP32 or ~15GB if using BF16.

Yes, you can run on CPU with the standard model.generate() instructions as long as you have enough RAM to hold the model weights. This is ~30GB if using FP32 or ~15GB if using BF16.

Thanks, Also I’m having trouble to load the model using oobabooga webui. Any instructions how can I use this model with the WebUI framework?

We leave it to the developers of oobabooga to add support for our model architecture.