Inference Speed
#8
by
andreasrath
- opened
Does anyone have any details on inference speeds on A100 or V100 or T4?
Hey @andreasrath , with our latest release you should expect the following
- A100/V100: faster than real-time inference
- T4: ~2x real-time today
We will be dropping support for streaming soon.
@sidroopdaska many thanks for your info!
Is there any hope for a more efficient model that can run in real-time CPUs? I used Candle to run this, even the quantized version and it took a minute for 2 seconds of audio on my AMD 7640U laptop
@soupslurpr this is not something on our roadmap for this release... feel free to find an issue on our github (https://github.com/metavoiceio/metavoice-src) and we'll be able to track it for a future release
vatsal-metavoice
changed discussion status to
closed