Spaces:
Runtime error
PyTorch implementation
Thanks for your amazing here!
It's awesome to see this space trending. Given the popularity and uniqueness of the application, I think it'd be great if you could add a PyTorch variant of the overall pipeline that is fully compatible with Diffusers. The community will benefit from that a lot I think.
Let us know if you need any assistance.
Regards,
Sayak
From the Diffusers team
There is a full pytorch implementation of the model in the repo linked to from the space.
https://github.com/lopho/makeavid-sd-tpu/blob/main/makeavid_sd/torch_impl/torch_unet_pseudo3d_condition.py
This is built with diffusers mixins, so it is usable with from_pretrained and from_config.
Weights on HF also include ptorch weights + safetensor weights next to JAX weights.
I am still working on clean torch inference + training code.
Also the model currently in use in the space has some architectural mistakes that need to be fixed first (which will require further training).
I would only implement this model in diffusers directly once the code is mature and those mistakes are fixed in both code an weights.
I will open a PR on diffusers by then.