|
--- |
|
base_model: |
|
- alibaba-pai/CogVideoX-Fun-5b-InP |
|
--- |
|
This video LoRA model upload is an early finetuned research version based on [CogVideoX-Fun-V1.1-5B-INP](https://huggingface.co/alibaba-pai/CogVideoX-Fun-V1.1-5b-InP) on a dataset of 65 captioned .mp4 video samples at a resolution of 720x480 based on the film Bladerunner 2049 for 2k steps on single H100 gpu using parameters provided by CogvideoX Fun's DeepSpeed finetune script. This finetune is an early version as we prepare to train better versions, in order to inference this currently it should be placed in a directory which can be read by the predict_t2v.py script provided by CogVideoX-Fun, as currently I believe there is no ComfyUI node for LoRAs. |
|
LoRA finetunes can be used on the CogVideoX-Fun Pose models as well, this is not just applicable to the InPaint model which this was trained on. |
|
|
|
Here are a few samples generated by the later finetuned model at 6.5k steps (These samples are more trained than the current model upload): |
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/632633824f4f4ea5a863d71a/j822PK7VDs9-Lutpvo8qp.mp4"></video> |
|
`The scene is in a control room where Ryan Gosling is standing in front of a large window that overlooks a city at night. He is looking at a woman who is sitting at a desk with various equipment. The woman is dressed in a black suit and appears to be focused on her work. Ryan Gosling's expression is one of concern as he talks to the woman. The video ends with Ryan Gosling continuing to observe the woman, his expression unchanged.` |
|
|
|
|
|
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/632633824f4f4ea5a863d71a/4LBjgg4_ne8JgW8mMhgFS.mp4"></video> |
|
`The video opens with Ryan Gosling sitting in the driver's seat of a car. He is wearing a green jacket with a fur collar and has a serious expression on his face. The car appears to be parked in a desolate area with foggy mountains in the background. As the video progresses, Gosling's expression changes to one of concern as he looks out of the car window. The camera then cuts to a close-up of Gosling's face, highlighting his furrowed brows and worried eyes. The video ends with a shot of the car's dashboard, which is covered in dust and debris, suggesting that the car has been abandoned for some time.` |
