
# Tune-A-Video - Modern Disney

## Model Description
This is a diffusers compatible checkpoint. When used with DiffusionPipeline, returns an instance of TuneAVideoPipeline

>df-cpt is used to indicate that its a diffusers compatible equivalent of Tune-A-Video-library/mo-di-bear-guitar .

- Base model: [nitrosocke/mo-di-diffusion](https://huggingface.co/nitrosocke/mo-di-diffusion)
- Training prompt: a bear is playing guitar.
![sample-train](samples/train.gif)

## Samples

![sample-500](samples/princess.gif)
Test prompt:  "A princess playing a guitar, modern disney style"

## Usage

### Loading with a pre-existing Text2Image checkpoint
```python
import torch
from diffusers import TuneAVideoPipeline, DDIMScheduler, UNet3DConditionModel
from diffusers.utils import export_to_video
from PIL import Image

# Use any pretrained Text2Image checkpoint based on stable diffusion
pretrained_model_path = "nitrosocke/mo-di-diffusion"
unet = UNet3DConditionModel.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", subfolder="unet", torch_dtype=torch.float16
).to("cuda")

pipe = TuneAVideoPipeline.from_pretrained(pretrained_model_path, unet=unet, torch_dtype=torch.float16).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)
```
### Loading a saved Tune-A-Video checkpoint
```python
import torch
from diffusers import DiffusionPipeline, DDIMScheduler
from diffusers.utils import export_to_video
from PIL import Image

pipe = DiffusionPipeline.from_pretrained(
    "Tune-A-Video-library/df-cpt-mo-di-bear-guitar", torch_dtype=torch.float16
).to("cuda")

prompt = "A princess playing a guitar, modern disney style"
generator = torch.Generator(device="cuda").manual_seed(42)

video_frames = pipe(prompt, video_length=3, generator=generator, num_inference_steps=50, output_type="np").frames

# Saving to gif.
pil_frames = [Image.fromarray(frame) for frame in video_frames]
duration = len(pil_frames) / 8
pil_frames[0].save(
    "animation.gif",
    save_all=True,
    append_images=pil_frames[1:],  # append rest of the images
    duration=duration * 1000,  # in milliseconds
    loop=0,
)

# Saving to video
video_path = export_to_video(video_frames)
```

## Related Papers:
- [Tune-A-Video](https://arxiv.org/abs/2212.11565): One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
- [Stable Diffusion](https://arxiv.org/abs/2112.10752): High-Resolution Image Synthesis with Latent Diffusion Models

{"license": "creativeml-openrail-m", "tags": ["tune-a-video", "text-to-video", "diffusers"], "base_model": "nitrosocke/mo-di-diffusion", "training_prompt": "A bear is playing guitar.", "inference": false}
diffusers tune-a-video text-to-video arxiv:2212.11565 arxiv:2112.10752 base_model:nitrosocke/mo-di-diffusion license:creativeml-openrail-m diffusers:TuneAVideoPipeline region:us
TAGS
#diffusers #tune-a-video #text-to-video #arxiv-2212.11565 #arxiv-2112.10752 #base_model-nitrosocke/mo-di-diffusion #license-creativeml-openrail-m #diffusers-TuneAVideoPipeline #region-us 


# Tune-A-Video - Modern Disney

## Model Description
This is a diffusers compatible checkpoint. When used with DiffusionPipeline, returns an instance of TuneAVideoPipeline

>df-cpt is used to indicate that its a diffusers compatible equivalent of Tune-A-Video-library/mo-di-bear-guitar .

- Base model: nitrosocke/mo-di-diffusion
- Training prompt: a bear is playing guitar.
!sample-train

## Samples

!sample-500
Test prompt:  "A princess playing a guitar, modern disney style"

## Usage

### Loading with a pre-existing Text2Image checkpoint

### Loading a saved Tune-A-Video checkpoint


## Related Papers:
- Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
- Stable Diffusion: High-Resolution Image Synthesis with Latent Diffusion Models

# Tune-A-Video - Modern Disney ## Model Description
This is a diffusers compatible checkpoint. When used with DiffusionPipeline, returns an instance of TuneAVideoPipeline

>df-cpt is used to indicate that its a diffusers compatible equivalent of Tune-A-Video-library/mo-di-bear-guitar .

- Base model: nitrosocke/mo-di-diffusion
- Training prompt: a bear is playing guitar.
!sample-train ## Samples

!sample-500
Test prompt:  "A princess playing a guitar, modern disney style" ## Usage ### Loading with a pre-existing Text2Image checkpoint ### Loading a saved Tune-A-Video checkpoint ## Related Papers:
- Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
- Stable Diffusion: High-Resolution Image Synthesis with Latent Diffusion Models
TAGS
#diffusers #tune-a-video #text-to-video #arxiv-2212.11565 #arxiv-2112.10752 #base_model-nitrosocke/mo-di-diffusion #license-creativeml-openrail-m #diffusers-TuneAVideoPipeline #region-us 
 # Tune-A-Video - Modern Disney ## Model Description
This is a diffusers compatible checkpoint. When used with DiffusionPipeline, returns an instance of TuneAVideoPipeline

>df-cpt is used to indicate that its a diffusers compatible equivalent of Tune-A-Video-library/mo-di-bear-guitar .

- Base model: nitrosocke/mo-di-diffusion
- Training prompt: a bear is playing guitar.
!sample-train ## Samples

!sample-500
Test prompt:  "A princess playing a guitar, modern disney style" ## Usage ### Loading with a pre-existing Text2Image checkpoint ### Loading a saved Tune-A-Video checkpoint ## Related Papers:
- Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
- Stable Diffusion: High-Resolution Image Synthesis with Latent Diffusion Models