Stable Video Diffusion img2vid

What it does

A Google Colab notebook that launches a GUI to use Stability AI's Stable Video Diffusion (SVD) model.

The SVD model is an image-to-video model. It takes the input image as the first frame of the video and generates a short video clip using a motion prior learned from a large video dataset.

How to use

1. Open the Colab Notebook.
2. Review options. Run the notebook.
3. Wait for the gradio.live link to appear. Click the link to start the GUI.
4. In the GUI, upload an initial image for the video.
5. Adjust crop offset to get the desired crop.
6. Press Run to generate a video.

A full tutorial can be found here.

Colab runtime

It takes 9 mins on a T4 and 2 mins on a V100.

High RAM is not strictly necessary but helps to avoid crashes.

Works on free Colab.

Advanced options

• Increase motion bucket id to increase the motion in the video.
• Increase fps id to increase the frame rate of the video.
• Fix the random as a positive integer to generate the same video.

Known issues

The notebook may crash after a few generations when NOT using the high RAM setting. (Such as in the Free Colab runtime.)

Credits

Thanks to these notebooks!

1. Camenduru's SVD Colab notebook
2. mkshing's SVD Colab notebook

Examples

000003.mp4

000002.mp4

000001.mp4

000000.2.mp4

000010.mp4