Image-to-Video Generation

Prerequisites: Text-to-Image Pipeline with Stable Diffusion; a CUDA GPU (24GB+ VRAM recommended, less with CPU offload); ffmpeg installed.

Targeted versions: Diffusers (current release); Stability AI's Stable Video Diffusion (SVD-XT) — verified against Hugging Face and Stability AI documentation, mid-2026.

Stable Video Diffusion (SVD) is a latent diffusion model that takes a single still image as a conditioning frame and generates a short video clip that continues plausibly from it — camera pans, subtle motion, environmental movement. It's an image-to-video model specifically, not text-to-video: it has no text encoder and can't be steered with a text prompt, only with the input image and a couple of motion-strength parameters.

You'll run SVD-XT (the 25-frame variant) through Diffusers, then chain it after an SDXL text-to-image step so the full pipeline goes from a text prompt straight to a video file. This project sticks to SVD because it's the model most directly runnable through Diffusers with a manageable VRAM footprint; if you outgrow it, newer open models like Wan 2.2 and LTX-Video push further on motion quality and resolution at a heavier compute cost.