metadata

license: apache-2.0
language:
  - en
base_model:
  - Wan-AI/Wan2.1-T2V-14B
pipeline_tag: text-to-video
tags:
  - text-to-video
  - text-to-image
  - lora
  - diffusers
  - template:diffusion-lora
widget:
  - text: >-
      cat video A short-haired tabby cat sits on a wooden fence, watching a
      butterfly flutter in front of it. The cat's ears twitch as it slowly lifts
      a paw, trying to swipe at the insect. The butterfly lands on its nose for
      a moment before flying away, leaving the cat slightly confused.
    output:
      url: example_videos/cat1.mp4
  - text: >-
      cat video A small orange kitten is jumping up at a dangling toy feather.
      Each time it leaps, it barely misses, but it keeps trying with
      determination. Its tail wiggles slightly before each jump, preparing for
      the next attempt.
    output:
      url: example_videos/cat2.mp4
  - text: >-
      ca45t cat video The video features a grey Scottish Fold cat lying on a
      couch and licking its paw. The cat is lying on its side with its head
      turned towards the camera. It has a collar on. It is using its front paw
      to clean its ear.  
    output:
      url: example_videos/cat3.mp4
  - text: >-
      ca45t cat video A small, white, fluffy, kitten with blue eyes, that looks
      like a British shorthair, is walking on a grey couch with a white wall
      behind it. It walks a few steps, and stops to look at something off-camera
      to its right. Then it resumes walking. The kitten's tail is sticking
      straight up into the air as it walks. 
    output:
      url: example_videos/cat4.mp4

Cat LoRA for Wan2.1 14B T2V

Overview

This LoRA is trained on the Wan2.1 14B T2V model and allows you to generate awesome videos of cats!

Features

Trained on the Wan2.1 14B T2V base model
Consistent results across different object types
Simple prompt structure that's easy to adapt

Community

Discord: Join our community to generate videos with this LoRA for free
Request LoRAs: We're training and open-sourcing Wan2.1 LoRAs for free - join our Discord to make requests!

Prompt: cat video A short-haired tabby cat sits on a wooden fence, watching a butterfly flutter in front of it. The cat's ears twitch as it slowly lifts a paw, trying to swipe at the insect. The butterfly lands on its nose for a moment before flying away, leaving the cat slightly confused.

Prompt: cat video A small orange kitten is jumping up at a dangling toy feather. Each time it leaps, it barely misses, but it keeps trying with determination. Its tail wiggles slightly before each jump, preparing for the next attempt.

Prompt: ca45t cat video The video features a grey Scottish Fold cat lying on a couch and licking its paw. The cat is lying on its side with its head turned towards the camera. It has a collar on. It is using its front paw to clean its ear.

Prompt: ca45t cat video A small, white, fluffy, kitten with blue eyes, that looks like a British shorthair, is walking on a grey couch with a white wall behind it. It walks a few steps, and stops to look at something off-camera to its right. Then it resumes walking. The kitten's tail is sticking straight up into the air as it walks.

Model File and Inference Workflow

📥 Download Links:

cats_10_epochs.safetensors - LoRA Model File
wan_txt2vid_lora_workflow.json - Wan T2V with LoRA Workflow for ComfyUI

Recommended Settings

LoRA Strength: 1.0
Embedded Guidance Scale: 6.0
Flow Shift: 5.0

Trigger Words

The key trigger phrase is: ca45t cat video

Prompt Template

For prompting, check out the example prompts; this way of prompting seems to work very well.

ComfyUI Workflow

This LoRA works with a modified version of Kijai's Wan Video Wrapper workflow. The main modification is adding a Wan LoRA node connected to the base model.

See the Downloads section above for the modified workflow.

Model Information

The model weights are available in Safetensors format. See the Downloads section above.

Training Details

Base Model: Wan2.1 14B T2V
Training Data: Trained on 2 minutes of video comprised of 27 short clips (each clip captioned separately) of cats.
Epochs: 10

Additional Information

Training was done using Diffusion Pipe for Training

Acknowledgments

Special thanks to Kijai for the ComfyUI Wan Video Wrapper and tdrussell for the training scripts!