Model Format and Loading Guide#

πŸ“– Overview#

LightX2V is a flexible video generation inference framework that supports multiple model sources and formats, providing users with rich options:

  • βœ… Wan Official Models: Directly compatible with officially released complete models from Wan2.1 and Wan2.2

  • βœ… Single-File Models: Supports single-file format models released by LightX2V (including quantized versions)

  • βœ… LoRA Models: Supports loading distilled LoRAs released by LightX2V

This document provides detailed instructions on how to use various model formats, configuration parameters, and best practices.


πŸ—‚οΈ Format 1: Wan Official Models#

Model Repositories#

Model Features#

  • Official Guarantee: Complete models officially released by Wan-AI with highest quality

  • Complete Components: Includes all necessary components (DIT, T5, CLIP, VAE)

  • Original Precision: Uses BF16/FP32 precision with no quantization loss

  • Strong Compatibility: Fully compatible with Wan official toolchain

Wan2.1 Official Models#

Directory Structure#

Using Wan2.1-I2V-14B-720P as an example:

Wan2.1-I2V-14B-720P/
β”œβ”€β”€ diffusion_pytorch_model-00001-of-00007.safetensors   # DIT model shard 1
β”œβ”€β”€ diffusion_pytorch_model-00002-of-00007.safetensors   # DIT model shard 2
β”œβ”€β”€ diffusion_pytorch_model-00003-of-00007.safetensors   # DIT model shard 3
β”œβ”€β”€ diffusion_pytorch_model-00004-of-00007.safetensors   # DIT model shard 4
β”œβ”€β”€ diffusion_pytorch_model-00005-of-00007.safetensors   # DIT model shard 5
β”œβ”€β”€ diffusion_pytorch_model-00006-of-00007.safetensors   # DIT model shard 6
β”œβ”€β”€ diffusion_pytorch_model-00007-of-00007.safetensors   # DIT model shard 7
β”œβ”€β”€ diffusion_pytorch_model.safetensors.index.json       # Shard index file
β”œβ”€β”€ models_t5_umt5-xxl-enc-bf16.pth                      # T5 text encoder
β”œβ”€β”€ models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth  # CLIP encoder
β”œβ”€β”€ Wan2.1_VAE.pth                                       # VAE encoder/decoder
β”œβ”€β”€ config.json                                          # Model configuration
β”œβ”€β”€ xlm-roberta-large/                                   # CLIP tokenizer
β”œβ”€β”€ google/                                              # T5 tokenizer
β”œβ”€β”€ assets/
└── examples/

Usage#

# Download model
huggingface-cli download Wan-AI/Wan2.1-I2V-14B-720P \
    --local-dir ./models/Wan2.1-I2V-14B-720P

# Configure launch script
model_path=./models/Wan2.1-I2V-14B-720P
lightx2v_path=/path/to/LightX2V

# Run inference
cd LightX2V/scripts
bash wan/run_wan_i2v.sh

Wan2.2 Official Models#

Directory Structure#

Using Wan2.2-I2V-A14B as an example:

Wan2.2-I2V-A14B/
β”œβ”€β”€ high_noise_model/                                    # High-noise model directory
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00001-of-00009.safetensors
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00002-of-00009.safetensors
β”‚   β”œβ”€β”€ ...
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00009-of-00009.safetensors
β”‚   └── diffusion_pytorch_model.safetensors.index.json
β”œβ”€β”€ low_noise_model/                                     # Low-noise model directory
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00001-of-00009.safetensors
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00002-of-00009.safetensors
β”‚   β”œβ”€β”€ ...
β”‚   β”œβ”€β”€ diffusion_pytorch_model-00009-of-00009.safetensors
β”‚   └── diffusion_pytorch_model.safetensors.index.json
β”œβ”€β”€ models_t5_umt5-xxl-enc-bf16.pth                      # T5 text encoder
β”œβ”€β”€ Wan2.1_VAE.pth                                       # VAE encoder/decoder
β”œβ”€β”€ configuration.json                                   # Model configuration
β”œβ”€β”€ google/                                              # T5 tokenizer
β”œβ”€β”€ assets/                                              # Example assets (optional)
└── examples/                                            # Example files (optional)

Usage#

# Download model
huggingface-cli download Wan-AI/Wan2.2-I2V-A14B \
    --local-dir ./models/Wan2.2-I2V-A14B

# Configure launch script
model_path=./models/Wan2.2-I2V-A14B
lightx2v_path=/path/to/LightX2V

# Run inference
cd LightX2V/scripts
bash wan22/run_wan22_moe_i2v.sh

Available Model List#

Wan2.1 Official Model List#

Model Name

Download Link

Wan2.1-I2V-14B-720P

Link

Wan2.1-I2V-14B-480P

Link

Wan2.1-T2V-14B

Link

Wan2.1-T2V-1.3B

Link

Wan2.1-FLF2V-14B-720P

Link

Wan2.1-VACE-14B

Link

Wan2.1-VACE-1.3B

Link

Wan2.2 Official Model List#

Model Name

Download Link

Wan2.2-I2V-A14B

Link

Wan2.2-T2V-A14B

Link

Wan2.2-TI2V-5B

Link

Wan2.2-Animate-14B

Link

Usage Tips#

πŸ’‘ Quantized Model Usage: To use quantized models, refer to the Model Conversion Script for conversion, or directly use pre-converted quantized models in Format 2 below

πŸ’‘ Memory Optimization: For devices with RTX 4090 24GB or smaller memory, it’s recommended to combine quantization techniques with CPU offload features:



πŸ—‚οΈ Format 3: LightX2V LoRA Models#

LoRA (Low-Rank Adaptation) models provide a lightweight model fine-tuning solution that enables customization for specific effects without modifying the base model.

Model Repositories#

Usage Methods#

Method 1: Offline Merging#

Merge LoRA weights offline into the base model to generate a new complete model file.

Steps:

Refer to the Model Conversion Documentation for offline merging.

Advantages:

  • βœ… No need to load LoRA during inference

  • βœ… Better performance

Disadvantages:

  • ❌ Requires additional storage space

  • ❌ Switching different LoRAs requires re-merging

Method 2: Online Loading#

Dynamically load LoRA weights during inference without modifying the base model.

LoRA Application Principle:

# LoRA weight application formula
# lora_scale = (alpha / rank)
# W' = W + lora_scale * B @ A
# Where: B = up_proj (out_features, rank)
#        A = down_proj (rank, in_features)

if weights_dict["alpha"] is not None:
    lora_scale = weights_dict["alpha"] / lora_down.shape[0]
elif alpha is not None:
    lora_scale = alpha / lora_down.shape[0]
else:
    lora_scale = 1.0

Configuration Method:

Wan2.1 LoRA Configuration:

{
  "lora_configs": [
    {
      "path": "wan2.1_i2v_lora_rank64_lightx2v_4step.safetensors",
      "strength": 1.0,
      "alpha": null
    }
  ]
}

Wan2.2 LoRA Configuration:

Since Wan2.2 uses a dual-model architecture (high-noise/low-noise), LoRA needs to be configured separately for both models:

{
  "lora_configs": [
    {
      "name": "low_noise_model",
      "path": "wan2.2_i2v_A14b_low_noise_lora_rank64_lightx2v_4step.safetensors",
      "strength": 1.0,
      "alpha": null
    },
    {
      "name": "high_noise_model",
      "path": "wan2.2_i2v_A14b_high_noise_lora_rank64_lightx2v_4step.safetensors",
      "strength": 1.0,
      "alpha": null
    }
  ]
}

Parameter Description:

Parameter

Description

Default

path

LoRA model file path

Required

strength

LoRA strength coefficient, range [0.0, 1.0]

1.0

alpha

LoRA scaling factor, uses model’s built-in value when null

null

name

(Wan2.2 only) Specifies which model to apply to

Required

Advantages:

  • βœ… Flexible switching between different LoRAs

  • βœ… Saves storage space

  • βœ… Can dynamically adjust LoRA strength

Disadvantages:

  • ❌ Additional loading time during inference

  • ❌ Slightly increases memory usage