[new-model] Added LingBot-World-Fast image-to-video support - #1665
[new-model] Added LingBot-World-Fast image-to-video support#1665Ishxn20 wants to merge 2 commits into
Conversation
Wired up FastVideo/LingBot-World-Fast-Diffusers, the old LingBot-World repo's distilled causal I2V checkpoint that was never ported. Reused the existing LingBot World 2 causal-fast DiT unchanged (verified all 1421 parameter names and shapes against the released checkpoint) and added the arch/pipeline configs, registry entries, preset, pipeline subclass, example, and SSIM test needed to run it. Fixed a model-family detector collision where the LingBot World (Cam) detector would have wrongly claimed this checkpoint's path and model_index name, since it only excluded the "causal-fast" spelling. Fixed WanCrossAttention in the shared LingBot World 2 causal-fast DiT to route through the attention() dispatcher instead of calling flash_attention() directly, so the already-advertised TORCH_SDPA backend actually works when flash-attn isn't installed. Self-attention already used this dispatcher; cross-attention did not. Verified end to end: weights load (18.54B params, 16 shards) and generation produces a coherent video with correct prompt-following camera motion. Not yet verified against the original implementation's numerics (activation-trace parity) or via the SSIM regression test, both of which need hardware unavailable in this environment. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI
🔴 PR merge requirementsWaiting for
This rule is failing.
|
Pre-commit checks failedHi @Ishxn20, the pre-commit checks have failed. To fix them locally: # Install pre-commit if you haven't already
uv pip install pre-commit
pre-commit install
# Run all checks and auto-fix what's possible
pre-commit run --all-filesCommon fixes:
After fixing, commit and push the changes. The checks will re-run automatically. For future commits, |
Ran pre-commit on the full PR diff after the fact and caught a line yapf wanted collapsed that a later edit had left split across two lines. No functional change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new LingBot-World-Fast image-to-video model variant to FastVideo by introducing a dedicated pipeline/config/preset/registry wiring, plus an SSIM similarity test and a small attention-path fix to make the shared causal-fast DiT work without FlashAttention installed.
Changes:
- Registers the new
FastVideo/LingBot-World-Fast-Diffusersvariant (configs + preset + pipeline) and adjusts LingBotWorld detector ordering/exclusions to avoid resolution collisions. - Adds a LingBot-World-Fast pipeline subclass that forcibly swaps in
FlowUniPCMultistepSchedulerto match the reference sampling path. - Updates LingBotWorld2 causal-fast cross-attention to route through the local
attention()dispatcher (FlashAttention if available, SDPA fallback otherwise) and adds an SSIM regression test for the new model.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| fastvideo/tests/ssim/test_lingbot_fast_similarity.py | New SSIM similarity test for LingBot-World-Fast I2V. |
| fastvideo/registry.py | Registers the new model family first and tightens LingBotWorld base detector exclusions; wires presets. |
| fastvideo/pipelines/basic/lingbotworld_fast/presets.py | Adds lingbotworld_fast_i2v inference preset for the new family. |
| fastvideo/pipelines/basic/lingbotworld_fast/fast_pipeline.py | New pipeline subclass that replaces the scheduler with FlowUniPCMultistepScheduler. |
| fastvideo/pipelines/basic/lingbotworld_fast/init.py | Exports/declares EntryClass for the new pipeline package. |
| fastvideo/models/registry.py | Adds a model-class alias mapping for the checkpoint’s shipped DiT class name. |
| fastvideo/models/dits/lingbotworld2/causal_fast.py | Routes cross-attention through attention() dispatcher (enabling SDPA fallback). |
| fastvideo/configs/pipelines/lingbotworld_fast.py | New pipeline config specializing LingBotWorld2 causal-fast config for LingBot-World-Fast. |
| fastvideo/configs/pipelines/init.py | Exposes the new pipeline config in the configs package exports. |
| fastvideo/configs/models/dits/lingbotworld_fast.py | Adds a LingBot-World-Fast DiT arch config with updated attention/sampling parameters. |
| fastvideo/configs/models/dits/init.py | Exposes the new DiT config in the DiT configs package exports. |
| examples/inference/basic/basic_lingbotworld_fast.py | Adds a basic end-to-end inference example script for the new model. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| logger = init_logger(__name__) | ||
|
|
||
| REQUIRED_GPUS = 2 | ||
|
|
||
| # The released checkpoint is 4-step distilled; see LingBotWorldFastArchConfig. | ||
| NUM_DISTILLED_STEPS = 4 |
Summary
FastVideo/LingBot-World-Fast-Diffusers, the old LingBot-World repo's distilled causal I2V checkpoint (generate_fast.py/LingBot-World-Fast) that was never ported.local_attn_size=-1(global attention, no eviction),chunk_size=3,timesteps_index=(0,179,358,679)."causal-fast"spelling, so it would have wrongly claimed this checkpoint's path and itslingbotworldcausaldmdpipelinemodel_index name. Registered LingBot-World-Fast first and tightened the exclusion.WanCrossAttentionin the shared LingBot World 2 causal-fast DiT to route through theattention()dispatcher instead of callingflash_attention()directly, so the already-advertisedTORCH_SDPAbackend actually works when flash-attn isn't installed. Self-attention already used this dispatcher; cross-attention did not.Test plan
update_model_archmatches the referencegenerate_fast.py(chunk_size=3,timesteps_index=(0,179,358,679),sample_shift=10.0)pre-commit run --files ...clean (yapf, ruff, codespell, mypy)[new-model](bootstrap mode)wan/modules/model_fast.py— the original also requires flash-attn, unavailable in dev environment