Skip to content

[ci]: run the full attention test directory in the unit lane - #1658

Open
SolitaryThinker wants to merge 1 commit into
mainfrom
swe-unit-lane-attention-dir
Open

[ci]: run the full attention test directory in the unit lane#1658
SolitaryThinker wants to merge 1 commit into
mainfrom
swe-unit-lane-attention-dir

Conversation

@SolitaryThinker

@SolitaryThinker SolitaryThinker commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Problem

run_unit_test collects whole directories — fastvideo/tests/api/,
.../train/, .../stages/, .../ops/ — but from
fastvideo/tests/attention/ it names exactly one file:

"./fastvideo/tests/attention/test_sdpa_metadata_mask_contract.py "

The other ten files in that directory run in no PR lane at all. Among
them are the regression tests for the attention selection surface —
test_selector_role_override.py (per-role backend overrides) and
test_attn_qat_infer_arch_gate.py (the per-arch ATTN_QAT_INFER capability
sets) — plus the custom-op boundary tests for FA2/FA3, the masked/varlen
path, and FP4 FA4.

Those are precisely the tests that catch a silently mis-resolved attention
backend, which is a failure mode that produces output rather than an error.
A file-by-file selection also means every new test added to that directory
starts life uncollected, and nothing says so.

What changes

Collect the directory:

"./fastvideo/tests/attention/ "

Every file in it is safe on the lane's L40S runner. The GPU/kernel-dependent
ones already gate themselves — pytest.skip on torch.cuda.is_available(),
on flash_attn / flash_attn.cute importability, on bf16 support, and a
module-level find_spec("flash_attn") guard in the resolver test. The rest
(test_attn_qat_infer_arch_gate.py, test_selector_role_override.py,
test_attn_qat_train_kernel.py, test_sdpa_head_sizes.py,
test_attention_metadata_base.py, test_video_sparse_attention_metadata.py)
are CPU tests that monkeypatch their device and import probes; the VSA
backend module guards its fastvideo_kernel import, so importing it without
the kernel is fine.

A test in test_pr_test.py now pins the directory form, so the collection
cannot silently narrow back to a file list.

Evidence

The lane itself. microscope-unit-tests on this PR runs the new selection —
if any of the newly collected files is not lane-safe, it fails here rather
than after a merge.

Note for reviewers

The open #1494 contains this same one-line widening as part of a much larger
change, and has been conflicting since mid-July. Landing it separately closes
the coverage hole now and removes one hunk from that PR's rebase; the
directory-form assertion added here is new.

@SolitaryThinker SolitaryThinker added type: ci CI/CD infrastructure scope: infra CI, tests, Docker, build labels Jul 28, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@mergify

mergify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

Protection Waiting on
🔴 PR merge requirements 👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

  • #approved-reviews-by>=1
  • check-success=full-suite-passed
This rule is failing.
  • #approved-reviews-by>=1
  • check-success=full-suite-passed
  • check-success=fastcheck-passed
  • check-success~=pre-commit
  • title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

@SolitaryThinker

Copy link
Copy Markdown
Collaborator Author

Lane result (microscope-unit-tests, both pipelines): 883 passed, 33 skipped, 0 failed.

All 11 files in the directory now collect — per file:

result file
17 passed test_attn_qat_infer_arch_gate.py
8 passed test_attention_metadata_base.py
5 passed test_video_sparse_attention_metadata.py
4 passed test_flash_attn_no_pad_resolver.py
3 passed test_attn_qat_train_kernel.py
3 passed, 9 skipped test_flash_attn_no_pad_custom_op.py
2 passed test_selector_role_override.py
2 passed test_sdpa_head_sizes.py
2 passed test_sdpa_metadata_mask_contract.py (previously the only one collected)
1 passed, 8 skipped test_flash_attn_cute_custom_op.py
16 skipped test_flash_attn_default_custom_op.py

The two that motivated the change — the per-arch ATTN_QAT_INFER capability
gate and the per-role selector override — both execute (17 and 2), so this is
real coverage, not just collection.

Worth stating plainly: the skips are not incidental. The unit-lane image has
no flash_attn, so test_flash_attn_default_custom_op.py skips entirely
(16/16) and the two other FA custom-op files skip most of their cases. Those
tests now at least report as skipped rather than being invisible, but they
still do not run anywhere in PR CI. Giving them a lane with flash_attn
installed is a separate question from this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: infra CI, tests, Docker, build type: ci CI/CD infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant