Skip to content

feat(ecvit): add multi-scale feature support - #950

Merged
liopeer merged 5 commits into
lightly-ai:mainfrom
lorinczszabolcs:add-multiscale-features-edgecrafter
Sep 21, 2026
Merged

liopeer merged 5 commits into
lightly-ai:mainfrom
lorinczszabolcs:add-multiscale-features-edgecrafter

Conversation

@lorinczszabolcs

Copy link
Copy Markdown
Contributor

Part of #928 and #925.

What has changed and why?

ECViTModelWrapper implemented ModelWrapper only, so the EdgeCrafter ECViT backbone could not expose its feature pyramid through the multi-scale interface that dense prediction heads use. This adapts the existing pyramid to that interface (the first item of #928).

  • ECViTModelWrapper now implements MultiScaleFeatureCNN, and EdgeCrafterPackage is a MultiScaleFeaturePackage.
  • forward_multiscale_features exposes the existing (P3, P4, P5) pyramid from forward. The strides are [8, 16, 32] and all three levels share the same channel dimension.
  • I used the CNN interface, because the pyramid has increasing strides.
  • ECViT now also works as a linear semantic segmentation backbone, so I added the linear-seg configs for the four ECViT presets, like the other packages.

The second item of #928 (multi-channel input) is left for a separate change.

How has it been tested?

New unit tests in tests/_models/ecvit/test_ecvit.py cover the feature dimensions and strides, output shapes, output order, invalid indices, the match with forward_features for the last level, and the uniform dimensions for the proj_dim=None presets. Run them with:

uv run --frozen pytest tests/_models/ecvit/

Did you update CHANGELOG.md?

  • Yes

Did you update the documentation?

  • Not needed (internal change without effects for user)

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

@mrpositron

Copy link
Copy Markdown
Contributor

/review

@liopeer liopeer changed the title Add multi-scale feature support to edgecrafter ECViT backbone feat(ecvit): add multi-scale feature support Aug 24, 2026
Make ECViTModelWrapper implement MultiScaleFeatureCNN and EdgeCrafterPackage a MultiScaleFeaturePackage, exposing the existing (P3, P4, P5) ECViT pyramid (strides 8, 16, 32) through forward_multiscale_features. Register the four ECViT linear semantic segmentation configs and add unit tests.

Part of lightly-ai#928 and lightly-ai#925.
Comment thread tests/_models/ecvit/test_ecvit.py

@liopeer liopeer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks a lot 🚀

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
liopeer added a commit to lorinczszabolcs/lightly-train that referenced this pull request Sep 21, 2026
…alized levels

Addresses review findings on lightly-ai#950/lightly-ai#951:

- ltdetr_object_detection: drop the edgecrafter special case that pinned
  _expected_input_channels to 3 while get_model was built for
  len(image_normalize["mean"]). A 4-channel image_normalize built a 4-channel
  stem but declared 3 in the preprocessor and the ONNX input spec.
- dinov3_ltdetr: forward num_input_channels to EDGE_CRAFTER_PACKAGE.get_model
  and drop the now-false comment claiming ECViT has no multi-channel support.
- ecvit_package: merge num_input_channels into model_args instead of passing it
  alongside **model_args, which raised TypeError when a caller set it in
  backbone_args. Matches DINOv3Package.get_model.
- ecvit: TODO on forward_multiscale_features recording that levels are not
  uniformly normalized when the preset proj_dim is None.
- docs: drop the edgecrafter note contradicting the new multi-channel support.
- tests: narrow patch_embed/conv types so mypy passes; reflow CHANGELOG.
@liopeer
liopeer enabled auto-merge (squash) September 21, 2026 17:22
@liopeer
liopeer merged commit 5863b70 into lightly-ai:main Sep 21, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants