Skip to content

Validate against all loaded namespaces via a namespace-aware dispatcher (fixes #608) #1529

Description

@rly

Problem

Directly addresses #608. Validation assumes a single namespace: ValidatorMap is built from one SpecNamespace and its get_validator raises when a builder's type is not in that namespace. A file's builder tree is validated against one namespace at a time, so a file that uses two independent extension namespaces fails.

Reproduction

A file whose root type comes from one extension and a child from another (e.g. ndx-multisubjects root NdxMultiSubjectsNWBFile + an ndx-sound AcousticWaveformSeries stimulus) raises during validation:

ValueError: data type 'NdxMultiSubjectsNWBFile' not found in namespace ndx-sound

This reproduces via pynwb.validate(path=...) (pynwb picks the "most specific" leaf namespaces and validates the whole tree against each) and is independent of nwbinspector. #608 also describes a related false-negative: a subtype defined in one extension namespace placed where its parent type is expected is wrongly rejected.

Design: a namespace-aware dispatcher (not a merged bare-name registry)

A file can legitimately contain two different types that share a name, defined in two independent namespaces. HDMF's read/construct path already differentiates them: every typed builder stores a namespace attribute and is resolved by (namespace, data_type) (TypeMap.get_clsget_builder_ns + get_dt_container_cls(dt, ns)). Validation must do the same. Merging all namespaces into one registry keyed by bare name would collapse distinct same-named types and misdispatch, so that approach is rejected.

Instead, validate the whole builder tree in one pass with a dispatcher that:

  • For each builder, reads its (namespace, data_type) from its attributes and resolves the validator from that namespace's ValidatorMap (each namespace's catalog is self-contained: its own types plus its transitively-included dependencies). Root (ndx-multisubjects, …) → ms validator; child (ndx-sound, …) → sound validator. This fixes the crash.
  • Resolves the Account for all loaded namespaces during validation #608 subtype false-negative by matching a child against a spec via the child's hierarchy computed in the child's own namespace (required_type_name in child_hierarchy), rather than enumerating subtypes of the required type across namespaces. This is collision-safe because a cross-namespace match only succeeds through a shared dependency type (the same type identity, not a name collision).
  • Routes the reference-validation reach-throughs (currently self.vmap.namespace.catalog.get_spec(...) / self.vmap.namespace.get_hierarchy(...) in AttributeValidator and DatasetValidator) so a referenced builder's hierarchy is computed in the referenced builder's own namespace, read from its namespace attribute.

Why not bare-name merging

Bare-name lookups across namespaces assume a name identifies the same type everywhere. That holds for shared dependencies (e.g. core VectorData) but not for independent leaves that reuse a name for different types. Dispatching by the builder's stored (namespace, data_type) keeps resolution namespace-scoped and mirrors the proven construct path.

Depends on

Acceptance criteria

  • A file rooted in one extension namespace with a child from an independent extension namespace validates without a "not found" error.
  • A subtype defined in a sibling namespace is accepted where its parent type is expected; an unrelated type still errors.
  • Two different types that share a name across namespaces are each validated against their own spec (no misdispatch).
  • New multi-namespace tests in tests/unit/validator_tests/test_validate.py.

Note on in-flight work

PRs #1480 (ValidationResult wrapper) and #1443 touch the validator return path and overlap with this change. Sequence after those land or coordinate to avoid conflicts.

The pynwb consumer (pynwb/src/pynwb/validation.py: drop the leaf-namespace selection, validate once via the dispatcher) is a follow-up in the pynwb repo.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions