Store exact history per batch element - #1502
Conversation
📖 Docs PreviewPreview of the documentation for this PR: 🔗 https://smokeshow.helpmanual.io/5l3l2x5m68393s5h320l/ Built from 6c3ee6d |
There was a problem hiding this comment.
Pull request overview
This PR refactors TorchIO’s batch transform history to store an exact, per-element history list (instead of shared history plus per-element slicing), and updates transform wrapping/unwrapping and inversion behavior to work with divergent element histories.
Changes:
- Introduces
_BatchedHistoryMixinto manage exact per-element histories forImagesBatchandSubjectsBatch, including divergent-history inversion behavior. - Updates
Transform.forward()to eagerly materialize clean per-element history traces (dropping internal batching bookkeeping keys from public history). - Updates tests and migration docs to use
histories/history(index)and new divergent-history semantics (e.g.,get_inverse_transform()raising on divergence).
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_swap.py | Update per-instance assertions to read per-element histories. |
| tests/test_spike.py | Update per-instance assertions to read per-element histories. |
| tests/test_spatial.py | Update per-instance spatial assertions to read per-element histories. |
| tests/test_per_instance.py | Update per-instance history expectations and out-of-range behavior. |
| tests/test_one_of.py | Update OneOf divergent history expectations and clearing behavior. |
| tests/test_normalize.py | Update per-instance normalize assertions to read per-element histories. |
| tests/test_noise.py | Update per-instance noise assertions to read per-element histories. |
| tests/test_motion.py | Update per-instance motion assertions to read per-element histories. |
| tests/test_labels_to_image.py | Update per-instance label-stat assertions to use history(index). |
| tests/test_ghosting.py | Update per-instance ghosting assertions to read per-element histories. |
| tests/test_gamma.py | Update per-instance gamma assertions to read per-element histories. |
| tests/test_flip.py | Update per-instance flip assertions to read per-element histories. |
| tests/test_blur.py | Update per-instance blur assertions to read per-element histories. |
| tests/test_bias_field.py | Update per-instance bias field assertions to read per-element histories. |
| tests/test_batch.py | Remove manual history adoption; rely on automatic preservation via per-element histories. |
| tests/test_anisotropy.py | Update per-instance anisotropy assertions to read per-element histories. |
| tests/conftest.py | Update vectorization test helper to replay params from per-element history. |
| src/torchio/transforms/transform.py | Build and append per-element history traces; propagate histories through wrap/unwrap paths. |
| src/torchio/transforms/monai_adapter.py | Remove manual history adoption after restacking subjects. |
| src/torchio/transforms/inverse.py | Delegate inverse application to batch types for correct divergent-history handling. |
| src/torchio/transforms/cornucopia_adapter.py | Remove manual history adoption after restacking subjects. |
| src/torchio/transforms/compose.py | Simplify per-element apply/rebatch logic; rely on automatic history preservation. |
| src/torchio/data/batch.py | Move batch containers to _BatchedHistoryMixin; remove legacy per-element history machinery. |
| src/torchio/data/batch_history.py | New mixin implementing exact per-element history storage and divergent-history inversion behavior. |
| docs/get-started/migration.md | Update migration guidance: restacking preserves exact per-element histories automatically. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
f992093 to
c49206f
Compare
9d4a1f0 to
a2250fc
Compare
a2250fc to
61c539c
Compare
61c539c to
fa4ffb3
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
9823221 to
80f7b1a
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: baf5f11a-f67c-44dc-804e-6be849fa9160
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 29 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (1)
src/torchio/transforms/transform.py:278
_build_history_traces()deep-copies the fullparamsdict once per batch element when_batched_keysis absent. For large (but batch-shared) parameter dicts this scales CPU+memory withbatch_sizeeven though every element’s trace is identical. You can deep-copyparamsonce and reuse the same copied dict across the per-elementAppliedTransformobjects (the objects can remain distinct to satisfy the independent-trace invariant).
batched_keys = params.get("_batched_keys")
if batched_keys is None:
return [
self._make_applied_transform(_copy.deepcopy(params))
for _ in range(batch_size)
]
[Generated by a coding agent]
What
Replace shared-plus-optional history with one exact transform history per batch element.
Why
The old model persisted
_batch_size,_batched_keys, and_keepinside public transform params, then reconstructed histories during unbatching. That representation required special cases in composition, adapters, inversion, replay, and batch containers.How
_per_element_history, history slicing, manual adoption, and OneOf/SomeOf freezingValidation
Replacement stack