Skip to content

Fix the Units waveforms test fixtures and assert the doubly indexed structure - #2240

Open
adityasingh2400 wants to merge 2 commits into
NeurodataWithoutBorders:devfrom
adityasingh2400:fix-2220-units-waveforms-fixture
Open

Fix the Units waveforms test fixtures and assert the doubly indexed structure#2240
adityasingh2400 wants to merge 2 commits into
NeurodataWithoutBorders:devfrom
adityasingh2400:fix-2220-units-waveforms-fixture

Conversation

@adityasingh2400

Copy link
Copy Markdown

Motivation

Addresses the test-fixture half of #2220. I have deliberately left misc.py and the waveforms_desc docstring alone so this does not conflict with #1936, which is already touching that text.

For the doubly indexed Units.waveforms column, add_unit interprets a 3-D per-unit input as (num_spikes, num_electrodes, num_samples). Dim 0 becomes waveforms_index_index, the spike events of a unit, dim 1 becomes waveforms_index, the waveforms of a spike event with one per electrode, and dim 2 is the samples of each waveform. That matches the schema, which says the waveforms_index column "indexes which waveforms in this column belong to the same spike event for a given unit, where each waveform was recorded from a different electrode" and the waveforms_index_index column "indexes the waveforms_index column to indicate which spike events belong to a given unit".

Both TestUnitsIO and TestUnitsWaveformsOnlyIO label these the other way round, with the outer dimension commented as electrodes and the middle one as spikes. Because of that the fixtures also contradict their own spike_times: TestUnitsIO declares 3 spike_times for each unit but encodes 2 spike events for the first unit and 3 for the second, and TestUnitsWaveformsOnlyIO declares 3 spike_times per unit but encodes 2 spike events. Neither test asserted anything about the waveforms columns, so nothing caught it.

Changes

Both fixtures are reordered to (num_spikes, num_electrodes, num_samples) so the number of spike events matches the number of spike_times, and the comments now describe what add_unit actually does. TestUnitsIO gets distinct sample values, 1 through 54, instead of the repeated [1, 2, 3] rows, so a transposed or mis-flattened write is detectable rather than looking identical.

A new test_waveforms_structure roundtrips the container and asserts waveforms_index_index, waveforms_index, and the 2-D waveforms dataset, plus the per-spike electrode counts. This is the first coverage of the doubly ragged structure of this column.

How to test the behavior?

Running the new assertions against the old fixture shows the inconsistency directly. The old first unit had 3 spike_times but only 2 spike events:

>       np.testing.assert_array_equal(waveforms_index_index.data[:], [3, 6])
E       AssertionError:
E       Arrays are not equal
E        [0]: 2 (ACTUAL), 3 (DESIRED)
E        [1]: 5 (ACTUAL), 6 (DESIRED)
E        ACTUAL: array([2, 5], dtype=uint8)
E        DESIRED: array([3, 6])

FAILED tests/integration/hdf5/test_misc.py::TestUnitsIO::test_waveforms_structure
1 failed, 16 deselected

With the corrected fixtures:

$ pytest tests/integration/hdf5/test_misc.py tests/unit/test_misc.py -q
61 passed, 662 subtests passed in 99.98s

The equivalent minimal repro of the ordering, on dev and unchanged by this PR:

import numpy as np
from pynwb.misc import Units

u = Units(name='units')
u.add_unit(spike_times=[0.], waveforms=np.arange(1 * 5 * 7).reshape(1, 5, 7))
wf = u['waveforms']
print(np.asarray(wf.target.target.data).shape)  # (5, 7)  -> (num_waveforms, num_samples)
print(list(np.asarray(wf.target.data)))         # [5]     -> 5 electrodes for the 1 spike event

Checklist

…tructure

TestUnitsIO and TestUnitsWaveformsOnlyIO labelled the outer dimension of the
3-D waveforms input to add_unit as electrodes and the middle dimension as spike
events. add_unit does the opposite: dim 0 becomes waveforms_index_index (spike
events per unit) and dim 1 becomes waveforms_index (waveforms, one per
electrode, per spike event), which matches the schema description of the
doubly indexed waveforms column.

Because of the swapped labelling the fixtures also contradicted their own
spike_times. TestUnitsIO declared 3 spike_times per unit but encoded 2 and 3
spike events, and TestUnitsWaveformsOnlyIO declared 3 spike_times per unit but
encoded 2 spike events. Neither test asserted anything about waveforms, so this
was never caught.

Reorder both fixtures to (num_spikes, num_electrodes, num_samples), give
TestUnitsIO distinct sample values so a transposition is detectable, and add
test_waveforms_structure asserting waveforms_index_index, waveforms_index, and
the 2-D waveforms dataset after a roundtrip.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant