Make date filtering for get_observations more inclusive

[taldcroft]

Description

Currently the date filtering for get_observations and get_starcats is the intersection of the supplied start and stop interval with the intervals obs_start to obs_stop for each observation. The obs_start/stop represent the time in nominally in Kalman tracking stars. This definition is not so convenient in practice, for instance something this gives a surprising and unhelpful result:

>>> import kadi.commands as kc
>>> obs = kc.get_observations(obsid=8008)[0]
>>> date = obs["starcat_date"]
>>> kc.get_starcats(date, date)
[]

This PR changes the filtering to go from manvr_start to obs_stop. This is going to return observations that are more in line with intuition in which the maneuver leading up to a NPNT dwell is all part of the observation.

Interface impacts

Date-based queries that start or end within a maneuver may return additional observations relative to the previous version.

Testing

Unit tests

• Mac

(ska3) ➜  kadi git:(more-inclusive-get-observations) pytest                         
================================================= test session starts =================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: anyio-4.12.1, timeout-2.4.0
collected 308 items                                                                                                   

kadi/commands/tests/test_commands.py ..........................................................s............... [ 24%]
...............                                                                                                 [ 28%]
kadi/commands/tests/test_commands_v2.py sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 51%]
sssssssssssssssss                                                                                               [ 57%]
kadi/commands/tests/test_filter_events.py ..                                                                    [ 58%]
kadi/commands/tests/test_states.py ...............................................x..........................   [ 82%]
kadi/commands/tests/test_validate.py ......................                                                     [ 89%]
kadi/tests/test_events.py ..........                                                                            [ 92%]
kadi/tests/test_occweb.py .......................                                                               [100%]

=============================== 218 passed, 89 skipped, 1 xfailed in 112.09s (0:01:52) ================================
(ska3) ➜  kadi git:(more-inclusive-get-observations) git rev-parse --short HEAD     
1057dc3

Commands v2

(ska3) ➜  kadi git:(more-inclusive-get-observations) git rev-parse --short HEAD                              
b554de3
(ska3) ➜  kadi git:(more-inclusive-get-observations) env KADI_CMDS_VERSION=2 pytest                          
================================================ test session starts ================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/aldcroft/git
configfile: pytest.ini
plugins: anyio-4.12.1, timeout-2.4.0
collected 308 items                                                                                                 

kadi/commands/tests/test_commands.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 23%]
sssssssssssssssss                                                                                             [ 28%]
kadi/commands/tests/test_commands_v2.py ..................................................................... [ 51%]
...................                                                                                           [ 57%]
kadi/commands/tests/test_filter_events.py ..                                                                  [ 58%]
kadi/commands/tests/test_states.py ...............................................x.......................... [ 82%]
                                                                                                              [ 82%]
kadi/commands/tests/test_validate.py ......................                                                   [ 89%]
kadi/tests/test_events.py ..........                                                                          [ 92%]
kadi/tests/test_occweb.py .......................                                                             [100%]

============================== 218 passed, 89 skipped, 1 xfailed in 109.86s (0:01:49) ===============================

Independent check of unit tests by Javier

• OSX

(ska3-flight) ~/SAO/git/kadi more-inclusive-get-observations $ git rev-parse HEAD
b554de31abe2c09344f366ac9d00e844d8a4395f
(ska3-flight) ~/SAO/git/kadi more-inclusive-get-observations $ pytest kadi       
============================================================ test session starts =============================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/javierg/SAO/git
configfile: pytest.ini
plugins: anyio-4.12.1, timeout-2.4.0
collected 308 items                                                                                                                          

kadi/commands/tests/test_commands.py sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss         [ 28%]
kadi/commands/tests/test_commands_v2.py ........................................................................................       [ 57%]
kadi/commands/tests/test_filter_events.py ..                                                                                           [ 58%]
kadi/commands/tests/test_states.py ...............................................x..........................                          [ 82%]
kadi/commands/tests/test_validate.py ......................                                                                            [ 89%]
kadi/tests/test_events.py ..........                                                                                                   [ 92%]
kadi/tests/test_occweb.py .......................                                                                                      [100%]

=========================================== 218 passed, 89 skipped, 1 xfailed in 131.42s (0:02:11) ===========================================
(ska3-flight) ~/SAO/git/kadi more-inclusive-get-observations $ export KADI_CMDS_VERSION=3
(ska3-flight) ~/SAO/git/kadi more-inclusive-get-observations $ pytest kadi               
============================================================ test session starts =============================================================
platform darwin -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/javierg/SAO/git
configfile: pytest.ini
plugins: anyio-4.12.1, timeout-2.4.0
collected 308 items                                                                                                                          

kadi/commands/tests/test_commands.py ..........................................................s..............................         [ 28%]
kadi/commands/tests/test_commands_v2.py ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss       [ 57%]
kadi/commands/tests/test_filter_events.py ..                                                                                           [ 58%]
kadi/commands/tests/test_states.py ...............................................x..........................                          [ 82%]
kadi/commands/tests/test_validate.py ......................                                                                            [ 89%]
kadi/tests/test_events.py ..........                                                                                                   [ 92%]
kadi/tests/test_occweb.py .......................                                                                                      [100%]

=========================================== 218 passed, 89 skipped, 1 xfailed in 132.83s (0:02:12) ===========================================

Functional tests

No functional testing.

[javierggt]

I totally agree with this change.

I wonder about the commands v2/v3 update. I think this change should work with commands v2, right? If I use KADI_CMDS_VERSION=2 I get a failure:

___________________________________________ test_get_observations_start_stop_inclusion ___________________________________________

    def test_get_observations_start_stop_inclusion():
        # Covers time from the middle of obsid 8008 to the middle of obsid 8009
        obss = get_observations("2007:002:05:00:00", "2007:002:20:00:01", scenario="flight")
        assert len(obss) == 2
    
        # One second in the middle of obsid 8008
        obss = get_observations("2007:002:05:00:00", "2007:002:05:00:01", scenario="flight")
        assert len(obss) == 1
    
        # During a maneuver
        obss = get_observations("2007:002:18:05:00", "2007:002:18:08:00", scenario="flight")
>       assert len(obss) == 0
E       AssertionError: assert 1 == 0
E        +  where 1 = len([{'manvr_start': '2007:002:18:04:39.216', 'npnt_enab': True, 'obs_start': '2007:002:18:08:39.216', 'obs_stop': '2007:003:07:12:50.440', ...}])

kadi/commands/tests/test_commands_v2.py:1056: AssertionError
==================================================== short test summary info =====================================================
FAILED kadi/commands/tests/test_commands_v2.py::test_get_observations_start_stop_inclusion - AssertionError: assert 1 == 0

I'm guessing it's just that the test needs to be updated as well.

[taldcroft]

@javierggt - I fixed the test and documented the v2 testing.