You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A parameter reaches the simulation and may still have no effect on the compared variable, and the search then spends a dimension and its share of the population on it:
x, the flow recession coefficient, enters only the routed runoff arn (_dynamic_model.py#L535-L538); a calibration with --variable rnf searches it for nothing.
A candidate can fail for a reason every candidate shares that the pre-checks cannot see (under the zones aggregation the station ids are not known before the run; a MODFLOW setup that never converges), and with the defaults the search spends 128 runs before the first generation line says so.
That the nine parameters are applied is guaranteed structurally, not by a check: the derived document writes them all (_worker.py#L286-L328), the format 1.0 model is strict with every field required, and the model reads each of them. What nothing says before the search is whether each one moves the compared series.
Proposed solution
rubem calibrate --sensitivity-check (sensitivity_check() in the runner, next to calibrate()) runs, in the pool of the calibration, the reference configuration (the starting point of the search) and one run per free parameter with that parameter moved by a fraction of its bound (a quarter of the range, toward the side with room), clipped to the bound and checked with is_admissible (moving w_1 up can leave w_3 negative; the weights are moved with w_3 derived). N + 1 runs, in parallel; the MODFLOW parameters named in --bound (Add the optional MODFLOW groundwater coupling #357) join like the others.
Each moved run is compared with the reference on the series the objective compares (aligned steps, the stations of the objective): maximum absolute difference and its value relative to the reference, plus the NSE of both. A parameter whose series is identical is reported as having no effect; a run that fails is reported with its error and does not end the check.
sensitivity.json in the run directory: the reference (parameters, NSE, station metrics), one entry per parameter (reference value, moved value, maximum absolute difference, relative difference, NSE, error), and the lists without_effect and failed. One log line per parameter and a closing warning naming the parameters without effect, with the advice to --fix them.
The command stops after the check (exit 0) and writes nothing else; the search is a separate invocation with the --fix the check suggested.
Tests that prove it: on the synthetic dataset x has no effect on rnf and an effect on arn; in a coupled configuration alpha_gw has no effect; the moved values are admissible; a moved run that fails is recorded and the check ends normally; sensitivity.json is written and the command exits 0.
Documentation: a "Sensitivity check" section in the calibration page, before "The search"; changelog.
A Morris (elementary effects) screening with several trajectories. More runs and more information; not needed for the question "does this parameter move the series at all".
A runtime assertion in the worker that the loaded configuration carries the candidate's parameters. Left out: the strict schema already refuses a wrong or missing key, so the assertion would only guard against a future format change.
Additional context
Part of the calibrator robustness work assessed on 2026-09-24. Independent of #358, #359, #360 and #361.
Description
A parameter reaches the simulation and may still have no effect on the compared variable, and the search then spends a dimension and its share of the population on it:
x, the flow recession coefficient, enters only the routed runoffarn(_dynamic_model.py#L535-L538); a calibration with--variable rnfsearches it for nothing.alpha_gwhas no effect; the documentation of the coupling tells the user to--fixit by hand.zonesaggregation the station ids are not known before the run; a MODFLOW setup that never converges), and with the defaults the search spends 128 runs before the first generation line says so.That the nine parameters are applied is guaranteed structurally, not by a check: the derived document writes them all (_worker.py#L286-L328), the format 1.0 model is strict with every field required, and the model reads each of them. What nothing says before the search is whether each one moves the compared series.
Proposed solution
rubem calibrate --sensitivity-check(sensitivity_check()in the runner, next tocalibrate()) runs, in the pool of the calibration, the reference configuration (the starting point of the search) and one run per free parameter with that parameter moved by a fraction of its bound (a quarter of the range, toward the side with room), clipped to the bound and checked withis_admissible(movingw_1up can leavew_3negative; the weights are moved withw_3derived). N + 1 runs, in parallel; the MODFLOW parameters named in--bound(Add the optional MODFLOW groundwater coupling #357) join like the others.sensitivity.jsonin the run directory: the reference (parameters, NSE, station metrics), one entry per parameter (reference value, moved value, maximum absolute difference, relative difference, NSE, error), and the listswithout_effectandfailed. One log line per parameter and a closing warning naming the parameters without effect, with the advice to--fixthem.--fixthe check suggested.Tests that prove it: on the synthetic dataset
xhas no effect onrnfand an effect onarn; in a coupled configurationalpha_gwhas no effect; the moved values are admissible; a moved run that fails is recorded and the check ends normally;sensitivity.jsonis written and the command exits 0.Documentation: a "Sensitivity check" section in the calibration page, before "The search"; changelog.
Alternative solutions
Additional context
Part of the calibrator robustness work assessed on 2026-09-24. Independent of #358, #359, #360 and #361.