Depending on PyNumDiff for differentiation: reduce maintenance burden, gain richer methods

[pavelkomarov]

Continuing the discussion from andgoldschmidt/derivative#55, where @Jacob-Stevens-Haas and I established that (a) variable step size isn't handled correctly in several derivative methods, (b) PySINDy's own SmoothedFiniteDifference could be offloaded to an external library, and (c) the right move may be for PySINDy to look to PyNumDiff rather than extending derivative further.

Making the case on the PySINDy side specifically:

What PySINDy could drop

PySINDy currently maintains its own differentiation submodule (~636 lines across FiniteDifference, SmoothedFiniteDifference, SpectralDerivative, SINDyDerivative) plus carries derivative as a dependency (~729 additional lines). PyNumDiff covers all of this — finite difference, smoothed finite difference, Savitzky-Golay, polynomial fitting, spline, spectral, TVR, Kalman — with a consistent API, correct irregular-step-size handling, NaN support, and multidimensional axis parameter across the board. PySINDy could in principle replace both its own submodule and the derivative dependency with direct calls to PyNumDiff or a thin adapter.

What PySINDy would gain

Beyond consolidation, PyNumDiff now includes methods derivative doesn't have: outlier-robust Kalman smoothing (robustdiff, Huber loss on both measurement and process residuals via CVXPY), and circular/wrapped domain support in rtsdiff (wraps the Kalman innovation to $[-\pi, \pi]$ for angle-valued states).

More importantly for SINDy users: PyNumDiff has a multi-objective hyperparameter optimization framework (pynumdiff.optimize) that balances data fidelity against derivative smoothness when ground-truth derivatives are unavailable — which is essentially always in real SINDy applications. The smoothness weight can be initialized from the signal's estimated cutoff frequency. This is exactly the knob SINDy users struggle to set by hand.

Integration path

BaseDifferentiation is already pluggable. The minimal adapter:

class PyNumDiffDerivative(BaseDifferentiation):
    def __init__(self, method, **kwargs):
        self.method = method
        self.kwargs = kwargs

    def _differentiate(self, x, t):
        _, x_dot = self.method(x, t if len(t) > 1 else t[1] - t[0], **self.kwargs)
        return x_dot

Or users bypass differentiation entirely by passing x_dot= to fit() after running PyNumDiff themselves — which is already an underappreciated pattern for getting best-in-class derivative estimates into a SINDy fit.

Happy to discuss where this should live (ideally PyNumDiff as a pysindy dependency, in my view). @Jacob-Stevens-Haas, @florisvb, @eigensteve tagging all of you since you should all be part of the broader conversation. There was talk of a big redesign anyway, a SINDy 3.0, which might be a good opportunity to harness the power of PyNumDiff and offload some code, thinking, and maintenance work.

— written largely by Claude Code to integrate all this info, with edits and color by me, the human

[pavelkomarov]

For concreteness, here's a method-by-method comparison:

Method	derivative pkg	PyNumDiff	Notes
Finite difference	✓	✓	PyNumDiff: variable step size, higher-order stencils
Savitzky-Golay / polynomial fit	✓	✓	PyNumDiff: derives directly from polynomial, not via FD
Spline	✓	✓	PyNumDiff: NaN support, variable step
Spectral / FFT	✓	✓	PyNumDiff: mirror-padding for aperiodic signals
Total variation regularization	✓	✓	PyNumDiff: orders 1–3 (velocity, acceleration, jerk)
Kalman / RTS smoothing	✓	✓	PyNumDiff: variable step, missing data, circular domains
Kernel / RBF	✓	✓	PyNumDiff: explicit sparsity for efficiency
Robust Kalman (Huber loss)	✗	✓
Local linear model	✗	✓
Hyperparameter optimization	✗	✓	Critical for SINDy without ground-truth derivatives

— Claude Code