Device missing alerting - #22
Open
ashleigh-byte wants to merge 4 commits into
Open
Conversation
Pure schema change with no consumers yet: nmap-derived MAC address, vendor, and hostname data will be tracked per device scan starting in a follow-up PR. Existing databases auto-migrate in place via _migrate_device_scans_columns() so upgrading doesn't require recreating the database.
…ty tracking Builds on the device_scans schema from #<PR1>. Extracts nmap-derived MAC address, vendor, and hostname per device (nmap only resolves MAC/vendor for hosts on the same local subnet it can ARP directly, which the existing sudo ARP scan already covers). Adds get_latest_devices_with_novelty(), which flags devices whose MAC hasn't been seen in any scan within the last 14 days -- devices with no resolvable MAC are never flagged as new, since there's no reliable identity to compare against. No reporting/AI changes yet -- this is data layer only. Surfacing this in the AI report is a separate follow-up PR.
The counterpart to the existing new-device novelty check: alert when a device that's normally reliably online (a NAS, server, etc.) vanishes, and again when it comes back. - DB.get_missing_devices() returns MACs seen in at least DEVICE_MISSING_RELIABILITY of scans over DEVICE_MISSING_LOOKBACK_DAYS (default 80% over 3 days) that are absent from the latest scan. Requires a minimum amount of history before evaluating at all, to avoid false positives on a freshly started netmon. - New per-MAC state machine in main.py mirrors the outage-alerting pattern: alerts once after DEVICE_MISSING_CONSECUTIVE_READINGS consecutive missing cycles (default 2), then again on reappearance with how long it was gone. Important fix found via testing: reliability is computed over a window that excludes the most recent few prior scans (not just the latest). Without this, a device's own ongoing disappearance drags its historical reliability score down cycle over cycle -- by the time enough consecutive-missing cycles had passed to actually confirm and alert on it, its reliability would already have silently fallen below the threshold, and it would stop being flagged right when the alert should fire. Caught this with an integration test scripting a real multi-cycle disappearance through the actual main() loop, not just unit tests of the query in isolation. Test plan: - Unit-tested get_missing_devices: reliable device flagged as missing, insufficient-history guard, sporadic (low-reliability) device never flagged, still-present device never flagged, and the self-dilution fix specifically (still correctly flagged on the 2nd consecutive gap cycle, not just the 1st) - Unit-tested DEVICE_MISSING_* config validation (defaults, overrides, boundary value 1.0, out-of-range/negative rejected) - Full integration test: stubbed Runner/AI/notifier, real in-memory sqlite.DB, ran the actual main() loop through a scripted 12-cycle sequence (8 stable cycles, 2-cycle gap, reappearance, 1 more stable cycle), asserting exactly one missing alert and exactly one reappeared alert
Adds a Missing-Device Alerting section explaining the reliability window and consecutive-check threshold, plus the three new env vars.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Known-device-missing alerting. Testing caught a real bug before it shipped: a device's own ongoing absence was diluting its historical reliability score, so it would silently stop being flagged right as the alert should fire — fixed by excluding recent scans from the reliability window. Verified with a full scripted run through the actual main() loop.