Ai json report and device watch - #15
Open
ashleigh-byte wants to merge 4 commits into
Open
Conversation
Pure schema change with no consumers yet: nmap-derived MAC address, vendor, and hostname data will be tracked per device scan starting in a follow-up PR. Existing databases auto-migrate in place via _migrate_device_scans_columns() so upgrading doesn't require recreating the database.
…ty tracking Builds on the device_scans schema from #<PR1>. Extracts nmap-derived MAC address, vendor, and hostname per device (nmap only resolves MAC/vendor for hosts on the same local subnet it can ARP directly, which the existing sudo ARP scan already covers). Adds get_latest_devices_with_novelty(), which flags devices whose MAC hasn't been seen in any scan within the last 14 days -- devices with no resolvable MAC are never flagged as new, since there's no reliable identity to compare against. No reporting/AI changes yet -- this is data layer only. Surfacing this in the AI report is a separate follow-up PR.
Combines PR #3 (Device Watch reporting) and #4 (AI report restructuring) since #3's third JSON field only makes sense on top of #4's restructured prompt. AI restructuring: the AI now returns only 3 short JSON fields (dynamics_analysis, device_watch, conclusion) instead of a full literal HTML template. Several capable local models (llama3.1:8b, qwen2.5:7b-instruct) reliably abandoned the old rigid template under load and fell back to generic assistant-style summaries; JSON output + code-assembled HTML (REPORT_TEMPLATE_SHELL) guarantees correct, consistent formatting regardless of which model is behind AI_BASE_URL. Malformed or missing JSON falls back to plain non-AI text per field rather than losing the whole report -- unchanged from the existing "AI failures degrade gracefully" behavior. Device Watch reporting: builds on the data layer from #2 -- a vendor-count breakdown of currently-online devices plus a short new-devices list (MACs not seen in the last 14 days) is now included in the AI's user message, so the "device_watch" field can call out anything genuinely new without being fed noisy per-IP detail for every device. Not included here: notifier truncation fixes, alert-then-crash on infra failures, and misc config additions (AI_CONTEXT_SIZE, SLEEP_TIME/ REPORT_CYCLE_COUNT, --test-ai, temperature) -- those are separate PRs.
Updates the Features & Workflow overview, adds a Device Watch section explaining MAC/vendor/hostname tracking and the new-device check, and updates the example detailed report to show the Device Watch field and note that the AI only ever produces short text fields -- the surrounding HTML is assembled in code, not generated by the model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI restructuring: the AI now returns only 3 short JSON fields (
dynamics_analysis,device_watch,conclusion) instead of a full literal HTML template. Several capable local models (llama3.1:8b, qwen2.5:7b-instruct) reliably abandoned the old rigid template under load and fell back to generic assistant-style summaries; JSON output + code-assembled HTML (REPORT_TEMPLATE_SHELL) guarantees correct, consistent formatting regardless of which model is behindAI_BASE_URL. Malformed or missing JSON falls back to plain non-AI text per field rather than losing the whole report — unchanged from the existing "AI failures degrade gracefully" behavior.Device Watch reporting: a vendor-count breakdown of currently-online devices plus a short new-devices list (MACs not seen in the last 14 days) is now included in the AI's user message, so
device_watchcan call out anything genuinely new without being fed noisy per-IP detail for every device.Not included: notifier truncation fixes, alert-then-crash on infra failures, or misc config additions (
AI_CONTEXT_SIZE,SLEEP_TIME/REPORT_CYCLE_COUNT,--test-ai, temperature) — those are separate PRs. Onlymain.pychanges.