fix: stop stray BLE notifications from desyncing reads and corrupting config - #155
Merged
Conversation
The notification queue has no request/response correlation, so a single leftover frame is read as the answer to the next command and desyncs every subsequent read by one. `disconnect()` cleared the client and characteristic but left the queue populated, and a BLEConnection is reusable — connect() after a drop keeps the same queue — so residue from a dead link could be returned as the first response on the next link. Flush from all three disconnect paths, only one of which goes through disconnect(): - `_on_disconnect()`, the stack-signalled callback, is the only path that runs on an unexpected drop (device reboot, out of range, proxy reset). Flushes before notifying the owner so it still runs if that callback raises. - `disconnect()` flushes outside its `is_connected` branch: the branch is skipped when the link already died underneath us, and not all bleak backends invoke disconnected_callback for a caller-initiated disconnect. - `_clear_cache_and_drop()` drops the client directly rather than via disconnect(), and a retried connect reuses the queue — an attempt that reached notification setup before failing could hand a frame to the next attempt. Also log each discarded frame's command echo instead of only a count. The echo leads every frame in the clear (encrypted responses carry it ahead of the nonce), so it distinguishes a duplicated response from a frame belonging to an entirely different exchange — a bare count reports that a desync happened but not why.
interrogate() trusted whatever the notification queue handed it. The queue
has no request/response correlation, so a frame from a different exchange —
a duplicated response, or one the firmware held in its TX ring until
notifications were enabled — could land mid-transfer, and nothing rejected
it: strip_command_echo() returns a non-matching frame *unchanged*, so its
2-byte echo was consumed as the chunk-number field and its body appended as
config data. The chunk loop only counts bytes, so this silently overshot
total_length and fed the trailing garbage to the TLV parser, which reported
it as an unrelated parse failure (or skipped it as an unknown packet type).
Screen every config-read frame with the command echo and skip the ones that
belong elsewhere, bounded by MAX_STRAY_CONFIG_FRAMES so a device streaming
unrelated frames fails with a clear error instead of renewing the read
timeout indefinitely. The no-config NACK ({0xFF,0x40,...}) is passed through
rather than skipped: it answers this command, and discarding it would turn a
clear "no stored configuration" error into a timeout.
strip_command_echo() keeps its lenient behavior — callers that cannot
receive foreign frames rely on it — but the mismatch case is now documented
and factored into matches_command_echo() for callers that must screen first.
Also name the command when a frame is dropped, here and in the queue drain.
describe_command_code() resolves the ACK high bit and the 0xFF NACK prefix
back to the originating command, so the frames most worth diagnosing do not
log as unrecognized codes:
Ignoring stray READ_FW_VERSION (0x0043) frame (25 B) while reading config
Discarded 2 stale notification(s) before command: READ_CONFIG (0x0040) ...
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The notification queue (
BLEConnection._notification_queue) is a plain FIFO of raw frames with no request/response correlation —read_response()returns whatever is at the front. One orphaned frame therefore shifts every subsequent read by one, permanently.drain_notifications()mitigates this before each write, but it only clears what has already arrived.Two gaps let stray frames through, both observed in the field:
1. Frames survived a disconnect.
disconnect()cleared_clientand_notification_characteristicbut left the queue populated. ABLEConnectionis reusable (connect()after a drop keeps the same queue), so residue from a dead link could be returned as the first response on the next link. Neither the stack-signalled drop nor the connect-retry teardown flushed either.2. A config read absorbed foreign frames silently.
strip_command_echo()returns a non-matching frame unchanged, so ininterrogate()a frame from another exchange had its 2-byte echo consumed as the chunk-number field and its body appended as config data. The chunk loop only counts bytes, so nothing detected it.Observed downstream, from a single session's logs:
total_length(707) matched the TLV wrapper's own length field; the extra 59 bytes were a stray frame spliced in, and the parser ran off the end of the real config into it. The same root cause surfaces elsewhere as a hard failure, e.g.Firmware version echo mismatch: expected 0x0043, got 0x0040— a leaked config chunk answering the firmware-version read.Changes
Flush the queue on every disconnect path — all three, only one of which goes through
disconnect():_on_disconnect(), the stack-signalled callback, is the only path that runs on an unexpected drop. Flushes before notifying the owner so it still runs if that callback raises.disconnect()flushes outside itsis_connectedbranch — the branch is skipped when the link already died, and not all bleak backends invokedisconnected_callbackfor a caller-initiated disconnect._clear_cache_and_drop()drops the client directly rather than viadisconnect(), and a retried connect reuses the queue.Screen config-read frames by command echo.
interrogate()now skips frames belonging to other exchanges rather than splicing them in, bounded byMAX_STRAY_CONFIG_FRAMESso a device streaming unrelated frames fails with a clear error instead of renewing the read timeout indefinitely. The no-config NACK ({0xFF,0x40,...}) is passed through rather than skipped — it answers this command, and discarding it would turn a clear "no stored configuration" error into a timeout.strip_command_echo()keeps its lenient behavior (callers that cannot receive foreign frames rely on it); the mismatch case is now documented and factored intomatches_command_echo()for callers that must screen first.Name the command when a frame is dropped, at both drop sites.
describe_command_code()resolves the ACK high bit and the0xFFNACK prefix back to the originating command, so the frames most worth diagnosing don't log as unrecognized codes:A bare count reported that a desync happened but not why.
Notes
strip_command_echo()itself, so no other call site is affected — it has only the two ininterrogate().interrogate()still discards the chunk-number field without validating it, and never truncatestlv_datatototal_length, so a correctly-echoed duplicate or overshooting final chunk can still get through. Happy to follow up separately.Test plan
ruff,ruff-format,mypy --strict,pylintclean viaprek run --all-files.