Release 4.10.12a1 - #187
Open
github-actions[bot] wants to merge 132 commits into
Open
Conversation
* feat(tests): add hivescope e2e skeleton Bootstraps tests/e2e/ with a hivescope-backed smoke test so the harness wiring is exercised end-to-end. Greenfield protocol/ACL/relay coverage will follow on top of this scaffold. * ci: run hivescope e2e tests in build workflow * test(e2e): add ACL + connection lifecycle suites - test_acl.py: blacklisted message types are dropped before delivery - test_broadcast.py: peer registration/deregistration on disconnect Pin hivescope to @dev so consumers pull the absorbed library. * ci: retrigger after hivescope dep fix * ci: retrigger after hivescope websockets dep added * ci: add standard GitHub Actions workflows AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: add missing CI/CD workflows and refresh stale ones - Impact: added coverage.yml, license_check.yml, pip_audit.yml, release-preview.yml, repo-health.yml; refreshed build_tests.yml, publish_stable.yml, release_workflow.yml to use OpenVoiceOS/gh-automations@dev; preserved e2e_tests job (pytest tests/e2e/ with hivescope) - Verified via: reviewed all templates and existing workflow content * build: include requirements.txt in sdist via MANIFEST.in setup.py reads requirements.txt at install time; without MANIFEST.in the file isn't shipped in the source tarball, so pip install from sdist fails with FileNotFoundError under the new build-tests workflow. * test: hoist pytest_plugins to repo-root conftest pytest 8 forbids 'pytest_plugins' in non-top-level conftest.py — moving the hivescope plugin declaration to the repo root keeps it active for both the full test matrix and the standalone tests/e2e/ runs. * test: only load hivescope plugin when hivescope is installed Coverage and unit-only workflows that don't install hivescope shouldn't fail at conftest import. Wrap the pytest_plugins declaration in a try/import so it's a no-op without hivescope. * ci: temporarily pin bus-client to feat/hivescope-e2e branch Until JarbasHiveMind/hivemind-websocket-client#112 (the protocol fix for handle_hello/handshake/bus assertions) lands and a release ships, HiveMind-core's CI must use the feature-branch version of the bus-client to validate end-to-end behaviour. Revert this pin once bus-client makes a release with the fix included. Replaces the reusable matrix job with an inline matrix so the pre-install step can run before the test extras. * ci: install pytest explicitly (no [test] extra in setup.py) * ci: skip pre-existing dead unit tests (HiveNodeClient, JsonDB) test_bus.py and test_db.py reference HiveNodeClient/JsonDB which no longer exist in current hivemind-bus-client / hivemind-core. They were already broken on dev; ignoring here so the e2e PR's CI can go green. Tracked separately. * test: remove dead unit tests superseded by tests/e2e/ test_bus.py imports jarbas_hive_mind.utils.emulation.FakeMycroft and HiveNodeClient — both gone. The scenarios it sketched (master/mid-relay/ end-satellite with bus/escalate/broadcast/propagate) are now properly covered by the hivescope-driven tests/e2e/ suite using TopologyBuilder. test_db.py imports JsonDB, RedisDB, and cast2client directly from hivemind_core.database, but those moved to the hivemind_plugin_manager plugin architecture. Database behaviour is exercised end-to-end through the e2e suite (every test creates an InMemoryClientDatabase via MasterNode.create). Both files were already broken on dev; the modern equivalents make them redundant. * ci: drop unit-test step (legacy tests removed; e2e covers behaviour) * ci: revert to shared reusable workflow with pre_install_pip Now that gh-automations#18 has landed, build-tests.yml and coverage.yml both expose a pre_install_pip input. Use it to preinstall the bus-client feature branch and drop the hand-rolled inline matrix. * ci: point coverage at tests/e2e and exclude hivemind-core from license check - coverage: legacy test/ dir was removed; coverage now runs the e2e suite. Pre-install hivescope alongside the bus-client feature pin. - license_check: exclude_packages '^(hivemind-core|hivemind_core)$' so the package's own AGPL listing doesn't trip the audit. * ci: route e2e tests through shared build-tests workflow * ci: collapse e2e job into build job — tests are tests * ci: publish_stable triggers on push to master, not on PR-merge Matches the skill template and every other repo. * ci(license_check): simplify exclude regex to single package name * build: switch to pyproject.toml; drop setup.py / requirements.txt / MANIFEST.in Move package metadata to pyproject.toml using setuptools.build_meta. Adds a __version__ string in version.py so the dynamic version attr can resolve to a single value. * test(e2e): address CodeRabbit review - Move start_all() inside the try-block so partial-start failures still call stop_all() in the finally clause. - ACL tests: poll for delivery (or sleep, in the negative case) instead of asserting immediately after send_to_satellite, so the result is deterministic regardless of dispatch timing. * test(e2e): expand protocol coverage - test_handshake_auth.py: registered key handshakes; session_id assigned; multi-satellite handshake completion. - test_broadcast_routing.py: send_to_satellite is targeted (siblings don't receive); send_to_all reaches every connected peer. - test_bus_routing.py: satellite-injected BUS message reaches master agent bus; session_id is preserved in the master-side bus context. All use hivescope's single_satellite / admin_satellite / three_satellites preset scenarios, focused on master-side protocol behaviour. * ci(license): broaden exclude regex (drop end anchor; both name variants) * ci(license): use (?i)^hivemind[-_]core.* exclude pattern * build: depend on hivemind-bus-client>=0.7.0a2 (released to PyPI) Drop the temp git-URL pin in CI now that the protocol fix is shipped. * ci: retrigger after PyPI cache refresh for 0.7.0a2 * ci: drop 'secrets: inherit'; pass PYPI_TOKEN/MATRIX_TOKEN explicitly only where needed
* fix: preserve client session pipeline * fix: address session pipeline review * fix: address bus message review * test(e2e): pin session-pipeline fix with hivescope (stacked on #79) (#82) * test(e2e): pin session-pipeline preservation via hivescope End-to-end coverage mirroring tests/test_session_pipeline.py through the real master/satellite stack: - no pipeline in payload is not invented from core config - explicit pipeline list is preserved end-to-end - explicit None pipeline is preserved - agent_bus_callback fires exactly once per BUS message (regression guard for the duplicate dispatch removed in this PR) - stale master-side pipeline is not reattached by _update_blacklist Verified by running against origin/dev (pre-fix): 4/5 fail; against this branch: 5/5 pass. * test(e2e): cover session_id='default' admin gate - non-admin client sending session_id='default' is disconnected - non-admin payload with no session at all (defaults to 'default') is disconnected - admin client sending session_id='default' is allowed and the message reaches the agent bus These guard the security branch in handle_bus_message that lives alongside the pipeline preservation logic. --------- Co-authored-by: JarbasAI <33701864+JarbasAl@users.noreply.github.com>
* Add client metadata option
* refactor: hard-require hivemind-plugin-manager>=0.5.0 for Client.metadata
0.5.0 ships Client.metadata, so the runtime feature-detection dance is
no longer needed. Simplifies the implementation and the tests.
Changes:
- pyproject.toml: bump hivemind-plugin-manager>=0.3.0 → >=0.5.0
- database.py: drop _client_supports_metadata(), CLIENT_SUPPORTS_METADATA,
METADATA_SUPPORT_REQUIRED, the RuntimeError guard, and the dict-unpack
workaround for the constructor. metadata is now passed as a normal
kwarg defaulting to {}.
- tests/test_client_metadata.py: rewritten and expanded
- drop tests for the removed fallback/rejection paths
- drop @skipif markers (metadata always supported now)
- parse_client_metadata: added empty-object, top-level
string/number/array rejection, nested-structure cases
- add_client: default-to-empty, no-wipe-on-update, dict-copy isolation
- CLI end-to-end via click.testing.CliRunner: happy path with metadata
flowing through to the DB, malformed-JSON rejection, top-level array
rejection, and absence of 'Metadata:' line when flag omitted
18 metadata tests, 44 total in suite, all passing.
* style: revert unrelated print/click.echo and f-string churn
The PR was scoped to adding --metadata, but had also converted two
'Password:' lines to click.echo while leaving every other print() in
the same blocks alone — resulting in a stylistic mix with no behavior
change. Revert to print() to match the surrounding code; if a
print->click.echo migration is wanted, it should be a separate scoped
change for both files.
---------
Co-authored-by: JarbasAi <jarbasai@mailfence.com>
* chore(license): replace LICENSE file with Apache-2.0 text * chore(license): update pyproject.toml to Apache-2.0 (PEP 639) * chore(license): switch source headers to SPDX Apache-2.0 * docs(license): remove COMMERCIAL-TERMS, consolidate trademark policy Under Apache-2.0 there is no copyleft to grant exemptions from. Trademark protection remains as the brand-licensing lever; consolidate all brand/trademark guidance in TRADEMARK-USAGE.md. * docs(readme): rewrite license & contribution sections for Apache-2.0 - Drop dual-license/commercial-track framing; HiveMind-core returns to Apache-2.0, matching the rest of the HiveMind ecosystem. - Reopen external contributions (Apache-2.0 inbound=outbound, no CLA needed). - Point brand-use questions at TRADEMARK-USAGE.md. - Replace the v4.0 'commercial notice' banner; note Apache-2.0 / AGPL-3.0 historical releases inline in the License section. * ci: update license_check workflow comment for Apache-2.0 * docs(changelog): record AGPL->Apache-2.0 relicense * fix(pyproject): drop License trove classifier (PEP 639 conflict) setuptools rejects pyproject.toml that has both an SPDX 'license' expression and a 'License :: OSI Approved :: ...' trove classifier under PEP 639 — they are mutually exclusive. The SPDX expression is authoritative; drop the redundant classifier.
HiveMind-core was the only repo in the family without a lint workflow. Matches the pattern used by hivemind-plugin-manager, hivemind-ovos-agent-plugin, hivemind-sqlite-database, hivemind-redis-database, and hivemind-json-db-plugin.
* feat: policy admission chain runner Implements Phase 2 of the policy plugin architecture spec (#85). Consumes the primitives shipped in hivemind-plugin-manager#27 (PolicyPlugin / Verdict / Mutation). - hivemind_core/policy.py: PolicyChain — runs review/review_binary/observe hooks in order, applies mutations, short-circuits on deny, fails closed on policy exceptions (with opt-in fail_open). - hivemind_core/protocol.py: chain is built from server config in HiveMindListenerProtocol.__post_init__; review hook before bus.emit in handle_inject_agent_msg; review_binary hook in handle_binary_message; observe hook after successful emit; new hive.policy.denied response message sent back to client on deny. - hivemind_core/config.py: new "policy" config block (chain + fail_open). - tests/test_policy_chain.py: 16 unit tests covering allow/deny/mutation/ fail-closed/fail-open paths, from_config loading, binary admission, observe error-swallowing. No built-in policies in this PR. The migration of static ACL enforcement (_update_blacklist + authorize) into a built-in ClientACLPolicy lives in the follow-up PR (Phase 3 of #85). * feat(policy): ClientACLPolicy + OVOSAgentPolicy default chain + e2e Phase 3 of the policy admission chain rollout (HiveMind-core#85), folded into the same PR as Phase 2. What's new: - hivemind_core/policy.py: ClientACLPolicy — built-in policy enforcing the per-client allowed_types whitelist. Replaces the static check that used to live in HiveMindClientConnection.authorize. Pure admission decision; no mutations, no DB sync. Wired as "hivemind-core-acl-policy" entry point. - hivemind_core/config.py: default policy chain is now [ClientACLPolicy, OVOSAgentPolicy]. The OVOS entry is conditional (skipped when the agent plugin isn't installed) but ships in the default config because hivemind-ovos-agent-plugin is the default agent_protocol. - hivemind_core/protocol.py: connection-level msg_blacklist / skill_blacklist / intent_blacklist fields preserved on HiveMindClientConnection for backwards compat (populated by OVOSAgentPolicy from Client.metadata at admission time). Outbound send()-side msg_blacklist filter restored. authorize() reduced to a default-True subclass hook. _update_blacklist stripped to session rewrite only — skill/intent injection moved to OVOSAgentPolicy. - hivemind_core/database.py: add_client(...) accepts the legacy intent_blacklist/skill_blacklist/message_blacklist kwargs and folds them into Client.metadata so CLI/external callers using the old signature keep working. - hivemind_core/scripts.py: 4 deprecated CLI commands restored (blacklist-skill / allow-skill / blacklist-intent / allow-intent). Each emits a deprecation echo on stderr and writes through Client.metadata rather than the deprecated top-level fields. - .github/workflows/build_tests.yml + coverage.yml: temporary pre_install_pip pins to plugin-manager#27 and ovos-agent-plugin#3 branches so the new entry points resolve in CI. Mark in PR body for removal after both upstream PRs release. Tests: - tests/test_policy_chain.py: 21 unit tests (16 chain + 5 TestClientACLPolicy). - tests/e2e/test_policy_admission.py: 7 new e2e tests via hivescope covering admission deny + hive.policy.denied notification, allow with mutation injection, empty-chain bypass, db.sync() picking up mid-session DB changes, and the outbound-filter backwards-compat invariant. - tests/e2e/test_acl.py: existing outbound-msg_blacklist tests still pass after the refactor. Requires hivemind-plugin-manager>=0.6.0a1 (the version that ships the Phase 1 primitives + metadata migration shims) and hivemind-ovos-agent-plugin>=0.2 (OVOSAgentPolicy). * fix(policy): install DenyAllPolicy fallback under fail_open=false The previous __post_init__ fallback installed an empty PolicyChain when PolicyChain.from_config raised — silently inverting an operator's fail_open=false (deny-by-default) intent into allow-everything. Now: - fail_open=false (default): install a DenyAllPolicy that rejects every inbound message + binary payload with code="policy_chain_unavailable", so clients see a loud signal and the broken chain can't accidentally permit traffic. - fail_open=true: install an empty allow-all chain as before. Tests cover both branches: TestDenyAllPolicy, TestFromConfigUnknownPlugin (unit), and a new e2e test that proves a misconfigured chain produces hive.policy.denied at the client and no message reaches the agent bus. Audit finding flagged at hivemind_core/protocol.py:246-252. * refactor(policy): address audit findings on PR #89 Follow-up to the policy admission chain rollout addressing the audit findings (no functional regressions; all 79 tests pass): - Drop dead caches: HiveMindClientConnection.skill_blacklist and intent_blacklist were write-only state. Only msg_blacklist is read (in send()), so the other two go. - Fix inverted-dependency on outbound msg_blacklist: previously only populated when the client first sent an inbound message (via OVOSAgentPolicy.review). Now populated at handle_new_client from Client.metadata["message_blacklist"], so the outbound filter is correct from t=0. OVOSAgentPolicy still refreshes it on each review to pick up mid-session DB changes. - OVOSAgentPolicy: stop writing skill_blacklist/intent_blacklist to the live connection — those flow through AddBlacklistedSkill / AddBlacklistedIntent mutations onto message.context["session"] and nowhere else (no leaky side-channel). - Rename _update_blacklist -> _install_client_session. The old name was misleading after the strip: it doesn't touch blacklists anymore. - New unit tests: TestDenyAllPolicy, TestFromConfigUnknownPlugin (fail- closed contract), legacy-constructor-kwarg migration, metadata-wins- over-legacy-kwarg conflict, stacklevel correctness. - New e2e tests: policy_error fail-closed path produces hive.policy.denied with code=policy_error; review_binary deny short- circuits the binary handler and notifies the client. - CI: pin hivescope to feat/drop-dead-acl-caches branch (companion hivescope#4) so its HiveMindClientConnection construction matches the new field set. Drop once hivescope#4 releases. * refactor(policy): drop msg_blacklist + fail_open; force always-on ClientACLPolicy Three architectural cleanups landing together: 1. **Drop msg_blacklist entirely.** The connection-level field, the outbound send()-side filter, and the handle_new_client population are all gone. The model is whitelist-only via allowed_types: deny by default, grant explicitly. There is no message blacklist anywhere in hivemind-core. 2. **Force always-on ClientACLPolicy.** The whitelist enforcement is not a configurable plugin — HiveMindListenerProtocol.__post_init__ always prepends ClientACLPolicy to the chain, dedup'ing if an operator listed it explicitly. The config key `policy.chain[].module = "hivemind-core-acl-policy"` is no longer needed (and is silently dedup'd). Operators cannot disable the allowed_types check. ClientACLPolicy gains an admin bypass — clients with `is_admin=True` (operator-controlled) skip the whitelist by definition. 3. **Drop fail_open.** The chain is unconditionally fail-closed: any exception in review/review_binary or in a mutation's apply becomes Verdict.deny("policy_error", ...). No operator knob. The hivemind bus is unauthenticated and private; lenient-on-error behaviour would be a security footgun. A policy that wants to swallow its own errors can do so internally and return Verdict.allow(). Other changes: - Default config drops both the fail_open knob and the hivemind-core-acl-policy chain entry (always-on now). - tests/e2e/test_acl.py deleted — tested the dropped outbound filter. - test_policy_admission e2e: drop the outbound-filter test, drop the empty-chain-passes-everything test, add test_empty_allowed_types_ denies_all to lock in deny-by-default. - New TestClientACLPolicy::test_admin_bypasses_whitelist. Requires hivemind-plugin-manager>=0.6.0a2 (deny-by-default Client) and hivemind-ovos-agent-plugin (no message_blacklist read). * ci: relax hivemind-plugin-manager floor to 0.5.0; add dev extras alias Two CI fixes: 1. Lower the hivemind-plugin-manager floor from >=0.6.0a1 to >=0.5.0 while #27 is in CI as a branch install. The gh-automations bump bot only updates version.py on merge to dev, so feature branches report the pre-bump version (0.5.0). Tighten the floor again post-release. 2. Add `[project.optional-dependencies] dev` alias. The upstream gh-automations workflow first attempts `pip install -e ".[dev]"` and falls through to a buggy `pip install test` literal branch when [dev] is missing — caused build_tests/coverage/pip_audit/ license_check to all fail. * docs: sync README after policy-admission-chain rollout (#85) AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: keep docs accurate after policy chain runner (PR #89) - Impact: updated README.md; added policy admission chain section with citations; updated default config block to include policy.chain; corrected add-client deny-by-default behaviour (was: "some messages allowed by default"); marked blacklist-skill/allow-skill/ blacklist-intent/allow-intent as deprecated in commands table and detail section; added make-admin/revoke-admin/allow-escalate/ blacklist-escalate/allow-propagate/blacklist-propagate to commands table; documented hive.policy.denied response format - Verified via: manual review against hivemind_core/policy.py, hivemind_core/protocol.py, hivemind_core/config.py, hivemind_core/scripts.py Ref: #85 * fix: audit follow-ups + CI green CI fixes: - Pre-pin hivemind-plugin-manager@feat/policy-plugins in ovos-agent-plugin workflows (ImportError: Mutation not yet on PyPI). - HPM: drop pkg_resources fallback in _iter_entrypoints (use stdlib importlib.metadata; pkg_resources isn't shipped on Python 3.12+ without an explicit setuptools dep). Fixes the 3.12/3.13 ModuleNotFoundError. Second-pass audit follow-ups: - ClientACLPolicy.review now refreshes is_admin + allowed_types from the DB on each call, so `revoke-admin` / `allow-msg` / `blacklist-msg` take effect mid-session without forcing a reconnect. Falls back to the cached connection values if the DB lookup fails. - get_server_config() back-compat for legacy installs: strips the removed `policy.fail_open` key with a loud LOG.warning, and seeds the default `policy.chain` when missing. Non-dict policy block is coerced to default. - `hivemind-core add-client` prints a hint when the resulting client has empty allowed_types ("this client will be denied on every message; run allow-msg ..."). - Stale docstrings cleaned: OVOSAgentPolicy module doc + test_policy_chain module doc. New unit tests: - test_policy_chain: ClientACLPolicy DB-refresh paths (revoke takes effect immediately, promote takes effect immediately, DB failure falls back to cached values). - test_config (new file): policy.fail_open is stripped, missing chain gets seeded with the default, non-dict policy is replaced. Audit findings not blocking and deferred: Client.__init__ monkey-patch (works, breaks IDE signature inspection only), pre-existing test_session_pipeline fragility on dev (CI already failing there before this PR). * feat(policy): chain enforces BYPASS_ADMIN; move default-session check out Two changes: 1. **Chain-runner admin bypass.** PolicyChain.review and review_binary skip policies whose class declares `BYPASS_ADMIN = True` when `client.is_admin` is truthy. Built-in ClientACLPolicy and OVOS plugin's OVOSAgentPolicy both opt in. The previous inline admin check inside ClientACLPolicy.review is removed — the policy now just enforces allowed_types unconditionally; the chain decides whether to invoke it for a given client. 2. **Default-session check moved to OVOSAgentPolicy.** handle_bus_message no longer disconnects non-admin clients on `session_id == "default"` payload injection. The check lives in OVOSAgentPolicy.review as a `Verdict.deny("session_id_default_forbidden", ...)`. Result: clean `hive.policy.denied` response instead of a connection drop. The HELLO-time default-session check stays in handle_hello_message (connection-establishment gate, before any policy chain runs). Tests updated: - TestClientACLPolicy: split into "policy enforces allowed_types unconditionally" + new TestChainAdminBypass class covering the chain-level bypass for BYPASS_ADMIN policies. - e2e test_non_admin_default_session_id_is_disconnected renamed to ..._is_denied_by_policy; asserts the message doesn't reach the agent bus (no longer asserts disconnect — that observable changed). - Unit test_session_pipeline._make_client marks the test client as admin so its connection-level tests don't hit the new chain enforcement (those tests target session-pipeline plumbing, not ACL). * docs: document AbstractDB v1->v2 migration story in README * refactor(policy): rename ClientACLPolicy to MessageTypeACLPolicy; drop runner-level is_admin handling - Rename ClientACLPolicy -> MessageTypeACLPolicy across hivemind_core, tests, README, and the hivemind.policy entry point. Old entry-point name 'hivemind-core-acl-policy' becomes 'hivemind-message-type-acl-policy'. - Expose DenyAllPolicy via the hivemind.policy entry-point group as 'hivemind-deny-all-policy' so operators can install it explicitly. - Drop BYPASS_ADMIN handling from PolicyChain.review / review_binary. Client.is_admin is informational; the runner gives it no special treatment. Policies that care branch on client.is_admin themselves. - MessageTypeACLPolicy no longer carries BYPASS_ADMIN — admins are subject to the whitelist like any other client. Operators grant message types via 'allow-msg' regardless of admin status. - Update tests to drop BYPASS_ADMIN assertions and exercise the self-branching pattern (a custom policy that checks client.is_admin inline). * feat(policy): chain hardening, structured deny data, CLI, docs - route hive.policy.denied via HiveMessageType.THIRDPRTY (out-of-band) instead of BUS to avoid re-entering the admission path - structured Verdict.deny data on policy_error: policy / mutation / error fields instead of an opaque reason string - mutation.apply may return a replacement Message; chain uses it - MessageTypeACLPolicy stashes the resolved client on message.context[RESOLVED_CLIENT_CTX_KEY] for downstream policies - bump DB-refresh failure log to LOG.warning - add PolicyChain.on_verdict trace hook (exception-swallowed) - add hivemind-core policy list / policy test CLI commands - adopt DenyCodes enum for built-in deny codes - add tests: mutation-apply failure, structured policy_error data, resolved-client memoisation, no-DB ACL path, observe-not-called contract, DenyAllPolicy review/review_binary, entry-point discovery, on_verdict hook - add docs/policy.md (architecture, authoring, built-ins, operator config, deny-code reference, CLI) * feat(policy): connection-scoped user cache, optional flag, chain-complete trace - HiveMindClientConnection.resolve_user(db, ttl, force) + invalidate_user cache the DB row on the connection; policies share the lookup across the chain pass instead of stashing it in message.context. - MessageTypeACLPolicy consumes client.resolve_user; drop the RESOLVED_CLIENT_CTX_KEY context dance. - PolicyChain accepts an 'optional: true' per-entry flag — exceptions from optional policies log a warning and continue the chain (fallback to Verdict.allow with no mutations). Mandatory policies stay fail-closed with POLICY_ERROR. The force-prepended MessageTypeACLPolicy is always mandatory. - PolicyChain.review fires on_verdict(None, final) on chain-complete allow so tracers see the full accumulated mutation set. - Document the optional flag and the hive.policy.denied wire format in docs/policy.md. - Tests: optional/mandatory exception behaviour, optional default, accumulated mutations on synthetic chain-complete verdict. Update existing MessageTypeACLPolicy DB tests for the resolve_user shape. * fix(policy): keep hive.policy.denied on the BUS channel; safe send() logging - _send_policy_denied routes through HiveMessageType.BUS (not THIRDPRTY); policy denial is core protocol, not third-party signalling - send() computes payload size from the full HiveMessage.serialize() so dict payloads do not crash the unencrypted-size log line - e2e test helper subscribes to BUS for hive.policy.denied capture - docs/policy.md wire-format updated to BUS routing * test(policy): mutation replacement path, two-path contract, protocol wiring - TestMutationReplacementPath: five cases covering the two-path mutation contract (None=in-place, non-None=replacement). Key tests: - replacement message is used by the next policy in the chain - tag mutation applied after a replacement runs on the NEW message - in-place mutation (None return) leaves the same object in the chain - TestProtocolWiring: five cases for handle_inject_agent_msg integration - deny verdict sends hive.policy.denied BUS message to client - allow verdict emits to agent bus - observe() called after emit - observe() exception swallowed, delivery unblocked - policy exception in wired chain → policy_error denial to client Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: restore version.py to dev baseline * fix: strip null session fields per SESSION-1 §2; rewrite non-conformant test _install_client_session now strips null-valued fields from the serialised session before writing it to message.context. SESSION-1 §2: a producer MUST NOT emit a field as JSON null; a consumer MUST treat null as a malformed value and behave as if the field were omitted (deployment default, not preserved null). test_explicit_none_pipeline_is_preserved was asserting spec-violating behaviour: that pipeline:null from the client is preserved as None on the agent bus. The correct conformant behaviour is that the bridge strips the null value. The test is renamed test_explicit_none_pipeline_is_treated_as_absent and now asserts that null is NOT preserved (SESSION-1 §2). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * ci+test: pin hivescope@fix/acl-resolve-user (allowed_types); treat null pipeline as absent per SESSION-1 * ci: pin hivescope/plugin-manager/ovos-agent-plugin to PyPI prereleases (drop all git branches) * ci: refresh pin comments (deps now on PyPI prereleases) * test(e2e): add BRIDGE-1 conformance suite at the core bridge (hivescope assertions); pin hivescope==0.3.0a1 * test: XPASS warning hook; §6 topology-hiding xfail -> skip (permanent MAY) * test(bridge1): anchor lang propagation to serialized session (robust to client canonicalization) * ci: broaden license exclude (AGPL hivemind family + filelock/tqdm license-detection false positives) * feat(cli): keep skill/intent blacklist commands as OVOS-policy-specific (drop deprecation); add set-metadata for arbitrary Client.metadata --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
#95) * feat: default new installs to SQLite (transactional, concurrency-safe, stdlib); keep JSON for existing deployments + add migrate-db; bump json_database>=0.10.2a1 (drops the bundled colliding plugin) * docs: document SQLite as the default backend + migrate-db * test: skip migrate-db test when sqlite plugin predates the schema-v2 Client model (published 0.2.1 still references message_blacklist) * build: require schema-v2 DB plugins (sqlite>=0.3.0a1, json>=0.0.3a1, hpm>=0.6.0a1) compatible with the current Client model; pin the published alphas in CI so the migrate-db test runs
…, cli, extending) — docs-only (#101)
…SCALATE up to the master and fan BROADCAST/PROPAGATE back down (replaces the hive.send.upstream bus event) (#98)
* fix: enforce min_protocol_version at handshake time (fail closed)
min_protocol_version was advisory only. handle_handshake_message() never
checked the configured floor; it was only advertised in the HELLO/HANDSHAKE
payload and only enforced via the min_version > max_version check at
connect time. That check cannot trigger for a password-capable client,
because a client with a pswd_handshake makes max_version = THREE (it is
"v3-capable"), even when it goes on to complete a legacy v2
PasswordHandShake instead of Noise. Result: a hub configured with
min_protocol_version=3 (Noise required, no plaintext key exchange) still
silently accepted a v2 password handshake, ignoring the operator's
security floor.
HIVEMIND-WIRE-1 §2 requires both peers to operate at the highest version
both support ("a version-3-capable peer whose counterpart's maximum is 2
or lower falls back to the version-2 handshake") but a v3-capable client
choosing to send a v2 envelope instead of negotiating Noise is not a
counterpart limitation - it's a downgrade the hub must refuse per
HIVEMIND-CRYPTO-1's fail-closed floor semantics (§3.4).
Fix: handle_handshake_message() now determines the protocol version the
client is actually completing (pubkey/RSA envelope = v1, password
envelope = v2, "noise" payload = v3, already exempt) and disconnects
before touching the legacy paths if that version is below the
configured min_protocol_version.
Back-compat: the shipped default (min_protocol_version=2 in config.py)
is unchanged, so behavior is identical for every deployment that has not
explicitly raised its floor to 3.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor: read the protocol floor through one helper, tolerate null config
Extracts the duplicated min_protocol_version read (HELLO advertisement +
handshake gate) into _configured_min_protocol_version(), and includes
TypeError in its fallback so a JSON null floor degrades to the default
instead of raising (CodeRabbit finding).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…d frames (#166) * fix: reject INTERCOM without a verifiable origin (fail closed) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: drop rejected INTERCOM instead of relaying it handle_intercom_message returns a bool that the BROADCAST, PROPAGATE and ESCALATE handlers read as "consumed, stop processing". False means "not addressed to me, keep relaying". The origin-authentication guard returned False on every rejection. The callers therefore fell through to the peer fan-out and to escalate_to_master, so an INTERCOM this node refused to authenticate was rebroadcast to every peer and pushed upstream. Rejecting a frame amplified it. Authentication failures now return True: the frame was handled here, by dropping it. The docstring states what each return value means. The "not for us" early return and the undecryptable-but-not-addressed-to-us path keep returning False, because those frames really do belong to someone else. Also report an invalid base64 signature as a signature verification failure instead of "failed to decrypt message!". Tests: one per caller path (broadcast, propagate, escalate) asserting a forged INTERCOM reaches neither peer sends nor the upstream master. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
handle_intercom_message verified the origin RSA signature only on the signed-envelope branch (payload dict with a "ciphertext" field). The two sibling branches -- an inner HiveMessage payload, and a plain payload deserialized with HiveMessage.deserialize -- dispatched the inner message to handle_bus_message / handle_propagate_message / handle_broadcast_message / handle_escalate_message / handle_binary_message / handle_client_shared_bus with no origin authentication. An attacker skipped the whole HIVEMIND-CRYPTO-1 §5 signature MUST by omitting one field. require_crypto (default True) already means "unencrypted payloads are not allowed" and is advertised to clients as crypto_required. Make that true for INTERCOM too: when require_crypto is set, the unsigned branches log a warning that names the peer and drop the frame. The drop returns True, so the frame is consumed here and is not relayed to peers nor escalated upstream. When require_crypto is False, plaintext INTERCOM (issue #117 / PR #123) still works. That is the escape hatch for deployments that want unauthenticated INTERCOM. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Found by a spec-conformance audit against HIVEMIND-POLICY-1. 1. Binary payloads bypassed the ACL gate (security). MessageTypeACLPolicy implemented only review(), so it inherited PolicyPlugin.review_binary, which allows everything. A client whose allowed_types whitelist denied every message type could still push RAW_AUDIO into the STT pipeline and FILE payloads to disk. POLICY-1 §2 puts every BUS message AND every BINARY payload through the full chain, and §4 makes the whitelist deny-by-default. MessageTypeACLPolicy now implements review_binary and denies with acl_disallowed_type when the resolved whitelist is empty. 2. A database error kept a revoked client's grants (security). The DB refresh was wrapped in try/except and fell back to the whitelist captured when the client connected, so a revocation issued while the DB was unreachable never took effect. POLICY-1 §4 wants the whitelist resolved freshly enough that a revocation lands without a reconnect, and §5 makes any policy error a deny. resolve_user's own docstring says exceptions propagate and callers fail closed; this caller did not. The error path now denies with policy_error. The deliberate 5s TTL cache on the connection is unchanged — only the error path changed. 3. The gate was removable by constructor injection (hardening). The force-prepend and dedupe ran only when policy_chain was None, so an embedder passing HiveMindListenerProtocol(policy_chain=...) got a chain with no ACL gate and no error. POLICY-1 §4 says the gate is always present, always first, and cannot be removed or reordered. The normalization moved into _with_acl_gate() and now runs on every chain. Nothing in-tree exploited this. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…elay floods (#172) Five related mesh-routing defects found by a HIVEMIND-NODE-1 spec-conformance audit. PROPAGATE and BROADCAST lost their outer envelope on downstream fan-out. Both handlers sent the inner message returned by _unpack_message to each peer, so a peer received a bare BUS instead of a PROPAGATE and had nothing to re-propagate: the flood reached only the origin's direct peers. NODE-1 §3.3 requires a relay to preserve the envelope, and §4 requires PROPAGATE to fan out across the whole reachable mesh. Both now rebuild the envelope, as handle_cascade_message already did. A relay fanned QUERY and CASCADE *responses* to every downstream node. query_from_master and cascade_from_master looped over all clients, including is_response chunks meant for one satellite, bypassing the route-aware _route_query_response. One peer's answers reached every sibling. NODE-1 §5.2 and AGENT-1 §3 forbid this. Responses now take the return path; only requests fan out. Loop prevention was not applied on the upstream to downstream relay path. propagate_from_master, query_from_master and cascade_from_master forwarded blindly, with no loop check and no self-hop, so the §3.4 machinery was wired into the downstream-origin handlers only. They now share one _relay_downstream helper. BROADCAST is left out on purpose: §4 makes it a single hop to the directly connected downstream nodes, never re-broadcast by recipients, so it only travels downstream and cannot cycle. Relayed ESCALATE, PROPAGATE and CASCADE lost metadata and site targeting. metadata, target_site_id and target_pubkey are constructor-only on HiveMessage, so rebuilding an envelope with only route copied dropped them. Site targeting reads the outer envelope, so a site-targeted message stopped being deliverable after one relay hop. A shared _rewrap helper now carries all of them, and the *_to_master methods take the finished envelope. The responsive PING never travelled upstream, contradicting its own docstring, so a master never learned of nodes below a relay. It now also goes to the upstream master, as the satellite side already does. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix: isolate peer sessions and surface backend failures Three defects found by a spec-conformance audit. Reserved "default" session reaching the OVOS bus (HIVEMIND-BRIDGE-1 §4.1, OVOS-SESSION-2 §5). Every connection is minted in the reserved session by hivemind-websocket-protocol, HELLO is not mandatory, and the disconnect notification emits the connection session onto the OVOS bus outside the policy chain. An unauthenticated peer could therefore write into the orchestrator-owned device-local session store. Core now moves a new connection off the reserved id, keeps a non-admin "default" session out of hive.client.disconnect, and enforces the reserved id in a new built-in DefaultSessionPolicy that sits next to MessageTypeACLPolicy instead of only in the operator-removable OVOSAgentPolicy. peer collisions (HIVEMIND-BRIDGE-1 §3, HIVEMIND-AGENT-1 §3). peer is "name::session_id"; name comes from the access key and session_id is chosen by the client in HELLO, so two connections sharing an access key could produce the same peer. The second evicted the first from self.clients and then received its responses. A colliding connection now gets a server-generated suffix (BRIDGE-1 §3.1). Registration also drops the stale entry a re-HELLO with a new session_id left behind. Unhandled ConnectionError from get_bus. The OVOS agent plugin now raises ConnectionError when the OVOS messagebus does not reconnect. No caller handled it, so the exception escaped into the websocket handlers and admitted messages disappeared with no signal. Inject, QUERY and CASCADE now answer the originating peer with a hive.policy.denied carrying the new stable code "backend_unavailable"; lifecycle notifications log and continue. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: retrigger CI Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: account for DefaultSessionPolicy in the built-in chain assertions The non-removable built-in prefix is now two policies, not one: MessageTypeACLPolicy followed by DefaultSessionPolicy. Update the composition assertions in TestACLGateIsNonRemovable accordingly, and have the reorder test also try to move DefaultSessionPolicy so both built-ins are covered. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Verified every command, config key, and code snippet in README.md and docs/ against the current source. Broken commands: - `add-client --db-backend` does not exist. Removed from README.md and docs/cli-reference.md. - `migrate-db --to sqlite` fails: --from/--to take a database plugin entry-point name. The --host/--port/--password flags do not exist. Corrected in README.md, docs/configuration.md, docs/cli-reference.md. Onboarding traps: - README Quick Start went from add-client straight to listen. A new client has an empty allowed_types whitelist, so every message is denied. Added a "Granting Message Types" step. - docs/cli.md claimed clients may send "a restricted set of message types" by default. The set is empty. Stale behavior: - docs/protocol.md documented handle_pong_message and a PONG wire message. Discovery is PING-only. Rewrote the section. - docs/protocol.md listed msg_blacklist/skill_blacklist/intent_blacklist on HiveMindClientConnection. Those fields are gone. - ClientDatabase.add_client takes `key`, not `access_key`. - delete-client does not write a tombstone. - Binary payloads now cross the allowed_types gate. - require_crypto documented as a protocol attribute, not a config key, with the INTERCOM drop behavior. - min_protocol_version is enforced at handshake completion. - Session ciphers: ChaCha20-Poly1305 is preferred over AES-GCM, and protocol v3 uses a Noise transport. Missing reference material: - docs/configuration.md lacked min_protocol_version, min_password_bits, runtime_password_strength_check, last_seen_update_interval, presence. - Command lists lacked print-config, derive-psk, export-clients, migrate-db, and the policy group. - add-client option tables lacked --admin, --metadata, --allow-weak-password. - plugins.md used the package name and wrong keys for the database config block. - plugin_development.md lacked the policy plugin type. Other: - Python version: 3.10+, was 3.8+ and 3.9+. - Replaced brittle file:line citations with symbol names. - Fixed 27 relative links that pointed outside the repository. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…abase and config (#180) * refactor: collapse duplicated CLI client lookup and trim restating docs Readability pass over the CLI, service, database and config modules. No behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: require plugin-manager 0.9.0a1 and drop the getattr backend probe ClientDatabase.get_client_by_api_key probed the backend with getattr before calling it, so a backend without the method silently degraded to a full scan instead of failing. Its neighbours get_client_by_id and refresh call through directly; the probe was inconsistent with them. AbstractDB grew get_client_by_api_key in hivemind-plugin-manager 0.9.0a1 but the floor was still 0.8.0a1, which is the actual cause. Raise the floor and call the method directly. The MemoryDB test fake did not inherit AbstractDB and so lacked the method; it now inherits it and is held to the backend contract. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Client.can_broadcast is a real column in the client database and the websocket transport plugin copies it onto the live connection, but HiveMindClientConnection never declared the field, so the assignment created a stray attribute nothing read. handle_broadcast_message gated on is_admin alone: revoking broadcast on a client changed nothing. Declare can_broadcast next to can_escalate/can_propagate and gate the handler on `is_admin and can_broadcast`. The DB default is True, so the grant can only narrow the admin privilege, never widen it to ordinary clients. Add allow-broadcast / blacklist-broadcast CLI commands so an operator can set it, mirroring the escalate/propagate commands. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…#183) HiveMindListenerProtocol.peer is the class default "master:0.0.0.0" and service.py never overrides it, so every node in every deployment announced the same string. Loop detection was already moved to self._node_id (the node public key) for that reason; the remaining uses were left behind and are now moved too: - the HELLO "node_id" a client stores as its master's identity - the Noise PSK salt (both sides derive it from the announced HELLO value, so the two stay consistent and no handshake breaks) - _unpack_message update_source_peer - the responsive PING payload "peer" - responder_peer on QUERY/CASCADE responses The peer field itself stays as a log label; nothing keys on it now. Also drops a duplicated context["source"] assignment left by a merge. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
PR #181 added a hand-built protocol fixture that sets only peer and clients. PR #183 then moved provenance stamping onto the node public key, so the broadcast path reads self.identity. Each passed alone and the pair broke dev. Test-only. The fixture now mirrors the others that build a protocol with object.__new__. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Human review requested!