Add Hierarchical two-stage intent variant by JarbasAl · Pull Request #53 · OpenVoiceOS/padacioso

JarbasAl · 2026-05-19T19:33:19Z

Summary

Adds a Hierarchical two-stage intent variant to padacioso, alongside the existing flat engine.

HierarchicalIntentContainer — groups intents into domains; a top-level classifier (itself a flat IntentContainer, trained automatically from the registered samples) selects exactly one domain, then only that domain's sub-container resolves the intent.
HierarchicalPadaciosoPipeline — OPM pipeline exposing the variant under the ovos-padacioso-hierarchical-pipeline entry point. Intents are filed under a domain == skill_id taken from the skill_id:intent label prefix.
domain_threshold config key turns the first stage into an off-topic rejection gate: queries whose best domain scores below the threshold are rejected before any sub-container runs (0.0 disables the gate).
benchmark/compare.py runs the flat and hierarchical engines against the padaos, padatious and nebulento baselines on the intents-for-eval and massive datasets.
Docs: docs/hierarchical_pipeline.md and docs/benchmark.md.

The engine ships two variants: the flat IntentContainer and the two-stage HierarchicalIntentContainer.

Test plan

pip install -e . then python -m pytest test/ — 85 tests pass
python benchmark/compare.py runs flat + hierarchical plus the three baselines

🤖 Generated with Claude Code

add_entity() stored samples as list; _match() used `str(v) not in list` which is O(V) per entity check. Changed to set so lookup is O(1). AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: eliminate linear scan in hot matching path - Impact: entity value validation now O(1) instead of O(V) - Verified via: python -m pytest test/test_padacioso.py -p no:ovoscope -q

Stacked penalties (wildcard + multiple unregistered entities) could push 1 - penalty below zero, producing negative confidence values. Added max(0.0, ...) at both return sites in the cased and uncased match paths. AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: prevent negative confidence values from reaching callers - Impact: conf is now guaranteed to be in [0.0, 1.0] - Verified via: python -m pytest test/test_padacioso.py -p no:ovoscope -q

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Replace flat 0.15 wildcard penalty with a proportional penalty that scales with the fraction of `*` tokens in the pattern. Range [0.05, 0.25]. Entity placeholders keep their own separate penalty path. - Impact: Stored in _regex_penalty dict; confidence rounded to 4dp to avoid float accumulation; test expectations updated to match new formula. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Substring match `s in query` incorrectly excluded intents when a keyword like "play" appeared inside "display" or "replay". Now uses a word-set lookup for single-word keywords and regex \b boundary for multi-word phrases. - Impact: _filter() no longer fires on partial substring hits. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: simplematch compiles {entity} to (?P<entity>.*) (greedy), causing the first entity to consume tokens that belong to later ones in patterns with ≥2 placeholders and no literal separator. Patching .* -> .*? via the Matcher.regex property setter (which triggers recompile) fixes the capture order. - Impact: _patch_nongreedy() applied at add_intent time and in lazy-init fallbacks. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: STT output is almost always lowercase; skipping the cased pass for those queries avoids a redundant regex evaluation per intent. Case-mismatch penalty (0.05) is only applied when query_has_upper and uncased match fires. Unregistered-entity penalty stays 0.04 (cased semantics) for pure-lowercase queries and 0.05 (case-mismatch semantics) when query contains uppercase. - Impact: _match() checks query_has_upper once per call; cased matchers skipped for the common lowercase case. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Stop evaluating intents once a 0.95-confidence match is found (good- enough threshold). Break ties deterministically by ascending wildcard penalty (more specific pattern wins), then by intent name as final tiebreaker. _match() now returns _matched_regex key (stripped from public output) to support penalty-based tie sorting. - Impact: calc_intent avoids evaluating all intents for clear-winner queries; ties are stable across Python dict ordering. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: maxsize=3 evicts immediately during a burst of ASR hypotheses; 128 keeps recent queries warm across a full recognition cycle. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Cover the 9 accuracy/speed improvements with targeted tests: word-boundary keyword exclusion, confidence clamping, proportional wildcard penalty values, multi-entity non-greedy split, and deterministic tie-breaking. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope (54 passed)

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: _match() now includes a private _matched_regex key for tie-breaking in calc_intent(), but opm.py calls calc_intents() directly and passes the raw dict to PadaciosoIntent(**intent), causing an unexpected keyword argument error. Strip the key before construction. - Verified via: uv run pytest test/ -q -p no:ovoscope (56 passed)

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Fuzzy matching was re-generating all single-word substitution variants on every query. Pre-compute them at add_intent() time. Add two O(1) pre-filters before the expensive simplematch+fuzzy_match pair: skip if word-count distance is too large, skip if the pattern shares no literal words with the query. - Impact: entity-match 12x faster, no-match 24x faster, near-miss 2.5x faster. - Verified via: uv run pytest test/test_padacioso.py -q -p no:ovoscope (54 passed)

…ces) AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: Provide a reproducible accuracy benchmark. 22 intents, 244 labelled match utterances, 25 no-match utterances. fuzz=False: 97.8% accuracy, 100% precision, 0 false positives. fuzz=True: 97.0% accuracy, 4 false positives. Results summarised in README. - Verified via: uv run python benchmark/accuracy.py

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: A 0.95-confidence entity match (e.g. add_shopping's "i need {item}" matching "i need help") was triggering early exit before a later intent's literal exact match (conf=1.0) could be evaluated. Fix on two levels: 1. Sort each intent's regexes literal-first so they short-circuit before entity patterns within the same intent. 2. Gate the _GOOD_ENOUGH early exit on best_is_literal — an entity match at 0.96 no longer blocks literal matches in later intents. 3. Tie-breaking prefers literal matches over entity/wildcard matches. - Impact: "i need help"→help, "play the next song"→next_track now correct. fuzz=False accuracy 97.8%→98.5%, F1 0.988→0.992. - Verified via: uv run pytest test/test_padacioso.py -q && python benchmark/accuracy.py

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: The previous benchmark used utterances that were obvious template fills ("set a timer for five minutes"). Real STT output uses contractions, idioms, indirect requests, and colloquialisms. The new dataset exposes padacioso's actual recall on natural speech: ~30% fuzz=False, ~51% fuzz=True. Precision stays 100%/97% respectively — it never misclassifies, it just doesn't cover phrasing not in the templates. README updated to present both datasets and explain the pattern-matcher tradeoff honestly.

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: surface comparative benchmark results (padaos/padacioso/padatious/rapidfuzz) - Impact: README now includes natural-language accuracy and latency comparison table - Verified via: uv run python benchmark/compare.py

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: correct config format - Impact: README config block now uses valid JSON syntax

Drop the erroneous f prefix from the LOG.debug call in __init__ so the lint / lint CI step no longer fails. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Sort stdlib imports alphabetically, add noqa comments for the intentional sys.path and logging.disable calls that must precede local imports, and drop the unused true_neg variable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The multi-token branch in _filter was running re.search against the raw (mixed-case) query while the keyword was lowercased, so a phrase like "Stop It now" would not be caught by an excluded keyword "stop it". Now both branches operate on q_lower for consistent case handling. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The previous early-exit broke immediately on the first literal match at >= 0.95 confidence, meaning a second intent with the exact same confidence would never be seen and the _tie_key sort had no effect. Now the loop continues collecting candidates that share the winning confidence and only stops when the next candidate arrives with a strictly lower confidence. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

The premature early-exit in calc_intents (conf == 1.0) would stop yielding after the first perfect match, preventing tied intents from reaching calc_intent's _tie_key sort. Remove the early-exit from calc_intents so that calc_intent's own smarter logic (which continues collecting ties at the top confidence level) remains in full control. Also strengthen test_tie_breaking_deterministic to assert that the alphabetically-first name always wins regardless of registration order, rather than only checking idempotence. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Two for-loops in add_intent and add_entity used the ambiguous single-letter variable l, which Ruff flags as E741. Rename to line for clarity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…+ RTF AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: benchmark nebulento (all 9 rapidfuzz strategies) against padaos/padacioso/padatious - Impact: run_rapidfuzz replaced by run_nebulento; summary table gains RTF column; nebulento added as dev dep - Verified via: uv run python benchmark/compare.py

…lts + RTF AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: surface complete nebulento strategy sweep in the benchmark README - Impact: table now covers 9 nebulento strategies; adds RTF column; explains tradeoffs - Verified via: uv run python benchmark/compare.py

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: show avg ms instead of RTF which is less intuitive - Impact: summary now shows Median + Mean columns in ms; RTF removed - Verified via: uv run python benchmark/compare.py

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: benchmark suite now lives in nebulento; remove from padacioso - Impact: benchmark/ directory emptied; dataset, compare, accuracy scripts gone

Mirrors the API shipped by nebulento.DomainIntentContainer and ovos_padatious.DomainIntentContainer: intents are grouped into domains, a top-level IntentContainer first picks the domain, then the domain's sub-container resolves the intent. API: from padacioso import DomainIntentContainer d = DomainIntentContainer() d.register_domain_intent('media', 'play', ['play {song}', 'put on {song}']) # ... seed the domain classifier with d.domain_engine.add_intent(...) d.calc_intent('play bohemian rhapsody') # -> {'name': 'play', 'entities': {'song': 'bohemian rhapsody'}, ...} Re-exported at package root. Slot extraction is preserved end-to-end — the domain-level match dict's entities still come from the matched template. Benchmarked against ovos-intent-benchmark's 50-intent / 10-domain dataset, hierarchical matching consistently lowers false-positive rate on out-of-domain chitchat because the top-level classifier rejects those utterances before any sub-container sees them.

coderabbitai · 2026-05-19T19:33:29Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 02d499a2-42a9-47f8-9f93-12f8ca2e15c5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch add-domain-intent-container

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-05-19T19:45:38Z

Greetings! The CI pipeline has delivered its findings. 🏗️

I've aggregated the results of the automated checks for this PR below.

📋 Repo Health

Checking for any potential repo regressions. 🔄

✅ All required files present.

Latest Version: 2.2.2a1

✅ padacioso/version.py — Version file
✅ README.md — README
✅ LICENSE.md — License file (consider renaming to LICENSE)
✅ pyproject.toml — pyproject.toml
⚠️ setup.py — setup.py
✅ CHANGELOG.md — Changelog
✅ padacioso/version.py has valid version block markers

⚖️ License Check

Ensuring our licenses are consistent and clear. 📄

✅ No license violations found.

Policy: Apache 2.0 (universal donor). StrongCopyleft / NetworkCopyleft / WeakCopyleft / Other / Error categories fail. MPL allowed.

🔌 Skill Tests (ovoscope)

Simulating real-world interactions with your skill. 🤖

✅ 20/20 passed

✅ TestDetach — 2/2
✅ TestEntityExtraction — 1/1
✅ TestLegacyStillConsumed — 1/1
✅ TestNegativeKeywordTopic — 1/1
✅ TestRegisteredIntentMatch — 4/4
✅ TestSessionBlacklist — 2/2
✅ TestSessionBlacklistAlias — 3/3
✅ TestSpecDeregister — 2/2
✅ TestSpecDisableEnable — 2/2
✅ TestSpecTemplateConsumed — 2/2

🔌 Plugin Detection

Scanning for any 'performance bottlenecks' in the plugin. 📉

❌ Plugin Status: ERRORS (1)

Plugin Info:

Name: padacioso
Description: dead simple intent parser

OPM Detection:

Plugin Type	Wheel	Editable
pipeline	✅	✅

Entry Point Validation:

Entry Point	Type	Import	Interface
`ovos-padacioso-domain-pipeline`	pipeline	✅	⊘
`ovos-padacioso-pipeline-plugin`	pipeline	✅ 725ms	⊘

⊘ No settingsmeta.json
✅ requires-python >=3.8 — running Python 3.11

Issues:

❌ Import time for ovos-padacioso-pipeline-plugin exceeds 500ms (725ms)
⚠️ No settingsmeta.json found
⚠️ No settingsmeta.json found

🚌 Bus Coverage

Mapping the signals and responses of your logic. 📡

⚠️ Bus coverage report unavailable — check the job log.

🔒 Security (pip-audit)

Ensuring our project remains safe and secure. 🛡️

✅ No known vulnerabilities found (49 packages scanned).

📊 Coverage

Is the code wearing its test-suit? Let's see. 👔

⚠️ 68.6% total coverage

Per-file coverage (4 files)

File	Coverage	Missing lines
`padacioso/version.py`	0.0%	5
`padacioso/domain_engine.py`	20.3%	55
`padacioso/opm.py`	70.0%	134
`padacioso/__init__.py`	79.4%	59

Full report: download the coverage-report artifact.

🔨 Build Tests

Checking if the code is ready for prime time. 📺

✅ All versions pass

Python	Build	Install	Tests
3.10	✅	✅	✅
3.11	✅	✅	✅
3.12	✅	✅	✅
3.13	✅	✅	✅
3.14	✅	✅	✅

🔍 Lint

I've finished the heavy lifting on this check. 🏋️‍♂️

❌ ruff: issues found — see job log

🏷️ Release Preview

I've checked the 'Known Issues' list for honesty. 😇

Current: 2.2.2a1 → Next: 2.3.0a1

Signal	Value
Label	`feature`
PR title	`Add Hierarchical two-stage intent variant`
Bump	minor

⚠️ No conventional commit prefix — alpha-only bump.
Suggested: fix: update the thing or feat: update the thing

🚀 Release Channel Compatibility

Predicted next version: 2.3.0a1

Channel	Status	Note	Current Constraint
Stable	⚪	Not in channel	-
Testing	⚪	Not in channel	-
Alpha	⚪	Not in channel	-

May your merges be conflict-free! 🕊️

Introduce overrideable hooks (_build_container, _container_add_intent / _add_entity / _remove_intent / _remove_entity / _has_intent, _calc_one) so subclasses can swap the underlying IntentContainer without duplicating the bus-handler plumbing. No behavioural change for the flat pipeline.

DomainPadaciosoPipeline subclasses PadaciosoPipeline and swaps the per-language container for DomainIntentContainer. Each intent label is routed to a domain == skill_id (extracted from the skill_id:intent prefix); inference picks the most likely domain via the top-level classifier and then resolves the intent inside that domain. detach_skill drops the whole domain in one shot. Exposed via the new `ovos-padacioso-domain-pipeline` OPM entry point (separate from the flat plugin) so the two can coexist in the same OVOS instance with independent config blocks.

…outing Add docs/domain_pipeline.md covering enablement, config key (intents.ovos_padacioso_domain_pipeline), skill_id-prefix routing, entity sharing across domains, and detach semantics. Link from README.

Cover the standalone ovos-padacioso-domain-pipeline OPM entry point: pipeline loading with a DomainIntentContainer, skill_id-prefix routing on register, orphan-label fallback, detach_intent / detach_skill, router-driven matching, and session blacklist gating.

Replace the top-level IntentContainer router with parallel evaluation across every domain sub-container, mirroring adapt's pattern. Strict regex is the wrong router: paraphrases that fail to match any router template silently block the sub-stage even when a sub-container has a literal hit. Parallel evaluation is strictly more permissive. For padacioso the cost is negligible: - cheap literal-token prefilter skips domains with zero overlap; - first sub-engine returning a 0.95 (literal) hit short-circuits. Also drops the unused training_data dict, adds calc_intents(top_k=...), and refreshes DomainPadaciosoPipeline + docs accordingly.

# Conflicts: # padacioso/opm.py

…ormalize_example dev deleted padacioso/bracket_expansion.py in favor of ovos_spec_tools-based normalization; domain_engine.py still imported the old module post-rebase.

handle_register_template/handle_register_entity/handle_disable_intent/ handle_enable_intent called the flat IntentContainer API directly, which DomainIntentContainer does not implement. Add _container_exclude_keywords and _container_get_intent_samples hooks (with domain-routing overrides) and use the existing hooks everywhere so the spec handlers work for both the flat and hierarchical/domain pipelines.

JarbasAl and others added 30 commits April 22, 2026 00:37

Delete status.md

070950f

docs: add README with usage guide and speed benchmarks

e202a9c

docs: add 10k-intent benchmark results to README

b5407a0

docs: add fuzzy vs non-fuzzy benchmark table to README

5541fa9

docs: fix mycroft.conf snippet — JSON not YAML

c505a45

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: correct config format - Impact: README config block now uses valid JSON syntax

fix: remove f-string with no placeholders (Ruff F541)

afac3d8

Drop the erroneous f prefix from the LOG.debug call in __init__ so the lint / lint CI step no longer fails. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

fix: rename ambiguous loop variable l → line (Ruff E741)

2f454ae

Two for-loops in add_intent and add_entity used the ambiguous single-letter variable l, which Ruff flags as E741. Rename to line for clarity. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

chore: remove benchmark suite (moved to nebulento repo)

2ce99a3

AI-Generated Change: - Model: claude-sonnet-4-6 - Intent: benchmark suite now lives in nebulento; remove from padacioso - Impact: benchmark/ directory emptied; dataset, compare, accuracy scripts gone

github-actions Bot added the feature label May 19, 2026

JarbasAl marked this pull request as draft May 19, 2026 19:43

JarbasAl requested review from femelo and mikejgray May 19, 2026 19:44

Merge branch 'dev' into add-domain-intent-container

03802fd

JarbasAl added 4 commits May 20, 2026 11:52

docs: document DomainPadaciosoPipeline entry point and hierarchical r…

304b4a4

…outing Add docs/domain_pipeline.md covering enablement, config key (intents.ovos_padacioso_domain_pipeline), skill_id-prefix routing, entity sharing across domains, and detach semantics. Link from README.

JarbasAl mentioned this pull request May 20, 2026

refactor: delegate template expansion to ovos_utils #54

Closed

2 tasks

JarbasAl changed the title ~~feat: add DomainIntentContainer for two-stage intent matching~~ Add Hierarchical two-stage intent variant May 22, 2026

JarbasAl force-pushed the add-domain-intent-container branch from 7bd47d4 to 4f9f857 Compare July 4, 2026 00:17

JarbasAl added 3 commits July 31, 2026 00:16

Merge remote-tracking branch 'origin/dev' into HEAD

347a6f2

# Conflicts: # padacioso/opm.py

fix: use padacioso._normalize in place of removed bracket_expansion.n…

1d2b628

…ormalize_example dev deleted padacioso/bracket_expansion.py in favor of ovos_spec_tools-based normalization; domain_engine.py still imported the old module post-rebase.

JarbasAl force-pushed the add-domain-intent-container branch from 4f9f857 to 4074840 Compare July 31, 2026 09:49

Navigation Menu

Uh oh!

Uh oh!

Add Hierarchical two-stage intent variant - #53

Add Hierarchical two-stage intent variant#53
JarbasAl wants to merge 40 commits into
devfrom
add-domain-intent-container

JarbasAl commented May 19, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented May 19, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented May 19, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

JarbasAl commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

coderabbitai Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greetings! The CI pipeline has delivered its findings. 🏗️

📋 Repo Health

⚖️ License Check

🔌 Skill Tests (ovoscope)

🔌 Plugin Detection

🚌 Bus Coverage

🔒 Security (pip-audit)

📊 Coverage

🔨 Build Tests

🔍 Lint

🏷️ Release Preview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

JarbasAl commented May 19, 2026 •

edited

Loading

coderabbitai Bot commented May 19, 2026 •

edited

Loading

github-actions Bot commented May 19, 2026 •

edited

Loading