From 7babd03e2b11791578b4792bc34f60ca02391cc1 Mon Sep 17 00:00:00 2001 From: JarbasAi Date: Fri, 31 Jul 2026 16:17:05 +0100 Subject: [PATCH] docs: rewrite README in Simplified Technical English Co-Authored-By: Claude Sonnet 5 --- README.md | 66 +++++++++++++---------- docs/audio-testing.md | 43 ++++++++------- docs/bus-coverage.md | 21 ++++---- docs/capture-session.md | 11 ++-- docs/ci-integration.md | 23 ++++---- docs/cli.md | 31 ++++++----- docs/end2end-test.md | 17 +++--- docs/gui-testing.md | 37 +++++++------ docs/index.md | 65 +++++++++++----------- docs/listener.md | 87 +++++++++++++++--------------- docs/media-provider-testing.md | 17 +++--- docs/media-testing.md | 99 ++++++++++++++++++---------------- docs/minicroft.md | 13 +++-- docs/ocp.md | 17 +++--- docs/phal.md | 29 +++++----- docs/pipeline.md | 25 +++++---- docs/pydantic-integration.md | 29 +++++----- docs/usage-guide.md | 95 ++++++++++++++++---------------- docs/voice-loop.md | 27 +++++----- 19 files changed, 411 insertions(+), 341 deletions(-) diff --git a/README.md b/README.md index 1804e72..caf2414 100644 --- a/README.md +++ b/README.md @@ -3,31 +3,34 @@ [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE) [![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/ovoscope/) # OvoScope + **End-to-end testing for [OVOS](https://openvoiceos.org) skills.** -OvoScope runs a full OVOS Core pipeline in-process using a `FakeBus` — no server, no audio -stack, no network. Load real skill plugins, emit a test utterance, and assert on every bus -message that comes back: type, data, routing context, session state, and message ordering. + +OvoScope runs a full OVOS Core pipeline in-process with a `FakeBus`. It needs no server, no +audio stack, and no network. Load real skill plugins, send a test utterance, and check every +bus message that comes back: type, data, routing context, session state, and message order. + ![image](https://github.com/user-attachments/assets/10a10ff5-64b7-42fd-86bd-cb6a5db769dd) -> Like a microscope for your OVOS skills. + --- ## Features | | | |---|---| | **Full pipeline** | Runs real intent pipeline plugins (Adapt, Padatious, Fallback, Converse, Common Query) | -| **Isolated** | Config isolation strips user preferences; deterministic `DEFAULT_TEST_PIPELINE` excludes AI/persona/OCP stages | -| **Ordered assertions** | Assert message type, data keys, routing context, and session state in sequence | -| **Recording mode** | Capture a live message sequence and save it as a JSON fixture — no manual construction needed | +| **Isolated** | Config isolation strips user preferences, and the deterministic `DEFAULT_TEST_PIPELINE` excludes AI, persona, and OCP stages | +| **Ordered assertions** | Checks message type, data keys, routing context, and session state in order | +| **Recording mode** | Captures a live message sequence and saves it as a JSON fixture. No manual construction needed | | **Multi-turn** | Pass a list of utterances to test full conversational flows | -| **pytest fixture** | `minicroft` class-scoped fixture auto-discovered via the `pytest11` entry point | -| **Inject skills** | `extra_skills={id: SkillClass}` to load inline test skills without a PyPI entry point | -| **Inject messages** | `MiniCroft.inject_message()` to trigger non-utterance handlers (GUI events, timers, API calls) | -| **Typed models** | Optional `ovoscope[pydantic]` bridge to `ovos-pydantic-models` for schema-validated messages | +| **pytest fixture** | The `minicroft` class-scoped fixture is auto-discovered through the `pytest11` entry point | +| **Inject skills** | Use `extra_skills={id: SkillClass}` to load inline test skills without a PyPI entry point | +| **Inject messages** | Use `MiniCroft.inject_message()` to trigger non-utterance handlers (GUI events, timers, API calls) | +| **Typed models** | The optional `ovoscope[pydantic]` bridge adds schema-validated messages through `ovos-pydantic-models` | --- ## Installation ```bash pip install ovoscope ``` -With optional typed message model support: +To add typed message model support: ```bash pip install ovoscope[pydantic] ``` @@ -62,11 +65,11 @@ class TestHelloWorld(unittest.TestCase): ], ).execute(timeout=10) ``` -Only keys you specify in `expected.data` and `expected.context` are checked — extra keys in the -received message are ignored. +OvoScope checks only the keys you list in `expected.data` and `expected.context`. It ignores +extra keys in the received message. --- ## Recording Mode -Don't know the exact message sequence yet? Record it from a live run: +If you do not know the exact message sequence yet, record it from a live run: ```python from ovoscope import End2EndTest test = End2EndTest.from_message( @@ -74,16 +77,16 @@ test = End2EndTest.from_message( skill_ids=[SKILL_ID], timeout=20, ) -test.save("tests/fixtures/hello_world.json") # anonymises location data by default +test.save("tests/fixtures/hello_world.json") # anonymizes location data by default ``` -Replay in CI: +Replay the fixture in CI: ```python End2EndTest.from_path("tests/fixtures/hello_world.json").execute(timeout=10) ``` --- ## pytest Fixture -The `minicroft` class-scoped fixture is auto-registered when ovoscope is installed. -No `setUp`/`tearDown` boilerplate needed: +OvoScope auto-registers the `minicroft` class-scoped fixture on install. You do not need +`setUp`/`tearDown` boilerplate: ```python class TestMySkill: skill_ids = ["my-skill.author"] @@ -97,11 +100,11 @@ class TestMySkill: ``` --- ## Pipeline Control -OvoScope exposes composable pipeline stage lists so tests are deterministic regardless of which -AI plugins are installed on the host: +OvoScope exposes composable pipeline stage lists so tests stay deterministic regardless of +which AI plugins are installed on the host: ```python from ovoscope import ADAPT_PIPELINE, PADATIOUS_PIPELINE, FALLBACK_PIPELINE, PERSONA_PIPELINE -# Adapt only — fastest +# Adapt only: fastest mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE) # Full intent chain mc = get_minicroft([SKILL_ID], @@ -109,14 +112,14 @@ mc = get_minicroft([SKILL_ID], # Opt in to persona for AI testing mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE) ``` -`DEFAULT_TEST_PIPELINE` (the default when `isolate_config=True`) includes all standard built-in -stages and deliberately excludes persona, Ollama, OCP, and m2v plugins. +`DEFAULT_TEST_PIPELINE` is the default when `isolate_config=True`. It includes all standard +built-in stages and leaves out persona, Ollama, OCP, and m2v plugins. --- ## Documentation | Document | | |---|---| -| [docs/usage-guide.md](docs/usage-guide.md) | **Start here** — 8 test patterns with full worked examples | -| [docs/ci-integration.md](docs/ci-integration.md) | Wiring ovoscope into GitHub Actions | +| [docs/usage-guide.md](docs/usage-guide.md) | **Start here**: 8 test patterns with full worked examples | +| [docs/ci-integration.md](docs/ci-integration.md) | Wiring OvoScope into GitHub Actions | | [docs/minicroft.md](docs/minicroft.md) | `MiniCroft` and `get_minicroft()` reference | | [docs/capture-session.md](docs/capture-session.md) | `CaptureSession` internals | | [docs/end2end-test.md](docs/end2end-test.md) | `End2EndTest` full parameter reference | @@ -129,6 +132,15 @@ stages and deliberately excludes persona, Ollama, OCP, and m2v plugins. --- +## Related Projects + +OvoScope is part of the [OpenVoiceOS](https://github.com/OpenVoiceOS) tooling suite: + +- [ovos-core](https://github.com/OpenVoiceOS/ovos-core): the OVOS assistant core that OvoScope tests skills against. +- [ovos-workshop](https://github.com/OpenVoiceOS/ovos-workshop): the skill base classes that OvoScope loads and drives. +- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client): the message bus client behind `FakeBus` and `Message`. +- [ovos-test-harness](https://github.com/OpenVoiceOS/ovos-test-harness): a companion test harness for OVOS components. + ## Credits Developed by [TigreGótico](https://tigregotico.pt) for @@ -152,7 +164,7 @@ under grant agreement No [101135429](https://cordis.europa.eu/project/id/1011354 ## Contributing -PRs are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. +PRs are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines. --- diff --git a/docs/audio-testing.md b/docs/audio-testing.md index c5067b5..e95df1f 100644 --- a/docs/audio-testing.md +++ b/docs/audio-testing.md @@ -16,10 +16,10 @@ classes provided in `ovoscope.audio`. ### AudioServiceHarness -`AudioServiceHarness` — `ovoscope/audio.py` +`AudioServiceHarness` (`ovoscope/audio.py`) -Wraps `AudioService` (from `ovos_audio.audio`) with a `MockAudioBackend` on a -`FakeBus`. Use it when your test exercises the audio routing layer — backend +`AudioServiceHarness` wraps `AudioService` (from `ovos_audio.audio`) with a `MockAudioBackend` on a +`FakeBus`. Use it when your test exercises the audio routing layer: backend selection by URI scheme, volume ducking on speech events, the 1-second stop guard, or session-source validation. @@ -37,9 +37,9 @@ with AudioServiceHarness() as h: ### PlaybackServiceHarness -`PlaybackServiceHarness` — `ovoscope/audio.py` +`PlaybackServiceHarness` (`ovoscope/audio.py`) -Wraps `PlaybackService` (from `ovos_audio.service`) with a `MockTTS` on a +`PlaybackServiceHarness` wraps `PlaybackService` (from `ovos_audio.service`) with a `MockTTS` on a `FakeBus`. Use it when testing TTS execution flow: `speak` messages, the `recognizer_loop:audio_output_start/end` lifecycle, and optional mic-listen triggers after speech. @@ -55,11 +55,11 @@ with PlaybackServiceHarness() as h: ## Stop Guard Pitfall -`AudioService._stop()` — `ovos-audio/ovos_audio/audio.py` — checks +`AudioService._stop()` (`ovos-audio/ovos_audio/audio.py`) checks `time.monotonic() - self.play_start_time > 1`. If stop is called within 1 second of `play()`, the stop command is silently ignored. -**Tests that call `stop()` must sleep at least 1.1 seconds after `play()`:** +Tests that call `stop()` must sleep at least 1.1 seconds after `play()`: ```python import time @@ -74,7 +74,7 @@ with AudioServiceHarness() as h: ## play_audio Patch Rationale -`PlaybackThread._play()` — `ovos-audio/ovos_audio/playback.py` — calls +`PlaybackThread._play()` (`ovos-audio/ovos_audio/playback.py`) calls `play_audio(data)` then waits on the returned process object. Without patching, this would invoke a real audio player binary (sox, aplay, paplay, mpg123). @@ -111,13 +111,13 @@ with AudioServiceHarness() as h: ``` `AudioServiceHarness.get_track_info()` and `list_backends()` already implement -this pattern internally — `ovoscope/audio.py`. +this pattern internally, in `ovoscope/audio.py`. ## API Reference ### MockAudioBackend -`MockAudioBackend` — `ovoscope/audio.py` +`MockAudioBackend` (`ovoscope/audio.py`) | Attribute / Method | Type | Description | |---|---|---| @@ -132,7 +132,7 @@ this pattern internally — `ovoscope/audio.py`. ### AudioServiceHarness -`AudioServiceHarness` — `ovoscope/audio.py` +`AudioServiceHarness` (`ovoscope/audio.py`) | Method | Description | |---|---| @@ -151,7 +151,7 @@ this pattern internally — `ovoscope/audio.py`. ### MockTTS -`MockTTS` — `ovoscope/audio.py` +`MockTTS` (`ovoscope/audio.py`) | Attribute / Method | Description | |---|---| @@ -162,7 +162,7 @@ this pattern internally — `ovoscope/audio.py`. ### PlaybackServiceHarness -`PlaybackServiceHarness` — `ovoscope/audio.py` +`PlaybackServiceHarness` (`ovoscope/audio.py`) | Method | Description | |---|---| @@ -175,7 +175,7 @@ this pattern internally — `ovoscope/audio.py`. ### AudioCaptureSession -`AudioCaptureSession` — `ovoscope/audio.py` +`AudioCaptureSession` (`ovoscope/audio.py`) | Method / Property | Description | |---|---| @@ -190,9 +190,12 @@ Default `track_prefixes` captures: `"mycroft.audio."`, ## Cross-References -- `AudioService` — `ovos-audio/ovos_audio/audio.py` -- `PlaybackService` — `ovos-audio/ovos_audio/service.py` -- `PlaybackThread` — `ovos-audio/ovos_audio/playback.py` -- `AudioBackend` (base class) — `ovos_plugin_manager.templates.audio.AudioBackend` -- `TTS` (base class) — `ovos_plugin_manager.templates.tts.TTS` -- End-to-end tests — `ovos-audio/test/end2end/` +- `AudioService` (`ovos-audio/ovos_audio/audio.py`) +- `PlaybackService` (`ovos-audio/ovos_audio/service.py`) +- `PlaybackThread` (`ovos-audio/ovos_audio/playback.py`) +- `AudioBackend`, the base class (`ovos_plugin_manager.templates.audio.AudioBackend`) +- `TTS`, the base class (`ovos_plugin_manager.templates.tts.TTS`) +- End-to-end tests: `ovos-audio/test/end2end/` + +--- +[← Pydantic Integration](pydantic-integration.md) · [Home](../README.md) · [Media Testing →](media-testing.md) diff --git a/docs/bus-coverage.md b/docs/bus-coverage.md index 3e84304..775c9f6 100644 --- a/docs/bus-coverage.md +++ b/docs/bus-coverage.md @@ -141,17 +141,17 @@ TOTAL 10/16 62.5% 10/15 6/15 In verbose mode (`--ovoscope-bus-cov-verbose`), `ovoscope` lists every message type: ``` -LISTENERS — my-skill.author - ✓ my-intent.intent 2 invocation(s) - ✗ some-unused-event NOT TESTED +LISTENERS: my-skill.author + [x] my-intent.intent 2 invocation(s) + [ ] some-unused-event NOT TESTED -EMITTERS — my-skill.author - ✓ speak observed 1x ✓ asserted - ✓ my-skill.done observed 1x ✗ not asserted +EMITTERS: my-skill.author + [x] speak observed 1x [x] asserted + [x] my-skill.done observed 1x [ ] not asserted ``` -* **✓ (Checked)**: The listener was triggered or the emitter was asserted. -* **✗ (Cross)**: The listener was never triggered or the emitter was seen but not checked in the test. +* **[x] (Checked)**: The listener was triggered or the emitter was asserted. +* **[ ] (Not checked)**: The listener was never triggered or the emitter was seen but not checked in the test. --- @@ -174,4 +174,7 @@ If you are building custom tooling, you can access these values via `SkillBusCov * `observed_emitter_pct`: `(observed_emitters / total_emitters) * 100` * `asserted_emitter_pct`: `(asserted_emitters / total_emitters) * 100` -Source: `SkillBusCoverage` — `ovoscope/bus_coverage.py` +Source: `SkillBusCoverage` (`ovoscope/bus_coverage.py`) + +--- +[← GUI Testing](gui-testing.md) · [Home](../README.md) diff --git a/docs/capture-session.md b/docs/capture-session.md index 209fe95..146cd43 100644 --- a/docs/capture-session.md +++ b/docs/capture-session.md @@ -1,13 +1,13 @@ # CaptureSession `CaptureSession` subscribes to all messages on the `FakeBus` and records them during a single test interaction. It handles synchronous responses (ordered, from the intent pipeline) and asynchronous responses (from external threads, unordered). -## Class: `CaptureSession` — `ovoscope/__init__.py` +## Class: `CaptureSession` (`ovoscope/__init__.py`) ```python from ovoscope import CaptureSession ``` A `dataclass` that wraps a `MiniCroft` and manages message collection for one test interaction. -`CaptureSession.finish` — `ovoscope/__init__.py` +`CaptureSession.finish` (`ovoscope/__init__.py`) -> **Idempotency:** `finish()` may be called multiple times safely — subsequent calls +> **Idempotency:** `finish()` may be called multiple times safely: subsequent calls > return the same message list without re-subscribing or clearing state. ### Fields | Field | Type | Default | Description | @@ -18,7 +18,7 @@ A `dataclass` that wraps a `MiniCroft` and manages message collection for one te | `eof_msgs` | `list[str]` | `["ovos.utterance.handled"]` | Message types that signal end of interaction | | `ignore_messages` | `list[str]` | `["ovos.skills.settings_changed"]` | Message types to discard | | `async_messages` | `list[str]` | `[]` | Message types to route to `async_responses` instead | -| `done` | `threading.Event` | — | Set when an EOF message is received | +| `done` | `threading.Event` |: | Set when an EOF message is received | ### Methods #### `capture(source_message, timeout=20)` Emits `source_message` on the bus and waits for an EOF message (or timeout). Subsequent calls on the same session accumulate into `responses`. @@ -86,3 +86,6 @@ capture.capture(follow_up, timeout=10) all_messages = capture.finish() ``` `End2EndTest` does this automatically when `source_message` is a list. + +--- +[← MiniCroft](minicroft.md) · [Home](../README.md) · [End2EndTest →](end2end-test.md) diff --git a/docs/ci-integration.md b/docs/ci-integration.md index 6692da4..c5b3c18 100644 --- a/docs/ci-integration.md +++ b/docs/ci-integration.md @@ -1,4 +1,4 @@ -# CI Integration — ovoscope +# CI Integration: ovoscope This document explains how to wire ovoscope end-to-end tests into a repo's CI pipeline using `gh-automations` reusable workflows, and how to structure test files and fixtures. --- @@ -16,7 +16,7 @@ my-skill-repo/ ├── setup.py (or pyproject.toml) └── ... ``` -Separate `end2end/` from `unittests/` so they can be run independently — end2end tests are +Separate `end2end/` from `unittests/` so they can be run independently: end2end tests are slower (they spin up a MiniCroft) and may require extra dependencies. --- ## pytest / unittest Configuration @@ -65,9 +65,9 @@ Fixture files generated by `End2EndTest.save()` (see [usage-guide.md](usage-guid contain the expected message sequence serialised as JSON. **When to commit fixtures:** - Commit fixtures that test stable, deterministic interactions (e.g., a specific dialog line). -- Do NOT commit fixtures where the `speak` utterance varies randomly — either omit the +- Do NOT commit fixtures where the `speak` utterance varies randomly: either omit the `utterance` key from expected data or use manual assertion instead. -- Always generate fixtures with `anonymize=True` (the default) — this strips real location data. +- Always generate fixtures with `anonymize=True` (the default): this strips real location data. **`.gitignore` pattern** (if you generate fixtures locally but don't want to commit them): ```gitignore test/end2end/fixtures/*.json @@ -77,7 +77,7 @@ Or selectively ignore only generated/recording artifacts: test/end2end/fixtures/recorded_*.json ``` --- -## GitHub Actions — End2End Job +## GitHub Actions: End2End Job Add an end2end job to your `release_workflow.yml` or a dedicated workflow. This example follows the `gh-automations` conventions used across all 203+ OVOS repos: ```yaml @@ -114,7 +114,7 @@ jobs: propose_release: true secrets: inherit ``` -The `build_tests` job runs before `publish_alpha` — a failing end2end test blocks the release. +The `build_tests` job runs before `publish_alpha`: a failing end2end test blocks the release. --- ## Standalone End2End Workflow If your repo only needs end2end tests (no release automation), use a simpler workflow: @@ -181,11 +181,14 @@ The ovoscope repository itself uses the standard OVOS workflow set: | **Release Alpha** | `release_workflow.yml` | PR merge to `dev` | Runs tests first, then calls `publish-alpha.yml` | | **Stable Release** | `publish_stable.yml` | Push to `master` | Calls `publish-stable.yml` with bot loop guard | | **Labels** | `conventional-label.yaml` | PR open/edit | Auto-labels PRs with conventional commit types | -The release workflow gates alpha publishing on test success — a failing test blocks the release. +The release workflow gates alpha publishing on test success: a failing test blocks the release. --- ## See Also -- [usage-guide.md](usage-guide.md) — tutorial walkthrough with all patterns -- [gh-automations/docs/workflow-reference.md](../../gh-automations/docs/workflow-reference.md) — full reusable workflow reference -- [gh-automations/docs/repo-setup.md](../../gh-automations/docs/repo-setup.md) — per-repo workflow setup +- [usage-guide.md](usage-guide.md): tutorial walkthrough with all patterns +- [gh-automations/docs/workflow-reference.md](../../gh-automations/docs/workflow-reference.md): full reusable workflow reference +- [gh-automations/docs/repo-setup.md](../../gh-automations/docs/repo-setup.md): per-repo workflow setup - Canonical examples: `Skills/ovos-skill-hello-world/test/test_helloworld.py` - Core examples: `ovos-core/test/end2end/` + +--- +[← CLI](cli.md) · [Home](../README.md) · [MiniCroft →](minicroft.md) diff --git a/docs/cli.md b/docs/cli.md index 771d2c1..fe29a70 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -16,10 +16,10 @@ ovoscope --help ## Subcommands -### `ovoscope record` — Record a fixture +### `ovoscope record`: Record a fixture **In-process recording** (default): loads the skill(s) inside the current -process using `MiniCroft` — `cli.py:cmd_record`. +process using `MiniCroft` (`cli.py:cmd_record`). ```bash ovoscope record \ @@ -30,7 +30,7 @@ ovoscope record \ --timeout 20 ``` -**Live recording** from a running OVOS instance (`RemoteRecorder` — +**Live recording** from a running OVOS instance (`RemoteRecorder`, in `remote_recorder.py:RemoteRecorder.record`): ```bash @@ -43,7 +43,7 @@ ovoscope record --live \ | Flag | Default | Description | |------|---------|-------------| -| `--skill-id` | — | OPM skill IDs to load (repeatable). | +| `--skill-id` | none | OPM skill IDs to load (repeatable). | | `--utterance` | **required** | User utterance text. | | `--output` | **required** | Output fixture JSON path. | | `--lang` | `en-US` | Language tag. | @@ -54,9 +54,9 @@ ovoscope record --live \ --- -### `ovoscope run` — Replay a fixture +### `ovoscope run`: Replay a fixture -Replays a saved fixture file and exits with code 1 on failure — +Replays a saved fixture file and exits with code 1 on failure, in `cli.py:cmd_run`. ```bash @@ -72,10 +72,10 @@ ovoscope run test/fixtures/hello.json --verbose --timeout 30 --- -### `ovoscope diff` — Compare two fixtures +### `ovoscope diff`: Compare two fixtures -Compares two fixture files and prints a colored report — -`diff.py:diff_fixtures`, `cli.py:cmd_diff`. +Compares two fixture files and prints a colored report, in +`diff.py:diff_fixtures` and `cli.py:cmd_diff`. ```bash ovoscope diff expected.json actual.json @@ -93,9 +93,9 @@ Exits 0 if identical, 1 if differences are found. --- -### `ovoscope validate` — Schema-validate fixtures +### `ovoscope validate`: Schema-validate fixtures -Validates one or more fixture files against the expected schema — +Validates one or more fixture files against the expected schema, in `cli.py:cmd_validate`. ```bash @@ -109,10 +109,10 @@ is a list) when the `pydantic` extra is not installed. --- -### `ovoscope coverage` — Ecosystem coverage scan +### `ovoscope coverage`: Ecosystem coverage scan -Scans a workspace root for OVOS plugin repos and reports E2E test coverage — -`coverage.py:scan_workspace`, `cli.py:cmd_coverage`. +Scans a workspace root for OVOS plugin repos and reports E2E test coverage, in +`coverage.py:scan_workspace` and `cli.py:cmd_coverage`. ```bash ovoscope coverage "OpenVoiceOS Workspace/" --format table @@ -186,3 +186,6 @@ With no explicit `--claude`/`--gemini` flag, the tool auto-detects which of |------|---------| | 0 | Success / no differences / all valid | | 1 | Failure / differences found / validation error | + +--- +[← Usage Guide](usage-guide.md) · [Home](../README.md) · [CI Integration →](ci-integration.md) diff --git a/docs/end2end-test.md b/docs/end2end-test.md index 7009cd9..28a749b 100644 --- a/docs/end2end-test.md +++ b/docs/end2end-test.md @@ -1,11 +1,11 @@ # End2EndTest `End2EndTest` is the primary API. It wires together `MiniCroft`, `CaptureSession`, and all assertion logic into a single declarative test object. -## Class: `End2EndTest` — `ovoscope/__init__.py` +## Class: `End2EndTest`: `ovoscope/__init__.py` ```python from ovoscope import End2EndTest ``` A `dataclass`. Configure once, call `.execute()` to run. -`End2EndTest.execute` — `ovoscope/__init__.py` +`End2EndTest.execute`: `ovoscope/__init__.py` --- ## Fields ### Core @@ -59,7 +59,7 @@ All default to `True`. Set to `False` to skip individual assertion categories: --- ## `execute(timeout=30)` Runs the test. Raises `AssertionError` on the first failing assertion. -If `minicroft` is `None`, creates one automatically (managed mode — stops it after the test). To run multiple tests against the same loaded skills, pass your own `MiniCroft`: +If `minicroft` is `None`, creates one automatically (managed mode: stops it after the test). To run multiple tests against the same loaded skills, pass your own `MiniCroft`: ```python from ovoscope import get_minicroft, End2EndTest croft = get_minicroft(["skill-weather.openvoiceos"]) @@ -82,19 +82,19 @@ For each `(expected, received)` pair: ```python assert expected.msg_type == received.msg_type ``` -**Data check** — subset match (expected keys must be present with matching values): +**Data check**: subset match (expected keys must be present with matching values): ```python for k, v in expected.data.items(): assert received.data[k] == v ``` -**Context check** — same subset pattern: +**Context check**: same subset pattern: ```python for k, v in expected.context.items(): assert received.context[k] == v ``` -**Routing check** — tracks rolling expected source/destination: +**Routing check**: tracks rolling expected source/destination: - Starts from `source_message[0].context["source"]` and `["destination"]` -- On `entry_points` message: flips (`e_src, e_dst = r_dst, r_src`) — the reply comes back the other way +- On `entry_points` message: flips (`e_src, e_dst = r_dst, r_src`): the reply comes back the other way - On `flip_points` message: updates expected from received, then swaps - `keep_original_src` always uses the original, regardless of flips ### Active skill tracking @@ -186,3 +186,6 @@ try: finally: croft.stop() ``` + +--- +[← Capture Session](capture-session.md) · [Home](../README.md) · [Pipeline →](pipeline.md) diff --git a/docs/gui-testing.md b/docs/gui-testing.md index 5a15919..bf0359d 100644 --- a/docs/gui-testing.md +++ b/docs/gui-testing.md @@ -7,7 +7,7 @@ and namespace teardown without cluttering the main message capture. ## Why GUI Messages Are Separate `End2EndTest` filters `gui.*` messages out by default (`ignore_gui=True`). This is -deliberate — GUI namespace churn (``gui.value.set``, ``gui.clear.namespace``) is +deliberate: GUI namespace churn (``gui.value.set``, ``gui.clear.namespace``) is high-frequency and rarely the focus of intent/dialogue tests. `GUICaptureSession` provides a complementary, opt-in capture layer for tests that *do* care about GUI state. @@ -65,7 +65,7 @@ mc.stop() ## Class: `GUICaptureSession` -`GUICaptureSession` — `ovoscope/__init__.py` +`GUICaptureSession`: `ovoscope/__init__.py` ```python from ovoscope import GUICaptureSession @@ -89,7 +89,7 @@ recording GUI-prefixed messages. ### Lifecycle Methods -`GUICaptureSession.start` — `ovoscope/__init__.py` +`GUICaptureSession.start`: `ovoscope/__init__.py` ```python gui = GUICaptureSession(mc.bus) @@ -103,7 +103,7 @@ gui.stop() | `start()` | Subscribe to the bus and begin capturing. | | `stop()` | Unsubscribe from the bus and stop capturing. | -`GUICaptureSession.__enter__` / `__exit__` — `ovoscope/__init__.py` +`GUICaptureSession.__enter__` / `__exit__`: `ovoscope/__init__.py` The preferred usage is as a context manager. `__enter__` calls `start()`; `__exit__` calls `stop()`. @@ -112,7 +112,7 @@ The preferred usage is as a context manager. `__enter__` calls `start()`; #### `assert_page_shown(namespace, page, timeout=2.0, exact=True)` -`GUICaptureSession.assert_page_shown` — `ovoscope/__init__.py` +`GUICaptureSession.assert_page_shown`: `ovoscope/__init__.py` Assert that a `gui.page.show` (or equivalent) message was emitted for the given namespace and page filename. @@ -138,7 +138,7 @@ that merely contains yours cannot satisfy the assertion. #### `assert_namespace_value(namespace, key, value, exact=True)` -`GUICaptureSession.assert_namespace_value` — `ovoscope/__init__.py` +`GUICaptureSession.assert_namespace_value`: `ovoscope/__init__.py` Assert that a `gui.value.set` or `gui.namespace.update` message set a specific key to a specific value in the given namespace. @@ -157,7 +157,7 @@ Raises `AssertionError` if no matching message is found. #### `assert_namespace_has_key(namespace, key, exact=True)` -`GUICaptureSession.assert_namespace_has_key` — `ovoscope/__init__.py` +`GUICaptureSession.assert_namespace_has_key`: `ovoscope/__init__.py` Assert that a `gui.value.set` or `gui.namespace.update` message set a specific key in the given namespace, regardless of value. Useful for @@ -177,7 +177,7 @@ Raises `AssertionError` if no matching message is found. #### `assert_namespace_cleared(namespace, exact=True)` -`GUICaptureSession.assert_namespace_cleared` — `ovoscope/__init__.py` +`GUICaptureSession.assert_namespace_cleared`: `ovoscope/__init__.py` Assert that a `gui.namespace.remove`, `gui.namespace.clear`, or `gui.clear.namespace` message was emitted for the given namespace — @@ -193,7 +193,7 @@ Raises `AssertionError` if no matching message is found. ## Message Filtering Only messages whose `msg_type` starts with one of the configured `prefixes` -are captured — `GUICaptureSession._on_message` — `ovoscope/__init__.py`. +are captured: `GUICaptureSession._on_message`: `ovoscope/__init__.py`. All other bus messages are ignored. Default captured message types (partial list): @@ -218,7 +218,7 @@ with GUICaptureSession(mc.bus) as gui: minicroft=mc, source_message=utterance, expected_messages=[...], - ignore_gui=True, # default — keeps End2EndTest clean + ignore_gui=True, # default: keeps End2EndTest clean ) test.execute() # Now assert GUI state separately @@ -251,20 +251,23 @@ with GUICaptureSession(mc.bus) as gui: ``` This is the recommended assertion for the template-based GUI: it does not care -which display backend (Qt, pyhtmx, …) renders the template — only that the skill +which display backend (Qt, pyhtmx, …) renders the template: only that the skill requested the right `SYSTEM_*` template with the right session data. ## What `GUICaptureSession` Does NOT Cover -- Full GUI rendering — only bus messages are captured; no QML engine is run. -- `ovos-gui` service behaviour — only the `FakeBus` in-process messages are +- Full GUI rendering: only bus messages are captured; no QML engine is run. +- `ovos-gui` service behaviour: only the `FakeBus` in-process messages are captured; messages sent to a real GUI over WebSocket are not included. - GUI framework events not prefixed with `gui.` or `mycroft.gui.` (these can be added via the `prefixes` constructor argument). ## Cross-References -- `CaptureSession` — `ovoscope/docs/capture-session.md` (ordered dialogue capture) -- `End2EndTest` — `ovoscope/docs/end2end-test.md` (full test runner) -- `MiniCroft` / `get_minicroft()` — `ovoscope/docs/minicroft.md` -- `GUI_IGNORED` message list — `ovoscope/__init__.py` +- `CaptureSession`: `ovoscope/docs/capture-session.md` (ordered dialogue capture) +- `End2EndTest`: `ovoscope/docs/end2end-test.md` (full test runner) +- `MiniCroft` / `get_minicroft()`: `ovoscope/docs/minicroft.md` +- `GUI_IGNORED` message list: `ovoscope/__init__.py` + +--- +[← Voice Loop](voice-loop.md) · [Home](../README.md) · [Bus Coverage →](bus-coverage.md) diff --git a/docs/index.md b/docs/index.md index a370ec3..77e23f1 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,24 +1,26 @@ # OvoScope Documentation -**OvoScope** is an end-to-end testing framework for OVOS skills. It runs a lightweight in-process OVOS Core using a `FakeBus`, loads real skill plugins, and captures every bus message produced in response to a test utterance — then asserts against the captured sequence. +**OvoScope** is an end-to-end testing framework for OVOS skills. It runs a lightweight in-process OVOS Core using a `FakeBus`, loads real skill plugins, and captures every bus message produced in response to a test utterance: then asserts against the captured sequence. ## Contents | Document | Description | |---|---| -| [usage-guide.md](usage-guide.md) | **Start here** — tutorial: from zero to your first end2end test | +| [usage-guide.md](usage-guide.md) | **Start here**: tutorial: from zero to your first end2end test | +| [cli.md](cli.md) | `ovoscope` command-line tool: record, run, diff, validate, coverage | | [ci-integration.md](ci-integration.md) | Wiring ovoscope into GitHub Actions CI with gh-automations | -| [minicroft.md](minicroft.md) | `MiniCroft` — in-process skill runtime | -| [capture-session.md](capture-session.md) | `CaptureSession` — message capture during a test | -| [end2end-test.md](end2end-test.md) | `End2EndTest` — full test runner reference | -| [e2e-pipeline-harness.md](e2e-pipeline-harness.md) | `E2EPipelineHarness`, `wait_for_match`, `make_utterance_message` — testing a single pipeline plugin (Adapt/Padatious/…) against raw bus messages | -| [intent-cases.md](intent-cases.md) | `IntentCase`, `register_intent_case_tests` — file-based intent test cases (`.intent.test`) with per-pipeline-family generated tests | +| [minicroft.md](minicroft.md) | `MiniCroft`: in-process skill runtime | +| [capture-session.md](capture-session.md) | `CaptureSession`: message capture during a test | +| [end2end-test.md](end2end-test.md) | `End2EndTest`: full test runner reference | +| [e2e-pipeline-harness.md](e2e-pipeline-harness.md) | `E2EPipelineHarness`, `wait_for_match`, `make_utterance_message`: testing a single pipeline plugin (Adapt/Padatious/...) against raw bus messages | +| [intent-cases.md](intent-cases.md) | `IntentCase`, `register_intent_case_tests`: file-based intent test cases (`.intent.test`) with per-pipeline-family generated tests | | [pydantic-integration.md](pydantic-integration.md) | Using `ovos-pydantic-models` with OvoScope | -| [audio-testing.md](audio-testing.md) | `AudioServiceHarness`, `PlaybackServiceHarness` — testing audio services | -| [media-testing.md](media-testing.md) | `OCPPlayerHarness`, `OCPCaptureSession`, `MockOCPBackend` — testing the `ovos-media` OCP player (and driving a real OCP backend) | -| [media-provider-testing.md](media-provider-testing.md) | `MediaProviderHarness` — testing `opm.media.provider` catalog/search plugins | -| [ocp.md](ocp.md) | `OCPTest` — testing legacy OCP search skills (`@ocp_search`) | -| [listener.md](listener.md) | `MiniListener`, `get_mini_listener`, `ListenerTest`, `MockVADEngine`, `MockHotWordEngine`, `VADTest`, `WakeWordTest` — testing audio transformer plugins, STT pipeline, VAD, and wake-word | -| [voice-loop.md](voice-loop.md) | `MiniVoiceLoop` / `MiniSimpleListener` / `MiniClassicListener` — file-driven bus-sequence testing for the ovos-dinkum, ovos-simple, and mycroft-classic listener services (wake-word → record-begin → utterance), with verifier-chain gating | -| [gui-testing.md](gui-testing.md) | `GUICaptureSession` — asserting GUI page navigation and namespace values | -| [bus-coverage.md](bus-coverage.md) | `BusCoverageTracker`, `BusCoverageReport` — measuring handler and emitter coverage per skill | +| [audio-testing.md](audio-testing.md) | `AudioServiceHarness`, `PlaybackServiceHarness`: testing audio services | +| [media-testing.md](media-testing.md) | `OCPPlayerHarness`, `OCPCaptureSession`, `MockOCPBackend`: testing the `ovos-media` OCP player (and driving a real OCP backend) | +| [media-provider-testing.md](media-provider-testing.md) | `MediaProviderHarness`: testing `opm.media.provider` catalog/search plugins | +| [ocp.md](ocp.md) | `OCPTest`: testing legacy OCP search skills (`@ocp_search`) | +| [phal.md](phal.md) | `MiniPHAL`, `PHALTest`: testing PHAL plugins without physical hardware | +| [listener.md](listener.md) | `MiniListener`, `get_mini_listener`, `ListenerTest`, `MockVADEngine`, `MockHotWordEngine`, `VADTest`, `WakeWordTest`: testing audio transformer plugins, STT pipeline, VAD, and wake-word | +| [voice-loop.md](voice-loop.md) | `MiniVoiceLoop` / `MiniSimpleListener` / `MiniClassicListener`: file-driven bus-sequence testing for the ovos-dinkum, ovos-simple, and mycroft-classic listener services (wake-word → record-begin → utterance), with verifier-chain gating | +| [gui-testing.md](gui-testing.md) | `GUICaptureSession`: asserting GUI page navigation and namespace values | +| [bus-coverage.md](bus-coverage.md) | `BusCoverageTracker`, `BusCoverageReport`: measuring handler and emitter coverage per skill | ## Conceptual Model ``` Test FakeBus @@ -30,7 +32,7 @@ source_message ──emit──► [MiniCroft + loaded skills] ▼ assert against expected_messages[] ``` -The key insight is that OVOS skill behaviour is fully observable through bus messages. OvoScope intercepts every message on the in-process `FakeBus`, so the entire skill interaction — intent matching, converse, fallback, speak, session changes — is captured and verifiable. +The key insight is that OVOS skill behaviour is fully observable through bus messages. OvoScope intercepts every message on the in-process `FakeBus`, so the entire skill interaction (intent matching, converse, fallback, speak, session changes), is captured and verifiable. ## Quick Start ```bash pip install ovoscope @@ -113,7 +115,7 @@ from ovoscope import SerializedMessage, SerializedTest Python 3.10+ is required (uses `match`/structural typing in ovos-core). ## Listener Pipeline Testing -`MiniListener` extends ovoscope to cover **audio transformer plugins** — the +`MiniListener` extends ovoscope to cover **audio transformer plugins**: the plugins that process raw audio before it reaches the intent engine. It wraps `AudioTransformersService` on a `FakeBus` so transformer behaviour is fully observable through bus messages. @@ -134,12 +136,12 @@ listener.shutdown() ## Listener-Service Bus-Sequence Testing -OVOS has several listener **services** — ovos-dinkum-listener, ovos-simple-listener, -and mycroft-classic-listener — each emitting the same `recognizer_loop:*` bus +OVOS has several listener **services**: ovos-dinkum-listener, ovos-simple-listener, +and mycroft-classic-listener: each emitting the same `recognizer_loop:*` bus events. `MiniVoiceLoop`, `MiniSimpleListener`, and `MiniClassicListener` each wire their real service to a `FakeBus` with mock mic/VAD/STT/wake-word plugins, drive it over an arbitrary audio file (or PCM frames), and capture the emitted -sequence — sharing one set of assertion helpers. `MiniVoiceLoop` also exercises +sequence: sharing one set of assertion helpers. `MiniVoiceLoop` also exercises the dinkum verifier-chain gate that decides whether a detection survives. See [voice-loop.md](voice-loop.md) for full API reference and usage patterns. @@ -158,10 +160,13 @@ with MiniVoiceLoop(ww_instances={"hey_mycroft": ww}, ``` ## What OvoScope Does NOT Do -- Does not start a real WebSocket MessageBus server — uses `FakeBus` (in-process pub/sub). -- `MiniCroft` / `End2EndTest` — the harness this page documents — does not load PHAL plugins or the audio service; it only loads skills and the intent pipeline. PHAL plugins and audio services have their own dedicated harnesses: [phal.md](phal.md) (`MiniPHAL`, `PHALTest`) and [audio-testing.md](audio-testing.md) (`AudioServiceHarness`, `PlaybackServiceHarness`). -- Does not test GUI rendering — GUI namespace messages are ignored by default (`ignore_gui=True`). -- Does not test TTS — operates at the `recognizer_loop:utterance` level (see [audio-testing.md](audio-testing.md) for TTS lifecycle testing). +- Does not start a real WebSocket MessageBus server: uses `FakeBus` (in-process pub/sub). +- `MiniCroft` / `End2EndTest`, the harness this page documents, does not load PHAL plugins or the + audio service; it only loads skills and the intent pipeline. PHAL plugins and audio services have + their own dedicated harnesses: [phal.md](phal.md) (`MiniPHAL`, `PHALTest`) and + [audio-testing.md](audio-testing.md) (`AudioServiceHarness`, `PlaybackServiceHarness`). +- Does not test GUI rendering: GUI namespace messages are ignored by default (`ignore_gui=True`). +- Does not test TTS: operates at the `recognizer_loop:utterance` level (see [audio-testing.md](audio-testing.md) for TTS lifecycle testing). - `MiniListener` covers `AudioTransformersService`, the STT pipeline, and mock VAD/WakeWord engines. `MiniVoiceLoop` / `MiniSimpleListener` / `MiniClassicListener` drive the dinkum, simple, and classic listener **services** from an audio file and capture the `recognizer_loop:*` bus sequence; the classic file drive is best-effort (energy-based pipeline). ## Quick Links | Resource | Path | @@ -172,10 +177,10 @@ with MiniVoiceLoop(ww_instances={"hey_mycroft": ww}, | Repo | Test location | Notes | |---|---|---| | `ovos-core` | `ovos-core/test/end2end/` | Adapt + Padatious pipeline tests, blacklist tests | -| `Skills/ovos-skill-hello-world` | `Skills/ovos-skill-hello-world/test/test_helloworld.py` | Canonical example — Adapt + Padatious match + no-match | +| `Skills/ovos-skill-hello-world` | `Skills/ovos-skill-hello-world/test/test_helloworld.py` | Canonical example: Adapt + Padatious match + no-match | ## Cross-References -- [ovos-core](https://github.com/OpenVoiceOS/ovos-core) — `SkillManager`, `IntentService` (runtime dependency) -- [ovos-utils](https://github.com/OpenVoiceOS/ovos-utils) — `FakeBus`, `ProcessState` -- [ovos-workshop](https://github.com/OpenVoiceOS/ovos-workshop) — `OVOSSkill` base class -- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client) — `Message`, `Session`, `SessionManager` -- [ovos-pydantic-models](https://github.com/OpenVoiceOS/ovos-pydantic-models) — optional typed message models (see [pydantic-integration.md](pydantic-integration.md)) +- [ovos-core](https://github.com/OpenVoiceOS/ovos-core): `SkillManager`, `IntentService` (runtime dependency) +- [ovos-utils](https://github.com/OpenVoiceOS/ovos-utils): `FakeBus`, `ProcessState` +- [ovos-workshop](https://github.com/OpenVoiceOS/ovos-workshop): `OVOSSkill` base class +- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client): `Message`, `Session`, `SessionManager` +- [ovos-pydantic-models](https://github.com/OpenVoiceOS/ovos-pydantic-models): optional typed message models (see [pydantic-integration.md](pydantic-integration.md)) diff --git a/docs/listener.md b/docs/listener.md index 96d6cef..239e023 100644 --- a/docs/listener.md +++ b/docs/listener.md @@ -1,7 +1,7 @@ -# MiniListener — Listener Pipeline Testing +# MiniListener: Listener Pipeline Testing `MiniListener` extends ovoscope's testing capability beyond the skill pipeline -to cover **audio transformer plugins** — the plugins that process raw audio +to cover **audio transformer plugins**: the plugins that process raw audio chunks before speech reaches the intent engine. ## Conceptual Model @@ -36,7 +36,7 @@ WAV file / bytes Rather than injecting a `recognizer_loop:utterance` (as `MiniCroft` does), `MiniListener` feeds **raw audio bytes** into `AudioTransformersService` — -`ovos_dinkum_listener/transformers.py` — which dispatches them to each +`ovos_dinkum_listener/transformers.py`: which dispatches them to each loaded plugin's `feed_audio_chunk()` / `feed_speech_chunk()` / `transform()` methods. All `Message` objects emitted on the internal `FakeBus` during that call are captured and returned. @@ -68,7 +68,7 @@ assert any(m.msg_type == "recognizer_loop:utterance" for m in msgs) listener.shutdown() ``` -**Streaming real ggwave audio** — the ggwave decoder only fires after it has +**Streaming real ggwave audio**: the ggwave decoder only fires after it has accumulated enough frames, so feed the whole waveform with `feed_audio_stream`, which keeps every message emitted across the stream (unlike `feed_audio`, which clears its buffer on each call): @@ -113,49 +113,49 @@ listener.shutdown() ## API Reference -### `MiniListener` — `ovoscope/listener.py` +### `MiniListener`: `ovoscope/listener.py` **Constructor parameters:** | Parameter | Type | Description | |-----------|------|-------------| | `config` | `dict` | Full OVOS config with `listener.audio_transformers` key | -| `plugin_instances` | `dict[str, Any]` | Pre-instantiated transformer plugins; bypasses OPM discovery | +| `plugin_instances` | `dict[str, Any]` | Pre-instantiated transformer plugins. Bypasses OPM discovery | | `stt_instance` | `Any` | Optional STT plugin to use in `listen()` | -| `vad_instance` | `Any` | Optional VAD engine (e.g. `MockVADEngine`) — `ovoscope/listener.py` | -| `ww_instances` | `dict[str, Any]` | Optional wake-word engines keyed by name — `ovoscope/listener.py` | +| `vad_instance` | `Any` | Optional VAD engine (e.g. `MockVADEngine`): `ovoscope/listener.py` | +| `ww_instances` | `dict[str, Any]` | Optional wake-word engines keyed by name: `ovoscope/listener.py` | **Audio transformer methods:** | Method | Signature | Description | |--------|-----------|-------------| -| `feed_audio(chunk)` — `ovoscope/listener.py` | `(bytes) → List[Message]` | Calls `AudioTransformersService.feed_audio()`. Requires `ovos-dinkum-listener`. | -| `feed_speech(chunk)` — `ovoscope/listener.py` | `(bytes) → List[Message]` | Calls `AudioTransformersService.feed_speech()`. Requires `ovos-dinkum-listener`. | +| `feed_audio(chunk)`: `ovoscope/listener.py` | `(bytes) → List[Message]` | Calls `AudioTransformersService.feed_audio()`. Requires `ovos-dinkum-listener`. | +| `feed_speech(chunk)`: `ovoscope/listener.py` | `(bytes) → List[Message]` | Calls `AudioTransformersService.feed_speech()`. Requires `ovos-dinkum-listener`. | | `feed_audio_stream(chunks, feed, chunk_size)` | `(bytes\|list[bytes], str, int) → List[Message]` | Streams frames in order **without** clearing between them; aggregates all emitted messages. Use for decoders that fire after many frames (ggwave). | -| `transform(chunk)` — `ovoscope/listener.py` | `(bytes) → tuple[bytes, dict, List[Message]]` | Full transform pipeline; returns `(audio, ctx, messages)`. Requires `ovos-dinkum-listener`. | -| `listen(audio, ...)` — `ovoscope/listener.py` | `(audio, language, stt_instance, ...) → List[Message]` | Full pipeline: audio → transformers → STT → utterance message. Requires `ovos-dinkum-listener`. | +| `transform(chunk)`: `ovoscope/listener.py` | `(bytes) → tuple[bytes, dict, List[Message]]` | Full transform pipeline; returns `(audio, ctx, messages)`. Requires `ovos-dinkum-listener`. | +| `listen(audio, ...)`: `ovoscope/listener.py` | `(audio, language, stt_instance, ...) → List[Message]` | Full pipeline: audio → transformers → STT → utterance message. Requires `ovos-dinkum-listener`. | **VAD methods:** | Method | Signature | Description | |--------|-----------|-------------| -| `is_silence(chunk)` — `ovoscope/listener.py` | `(bytes) → bool` | Delegates to the injected VAD engine. Raises `RuntimeError` if no VAD engine set. | -| `extract_speech(audio)` — `ovoscope/listener.py` | `(bytes) → bytes` | Returns only speech frames from `audio`. Raises `RuntimeError` if no VAD engine set. | +| `is_silence(chunk)`: `ovoscope/listener.py` | `(bytes) → bool` | Delegates to the injected VAD engine. Raises `RuntimeError` if no VAD engine set. | +| `extract_speech(audio)`: `ovoscope/listener.py` | `(bytes) → bytes` | Returns only speech frames from `audio`. Raises `RuntimeError` if no VAD engine set. | **Wake-word methods:** | Method | Signature | Description | |--------|-----------|-------------| -| `detect_wakeword(chunk, ww_name=None)` — `ovoscope/listener.py` | `(bytes, str?) → bool` | Feed `chunk` to the named engine (or first engine if `ww_name=None`). Returns `True` if the engine fires. | -| `scan_for_wakeword(audio, frame_size=2048, ww_name=None)` — `ovoscope/listener.py` | `(bytes\|List[bytes], int, str?) → (bool, int?)` | Feed each frame sequentially; return `(True, frame_index)` on first detection, or `(False, None)` if threshold never reached. | +| `detect_wakeword(chunk, ww_name=None)`: `ovoscope/listener.py` | `(bytes, str?) → bool` | Feed `chunk` to the named engine (or first engine if `ww_name=None`). Returns `True` if the engine fires. | +| `scan_for_wakeword(audio, frame_size=2048, ww_name=None)`: `ovoscope/listener.py` | `(bytes\|List[bytes], int, str?) → (bool, int?)` | Feed each frame sequentially; return `(True, frame_index)` on first detection, or `(False, None)` if threshold never reached. | **Lifecycle:** | Method | Description | |--------|-------------| -| `shutdown()` — `ovoscope/listener.py` | Gracefully shuts down transformer plugins and all wake-word engines. | +| `shutdown()`: `ovoscope/listener.py` | Gracefully shuts down transformer plugins and all wake-word engines. | -#### `listen()` — `ovoscope/listener.py` +#### `listen()`: `ovoscope/listener.py` ``` listen( @@ -170,36 +170,36 @@ listen( Runs the complete listener pipeline: 1. Reads WAV file (or accepts raw bytes) -2. Passes bytes through `AudioTransformersService.transform()` — all loaded transformer plugins run -3. Converts the (possibly modified) bytes to `AudioData` via `_wav_to_audio_data()` — `listener.py` +2. Passes bytes through `AudioTransformersService.transform()`: all loaded transformer plugins run +3. Converts the (possibly modified) bytes to `AudioData` via `_wav_to_audio_data()`: `listener.py` 4. Calls `stt_instance.execute(audio_data, language)` if provided 5. Emits `recognizer_loop:utterance` on the FakeBus if the transcript is non-empty 6. Returns all captured messages (from transformers **and** the utterance step) -`_wav_to_audio_data(audio, sample_rate, sample_width)` — `listener.py`: +`_wav_to_audio_data(audio, sample_rate, sample_width)`: `listener.py`: - File path → `AudioData.from_file(path)` (handles WAV/AIFF/FLAC headers) -- Raw bytes → parses WAV header via `wave` stdlib; falls back to raw PCM if not a valid WAV +- Raw bytes → parses the WAV header with the `wave` stdlib module, or falls back to raw PCM if not a valid WAV **Constructor parameters:** | Parameter | Type | Description | |-----------|------|-------------| | `config` | `dict` | Full OVOS config with `listener.audio_transformers` key | -| `plugin_instances` | `dict[str, Any]` | Pre-instantiated plugins; bypasses OPM discovery | +| `plugin_instances` | `dict[str, Any]` | Pre-instantiated plugins. Bypasses OPM discovery | -### `get_mini_listener()` — `ovoscope/listener.py` +### `get_mini_listener()`: `ovoscope/listener.py` Factory function. Two usage modes: -**Mode A — OPM discovery** (plugin registered as entry point): +**Mode A: OPM discovery** (plugin registered as entry point): ```python listener = get_mini_listener( transformer_plugins=["ovos-audio-transformer-plugin-ggwave"] ) ``` -**Mode B — direct injection** (bypass OPM, full control over plugin config): +**Mode B: direct injection** (bypass OPM, full control over plugin config): ```python plugin = GGWavePlugin(config={"start_enabled": True}) listener = get_mini_listener( @@ -207,7 +207,7 @@ listener = get_mini_listener( ) ``` -**Mode C — VAD / WakeWord injection:** +**Mode C: VAD / WakeWord injection:** ```python from ovoscope.listener import get_mini_listener, MockVADEngine, MockHotWordEngine @@ -226,7 +226,7 @@ listener = get_mini_listener( | `ww_plugin` | `str` | OPM WakeWord plugin name to load via `OVOSWakeWordFactory` | | `ww_instances` | `dict[str, Any]` | Pre-built WakeWord engines keyed by phrase name | -### `ListenerTest` — `ovoscope/listener.py` +### `ListenerTest`: `ovoscope/listener.py` Declarative test runner, analogous to `End2EndTest`. @@ -240,12 +240,12 @@ Declarative test runner, analogous to `End2EndTest`. | `expected_types` | `list[str]` | `[]` | Message types that must appear | | `forbidden_types` | `list[str]` | `[]` | Message types that must NOT appear | -`execute()` — runs the test, raises `AssertionError` on failure, returns the +`execute()`: runs the test, raises `AssertionError` on failure, returns the captured message list on success. ## Plugin Injection vs OPM Discovery -`AudioTransformersService.load_plugins()` — `transformers.py` — uses +`AudioTransformersService.load_plugins()`: `transformers.py`: uses `find_audio_transformer_plugins()` from `ovos-plugin-manager` to discover plugins by entry point. If a plugin is registered under a legacy group (e.g. `neon.plugin.audio` instead of `opm.plugin.audio_transformer`), or is not @@ -260,13 +260,13 @@ of how the plugin was loaded. `MiniListener` supports **in-process VAD and WakeWord testing** without loading real models or hardware. -### `MockVADEngine` — `ovoscope/listener.py` +### `MockVADEngine`: `ovoscope/listener.py` A zero-dependency VAD stub: - **Silence** = chunk is all `\x00` bytes - **Speech** = any non-zero byte present -- Tracks `chunks_processed` counter; `reset()` zeroes it. +- Tracks the `chunks_processed` counter. `reset()` zeroes it. ```python from ovoscope.listener import MockVADEngine, MiniListener @@ -280,7 +280,7 @@ print(listener.extract_speech(b"\x00" * 512 + b"\x01" * 512)) # → b"\x01" * 5 listener.shutdown() ``` -### `MockHotWordEngine` — `ovoscope/listener.py` +### `MockHotWordEngine`: `ovoscope/listener.py` A controllable WakeWord stub: @@ -297,13 +297,13 @@ listener = MiniListener( ww_instances={"hey_mycroft": ww}, ) -# Feed 5 frames; detection fires on frame index 2 (0-indexed) +# Feed 5 frames. Detection fires on frame index 2 (0-indexed) found, frame = listener.scan_for_wakeword([b"\x00" * 512] * 5) assert found and frame == 2 listener.shutdown() ``` -### `VADTest` — `ovoscope/listener.py` +### `VADTest`: `ovoscope/listener.py` Declarative VAD test helper: @@ -326,7 +326,7 @@ VADTest( ).execute() ``` -### `WakeWordTest` — `ovoscope/listener.py` +### `WakeWordTest`: `ovoscope/listener.py` Declarative WakeWord test helper: @@ -351,14 +351,17 @@ WakeWordTest( ## What MiniListener Does NOT Cover -- Full `DinkumVoiceLoop` state machine — only `AudioTransformersService` and mock VAD/WW engines -- Real hardware audio — inject a WAV file path or raw bytes instead -- Real STT models — `listen()` accepts a mock or real STT plugin, but does not load one automatically +- Full `DinkumVoiceLoop` state machine: only `AudioTransformersService` and mock VAD/WW engines +- Real hardware audio: inject a WAV file path or raw bytes instead +- Real STT models: `listen()` accepts a mock or real STT plugin, but does not load one automatically ## Cross-References -- `AudioTransformersService` — `ovos-dinkum-listener/ovos_dinkum_listener/transformers.py` -- `AudioData` — `ovos-plugin-manager/ovos_plugin_manager/utils/audio.py` -- `MiniCroft` / `get_minicroft()` — `ovoscope/docs/minicroft.md` (skill pipeline equivalent) +- `AudioTransformersService`: `ovos-dinkum-listener/ovos_dinkum_listener/transformers.py` +- `AudioData`: `ovos-plugin-manager/ovos_plugin_manager/utils/audio.py` +- `MiniCroft` / `get_minicroft()`: `ovoscope/docs/minicroft.md` (skill pipeline equivalent) - Audio transformer E2E test: `Transformer plugins/ovos-audio-transformer-plugin-ggwave/test/end2end/test_ggwave_transformer.py` - STT pipeline E2E test: `STT plugins/ovos-stt-plugin-rover/test/end2end/test_rover_listener_e2e.py` + +--- +[← PHAL](phal.md) · [Home](../README.md) · [Voice Loop →](voice-loop.md) diff --git a/docs/media-provider-testing.md b/docs/media-provider-testing.md index 4e84a98..54bfc0a 100644 --- a/docs/media-provider-testing.md +++ b/docs/media-provider-testing.md @@ -1,7 +1,7 @@ # MediaProvider (Search) Testing with ovoscope `MediaProviderHarness` (`ovoscope.media_provider`) tests `opm.media.provider` -plugins — the in-process **catalog/search** providers introduced by the +plugins: the in-process **catalog/search** providers introduced by the ovos-media sprint to replace OCP search skills. It is the search-side counterpart to [`OCPPlayerHarness`](media-testing.md), which drives the *player*. @@ -45,7 +45,7 @@ h.assert_not_routes(Signals(medium=MediaType.MUSIC), QueryContext(supported_playback_types={"video"})) # audio-only provider h.assert_not_routes(Signals(medium=MediaType.MOVIE)) -# the never-raising search the pipeline calls — ranked, playable results +# the never-raising search the pipeline calls: ranked, playable results releases = h.assert_returns_playables(Signals(title="worms")) assert all(r.uri.startswith("library://") for r in releases) ``` @@ -55,7 +55,7 @@ assert all(r.uri.startswith("library://") for r in releases) | Constructor | Use | |---|---| | `MediaProviderHarness.from_entrypoint(name, config=None, group="opm.media.provider", mock_api=None, api_attr="_api")` | Discover the provider through its installed entry-point (the real e2e). Raises `AssertionError` if the entry-point is missing or ambiguous. | -| `MediaProviderHarness.from_class(provider_cls, config=None, mock_api=None, api_attr="_api")` | Wrap a class you already hold (no packaging needed) — handy for unit tests. | +| `MediaProviderHarness.from_class(provider_cls, config=None, mock_api=None, api_attr="_api")` | Wrap a class you already hold (no packaging needed): handy for unit tests. | `mock_api` is set onto `provider.` (default `_api`) to bypass the lazy, network-backed client a provider builds on first use. @@ -91,7 +91,10 @@ Thin pass-throughs mirroring the `MediaProvider` contract: ## Cross-references -- `MediaProvider` / `QueryContext` — `ovos_plugin_manager.templates.media_provider` -- `Signals`, `Release`, `MediaType` — `mediavocab` -- Player-side harness — [media-testing.md](media-testing.md) -- OCP *search skill* testing (legacy stack) — [ocp.md](ocp.md) +- `MediaProvider` / `QueryContext` (`ovos_plugin_manager.templates.media_provider`) +- `Signals`, `Release`, `MediaType` (`mediavocab`) +- Player-side harness: [media-testing.md](media-testing.md) +- OCP *search skill* testing (legacy stack): [ocp.md](ocp.md) + +--- +[← Media Testing](media-testing.md) · [Home](../README.md) · [OCP →](ocp.md) diff --git a/docs/media-testing.md b/docs/media-testing.md index 0d2ec3e..80a7e2e 100644 --- a/docs/media-testing.md +++ b/docs/media-testing.md @@ -1,7 +1,7 @@ # Media / OCP Testing with ovoscope -This document describes how to test `ovos-media` services — specifically the -`OCPMediaPlayer` state machine — using the harness classes provided in +This document describes how to test `ovos-media` services: specifically the +`OCPMediaPlayer` state machine: using the harness classes provided in `ovoscope.media`. > **Prerequisite:** Media testing harnesses require `ovos-media` to be installed. @@ -19,21 +19,21 @@ This document describes how to test `ovos-media` services — specifically the ## OCPPlayerHarness -`OCPPlayerHarness` — `ovoscope/media.py` +`OCPPlayerHarness` (`ovoscope/media.py`) Wraps a real `OCPMediaPlayer` (`ovos_media.player`) with a `MockOCPBackend` on a `FakeBus`. All heavy dependencies are patched out: -- `ovos_media.player.AudioService` — mocked; `MockOCPBackend` injected as the +- `ovos_media.player.AudioService`: mocked. `MockOCPBackend` is injected as the sole audio backend -- `ovos_media.player.VideoService` — mocked -- `ovos_media.player.WebService` — mocked -- `ovos_media.player.OcpMprisExporter` — mocked (no D-Bus session required) -- `ovos_media.player.GUIInterface` — mocked (exposed as `harness.gui`), only - on `ovos-media` builds that define it; builds without in-core GUI - integration skip this patch -- `ovos_media.player.OCPMediaCatalog` — mocked -- `ovos_media.player.Configuration` — returns `{"media": {}}` +- `ovos_media.player.VideoService`: mocked +- `ovos_media.player.WebService`: mocked +- `ovos_media.player.OcpMprisExporter`: mocked (no D-Bus session required) +- `ovos_media.player.GUIInterface`: mocked (exposed as `harness.gui`), only + on `ovos-media` builds that define it. Builds without in-core GUI + integration skip this patch. +- `ovos_media.player.OCPMediaCatalog`: mocked +- `ovos_media.player.Configuration`: returns `{"media": {}}` ### Basic Usage @@ -79,7 +79,8 @@ with OCPPlayerHarness() as h: "media": track1.as_dict, "playlist": [track1.as_dict, track2.as_dict], })) - import time; time.sleep(0.05) + import time + time.sleep(0.05) h.assert_now_playing_uri("http://example.com/1.mp3") h.next_track() @@ -92,10 +93,10 @@ with OCPPlayerHarness() as h: interruptions. Understanding the difference is essential for writing correct tests. -#### Ducking — lower volume, keep playing +#### Ducking: lower volume, keep playing Ducking happens when the assistant **speaks** (TTS output). The player stays -in ``PLAYING`` state; only the audio backend volume is reduced. +in ``PLAYING`` state. Only the audio backend volume is reduced. | Bus message | Handler | Effect | |---|---|---| @@ -117,15 +118,15 @@ with OCPPlayerHarness() as h: entry = MediaEntry(uri="http://example.com/song.mp3", playback=PlaybackType.AUDIO) h.play(entry) - h.duck() # lower_volume called; player stays PLAYING + h.duck() # lower_volume called, player stays PLAYING h.assert_player_state(PlayerState.PLAYING) assert h.player._paused_on_duck # flag set - h.unduck() # restore_volume called; _paused_on_duck cleared + h.unduck() # restore_volume called, _paused_on_duck cleared h.assert_player_state(PlayerState.PLAYING) assert not h.player._paused_on_duck ``` -#### Corking — pause the player, resume after listening +#### Corking: pause the player, resume after listening Corking happens when the **microphone opens** (wake-word recognised, user speaking). The player is fully **paused** and resumes after the interaction. @@ -134,7 +135,7 @@ speaking). The player is fully **paused** and resumes after the interaction. |---|---|---| | `recognizer_loop:record_begin` / `ovos.common_play.cork` | `handle_cork_request` | Pauses player, sets `_paused_on_duck=True` | | `ovos.common_play.uncork` | `handle_uncork_request` | Resumes player **only if PAUSED and `_paused_on_duck`** | -| `recognizer_loop:record_end` | `handle_record_end` | Waits up to 8 s for `speak`; if none → uncork | +| `recognizer_loop:record_end` | `handle_record_end` | Waits up to 8 s for `speak`. If none, uncork | ```python from ovoscope.media import OCPPlayerHarness @@ -162,8 +163,8 @@ no-op, preventing a spurious resume. ```python with OCPPlayerHarness() as h: h.play(entry) - h.pause() # manual pause — _paused_on_duck stays False - h.uncork() # no-op — _paused_on_duck is False + h.pause() # manual pause: _paused_on_duck stays False + h.uncork() # no-op: _paused_on_duck is False h.assert_player_state(PlayerState.PAUSED) ``` @@ -181,7 +182,8 @@ with OCPPlayerHarness() as h: h.cork() with patch.object(h.bus, "wait_for_message", return_value=None): h.bus.emit(Message("recognizer_loop:record_end")) - import time; time.sleep(0.05) + import time + time.sleep(0.05) h.assert_player_state(PlayerState.PLAYING) ``` @@ -203,8 +205,8 @@ with OCPPlayerHarness() as h: By default `OCPPlayerHarness` injects a `MockOCPBackend` and mocks out `AudioService`, so it exercises the **player state machine** but never the real -backend routing. To test a **real** OCP audio backend end-to-end — e.g. assert -that playing a uri makes a Music Assistant backend call its server — pass a +backend routing. To test a **real** OCP audio backend end-to-end: e.g. assert +that playing a uri makes a Music Assistant backend call its server: pass a `backend_factory`: a `bus -> AudioBackend` callable. The harness then wires a *real* `AudioService` (no autoload) with your backend as its sole service, so the player's `play -> load_track -> LOADED_MEDIA -> backend.play()` path actually @@ -228,19 +230,19 @@ with OCPPlayerHarness(backend_factory=make_backend) as h: Notes: - The factory **owns mocking** any network client the real backend would reach. -- Deferred uris (`library://`, `{sei}//…`) are resolved by the OCP pipeline's - stream extractors *before* the player in production; the harness loads no - extractor plugins, so it bypasses the player's stream validation when a backend - factory is used. -- `name`/`namespace` are supplied by the harness if the backend lacks them +- The OCP pipeline's stream extractors resolve deferred uris (`library://`, `{sei}//…`) + *before* the player in production. The harness loads no + extractor plugins, so it bypasses the player's stream validation when you use a + backend factory. +- The harness supplies `name`/`namespace` if the backend lacks them (normally set by `BaseMediaService.load_services()`, which the harness bypasses). - The mock-only helpers (`assert_backend_paused`, `backend.played_uris`) assume a - `MockOCPBackend` and may not apply to a real backend — assert on the backend's + `MockOCPBackend` and may not apply to a real backend: assert on the backend's own state/spies instead. ## OCPCaptureSession -`OCPCaptureSession` — `ovoscope/media.py` +`OCPCaptureSession` (`ovoscope/media.py`) Captures all `ovos.common_play.*` and `ovos.audio.*` bus messages during a block of code and lets you assert that specific message types appeared in order. @@ -275,7 +277,7 @@ with OCPPlayerHarness() as h: ### MockOCPBackend -`MockOCPBackend` — `ovoscope/media.py` +`MockOCPBackend` (`ovoscope/media.py`) | Attribute / Method | Type | Description | |---|---|---| @@ -291,10 +293,10 @@ with OCPPlayerHarness() as h: ### OCPPlayerHarness -`OCPPlayerHarness` — `ovoscope/media.py` +`OCPPlayerHarness` (`ovoscope/media.py`) **Constructor:** `OCPPlayerHarness(backend_namespace="audio", backend_factory=None)`. -`backend_factory` is an optional `bus -> AudioBackend` callable; when given, the +`backend_factory` is an optional `bus -> AudioBackend` callable. When given, the harness drives that real backend through a real `AudioService` (see [Driving a Real OCP Backend](#driving-a-real-ocp-backend)) instead of the default `MockOCPBackend`. @@ -309,10 +311,10 @@ harness drives that real backend through a real `AudioService` (see | `stop()` | `ovos.common_play.stop` | | `next_track()` | `ovos.common_play.next` | | `prev_track()` | `ovos.common_play.previous` | -| `duck()` | `ovos.audio.output.started` — lower volume, player stays PLAYING | -| `unduck()` | `ovos.audio.output.ended` — restore volume whenever `_paused_on_duck` is True (duck or cork path) | -| `cork()` | `ovos.common_play.cork` — pause player, set `_paused_on_duck=True` | -| `uncork()` | `ovos.common_play.uncork` — resume player if PAUSED and `_paused_on_duck` | +| `duck()` | `ovos.audio.output.started`: lower volume, player stays PLAYING | +| `unduck()` | `ovos.audio.output.ended`: restore volume whenever `_paused_on_duck` is True (duck or cork path) | +| `cork()` | `ovos.common_play.cork`: pause player, set `_paused_on_duck=True` | +| `uncork()` | `ovos.common_play.uncork`: resume player if PAUSED and `_paused_on_duck` | | `simulate_track_end()` | `ovos.common_play.media.state` END_OF_MEDIA | | `simulate_invalid_stream()` | `ovos.common_play.media.state` INVALID_MEDIA | @@ -338,7 +340,7 @@ harness drives that real backend through a real `AudioService` (see ### OCPCaptureSession -`OCPCaptureSession` — `ovoscope/media.py` +`OCPCaptureSession` (`ovoscope/media.py`) | Method / Property | Description | |---|---| @@ -354,7 +356,7 @@ Default `track_prefixes` captures: `"ovos.common_play."`, `"ovos.audio."`. - **No real audio**: `MockOCPBackend` never plays audio. Use `simulate_end()` to trigger end-of-track logic. -- **No MPRIS**: `OcpMprisExporter` is mocked out — MPRIS D-Bus integration is +- **No MPRIS**: `OcpMprisExporter` is mocked out: MPRIS D-Bus integration is not exercised. - **No GUI rendering**: on `ovos-media` builds that still define `GUIInterface`, it is patched with a `MagicMock`; test GUI calls via @@ -365,13 +367,16 @@ Default `track_prefixes` captures: `"ovos.common_play."`, `"ovos.audio."`. is wired with a real mock backend. - **FakeBus is synchronous**: Handlers run in the same thread that calls `bus.emit()`. The `time.sleep(0.05)` in control methods is sufficient for - synchronous delivery; async or threaded handlers may need explicit waits. + synchronous delivery. Async or threaded handlers may need explicit waits. ## Cross-References -- `OCPMediaPlayer` — `ovos-media/ovos_media/player.py` -- `BaseMediaService` — `ovos-media/ovos_media/media_backends/base.py` -- `AudioBackend` (base class) — `ovos_plugin_manager.templates.audio.AudioBackend` -- `MediaEntry`, `PlayerState`, `MediaState` — `ovos_utils.ocp` -- `MockAudioBackend` / `AudioServiceHarness` (audio pattern) — `ovoscope/audio.py` -- End-to-end tests — `ovos-media/test/end2end/test_ocp_player.py` +- `OCPMediaPlayer` (`ovos-media/ovos_media/player.py`) +- `BaseMediaService` (`ovos-media/ovos_media/media_backends/base.py`) +- `AudioBackend` (base class) (`ovos_plugin_manager.templates.audio.AudioBackend`) +- `MediaEntry`, `PlayerState`, `MediaState` (`ovos_utils.ocp`) +- `MockAudioBackend` / `AudioServiceHarness` (audio pattern): `ovoscope/audio.py` +- End-to-end tests: `ovos-media/test/end2end/test_ocp_player.py` + +--- +[← Audio Testing](audio-testing.md) · [Home](../README.md) · [Media Provider Testing →](media-provider-testing.md) diff --git a/docs/minicroft.md b/docs/minicroft.md index 9cc05da..db9821a 100644 --- a/docs/minicroft.md +++ b/docs/minicroft.md @@ -49,7 +49,7 @@ MiniCroft( | `boot_messages` | `list[Message]` | All messages captured during startup | | `status` | `ProcessState` | Current lifecycle state | ### `MiniCroft.run()` -Loads plugins and marks the runtime as ready. Called internally by `start()`. Does not block — returns after all skills are loaded. +Loads plugins and marks the runtime as ready. Called internally by `start()`. Does not block: returns after all skills are loaded. ### `MiniCroft.stop()` Shuts down skills and closes the bus. --- @@ -119,8 +119,11 @@ All overrides are restored to their original values in `MiniCroft.stop()`. --- ## Boot Sequence On startup, MiniCroft captures all messages emitted during skill loading into `boot_messages`. These can be asserted in `End2EndTest.expected_boot_sequence`. The typical boot sequence includes: -1. `mycroft.skills.train` — intent pipeline training request -2. `mycroft.skills.initialized` — skills initialized -3. `mycroft.skills.ready` — skills service ready -4. `mycroft.ready` — all core services ready +1. `mycroft.skills.train`: intent pipeline training request +2. `mycroft.skills.initialized`: skills initialized +3. `mycroft.skills.ready`: skills service ready +4. `mycroft.ready`: all core services ready Skills that participate in `converse` or `fallback` registration also emit messages during boot (e.g. `ovos.skills.fallback.register`). + +--- +[← CI Integration](ci-integration.md) · [Home](../README.md) · [Capture Session →](capture-session.md) diff --git a/docs/ocp.md b/docs/ocp.md index dffbaf5..0b61105 100644 --- a/docs/ocp.md +++ b/docs/ocp.md @@ -13,9 +13,9 @@ recognizer_loop:utterance → ovos.common_play.start (selected track) ``` -## `OCPTest` — Declarative Style +## `OCPTest`: Declarative Style -`OCPTest` — `ocp.py:OCPTest` +`OCPTest` (`ocp.py:OCPTest`) ```python from ovoscope.ocp import OCPTest @@ -45,12 +45,12 @@ result = OCPTest( | `timeout` | `float` | `20.0` | Max wait in seconds. | | `patch_targets` | `List[str]` | `[]` | Additional `requests`-like module paths to patch (dotted Python path to the callable to replace). | -### `execute()` — `ovoscope/ocp.py` +### `execute()`: `ovoscope/ocp.py` -Returns `List[Message]` — all bus messages captured during the interaction +Returns `List[Message]`: all bus messages captured during the interaction (same format as `CaptureSession.responses`). -## HTTP Mocking — `ovoscope/ocp.py` +## HTTP Mocking: `ovoscope/ocp.py` HTTP calls are intercepted via `unittest.mock.patch` on `requests.Session.get` and `requests.get` by default. @@ -78,14 +78,14 @@ OCPTest( ).execute() ``` -The format is the same as `unittest.mock.patch` target strings — the dotted +The format is the same as `unittest.mock.patch` target strings: the dotted path to where the symbol is **used** (not where it is defined). See [unittest.mock patch docs](https://docs.python.org/3/library/unittest.mock.html#unittest.mock.patch) for details. ## `assert_ocp_query_response` -`assert_ocp_query_response` — `ocp.py:assert_ocp_query_response` +`assert_ocp_query_response` (`ocp.py:assert_ocp_query_response`) ```python from ovoscope.ocp import assert_ocp_query_response @@ -106,3 +106,6 @@ assert_ocp_query_response( | `media_type` | All items must have this `media_type`. | | `expected_media` | Partial-dict subset matching. | | `stream_url_contains` | Substring in `ovos.common_play.start` URI. | + +--- +[← Media Provider Testing](media-provider-testing.md) · [Home](../README.md) · [PHAL →](phal.md) diff --git a/docs/phal.md b/docs/phal.md index 83e346c..7461f05 100644 --- a/docs/phal.md +++ b/docs/phal.md @@ -24,13 +24,13 @@ device access is required. Plugins that require physical hardware are **not suitable** for in-process testing and should use hardware-in-the-loop integration tests instead: -- `ovos-PHAL-plugin-alsa` — requires ALSA audio subsystem -- `ovos-PHAL-plugin-mk1` — requires Mark 1 hardware -- `ovos-PHAL-plugin-dotstar` — requires APA102 LED ring +- `ovos-PHAL-plugin-alsa`: requires ALSA audio subsystem +- `ovos-PHAL-plugin-mk1`: requires Mark 1 hardware +- `ovos-PHAL-plugin-dotstar`: requires APA102 LED ring -## `MiniPHAL` — Context Manager +## `MiniPHAL`: Context Manager -`MiniPHAL` — `ovoscope/phal.py` +`MiniPHAL`: `ovoscope/phal.py` ```python from ovos_utils.messagebus import Message @@ -54,16 +54,16 @@ with MiniPHAL( ### Methods -`MiniPHAL.emit` — `ovoscope/phal.py` +`MiniPHAL.emit`: `ovoscope/phal.py` | Method | Description | |--------|-------------| | `emit(msg, wait=0.05)` | Emit `msg` on the internal bus then sleep `wait` seconds so async handlers have time to fire before the next assertion. Set `wait=0` to disable the sleep. | -| `assert_emitted(msg_type, timeout=2.0)` | Poll captured messages up to `timeout` seconds; return the first matching `Message`. Raises `AssertionError` on timeout. — `ovoscope/phal.py` | -| `assert_not_emitted(msg_type, wait=0.2)` | Sleep `wait` seconds then assert no captured message has `msg_type`. Raises `AssertionError` if one was captured. — `ovoscope/phal.py` | -| `clear_captured()` | Clear the captured message list. Useful between sequential assertions in the same `with` block. — `ovoscope/phal.py` | +| `assert_emitted(msg_type, timeout=2.0)` | Poll captured messages up to `timeout` seconds; return the first matching `Message`. Raises `AssertionError` on timeout.: `ovoscope/phal.py` | +| `assert_not_emitted(msg_type, wait=0.2)` | Sleep `wait` seconds then assert no captured message has `msg_type`. Raises `AssertionError` if one was captured.: `ovoscope/phal.py` | +| `clear_captured()` | Clear the captured message list. Useful between sequential assertions in the same `with` block.: `ovoscope/phal.py` | -#### `emit(wait=...)` — settling delay +#### `emit(wait=...)`: settling delay The `wait` parameter (default `0.05` s) controls how long `MiniPHAL` sleeps after calling `bus.emit()`. PHAL plugin handlers may run on a background thread, @@ -72,7 +72,7 @@ for plugins with higher latency; set `wait=0` to suppress the sleep entirely whe the handler is known to be synchronous. ```python -# Default — 50 ms settle time +# Default: 50 ms settle time phal.emit(Message("network.connected")) # Custom settle time (slower plugin) @@ -82,9 +82,9 @@ phal.emit(Message("system.reboot"), wait=0.5) phal.emit(Message("config.get"), wait=0) ``` -## `PHALTest` — Declarative Style +## `PHALTest`: Declarative Style -`PHALTest` — `phal.py:PHALTest` +`PHALTest` (`phal.py:PHALTest`) ```python from ovos_utils.messagebus import Message @@ -110,3 +110,6 @@ PHALTest( | `plugin_instances` | `Dict` | `{}` | Pre-built instances. | | `config` | `Dict` | `{}` | Per-plugin config. | | `timeout` | `float` | `5.0` | Wait timeout in seconds. | + +--- +[← OCP](ocp.md) · [Home](../README.md) · [Listener →](listener.md) diff --git a/docs/pipeline.md b/docs/pipeline.md index 30e3d4f..5fe1e6d 100644 --- a/docs/pipeline.md +++ b/docs/pipeline.md @@ -1,7 +1,7 @@ # Pipeline Plugin Testing `ovoscope.pipeline` provides `PipelineHarness` for testing intent / pipeline -plugins in isolation — no skill is needed. +plugins in isolation: no skill is needed. ## What Is Tested @@ -9,9 +9,9 @@ Pipeline plugins (Adapt, Padatious, Padacioso, OCP, etc.) match utterances to intents. `PipelineHarness` loads the specified stages on a `MiniCroft` that has no skills, so only the pipeline matching logic is exercised. -## `_SinkSkill` — Internal Catch-all +## `_SinkSkill`: Internal Catch-all -`_SinkSkill` — `ovoscope/pipeline.py` +`_SinkSkill`: `ovoscope/pipeline.py` When `PipelineHarness` creates a `MiniCroft`, it injects an internal `__ovoscope_sink__` skill as a routing target for matched intents. This is @@ -21,9 +21,9 @@ intent message and signals the waiting `match()` call. Users never interact with `_SinkSkill` directly. -## `PipelineHarness` — Context Manager +## `PipelineHarness`: Context Manager -`PipelineHarness` — `ovoscope/pipeline.py` +`PipelineHarness`: `ovoscope/pipeline.py` ```python from ovoscope.pipeline import PipelineHarness @@ -57,13 +57,13 @@ with PipelineHarness( OVOS evaluates pipeline stages in the order listed in `pipeline`. The first stage that returns a non-empty match list wins; remaining stages are skipped. -**Success signal**: `intent.service.skills.activated` bus message — emitted +**Success signal**: `intent.service.skills.activated` bus message: emitted when a stage commits to handling the utterance. **Failure signal**: `intent_failure` or `mycroft.skill.handler.start` bus -messages — emitted when no stage matched after all stages have been consulted. +messages: emitted when no stage matched after all stages have been consulted. -`match()` — `ovoscope/pipeline.py` — uses separate `threading.Event` +`match()`: `ovoscope/pipeline.py`: uses separate `threading.Event` objects for success and failure so that an `intent_failure` arriving first does not mask a subsequent late success match. On timeout or failure the method returns `None`; on success it returns the captured `Message`. @@ -117,7 +117,7 @@ with PipelineHarness( # Pass: msg_type "padatious:0.95:LightsOnIntent" contains "LightsOnIntent" msg = harness.assert_matches("turn on the lights", intent_type="LightsOnIntent") -# Pass: no intent_type check — any match accepted +# Pass: no intent_type check: any match accepted msg = harness.assert_matches("turn on the lights") # Fail: "LightsOffIntent" not in "padatious:0.95:LightsOnIntent" @@ -127,11 +127,14 @@ msg = harness.assert_matches("turn on the lights", intent_type="LightsOffIntent" ## Implementation Notes -`PipelineHarness.__enter__` — `ovoscope/pipeline.py` — creates a +`PipelineHarness.__enter__`: `ovoscope/pipeline.py`: creates a `MiniCroft` with `skill_ids=[]` and the specified pipeline. -`PipelineHarness.match()` — `ovoscope/pipeline.py` — subscribes to +`PipelineHarness.match()`: `ovoscope/pipeline.py`: subscribes to `intent.service.skills.activated` (success) and `intent_failure` / `mycroft.skill.handler.start` (failure) before emitting the utterance, then waits on a `threading.Event` with the given timeout. Bus handlers are removed after the wait completes to avoid cross-test leakage. + +--- +[← End2EndTest](end2end-test.md) · [Home](../README.md) · [Pydantic Integration →](pydantic-integration.md) diff --git a/docs/pydantic-integration.md b/docs/pydantic-integration.md index 6e713af..448c76f 100644 --- a/docs/pydantic-integration.md +++ b/docs/pydantic-integration.md @@ -1,10 +1,10 @@ # OvoScope + ovos-pydantic-models Integration -OvoScope currently operates on untyped `ovos_bus_client.message.Message` objects — dicts with string keys. `ovos-pydantic-models` provides typed Pydantic v2 models for every OVOS message type. This document describes how they can be used together and what a deeper integration could look like. +OvoScope currently operates on untyped `ovos_bus_client.message.Message` objects: dicts with string keys. `ovos-pydantic-models` provides typed Pydantic v2 models for every OVOS message type. This document describes how they can be used together and what a deeper integration could look like. --- ## The Problem Today Writing test fixtures by hand is verbose and error-prone: ```python -# untyped — no validation, any typo silently passes +# untyped: no validation, any typo silently passes expected = Message("recognizer_loop:utterance", {"utterances": ["hello"], "lang": "en-us"}, {}) ``` `Message` is a raw dict wrapper. There is no validation of field names, no type checking, and no autocomplete. A typo in a field name (`"utterance"` instead of `"utterances"`) silently produces a wrong test. @@ -39,7 +39,7 @@ from ovoscope import End2EndTest from ovos_bus_client.message import Message from ovos_bus_client.session import Session from ovos_pydantic_models import RecognizerLoopUtteranceMessage, RecognizerLoopUtteranceData -# typed construction — validated at instantiation +# typed construction: validated at instantiation utterance_model = RecognizerLoopUtteranceMessage( data=RecognizerLoopUtteranceData(utterances=["what is the weather?"], lang="en-us"), ) @@ -75,7 +75,7 @@ End2EndTest( expected_messages=expected, ).execute() ``` -Because `End2EndTest` checks only the data keys you specify (subset match), you can omit optional fields in expected messages — this works the same as before, but field names are now validated at Python parse time. +Because `End2EndTest` checks only the data keys you specify (subset match), you can omit optional fields in expected messages: this works the same as before, but field names are now validated at Python parse time. --- ## Usage Pattern 3: Typed Assertions on Received Messages After a test captures messages, convert received `Message` objects to their typed counterparts for richer assertions: @@ -94,7 +94,7 @@ typed_speak = from_bus_message(speak_msgs[0], SpeakMessage) assert "london" in typed_speak.data.utterance.lower() assert typed_speak.data.expect_response is False ``` -This is cleaner than `msg.data["utterance"]` — you get IDE autocomplete and the field contract is explicit. +This is cleaner than `msg.data["utterance"]`: you get IDE autocomplete and the field contract is explicit. --- ## Usage Pattern 4: Type-safe Test Helpers Build helpers that combine the two: @@ -117,7 +117,7 @@ def make_utterance(text: str, lang: str = "en-us", session: Session | None = Non ``` --- ## Deeper Integration: What OvoScope Could Gain -The patterns above work today with no changes to OvoScope. A deeper integration would add native support for pydantic models as a first-class alternative to `Message`: +The patterns above work today with no changes to OvoScope. A deeper integration would add native support for pydantic models as an alternative to `Message`: ### Idea 1: Accept pydantic models directly in `End2EndTest` ```python # instead of requiring to_bus_message() manually: @@ -161,7 +161,7 @@ Install with: pip install ovoscope[pydantic] ``` The bridge functions (`to_bus_message`, `from_bus_message`, `validate_fixture`) live in -`ovoscope.pydantic_helpers` and guard their imports conditionally — the module can be imported +`ovoscope.pydantic_helpers` and guard their imports conditionally: the module can be imported without `ovos-pydantic-models` installed, but calling any function raises a clear `ImportError` pointing to the extras install command: ```python @@ -172,10 +172,13 @@ from ovoscope.pydantic_helpers import to_bus_message # ImportError only on call ## Summary | Pattern | What you get | Status | |---|---|---| -| Typed source messages via `to_bus_message()` | Validation at construction | ✅ `ovoscope.pydantic_helpers` | -| Typed expected messages via `to_bus_message()` | Field name validation | ✅ `ovoscope.pydantic_helpers` | -| Typed assertions via `from_bus_message()` | IDE autocomplete, field contracts | ✅ `ovoscope.pydantic_helpers` | -| Fixture validation via `validate_fixture()` | Clear errors on malformed JSON | ✅ `ovoscope.pydantic_helpers` | -| Native pydantic in `End2EndTest` | Seamless API (no `to_bus_message` call) | 💡 Future: `__post_init__` auto-conversion | -| Schema validation in assertions | Catch malformed skill messages | 💡 Future: `validate_schemas=True` flag | +| Typed source messages via `to_bus_message()` | Validation at construction | Done, in `ovoscope.pydantic_helpers` | +| Typed expected messages via `to_bus_message()` | Field name validation | Done, in `ovoscope.pydantic_helpers` | +| Typed assertions via `from_bus_message()` | IDE autocomplete, field contracts | Done, in `ovoscope.pydantic_helpers` | +| Fixture validation via `validate_fixture()` | Clear errors on malformed JSON | Done, in `ovoscope.pydantic_helpers` | +| Native pydantic in `End2EndTest` | Direct API, no `to_bus_message` call | Future: `__post_init__` auto-conversion | +| Schema validation in assertions | Catch malformed skill messages | Future: `validate_schemas=True` flag | Install the extras to use the implemented patterns: `pip install ovoscope[pydantic]` + +--- +[← Pipeline](pipeline.md) · [Home](../README.md) · [Audio Testing →](audio-testing.md) diff --git a/docs/usage-guide.md b/docs/usage-guide.md index 1e844cb..0c4e06e 100644 --- a/docs/usage-guide.md +++ b/docs/usage-guide.md @@ -5,7 +5,7 @@ assumes familiarity with Python's `unittest` and the OVOS bus message model. ## Prerequisites Install ovoscope and the skill under test in the same virtual environment: ```bash -# editable installs — recommended during development +# editable installs: recommended during development uv pip install -e ovoscope/ -e Skills/ovos-skill-hello-world/ # or via PyPI pip install ovoscope ovos-skill-hello-world @@ -32,18 +32,18 @@ python -c "from ovos_plugin_manager.skills import find_skill_plugins; print(list | Test session state after an interaction | **ovoscope** | | Test multi-turn dialogue (converse / fallback) | **ovoscope** | | Test that a skill is blacklisted and does NOT match | **ovoscope** | -**Rule of thumb**: if you are asserting on *what gets emitted on the bus* — type, order, data, or -routing — use ovoscope. If you are testing the internal Python logic of a handler in isolation, +**Rule of thumb**: if you are asserting on *what gets emitted on the bus*: type, order, data, or +routing: use ovoscope. If you are testing the internal Python logic of a handler in isolation, use FakeBus unit tests. FakeBus reference: ```python from ovos_utils.fakebus import FakeBus # ovos-utils ``` --- -## Quick Start — Hello World +## Quick Start: Hello World The canonical example skill is `ovos-skill-hello-world.openvoiceos`. It has two intents: -- **HelloWorldIntent** (Adapt) — triggered by "hello world" -- **Greetings.intent** (Padatious) — triggered by greetings like "good morning" +- **HelloWorldIntent** (Adapt): triggered by "hello world" +- **Greetings.intent** (Padatious): triggered by greetings like "good morning" ```python import unittest from ovos_bus_client.message import Message @@ -84,10 +84,10 @@ class TestHelloWorldQuickStart(unittest.TestCase): ) test.execute(timeout=10) ``` -`test.execute()` raises `AssertionError` on any mismatch. No return value is used — use pytest or +`test.execute()` raises `AssertionError` on any mismatch. No return value is used: use pytest or `unittest.TestCase` assertions normally. --- -## Pattern 1 — Manual Assertion (Adapt Intent Match) +## Pattern 1: Manual Assertion (Adapt Intent Match) Write each expected `Message` explicitly. This is the most readable pattern and the easiest to debug. ```python @@ -128,10 +128,10 @@ test = End2EndTest( ) test.execute(timeout=10) ``` -Only keys present in `expected.data` and `expected.context` are checked — extra keys in the +Only keys present in `expected.data` and `expected.context` are checked: extra keys in the received message are ignored. This lets you assert on exactly the fields you care about. --- -## Pattern 2 — Padatious Intent Match +## Pattern 2: Padatious Intent Match Padatious uses `.intent` file names as the message type. Restrict the session pipeline to Padatious only so Adapt doesn't shadow the match: ```python @@ -168,10 +168,10 @@ test = End2EndTest( test.execute(timeout=10) ``` Note: for Padatious the `speak` message's `utterance` key may vary (depends on the dialog file -randomisation), so omit `"utterance"` from `expected.data` if it is non-deterministic — only +randomisation), so omit `"utterance"` from `expected.data` if it is non-deterministic: only assert on `lang` and `meta`. --- -## Pattern 3 — Recording Mode (Bootstrap Fixtures) +## Pattern 3: Recording Mode (Bootstrap Fixtures) Don't know the exact message sequence yet? Let ovoscope record it for you: ```python from ovoscope import End2EndTest @@ -195,14 +195,14 @@ test = End2EndTest.from_message( test.save("tests/fixtures/hello_world_adapt.json", anonymize=True) ``` `anonymize=True` (default) strips real location / personal data from the session context before -saving — safe to commit. +saving: safe to commit. Then in your test suite: ```python test = End2EndTest.from_path("tests/fixtures/hello_world_adapt.json") test.execute(timeout=10) ``` --- -## Pattern 4 — Replay from JSON Fixture +## Pattern 4: Replay from JSON Fixture Committed JSON fixtures make tests fully self-contained: no network, no live skill discovery, no non-determinism in expected messages. ```python @@ -217,10 +217,10 @@ class TestFromFixture(unittest.TestCase): test.execute(timeout=10) ``` Note: skills still need to be installed (the JSON stores `skill_ids`, and `execute()` calls -`get_minicroft()` which loads the real plugin). The fixture stores the expected message sequence -— not the skill code. +`get_minicroft()` which loads the real plugin). The fixture stores the expected message sequence, +not the skill code. --- -## Pattern 5 — Reusing MiniCroft Across Multiple Tests +## Pattern 5: Reusing MiniCroft Across Multiple Tests Creating a `MiniCroft` is expensive (it trains intent models). Reuse it across tests in the same class with `setUp` / `tearDown`: ```python @@ -247,7 +247,7 @@ class TestHelloWorldSharedRuntime(unittest.TestCase): {"session": session.serialize(), "source": "A", "destination": "B"}, ) return End2EndTest( - minicroft=self.minicroft, # pass existing MiniCroft — not managed, not stopped + minicroft=self.minicroft, # pass existing MiniCroft: not managed, not stopped skill_ids=[SKILL_ID], source_message=message, expected_messages=expected_messages, @@ -287,7 +287,7 @@ When you pass `minicroft=self.minicroft` explicitly, `End2EndTest` sets `managed **not** call `minicroft.stop()` at the end of `execute()`. Your `tearDown` is responsible for cleanup. --- -## Pattern 6 — Multi-Turn Conversation +## Pattern 6: Multi-Turn Conversation Pass a **list** of `Message` objects as `source_message` to test a dialogue sequence. ovoscope emits them in order, propagating session state between turns: ```python @@ -302,7 +302,7 @@ turn1 = Message( turn2 = Message( "recognizer_loop:utterance", {"utterances": ["good morning"], "lang": "en-US"}, - {"source": "A", "destination": "B"}, # no "session" key — will be filled by ovoscope + {"source": "A", "destination": "B"}, # no "session" key: will be filled by ovoscope ) test = End2EndTest( skill_ids=[SKILL_ID], @@ -320,9 +320,9 @@ test = End2EndTest( test.execute(timeout=20) ``` Session propagation: if turn 2 has no `"session"` key in context, ovoscope copies the session -from the last received message — simulating how a real OVOS client propagates session updates. +from the last received message: simulating how a real OVOS client propagates session updates. --- -## Pattern 7 — Testing Fallback Skills +## Pattern 7: Testing Fallback Skills Fallback skills receive a `"ovos.skills.fallback.ping"` message to probe for a handler, and then the main fallback message. The expected sequence is longer than a normal intent match: ```python @@ -344,15 +344,15 @@ test = End2EndTest( # ... handler messages ... Message("ovos.utterance.handled", {}), ], - # "ovos.skills.fallback.ping" is in DEFAULT_KEEP_SRC — its routing is checked against + # "ovos.skills.fallback.ping" is in DEFAULT_KEEP_SRC: its routing is checked against # the original source_message context, not the rolling flip-point tracker ) test.execute(timeout=15) ``` -See `DEFAULT_KEEP_SRC` in `ovoscope/__init__.py` — it pre-populates `keep_original_src` so +See `DEFAULT_KEEP_SRC` in `ovoscope/__init__.py`: it pre-populates `keep_original_src` so fallback ping routing is always validated against the original source message context. --- -## Pattern 8 — Session State Validation +## Pattern 8: Session State Validation Use `final_session` and `inject_active` to assert on session state at the end of a test: ```python from ovos_bus_client.session import Session @@ -403,7 +403,7 @@ test = End2EndTest( test_async_message_number=True, # assert exactly 1 async message received ) ``` -Async messages are collected in `CaptureSession.async_responses` — they are NOT in the main +Async messages are collected in `CaptureSession.async_responses`: they are NOT in the main `responses` list and are NOT included in `test_message_number` count. --- ## Disabling Assertions @@ -420,7 +420,7 @@ Some assertion groups can be turned off individually when a message is noisy or | `test_async_messages` | `True` | Assert async message types | | `test_async_message_number` | `True` | Assert async message count | | `test_final_session` | `True` | Assert final session state | -Example — disable data and routing checks for a noisy third-party message: +Example: disable data and routing checks for a noisy third-party message: ```python test = End2EndTest( ... @@ -430,11 +430,11 @@ test = End2EndTest( ``` --- ## Troubleshooting -### Timeout — no messages received +### Timeout: no messages received - The skill plugin is not loaded. Verify `find_skill_plugins()` returns your skill ID. - The session pipeline is empty or does not include the right plugin. Set `session.pipeline = [...]` explicitly. -- The EOF message (`ovos.utterance.handled`) never fires — check if the intent matched at all +- The EOF message (`ovos.utterance.handled`) never fires: check if the intent matched at all by setting `verbose=True` and inspecting stdout. ### Skill not loading ``` @@ -457,7 +457,7 @@ entry_points={ - For Padatious: training happens at `MiniCroft.run()` via `mycroft.skills.train`. If training fails silently, check the Padatious model files exist under `~/.local/share/`. ### Wrong message count -Enable `verbose=True` (default) — ovoscope prints every received message with its index. Compare +Enable `verbose=True` (default): ovoscope prints every received message with its index. Compare against the expected list to find the first divergence. ### `get_minicroft()` hangs `get_minicroft()` polls `croft.status.state` in a tight loop (0.1s sleep). If it hangs @@ -468,13 +468,13 @@ watch for tracebacks. ### Test lifecycle constants ```python from ovoscope import ( - DEFAULT_EOF, # ["ovos.utterance.handled"] — end-of-test trigger - DEFAULT_IGNORED, # ["ovos.skills.settings_changed"] — filtered out + DEFAULT_EOF, # ["ovos.utterance.handled"]: end-of-test trigger + DEFAULT_IGNORED, # ["ovos.skills.settings_changed"]: filtered out GUI_IGNORED, # GUI namespace messages ignored when ignore_gui=True - DEFAULT_ENTRY_POINTS, # ["recognizer_loop:utterance"] — routing reset points - DEFAULT_FLIP_POINTS, # [] — routing flip points - DEFAULT_KEEP_SRC, # ["ovos.skills.fallback.ping"] — always check vs original source - DEFAULT_ACTIVATION, # [] — activation check points + DEFAULT_ENTRY_POINTS, # ["recognizer_loop:utterance"]: routing reset points + DEFAULT_FLIP_POINTS, # []: routing flip points + DEFAULT_KEEP_SRC, # ["ovos.skills.fallback.ping"]: always check vs original source + DEFAULT_ACTIVATION, # []: activation check points DEFAULT_DEACTIVATION, # ["intent.service.skills.deactivate"] ) ``` @@ -490,7 +490,7 @@ from ovoscope import ( FALLBACK_PIPELINE, # ["ovos-fallback-pipeline-plugin-high", ...medium, ...low] COMMON_QUERY_PIPELINE, # ["ovos-common-query-pipeline-plugin"] PERSONA_PIPELINE, # ["ovos-persona-pipeline-plugin-high", ...low] - DEFAULT_TEST_PIPELINE, # all standard stages, no AI/persona/OCP — the default + DEFAULT_TEST_PIPELINE, # all standard stages, no AI/persona/OCP: the default ) ``` `DEFAULT_TEST_PIPELINE` is the default value of `MiniCroft.default_pipeline` when @@ -498,14 +498,14 @@ from ovoscope import ( reproducible results regardless of which AI plugins are installed. **Composing custom pipelines:** ```python -# Adapt intent only — fastest, no fallback +# Adapt intent only: fastest, no fallback mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE) -# Full intent chain with fallback — typical skill testing +# Full intent chain with fallback: typical skill testing mc = get_minicroft([SKILL_ID], default_pipeline=CONVERSE_PIPELINE + ADAPT_PIPELINE + FALLBACK_PIPELINE) -# Include persona pipeline — when testing AI persona behaviour +# Include persona pipeline: when testing AI persona behaviour mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE) -# No override — use whatever the system config says (includes OCP, m2v, etc.) +# No override: use whatever the system config says (includes OCP, m2v, etc.) mc = get_minicroft([SKILL_ID], default_pipeline=None) ``` Sessions created without an explicit `session` in their message context inherit @@ -513,13 +513,13 @@ Sessions created without an explicit `session` in their message context inherit The original pipeline is restored when `mc.stop()` is called. **When to use `PERSONA_PIPELINE`:** Only add persona stages when you are explicitly testing persona behaviour. Persona plugins make network calls to AI APIs and are -non-deterministic — they are intentionally excluded from `DEFAULT_TEST_PIPELINE`. +non-deterministic: they are intentionally excluded from `DEFAULT_TEST_PIPELINE`. --- ## See Also -- [end2end-test.md](end2end-test.md) — full `End2EndTest` parameter reference -- [minicroft.md](minicroft.md) — `MiniCroft` / `get_minicroft()` reference -- [capture-session.md](capture-session.md) — `CaptureSession` internals -- [ci-integration.md](ci-integration.md) — wiring ovoscope into GitHub Actions CI +- [end2end-test.md](end2end-test.md): full `End2EndTest` parameter reference +- [minicroft.md](minicroft.md): `MiniCroft` / `get_minicroft()` reference +- [capture-session.md](capture-session.md): `CaptureSession` internals +- [ci-integration.md](ci-integration.md): wiring ovoscope into GitHub Actions CI - Canonical examples: `Skills/ovos-skill-hello-world/test/test_helloworld.py` - Core examples: `ovos-core/test/end2end/` @@ -618,3 +618,6 @@ mc.stop() ``` See [ovoscope/__init__.py](../ovoscope/__init__.py) for `GUICaptureSession` API. + +--- +[Home](../README.md) · [CLI →](cli.md) diff --git a/docs/voice-loop.md b/docs/voice-loop.md index aee94df..a8dcc5a 100644 --- a/docs/voice-loop.md +++ b/docs/voice-loop.md @@ -22,14 +22,14 @@ captures the emitted bus sequence. The mocks live in Every harness inherits these (each takes an optional message list, defaulting to the last feed result, and returns the checked list): -- `assert_record_begin_emitted()` — `recognizer_loop:record_begin` present. -- `assert_wakeword_detected()` — both `recognizer_loop:wakeword` and `…:record_begin`. -- `assert_wakeword_suppressed()` — neither wake-word nor record-begin present. -- `assert_utterance_emitted(utterance=None)` — a `recognizer_loop:utterance` (optionally with the given text). +- `assert_record_begin_emitted()` (`recognizer_loop:record_begin`) present. +- `assert_wakeword_detected()`: both `recognizer_loop:wakeword` and `…:record_begin`. +- `assert_wakeword_suppressed()`: neither wake-word nor record-begin present. +- `assert_utterance_emitted(utterance=None)`: a `recognizer_loop:utterance` (optionally with the given text). --- -## ovos-dinkum-listener — `MiniVoiceLoop` +## ovos-dinkum-listener: `MiniVoiceLoop` ### Wake-word / verifier gate (`feed_chunks`) @@ -62,14 +62,14 @@ with loop([boom]) as vl: # fail-open | Sequence | Expected bus events | |---|---| | WW detected + all verifiers accept | `recognizer_loop:wakeword` + `…:record_begin` | -| WW detected + a verifier rejects | suppressed — no `recognizer_loop:*` | +| WW detected + a verifier rejects | suppressed: no `recognizer_loop:*` | | WW detected + a verifier raises (fail-open) | `…:record_begin` emitted | | No WW detected | no `recognizer_loop:*` | The verifier gate lives inside `DinkumVoiceLoop._detect_ww` and is only present in ovos-dinkum-listener builds that ship the hotword-verifier feature (`HotwordContainer.verify`). On a build without it the gate is absent and a -detection is never suppressed — assert accordingly for the version under test. +detection is never suppressed: assert accordingly for the version under test. ### Full loop from an audio file (`feed_file`) @@ -95,7 +95,7 @@ ovos-dinkum-listener is not installed. --- -## ovos-simple-listener — `MiniSimpleListener` +## ovos-simple-listener: `MiniSimpleListener` Drives the real `SimpleListener` thread with the canonical bus callbacks (a per-instance mirror of `OVOSCallbacks`). @@ -119,7 +119,7 @@ mimic production. Raises `RuntimeError` when ovos-simple-listener is absent. --- -## mycroft-classic-listener — `MiniClassicListener` +## mycroft-classic-listener: `MiniClassicListener` The classic listener is a threaded, energy-based pipeline. Two entry points: @@ -163,10 +163,10 @@ absent. --- -## Declarative helper — `VoiceLoopTest` +## Declarative helper: `VoiceLoopTest` -For the dinkum backend, `VoiceLoopTest` runs a scenario and asserts in one call — -via `feed_chunks` by default, or `feed_file` when `audio_file` is set: +For the dinkum backend, `VoiceLoopTest` runs a scenario and asserts in one call, +using `feed_chunks` by default, or `feed_file` when `audio_file` is set: ```python from unittest.mock import Mock @@ -201,3 +201,6 @@ VoiceLoopTest( | `bridge_recognizer_loop_to_bus` / `classic_listener_available` | Classic event-bridge + capability probe. | | `MockFileMicrophone`, `MockStreamingSTT`, `MockVADEngine`, `MockHotWordEngine` | Mock plugins shared across backends. | | `VoiceLoopTest` | Declarative dinkum scenario runner. | + +--- +[← Listener](listener.md) · [Home](../README.md) · [GUI Testing →](gui-testing.md)