Skip to content

docs: rewrite README in Simplified Technical English #119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JarbasAl merged 1 commit into from
Sep 1, 2026
Merged

docs: rewrite README in Simplified Technical English #119

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 39 additions & 27 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,34 @@
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%2B-blue)](https://pypi.org/project/ovoscope/)
# OvoScope

**End-to-end testing for [OVOS](https://openvoiceos.org) skills.**
OvoScope runs a full OVOS Core pipeline in-process using a `FakeBus` — no server, no audio
stack, no network. Load real skill plugins, emit a test utterance, and assert on every bus
message that comes back: type, data, routing context, session state, and message ordering.

OvoScope runs a full OVOS Core pipeline in-process with a `FakeBus`. It needs no server, no
audio stack, and no network. Load real skill plugins, send a test utterance, and check every
bus message that comes back: type, data, routing context, session state, and message order.

![image](https://github.com/user-attachments/assets/10a10ff5-64b7-42fd-86bd-cb6a5db769dd)
> Like a microscope for your OVOS skills.

---
## Features
| | |
|---|---|
| **Full pipeline** | Runs real intent pipeline plugins (Adapt, Padatious, Fallback, Converse, Common Query) |
| **Isolated** | Config isolation strips user preferences; deterministic `DEFAULT_TEST_PIPELINE` excludes AI/persona/OCP stages |
| **Ordered assertions** | Assert message type, data keys, routing context, and session state in sequence |
| **Recording mode** | Capture a live message sequence and save it as a JSON fixture — no manual construction needed |
| **Isolated** | Config isolation strips user preferences, and the deterministic `DEFAULT_TEST_PIPELINE` excludes AI, persona, and OCP stages |
| **Ordered assertions** | Checks message type, data keys, routing context, and session state in order |
| **Recording mode** | Captures a live message sequence and saves it as a JSON fixture. No manual construction needed |
| **Multi-turn** | Pass a list of utterances to test full conversational flows |
| **pytest fixture** | `minicroft` class-scoped fixture auto-discovered via the `pytest11` entry point |
| **Inject skills** | `extra_skills={id: SkillClass}` to load inline test skills without a PyPI entry point |
| **Inject messages** | `MiniCroft.inject_message()` to trigger non-utterance handlers (GUI events, timers, API calls) |
| **Typed models** | Optional `ovoscope[pydantic]` bridge to `ovos-pydantic-models` for schema-validated messages |
| **pytest fixture** | The `minicroft` class-scoped fixture is auto-discovered through the `pytest11` entry point |
| **Inject skills** | Use `extra_skills={id: SkillClass}` to load inline test skills without a PyPI entry point |
| **Inject messages** | Use `MiniCroft.inject_message()` to trigger non-utterance handlers (GUI events, timers, API calls) |
| **Typed models** | The optional `ovoscope[pydantic]` bridge adds schema-validated messages through `ovos-pydantic-models` |
---
## Installation
```bash
pip install ovoscope
```
With optional typed message model support:
To add typed message model support:
```bash
pip install ovoscope[pydantic]
```
Expand Down Expand Up @@ -62,28 +65,28 @@ class TestHelloWorld(unittest.TestCase):
],
).execute(timeout=10)
```
Only keys you specify in `expected.data` and `expected.context` are checked — extra keys in the
received message are ignored.
OvoScope checks only the keys you list in `expected.data` and `expected.context`. It ignores
extra keys in the received message.
---
## Recording Mode
Don't know the exact message sequence yet? Record it from a live run:
If you do not know the exact message sequence yet, record it from a live run:
```python
from ovoscope import End2EndTest
test = End2EndTest.from_message(
message=utterance,
skill_ids=[SKILL_ID],
timeout=20,
)
test.save("tests/fixtures/hello_world.json") # anonymises location data by default
test.save("tests/fixtures/hello_world.json") # anonymizes location data by default
```
Replay in CI:
Replay the fixture in CI:
```python
End2EndTest.from_path("tests/fixtures/hello_world.json").execute(timeout=10)
```
---
## pytest Fixture
The `minicroft` class-scoped fixture is auto-registered when ovoscope is installed.
No `setUp`/`tearDown` boilerplate needed:
OvoScope auto-registers the `minicroft` class-scoped fixture on install. You do not need
`setUp`/`tearDown` boilerplate:
```python
class TestMySkill:
skill_ids = ["my-skill.author"]
Expand All @@ -97,26 +100,26 @@ class TestMySkill:
```
---
## Pipeline Control
OvoScope exposes composable pipeline stage lists so tests are deterministic regardless of which
AI plugins are installed on the host:
OvoScope exposes composable pipeline stage lists so tests stay deterministic regardless of
which AI plugins are installed on the host:
```python
from ovoscope import ADAPT_PIPELINE, PADATIOUS_PIPELINE, FALLBACK_PIPELINE, PERSONA_PIPELINE
# Adapt only — fastest
# Adapt only: fastest
mc = get_minicroft([SKILL_ID], default_pipeline=ADAPT_PIPELINE)
# Full intent chain
mc = get_minicroft([SKILL_ID],
default_pipeline=ADAPT_PIPELINE + PADATIOUS_PIPELINE + FALLBACK_PIPELINE)
# Opt in to persona for AI testing
mc = get_minicroft([SKILL_ID], default_pipeline=DEFAULT_TEST_PIPELINE + PERSONA_PIPELINE)
```
`DEFAULT_TEST_PIPELINE` (the default when `isolate_config=True`) includes all standard built-in
stages and deliberately excludes persona, Ollama, OCP, and m2v plugins.
`DEFAULT_TEST_PIPELINE` is the default when `isolate_config=True`. It includes all standard
built-in stages and leaves out persona, Ollama, OCP, and m2v plugins.
---
## Documentation
| Document | |
|---|---|
| [docs/usage-guide.md](docs/usage-guide.md) | **Start here** — 8 test patterns with full worked examples |
| [docs/ci-integration.md](docs/ci-integration.md) | Wiring ovoscope into GitHub Actions |
| [docs/usage-guide.md](docs/usage-guide.md) | **Start here**: 8 test patterns with full worked examples |
| [docs/ci-integration.md](docs/ci-integration.md) | Wiring OvoScope into GitHub Actions |
| [docs/minicroft.md](docs/minicroft.md) | `MiniCroft` and `get_minicroft()` reference |
| [docs/capture-session.md](docs/capture-session.md) | `CaptureSession` internals |
| [docs/end2end-test.md](docs/end2end-test.md) | `End2EndTest` full parameter reference |
Expand All @@ -129,6 +132,15 @@ stages and deliberately excludes persona, Ollama, OCP, and m2v plugins.

---

## Related Projects

OvoScope is part of the [OpenVoiceOS](https://github.com/OpenVoiceOS) tooling suite:

- [ovos-core](https://github.com/OpenVoiceOS/ovos-core): the OVOS assistant core that OvoScope tests skills against.
- [ovos-workshop](https://github.com/OpenVoiceOS/ovos-workshop): the skill base classes that OvoScope loads and drives.
- [ovos-bus-client](https://github.com/OpenVoiceOS/ovos-bus-client): the message bus client behind `FakeBus` and `Message`.
- [ovos-test-harness](https://github.com/OpenVoiceOS/ovos-test-harness): a companion test harness for OVOS components.

## Credits

Developed by [TigreGótico](https://tigregotico.pt) for
Expand All @@ -152,7 +164,7 @@ under grant agreement No [101135429](https://cordis.europa.eu/project/id/1011354

## Contributing

PRs are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
PRs are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.

---

Expand Down
43 changes: 23 additions & 20 deletions docs/audio-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ classes provided in `ovoscope.audio`.

### AudioServiceHarness

`AudioServiceHarness` — `ovoscope/audio.py`
`AudioServiceHarness` (`ovoscope/audio.py`)

Wraps `AudioService` (from `ovos_audio.audio`) with a `MockAudioBackend` on a
`FakeBus`. Use it when your test exercises the audio routing layer — backend
`AudioServiceHarness` wraps `AudioService` (from `ovos_audio.audio`) with a `MockAudioBackend` on a
`FakeBus`. Use it when your test exercises the audio routing layer: backend
selection by URI scheme, volume ducking on speech events, the 1-second stop
guard, or session-source validation.

Expand All @@ -37,9 +37,9 @@ with AudioServiceHarness() as h:

### PlaybackServiceHarness

`PlaybackServiceHarness` — `ovoscope/audio.py`
`PlaybackServiceHarness` (`ovoscope/audio.py`)

Wraps `PlaybackService` (from `ovos_audio.service`) with a `MockTTS` on a
`PlaybackServiceHarness` wraps `PlaybackService` (from `ovos_audio.service`) with a `MockTTS` on a
`FakeBus`. Use it when testing TTS execution flow: `speak` messages, the
`recognizer_loop:audio_output_start/end` lifecycle, and optional mic-listen
triggers after speech.
Expand All @@ -55,11 +55,11 @@ with PlaybackServiceHarness() as h:

## Stop Guard Pitfall

`AudioService._stop()` — `ovos-audio/ovos_audio/audio.py` — checks
`AudioService._stop()` (`ovos-audio/ovos_audio/audio.py`) checks
`time.monotonic() - self.play_start_time > 1`. If stop is called within 1
second of `play()`, the stop command is silently ignored.

**Tests that call `stop()` must sleep at least 1.1 seconds after `play()`:**
Tests that call `stop()` must sleep at least 1.1 seconds after `play()`:

```python
import time
Expand All @@ -74,7 +74,7 @@ with AudioServiceHarness() as h:

## play_audio Patch Rationale

`PlaybackThread._play()` — `ovos-audio/ovos_audio/playback.py` — calls
`PlaybackThread._play()` (`ovos-audio/ovos_audio/playback.py`) calls
`play_audio(data)` then waits on the returned process object. Without patching,
this would invoke a real audio player binary (sox, aplay, paplay, mpg123).

Expand Down Expand Up @@ -111,13 +111,13 @@ with AudioServiceHarness() as h:
```

`AudioServiceHarness.get_track_info()` and `list_backends()` already implement
this pattern internally — `ovoscope/audio.py`.
this pattern internally, in `ovoscope/audio.py`.

## API Reference

### MockAudioBackend

`MockAudioBackend` — `ovoscope/audio.py`
`MockAudioBackend` (`ovoscope/audio.py`)

| Attribute / Method | Type | Description |
|---|---|---|
Expand All @@ -132,7 +132,7 @@ this pattern internally — `ovoscope/audio.py`.

### AudioServiceHarness

`AudioServiceHarness` — `ovoscope/audio.py`
`AudioServiceHarness` (`ovoscope/audio.py`)

| Method | Description |
|---|---|
Expand All @@ -151,7 +151,7 @@ this pattern internally — `ovoscope/audio.py`.

### MockTTS

`MockTTS` — `ovoscope/audio.py`
`MockTTS` (`ovoscope/audio.py`)

| Attribute / Method | Description |
|---|---|
Expand All @@ -162,7 +162,7 @@ this pattern internally — `ovoscope/audio.py`.

### PlaybackServiceHarness

`PlaybackServiceHarness` — `ovoscope/audio.py`
`PlaybackServiceHarness` (`ovoscope/audio.py`)

| Method | Description |
|---|---|
Expand All @@ -175,7 +175,7 @@ this pattern internally — `ovoscope/audio.py`.

### AudioCaptureSession

`AudioCaptureSession` — `ovoscope/audio.py`
`AudioCaptureSession` (`ovoscope/audio.py`)

| Method / Property | Description |
|---|---|
Expand All @@ -190,9 +190,12 @@ Default `track_prefixes` captures: `"mycroft.audio."`,

## Cross-References

- `AudioService` — `ovos-audio/ovos_audio/audio.py`
- `PlaybackService` — `ovos-audio/ovos_audio/service.py`
- `PlaybackThread` — `ovos-audio/ovos_audio/playback.py`
- `AudioBackend` (base class) — `ovos_plugin_manager.templates.audio.AudioBackend`
- `TTS` (base class) — `ovos_plugin_manager.templates.tts.TTS`
- End-to-end tests — `ovos-audio/test/end2end/`
- `AudioService` (`ovos-audio/ovos_audio/audio.py`)
- `PlaybackService` (`ovos-audio/ovos_audio/service.py`)
- `PlaybackThread` (`ovos-audio/ovos_audio/playback.py`)
- `AudioBackend`, the base class (`ovos_plugin_manager.templates.audio.AudioBackend`)
- `TTS`, the base class (`ovos_plugin_manager.templates.tts.TTS`)
- End-to-end tests: `ovos-audio/test/end2end/`

---
[← Pydantic Integration](pydantic-integration.md) · [Home](../README.md) · [Media Testing →](media-testing.md)
21 changes: 12 additions & 9 deletions docs/bus-coverage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,17 +141,17 @@ TOTAL 10/16 62.5% 10/15 6/15
In verbose mode (`--ovoscope-bus-cov-verbose`), `ovoscope` lists every message type:

```
LISTENERS — my-skill.author
✓ my-intent.intent 2 invocation(s)
✗ some-unused-event NOT TESTED
LISTENERS: my-skill.author
[x] my-intent.intent 2 invocation(s)
[ ] some-unused-event NOT TESTED

EMITTERS — my-skill.author
✓ speak observed 1x ✓ asserted
✓ my-skill.done observed 1x ✗ not asserted
EMITTERS: my-skill.author
[x] speak observed 1x [x] asserted
[x] my-skill.done observed 1x [ ] not asserted
```

* **✓ (Checked)**: The listener was triggered or the emitter was asserted.
* **✗ (Cross)**: The listener was never triggered or the emitter was seen but not checked in the test.
* **[x] (Checked)**: The listener was triggered or the emitter was asserted.
* **[ ] (Not checked)**: The listener was never triggered or the emitter was seen but not checked in the test.

---

Expand All @@ -174,4 +174,7 @@ If you are building custom tooling, you can access these values via `SkillBusCov
* `observed_emitter_pct`: `(observed_emitters / total_emitters) * 100`
* `asserted_emitter_pct`: `(asserted_emitters / total_emitters) * 100`

Source: `SkillBusCoverage` — `ovoscope/bus_coverage.py`
Source: `SkillBusCoverage` (`ovoscope/bus_coverage.py`)

---
[← GUI Testing](gui-testing.md) · [Home](../README.md)
11 changes: 7 additions & 4 deletions docs/capture-session.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
# CaptureSession
`CaptureSession` subscribes to all messages on the `FakeBus` and records them during a single test interaction. It handles synchronous responses (ordered, from the intent pipeline) and asynchronous responses (from external threads, unordered).
## Class: `CaptureSession` — `ovoscope/__init__.py`
## Class: `CaptureSession` (`ovoscope/__init__.py`)
```python
from ovoscope import CaptureSession
```
A `dataclass` that wraps a `MiniCroft` and manages message collection for one test interaction.
`CaptureSession.finish` — `ovoscope/__init__.py`
`CaptureSession.finish` (`ovoscope/__init__.py`)

> **Idempotency:** `finish()` may be called multiple times safely — subsequent calls
> **Idempotency:** `finish()` may be called multiple times safely: subsequent calls
> return the same message list without re-subscribing or clearing state.
### Fields
| Field | Type | Default | Description |
Expand All @@ -18,7 +18,7 @@ A `dataclass` that wraps a `MiniCroft` and manages message collection for one te
| `eof_msgs` | `list[str]` | `["ovos.utterance.handled"]` | Message types that signal end of interaction |
| `ignore_messages` | `list[str]` | `["ovos.skills.settings_changed"]` | Message types to discard |
| `async_messages` | `list[str]` | `[]` | Message types to route to `async_responses` instead |
| `done` | `threading.Event` | — | Set when an EOF message is received |
| `done` | `threading.Event` |: | Set when an EOF message is received |
### Methods
#### `capture(source_message, timeout=20)`
Emits `source_message` on the bus and waits for an EOF message (or timeout). Subsequent calls on the same session accumulate into `responses`.
Expand Down Expand Up @@ -86,3 +86,6 @@ capture.capture(follow_up, timeout=10)
all_messages = capture.finish()
```
`End2EndTest` does this automatically when `source_message` is a list.

---
[← MiniCroft](minicroft.md) · [Home](../README.md) · [End2EndTest →](end2end-test.md)
23 changes: 13 additions & 10 deletions docs/ci-integration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# CI Integration — ovoscope
# CI Integration: ovoscope
This document explains how to wire ovoscope end-to-end tests into a repo's CI pipeline using
`gh-automations` reusable workflows, and how to structure test files and fixtures.
---
Expand All @@ -16,7 +16,7 @@ my-skill-repo/
├── setup.py (or pyproject.toml)
└── ...
```
Separate `end2end/` from `unittests/` so they can be run independently — end2end tests are
Separate `end2end/` from `unittests/` so they can be run independently: end2end tests are
slower (they spin up a MiniCroft) and may require extra dependencies.
---
## pytest / unittest Configuration
Expand Down Expand Up @@ -65,9 +65,9 @@ Fixture files generated by `End2EndTest.save()` (see [usage-guide.md](usage-guid
contain the expected message sequence serialised as JSON.
**When to commit fixtures:**
- Commit fixtures that test stable, deterministic interactions (e.g., a specific dialog line).
- Do NOT commit fixtures where the `speak` utterance varies randomly — either omit the
- Do NOT commit fixtures where the `speak` utterance varies randomly: either omit the
`utterance` key from expected data or use manual assertion instead.
- Always generate fixtures with `anonymize=True` (the default) — this strips real location data.
- Always generate fixtures with `anonymize=True` (the default): this strips real location data.
**`.gitignore` pattern** (if you generate fixtures locally but don't want to commit them):
```gitignore
test/end2end/fixtures/*.json
Expand All @@ -77,7 +77,7 @@ Or selectively ignore only generated/recording artifacts:
test/end2end/fixtures/recorded_*.json
```
---
## GitHub Actions — End2End Job
## GitHub Actions: End2End Job
Add an end2end job to your `release_workflow.yml` or a dedicated workflow. This example follows
the `gh-automations` conventions used across all 203+ OVOS repos:
```yaml
Expand Down Expand Up @@ -114,7 +114,7 @@ jobs:
propose_release: true
secrets: inherit
```
The `build_tests` job runs before `publish_alpha` — a failing end2end test blocks the release.
The `build_tests` job runs before `publish_alpha`: a failing end2end test blocks the release.
---
## Standalone End2End Workflow
If your repo only needs end2end tests (no release automation), use a simpler workflow:
Expand Down Expand Up @@ -181,11 +181,14 @@ The ovoscope repository itself uses the standard OVOS workflow set:
| **Release Alpha** | `release_workflow.yml` | PR merge to `dev` | Runs tests first, then calls `publish-alpha.yml` |
| **Stable Release** | `publish_stable.yml` | Push to `master` | Calls `publish-stable.yml` with bot loop guard |
| **Labels** | `conventional-label.yaml` | PR open/edit | Auto-labels PRs with conventional commit types |
The release workflow gates alpha publishing on test success — a failing test blocks the release.
The release workflow gates alpha publishing on test success: a failing test blocks the release.
---
## See Also
- [usage-guide.md](usage-guide.md) — tutorial walkthrough with all patterns
- [gh-automations/docs/workflow-reference.md](../../gh-automations/docs/workflow-reference.md) — full reusable workflow reference
- [gh-automations/docs/repo-setup.md](../../gh-automations/docs/repo-setup.md) — per-repo workflow setup
- [usage-guide.md](usage-guide.md): tutorial walkthrough with all patterns
- [gh-automations/docs/workflow-reference.md](../../gh-automations/docs/workflow-reference.md): full reusable workflow reference
- [gh-automations/docs/repo-setup.md](../../gh-automations/docs/repo-setup.md): per-repo workflow setup
- Canonical examples: `Skills/ovos-skill-hello-world/test/test_helloworld.py`
- Core examples: `ovos-core/test/end2end/`

---
[← CLI](cli.md) · [Home](../README.md) · [MiniCroft →](minicroft.md)
Loading
Loading