test: isolate the prompt_toolkit app session so UI tests run on Windows - #921
Open
MohammedAlkindi wants to merge 1 commit into
Open
test: isolate the prompt_toolkit app session so UI tests run on Windows#921MohammedAlkindi wants to merge 1 commit into
MohammedAlkindi wants to merge 1 commit into
Conversation
Building a prompt_toolkit Application binds the ambient AppSession's output, and constructing that output probes the terminal. Under pytest's captured stdout on Windows the probe raises NoConsoleScreenBufferError, so every test that builds UI fails there. Linux CI never sees it because the POSIX Vt100 output has no equivalent probe. Bind a DummyOutput and a pipe input for the duration of each test, which is the documented prompt_toolkit pattern for this and also stops the tests reaching for the real terminal when run locally. On Windows this takes tests/unit from 231 failures to 200, and removes all 62 NoConsoleScreenBufferError occurrences. Totals reconcile at 7408 either way, so no previously-passing test changes state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
On Windows, 62 occurrences of
prompt_toolkit.output.win32.NoConsoleScreenBufferErrortake out 31 tests intests/unit— the suite yourchecks.ymlruns asuv run pytest tests/unit -v.Building a prompt_toolkit
Applicationbinds the ambientAppSession's output, and constructing that output probes the terminal for its screen buffer. Under pytest's captured stdout that probe fails on Windows:Linux CI never sees this, because the POSIX
Vt100_Outputhas no equivalent probe — the failure needs a Windows host and a non-console stdout, which is exactly the pair CI cannot produce.Changes
One autouse fixture in
tests/conftest.py, alongside the existingisolate_herdr_lifecycle, binding aDummyOutputand a pipe input for the duration of each test. This is the documented prompt_toolkit pattern for testing, and it makes the session explicit rather than ambient — which also stops the tests reaching for the real terminal when run locally.Testing
Windows 11 (10.0.26200), Python 3.12.10, clean clone of
mainat0b8a364,pip install -e .pluspytest/pytest-asyncio.pytest tests/unitNoConsoleScreenBufferErroroccurrencesTotals reconcile at 7408 collected either way, and failures fell by exactly the number passes rose (31), so no previously-passing test changes state on Windows.
What is deliberately unchanged
This fixes one cause, not the Windows run as a whole — 200 failures remain and I have not touched them. From a quick triage they are mostly separate concerns: tests asserting POSIX path strings (
tests/unit/fast_agent/utils/test_path_display.pyexpectssrc/app.pywhere Windows correctly producessrc\app.py), a uvloop fallback test, and docs-snapshot comparisons. Several look like test-side platform assumptions rather than product bugs, and some may be intentional. I did not want to bundle judgement calls about those into a change that is mechanically verifiable on its own.Limits I want to be upfront about
I could not run this on Linux, so I have not verified there. The change is test-only and
DummyOutputis prompt_toolkit's own testing primitive, but a global autouse fixture does alter the ambient session for every test — if any Linux-only test depends on the real output object, CI will surface it and I will happily scope the fixture down to the UI packages instead. Your CI is the authority here, not my run.Note on AI assistance
I used an AI assistant while working on this. The before/after suite runs, the failure-cause counts, and the reconciliation of the totals quoted above were all executed by me on the Windows machine described, and I confirmed the specific traceback in
win32.py:220rather than inferring the cause from the test names.