Skip to content

fix(orchestrator): handle missing framework before skill preflight - #1347

Open
gewenyu99 wants to merge 7 commits into
mainfrom
codex/fix-integration-no-framework
Open

gewenyu99 wants to merge 7 commits into
mainfrom
codex/fix-integration-no-framework

Conversation

@gewenyu99

@gewenyu99 gewenyu99 commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

When framework detection finds nothing, the main integration flow continues into orchestrator preflight and reports that setup instructions failed to download. The scout report attributes 37 production users in seven days to this misleading error, including version 2.77.0.

Changes

  • Preserve the manual framework picker when interactive auto-detection finds nothing. If the integration program reaches its run entry without a framework configuration, abort with DetectNoFramework and app-root guidance before agent execution or skill preflight. CI uses the same no-framework outcome during pre-run detection.
  • Include the replay-vision no-framework guard and actionable message from fix(replay-vision): clear abort when no framework is detected #1205.
  • Use a shared abortNoFrameworkDetected helper for integration run/CI checks and replay vision's interactive/CI detection. Keep replay-specific headings and documentation in the replay program; CI receives the same actionable guidance as interactive runs.
  • Render a short red heading, a separate retry instruction and explanation, and a labeled documentation link, with blank lines between sections and extra spacing before “Press any key to exit.” Carry recovery instructions through the logging UI as well.
  • Exercise detection, the real terminal framework picker, run configuration, and error outros. A regression test simulates a detection miss in a Next.js project, selects Next.js by keyboard, and verifies the integration run can proceed.
  • Route early error outros past incomplete intro/setup/auth screens and exit on dismissal, so an abort does not wait behind a detection spinner. Regression tests verify the visible screen and real dismissal wait.
  • Preserve the generic missing-variant safety guard.

Credit

Thank you to Tal Gluck (@talagluck), who diagnosed and implemented the replay-vision fix in #1205. The replay-vision guard and actionable guidance originated with Tal. This PR carries that work forward through a shared abort helper and a shorter, spaced-out presentation. That contribution directly helped this fix.

This PR incorporates that work into a signed commit, with Tal credited as a co-author, so both affected flows can be fixed without waiting for the original commits to be signed. It supersedes #1205's code change and adds the main integration fix and regression coverage for both paths.

Test plan

  • Focused integration detection, manual-picker/no-framework, program adapter, and router suites: 111 tests passed. Confirmed the new picker and missing-framework run regressions fail before the fix.
  • Manually checked the rebuilt CLI in an empty folder: detection finishes, the manual picker appears, and choosing Next.js reaches Continue. Replay-vision error layout and dismissal were also verified in the terminal.
  • Shared error-layout, abort, router, and frame suites passed 177 tests when the presentation was added; orchestrator variant resolution passed 11 tests.
  • pnpm typecheck passed.
  • pnpm build passed, including CLI and warlock smoke tests.
  • Changed-file Prettier, ESLint, and git diff --check passed, with two existing non-null assertion warnings in the warehouse-reporting test.

LLM context

Implemented and verified with Codex from the supplied scout report; replay-vision behavior and wording were carried over from Tal's PR #1205.

Stop the main integration flow with DetectNoFramework when detection finds
no framework, before it can surface a misleading missing-skill error.

Include Tal Gluck's replay-vision fix and actionable guidance from
#1205. Thank you, Tal, for identifying
and fixing the replay-vision path. Carry that work in this signed commit
alongside the main integration fix and regression coverage for both flows.

Co-authored-by: Tal Gluck <talagluck@gmail.com>
@gewenyu99
gewenyu99 requested review from a team as code owners September 24, 2026 15:25
@gewenyu99
gewenyu99 requested review from TueHaulund, arnohillen, fasyy612 and ksvat and removed request for a team September 24, 2026 15:25
@github-actions

Copy link
Copy Markdown

🧙 Wizard CI

Run the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands:

Test all apps:

  • /wizard-ci all

Test all apps in a directory:

  • /wizard-ci ai-observability
  • /wizard-ci basic-integration
  • /wizard-ci feature-flags
  • /wizard-ci mcp-analytics
  • /wizard-ci replay-vision
  • /wizard-ci revenue
  • /wizard-ci self-driving
  • /wizard-ci warehouse
  • /wizard-ci warehouse-seeded

Test an individual app:

  • /wizard-ci ai-observability/anthropic
  • /wizard-ci ai-observability/google-adk
  • /wizard-ci ai-observability/groq
Show more apps
  • /wizard-ci ai-observability/manual-capture
  • /wizard-ci ai-observability/openai
  • /wizard-ci ai-observability/openai-agents
  • /wizard-ci ai-observability/opentelemetry
  • /wizard-ci ai-observability/vercel-ai
  • /wizard-ci basic-integration/android
  • /wizard-ci basic-integration/angular
  • /wizard-ci basic-integration/astro
  • /wizard-ci basic-integration/django
  • /wizard-ci basic-integration/fastapi
  • /wizard-ci basic-integration/flask
  • /wizard-ci basic-integration/flutter
  • /wizard-ci basic-integration/javascript-node
  • /wizard-ci basic-integration/javascript-web
  • /wizard-ci basic-integration/laravel
  • /wizard-ci basic-integration/next-js
  • /wizard-ci basic-integration/nuxt
  • /wizard-ci basic-integration/python
  • /wizard-ci basic-integration/rails
  • /wizard-ci basic-integration/react-native
  • /wizard-ci basic-integration/react-router
  • /wizard-ci basic-integration/sveltekit
  • /wizard-ci basic-integration/swift
  • /wizard-ci basic-integration/tanstack-router
  • /wizard-ci basic-integration/tanstack-start
  • /wizard-ci basic-integration/vue
  • /wizard-ci feature-flags/django
  • /wizard-ci feature-flags/next-js
  • /wizard-ci mcp-analytics/custom-dispatcher
  • /wizard-ci mcp-analytics/typescript-sdk
  • /wizard-ci replay-vision/javascript-node
  • /wizard-ci replay-vision/next-js
  • /wizard-ci replay-vision/react-vite
  • /wizard-ci revenue/stripe
  • /wizard-ci self-driving/astro
  • /wizard-ci self-driving/fastapi
  • /wizard-ci self-driving/nuxt
  • /wizard-ci self-driving/react-router
  • /wizard-ci self-driving/sveltekit
  • /wizard-ci warehouse/monorepo-env
  • /wizard-ci warehouse/multi-source-next
  • /wizard-ci warehouse/stripe-node
  • /wizard-ci warehouse/zero-source
  • /wizard-ci warehouse-seeded/next-stripe
  • /wizard-ci warehouse-seeded/next-stripe-declined

Test against a Context Mill branch:

  • /wizard-ci all context-mill:my-branch

Add context-mill:<branch> to any command above to pin the Context Mill branch. It defaults to main.

Results will be posted here when complete.

Route early error outros before walking the program's incomplete screens.
Framework-detection aborts otherwise wait for dismissal while the intro
keeps displaying its detection spinner. Exit after dismissal and preserve
the existing failed-run handoff.

Exercise the real outro dismissal wait for both integration and replay
vision, and check early-error routing across registered programs.
ctx.setUnsupportedVersion(versionResult.supported);
}
} else {
await wizardAbort({

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

astra this is not dry do beter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe not entirely relevant but I ban else statements entirely XD

Use one abort helper for integration and replay-vision detection in both
interactive and CI paths. Keep replay-specific guidance with its program
and provide the same actionable message to CI runs.

Extend the no-framework regression coverage to both CI pre-run hooks.

@sarahxsanders sarahxsanders left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the abort looks like it would remove manual framework picking

Comment on lines 81 to 84
} else {
await abortNoFrameworkDetected();
return;
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we keep the manual framework picker when a project exists but detection returns no result?

this exit also blocks users who could select their supported framework and continue

@gewenyu99 gewenyu99 Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch (robot slop removed)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in be6ea8d. Interactive detection now finishes and shows the manual framework picker when it finds nothing. The no-framework guard runs at the integration program's run entry, before agent execution; CI still aborts during pre-run detection.

The regression test simulates a missed Next.js project, selects Next.js through the real terminal picker, and verifies the resulting run configuration. All 111 focused tests, typecheck, and build pass. I also checked the rebuilt CLI manually. The commit is signed and verified by GitHub.

It's done, NotVincent will tell Vincent to double check.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manual framework picker after auto-detection finds no framework

Separate the short error heading, recovery instruction, supporting detail,
and labeled documentation link. Give the terminal exit hint extra space
and name the actual exit action. Preserve recovery instructions in CI logs.

Use structured outro data for the shared no-framework abort and keep
replay-specific headings and links in the replay program.
gewenyu99 and others added 3 commits September 24, 2026 12:07
Let interactive detection finish without a framework so users can select
one manually. Guard the integration run entry before constructing a run
when no framework configuration exists. Keep CI and replay guards.

Cover detection miss, keyboard selection, and missing-framework run abort.
johncwaters added a commit to johncwaters/glimmervoid that referenced this pull request Sep 25, 2026
…ndbox

Team PR reviews now run the operator's own pr-review skill, so a lane
review holds the same standard as a hand-run one, and the skill's Step 4
posting JSON is kept and posted verbatim after the operator approves. The
posted shape matches hand-run reviews: the automated-review note, a
**[tag] SEVERITY** header and multi-paragraph bodies with a suggested fix.
The code-review block supplies verdict, findings and summary and is the
fallback render; verdicts use code-review's vocabulary, and drafts saved
with the old verdicts still load.

The session reviews untrusted PR text with permissions bypassed, so it
starts in an empty work dir and reaches the checkout by absolute path only
(an --add-dir loads the checkout's .claude/skills). GitHub credentials,
git credential helpers, ssh and push urls and GLIMMERVOID secrets are
withheld; MCP servers are off; and Claude Code's Bash sandbox pins egress
to OpenAI, ChatGPT and api.github.com, blocks ssh, gh, keychain and
git-credentials reads, and fails closed: a session that cannot apply its
sandbox never spawns. The prompt states Codex lanes are expected, and
Codex runs inside the sandbox.

Constraint: api.github.com stays on the egress list because Codex 0.155 syncs curated plugins from it at startup and stalls when blocked; no GitHub credential is reachable inside
Constraint: Codex reads ~/.codex/auth.json inside the sandbox only because the dotfiles Read deny on it was lifted (dotfiles fa25be3)
Rejected: a separate CODEX_HOME login for the lane | one more credential to keep alive
Rejected: prefix deny rules as the only barrier | bypassable via command, absolute paths, bash -c and python
Directive: HIGH round-3 fix (sandbox fail-open when the hooks settings file is not written, session/session-hook-lifecycle.ts) landed after the last review; not promoted until the operator has looked at it
Confidence: medium
Scope-risk: broad
Not-tested: round-3 fixes (Session refuses to spawn when a requested sandbox is not applied; server/AGENTS.md invariant rewording) were not re-reviewed, only unit-tested and exercised by a sandboxed live run on PostHog/wizard#1347; Linux bubblewrap path not run; suite failures limited to pre-existing trace-wiring (26) and a timing-flaky integration-ref-watch

@johncwaters johncwaters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

The no-framework abort is small, guarded and consistent across the integration, CI, error-tracking and replay-vision paths. One question on exit status below; otherwise looks good.

Comment thread src/ui/tui/router.ts
session.runPhase === RunPhase.Error &&
session.outroData?.kind === OutroKind.Error
) {
return session.outroDismissed ? ScreenId.Exit : ScreenId.Outro;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

After dismissal this routes to ScreenId.Exit, and ExitScreen calls process.exit(0) unless a mint handoff is set, while wizardAbort also resumes on the same dismissal and exits with code 1 (after emitting its error line). If the Exit screen wins, an aborted run exits 0. Ordering probably favors wizardAbort, but nothing pins it. Could a test assert the exit code on dismissal?

Comment thread src/ui/tui/router.ts
@@ -78,24 +78,19 @@ export class WizardRouter {
return this.overlays[this.overlays.length - 1];
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Automated review. Not written by a human.

Nit: this branch now applies to any run in RunPhase.Error with an error outro, replacing the Auth-only special case, so later post-run screens are skipped after any error abort. Probably intended, just confirming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants