diff --git a/src/agent/__tests__/__snapshots__/commandments.test.ts.snap b/src/agent/__tests__/__snapshots__/commandments.test.ts.snap index a1c395e11..7d230efe9 100644 --- a/src/agent/__tests__/__snapshots__/commandments.test.ts.snap +++ b/src/agent/__tests__/__snapshots__/commandments.test.ts.snap @@ -22,7 +22,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en - For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`). - Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`). - Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running. -After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields. +After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered. +Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking. Use the Task tools to plan and track the whole run so the user always sees where you are. Create the task list once you understand the work — after you load and skim the skill workflow, not before — in a single tool call, in the order you will run them, with one task per stage covering the whole run through to instrumenting events, creating the dashboard, and writing the setup report. Give each an imperative subject AND an \`activeForm\` (the present-continuous label the panel shows while it runs, e.g. subject "Install SDK" / activeForm "Installing SDK"). Keep the list current: add a task the moment you discover work it is missing. Try to keep exactly ONE task \`in_progress\`. \`TaskUpdate\` it to \`in_progress\` right before you start that stage, and to \`completed\` the instant you finish it — one at a time, never batched at the end. Only mark \`completed\` when the work is genuinely done; if the build fails, a step is partial, or you hit a blocker, keep it \`in_progress\` and add a task for the fix. After you complete a task, take the next one in order (lowest id first — earlier stages set up later ones), mark it \`in_progress\`, and continue. Driving the list in order top to bottom is how you finish every stage. @@ -51,7 +52,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en - For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`). - Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`). - Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running. -After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields." +After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered. +Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking." `; exports[`commandments by axis > 'pi' + 'linear' > matches the published prompt 1`] = ` @@ -76,7 +78,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en - For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`). - Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`). - Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running. -After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields. +After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered. +Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking. Use the Task tools to plan and track the whole run so the user always sees where you are. Create the task list once you understand the work — after you load and skim the skill workflow, not before — in a single tool call, in the order you will run them, with one task per stage covering the whole run through to instrumenting events, creating the dashboard, and writing the setup report. Give each an imperative subject AND an \`activeForm\` (the present-continuous label the panel shows while it runs, e.g. subject "Install SDK" / activeForm "Installing SDK"). Keep the list current: add a task the moment you discover work it is missing. Try to keep exactly ONE task \`in_progress\`. \`TaskUpdate\` it to \`in_progress\` right before you start that stage, and to \`completed\` the instant you finish it — one at a time, never batched at the end. Only mark \`completed\` when the work is genuinely done; if the build fails, a step is partial, or you hit a blocker, keep it \`in_progress\` and add a task for the fix. After you complete a task, take the next one in order (lowest id first — earlier stages set up later ones), mark it \`in_progress\`, and continue. Driving the list in order top to bottom is how you finish every stage. @@ -129,7 +132,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en - For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`). - Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`). - Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running. -After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields. +After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered. +Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking. ## This runtime Below are important guidance on the harness constraints you are bound to. Follow them as commandments. @@ -172,7 +176,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en - For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`). - Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`). - Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running. -After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields. +After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered. +Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking. ALWAYS surface a custom-scout proposal in step 6b: bring the user your one or two strongest candidate scouts even when the built-in troop looks sufficient. The proposal ask leads with a "None — keep the built-in troop" option, so declining costs the user one keystroke — but a proposal you silently skip is coverage they never got to see or judge. Where the skill says to skip the ask when the gap analysis finds no candidate, do NOT skip: pick your best candidates anyway and let the user decide. Rank candidates at the discriminator level, not the category level. "Covered" only means an enabled scout would actually FIRE for that failure mode: a conversion-rate watcher does not catch entry volume collapsing; a Stripe-transaction watcher does not catch a lead form going silent. A surface whose failure mode has no firing condition among the enabled scouts is your strongest candidate. Be honest in the option descriptions: if a candidate overlaps something an enabled scout partially watches, say so in its description rather than dropping the candidate. The user chooses with full information; you do not gatekeep on their behalf. diff --git a/src/agent/__tests__/commandments.test.ts b/src/agent/__tests__/commandments.test.ts index 80db96e59..3e4f22114 100644 --- a/src/agent/__tests__/commandments.test.ts +++ b/src/agent/__tests__/commandments.test.ts @@ -78,6 +78,18 @@ describe('commandments by axis', () => { ); }); }); + describe('wizard_ask correction rules', () => { + const text = WIZARD_COMMANDMENTS.join('\n'); + + it('lets the agent re-ask the fields a downstream call rejected', () => { + expect(text).toMatch(/rejected/i); + expect(text).toMatch(/reuse the same `subject`/); + }); + + it('keeps a dismissed or timed-out ask a decline, not a correction', () => { + expect(text).toMatch(/dismissed or timed-out ask is not this case/i); + }); + }); }); /** diff --git a/src/agent/commandments.ts b/src/agent/commandments.ts index b45593cc8..8089f6f80 100644 --- a/src/agent/commandments.ts +++ b/src/agent/commandments.ts @@ -45,6 +45,7 @@ export const WIZARD_COMMANDMENTS = [ ' - For `single` and `multi`, extract the alternatives from the prose into `options` as `{ label, value }` pairs. Use the human phrase as `label` and a lowercase-hyphenated form as `value` (e.g., `label: "Vanilla JS"`, `value: "vanilla-js"`).', ' - Use a kebab-case slug of the question label as `id` (e.g., "Tech stack" → `tech-stack`, "Show frequency" → `show-frequency`).', ' - Do not invent fields the schema does not define (no `source`, `category`, `priority`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.', - 'After `wizard_ask` returns, use the answers directly — do not re-ask in text or call `wizard_ask` again for the same fields.', + 'After `wizard_ask` returns, use the answers directly — do not re-ask in text, and do not call `wizard_ask` again to confirm a field the user already answered.', + 'Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same `subject`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.', ].join('\n'), ];