Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 10 additions & 5 deletions src/agent/__tests__/__snapshots__/commandments.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en
- For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`).
- Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`).
- Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered.
Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.
Use the Task tools to plan and track the whole run so the user always sees where you are. Create the task list once you understand the work — after you load and skim the skill workflow, not before — in a single tool call, in the order you will run them, with one task per stage covering the whole run through to instrumenting events, creating the dashboard, and writing the setup report. Give each an imperative subject AND an \`activeForm\` (the present-continuous label the panel shows while it runs, e.g. subject "Install SDK" / activeForm "Installing SDK"). Keep the list current: add a task the moment you discover work it is missing.
Try to keep exactly ONE task \`in_progress\`. \`TaskUpdate\` it to \`in_progress\` right before you start that stage, and to \`completed\` the instant you finish it — one at a time, never batched at the end. Only mark \`completed\` when the work is genuinely done; if the build fails, a step is partial, or you hit a blocker, keep it \`in_progress\` and add a task for the fix.
After you complete a task, take the next one in order (lowest id first — earlier stages set up later ones), mark it \`in_progress\`, and continue. Driving the list in order top to bottom is how you finish every stage.
Expand Down Expand Up @@ -51,7 +52,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en
- For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`).
- Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`).
- Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields."
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered.
Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking."
`;

exports[`commandments by axis > 'pi' + 'linear' > matches the published prompt 1`] = `
Expand All @@ -76,7 +78,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en
- For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`).
- Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`).
- Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered.
Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.
Use the Task tools to plan and track the whole run so the user always sees where you are. Create the task list once you understand the work — after you load and skim the skill workflow, not before — in a single tool call, in the order you will run them, with one task per stage covering the whole run through to instrumenting events, creating the dashboard, and writing the setup report. Give each an imperative subject AND an \`activeForm\` (the present-continuous label the panel shows while it runs, e.g. subject "Install SDK" / activeForm "Installing SDK"). Keep the list current: add a task the moment you discover work it is missing.
Try to keep exactly ONE task \`in_progress\`. \`TaskUpdate\` it to \`in_progress\` right before you start that stage, and to \`completed\` the instant you finish it — one at a time, never batched at the end. Only mark \`completed\` when the work is genuinely done; if the build fails, a step is partial, or you hit a blocker, keep it \`in_progress\` and add a task for the fix.
After you complete a task, take the next one in order (lowest id first — earlier stages set up later ones), mark it \`in_progress\`, and continue. Driving the list in order top to bottom is how you finish every stage.
Expand Down Expand Up @@ -129,7 +132,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en
- For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`).
- Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`).
- Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered.
Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.

## This runtime
Below are important guidance on the harness constraints you are bound to. Follow them as commandments.
Expand Down Expand Up @@ -172,7 +176,8 @@ When a skill provides a numbered or bulleted list of questions, translate the en
- For \`single\` and \`multi\`, extract the alternatives from the prose into \`options\` as \`{ label, value }\` pairs. Use the human phrase as \`label\` and a lowercase-hyphenated form as \`value\` (e.g., \`label: "Vanilla JS"\`, \`value: "vanilla-js"\`).
- Use a kebab-case slug of the question label as \`id\` (e.g., "Tech stack" → \`tech-stack\`, "Show frequency" → \`show-frequency\`).
- Do not invent fields the schema does not define (no \`source\`, \`category\`, \`priority\`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text or call \`wizard_ask\` again for the same fields.
After \`wizard_ask\` returns, use the answers directly — do not re-ask in text, and do not call \`wizard_ask\` again to confirm a field the user already answered.
Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same \`subject\`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.
ALWAYS surface a custom-scout proposal in step 6b: bring the user your one or two strongest candidate scouts even when the built-in troop looks sufficient. The proposal ask leads with a "None — keep the built-in troop" option, so declining costs the user one keystroke — but a proposal you silently skip is coverage they never got to see or judge. Where the skill says to skip the ask when the gap analysis finds no candidate, do NOT skip: pick your best candidates anyway and let the user decide.
Rank candidates at the discriminator level, not the category level. "Covered" only means an enabled scout would actually FIRE for that failure mode: a conversion-rate watcher does not catch entry volume collapsing; a Stripe-transaction watcher does not catch a lead form going silent. A surface whose failure mode has no firing condition among the enabled scouts is your strongest candidate.
Be honest in the option descriptions: if a candidate overlaps something an enabled scout partially watches, say so in its description rather than dropping the candidate. The user chooses with full information; you do not gatekeep on their behalf.
Expand Down
12 changes: 12 additions & 0 deletions src/agent/__tests__/commandments.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,18 @@ describe('commandments by axis', () => {
);
});
});
describe('wizard_ask correction rules', () => {
const text = WIZARD_COMMANDMENTS.join('\n');

it('lets the agent re-ask the fields a downstream call rejected', () => {
expect(text).toMatch(/rejected/i);
expect(text).toMatch(/reuse the same `subject`/);
});

it('keeps a dismissed or timed-out ask a decline, not a correction', () => {
expect(text).toMatch(/dismissed or timed-out ask is not this case/i);
});
});
});

/**
Expand Down
3 changes: 2 additions & 1 deletion src/agent/commandments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ export const WIZARD_COMMANDMENTS = [
' - For `single` and `multi`, extract the alternatives from the prose into `options` as `{ label, value }` pairs. Use the human phrase as `label` and a lowercase-hyphenated form as `value` (e.g., `label: "Vanilla JS"`, `value: "vanilla-js"`).',
' - Use a kebab-case slug of the question label as `id` (e.g., "Tech stack" → `tech-stack`, "Show frequency" → `show-frequency`).',
' - Do not invent fields the schema does not define (no `source`, `category`, `priority`, etc.) — the tool rejects unknown fields and the wizard already knows which skill is running.',
'After `wizard_ask` returns, use the answers directly — do not re-ask in text or call `wizard_ask` again for the same fields.',
'After `wizard_ask` returns, use the answers directly — do not re-ask in text, and do not call `wizard_ask` again to confirm a field the user already answered.',
'Correcting a rejected value is the one exception, and it is expected: when whatever you handed an answer to rejects it — a connection that fails, a host or key the API calls invalid, a permission error — ask again for only the fields that were wrong, reuse the same `subject`, and say what was rejected and what a valid value looks like. A value the user could fix in one keystroke is never a reason to abandon the step for a link they have to follow themselves. Stop after a couple of attempts on one subject. A dismissed or timed-out ask is not this case — the result names the fields it did not collect and why, and that is a decline, so fall back instead of re-asking.',
].join('\n'),
];
Loading