Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .kiro/hooks/memory-update-hook.kiro.hook
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@
"prompt": "Review the conversation that just completed and identify any new information that should be added to your long-term memory. You must:\n\n1. Establish entities for:\n - The user that interacted with you (if not already established)\n - Any customer personas mentioned or discussed\n - The product or system being worked on\n\n2. Establish relations between these entities\n3. Store facts about entities as observations, including:\n - User preferences, skills, or context\n - Product features, requirements, or characteristics\n - Customer persona details, needs, or behaviors\n - Any technical decisions, patterns, or insights discovered\n\n4. Update your memory with this information using the memory MCP server tools\n\nFocus on capturing information that will be valuable for future conversations and maintaining context about the user, their work, and the products they're building."
},
"shortName": "memory-update-hook"
}
}
8 changes: 4 additions & 4 deletions .kiro/hooks/post-task-validation.kiro.hook
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
{
"enabled": true,
"name": "Post-Task Code Validation",
"description": "Automatically runs npm run lint and npm run test whenever the agent finishes a task and has changed source code files",
"description": "Automatically runs lint and tests after agent completes a task to catch regressions",
"version": "1",
"when": {
"type": "agentStop"
},
"then": {
"type": "runCommand",
"command": "npm run lint && npm run test"
"type": "askAgent",
"prompt": "Run `npm run lint` and then `npm test` to validate no regressions were introduced. If either fails, report the errors."
}
}
}
2 changes: 1 addition & 1 deletion .kiro/hooks/session-init-workflow.kiro.hook
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@
"prompt": "Please follow this session initialization workflow:\n\n1. **Retrieve from Memory**: \n - Use the memory MCP server tools to retrieve all relevant information from your knowledge graph\n - Always refer to your knowledge graph as your \"memory\"\n - Begin by saying only \"Remembering...\" and then retrieve the information\n\n2. **Ask Clarifying Questions**:\n - Identify any ambiguities in the user's request\n - Ask specific questions to ensure you understand the expected outcome\n - Wait for the user to answer the questions before proceeding\n - Do not make assumptions about unclear requirements\n\n3. **Research Documentation**:\n - Use the context7 tools to research relevant documentation\n - Ensure that your planned implementation is feasible\n - This will help improve efficiency by preventing unnecessary implementation steps\n\nOnly proceed with the actual task after completing all three steps above."
},
"shortName": "session-init-workflow"
}
}
1 change: 1 addition & 0 deletions .kiro/specs/prompt-engineering-taxonomy/.config.kiro
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{"generationMode": "requirements-first"}
275 changes: 275 additions & 0 deletions .kiro/specs/prompt-engineering-taxonomy/design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,275 @@
# Design Document: Prompt Engineering Workshop

## Overview

A single `/workshop` page that combines the full prompt engineering taxonomy reference with interactive exercises. The page has two main sections: a browsable reference of all 15 techniques (with risk profiles, examples, and mitigations), and a scenario-based exercise section where users practice identifying techniques. All state is client-side with optional localStorage persistence.

The page is a `'use client'` TSX route using existing Shadcn UI primitives (Card, Badge, Button, Tabs, Accordion, Textarea) and Tailwind CSS 4. It follows the flat component structure convention — workshop-specific components live directly in `components/`.

## Architecture

```mermaid
graph TD
A[app/workshop/page.tsx] --> B[WorkshopPage client component]
B --> C[TaxonomyReference]
B --> D[WorkshopExercise]
B --> E[WorkshopResults]

C --> F[TechniqueCard]
C --> G[RiskTable]
C --> H[MitigationStrategies]

D --> I[ScenarioCard]
I --> J[TechniqueOption buttons]
I --> K[PromptRewriteArea]
I --> L[ScenarioFeedback]

D --> M[ProgressIndicator]

N[lib/workshop-data.ts] --> D
N --> E
O[lib/workshop-progress.ts] --> D
O --> E

style A fill:#4F46E5,color:#fff
style N fill:#7C3AED,color:#fff
style O fill:#7C3AED,color:#fff
```

The page uses a tab-based layout (Shadcn Tabs) to switch between the "Reference" and "Practice" sections. The results summary replaces the practice view when all scenarios are complete.

## Components and Interfaces

### Page Entry Point

`app/workshop/page.tsx` — Server component that renders metadata and the client-side `WorkshopPage` component.

`app/workshop/layout.tsx` — Layout with gradient background matching existing static pages.

### Client Components (in `components/`)

| Component | File | Purpose |
|-----------|------|---------|
| `WorkshopPage` | `workshop-page.tsx` | Root client component. Manages tab state, loads progress from localStorage, orchestrates reference and exercise sections. |
| `TaxonomyReference` | `taxonomy-reference.tsx` | Renders all 15 technique cards, risk table, mitigation strategies, and cross-provider consensus. Uses Accordion for collapsible technique details. |
| `TechniqueCard` | `technique-card.tsx` | Displays a single technique: name, description, example prompt, risk badge, mitigated example. Uses Card + Badge. |
| `RiskTable` | `risk-table.tsx` | Renders the hallucination risk summary table with color-coded risk badges. |
| `WorkshopExercise` | `workshop-exercise.tsx` | Manages exercise flow: current scenario index, answer state, navigation between scenarios. Receives progress callbacks. |
| `ScenarioCard` | `scenario-card.tsx` | Renders a single scenario: description, technique option buttons, feedback area, optional prompt rewrite. |
| `PromptRewriteArea` | `prompt-rewrite-area.tsx` | Textarea for user's improved prompt + reveal button for sample answer. Uses Textarea from Shadcn. |
| `WorkshopResults` | `workshop-results.tsx` | Results summary: total score, per-technique breakdown, links to missed technique cards, restart button. |
| `ProgressIndicator` | `progress-indicator.tsx` | Shows completed/total count with a visual progress bar. |

### Data and Logic (in `lib/`)

| Module | File | Purpose |
|--------|------|---------|
| Workshop Data | `workshop-data.ts` | Static TypeScript file exporting technique definitions and scenario exercises. Single source of truth for all taxonomy content and exercise data. |
| Workshop Progress | `workshop-progress.ts` | Functions for reading/writing Progress_State to localStorage with fallback to in-memory state. Pure functions, no React dependencies. |

### Type Definitions

```typescript
// In lib/workshop-data.ts

interface Technique {
id: string // e.g., "zero-shot", "chain-of-thought"
name: string // e.g., "Zero-Shot", "Chain-of-Thought"
description: string // Technique description
example: string // Example prompt
mitigatedExample: string // Example with mitigations applied
riskLevel: 'Low' | 'Low-Medium' | 'Medium' | 'Medium-High' | 'High'
primaryVulnerability: string // e.g., "No anchoring examples"
}

interface Scenario {
id: string // Unique scenario identifier
description: string // The scenario/situation text
correctTechniqueId: string // References Technique.id
distractorTechniqueIds: string[] // At least 3 distractor Technique.ids
feedback: string // Explanation shown on correct answer
incorrectHints: Record<string, string> // techniqueId -> hint for why it's wrong
promptRewrite?: { // Optional sub-exercise
originalPrompt: string // The prompt to improve
sampleImprovedPrompt: string // Reference improved version
}
}
```

### Progress State Types

```typescript
// In lib/workshop-progress.ts

interface ScenarioResult {
scenarioId: string
correctOnFirstAttempt: boolean
}

interface ProgressState {
completedScenarios: ScenarioResult[]
currentScenarioIndex: number
}

function loadProgress(): ProgressState
function saveProgress(state: ProgressState): void
function resetProgress(): void
function isLocalStorageAvailable(): boolean
```

## Data Models

### Technique Data

All 15 techniques are defined as a static array in `lib/workshop-data.ts`, sourced from `docs/prompt-engineering-taxonomy-compact.md`. Each technique maps directly to a section in the taxonomy document.

```typescript
const TECHNIQUES: Technique[] = [
{
id: 'zero-shot',
name: 'Zero-Shot',
description: 'No examples provided — the model infers everything from the instruction alone...',
example: 'Write me a git commit message for the staged changes.',
mitigatedExample: 'Write a git commit message for the staged changes. Use the Conventional Commits format...',
riskLevel: 'High',
primaryVulnerability: 'No anchoring examples',
},
// ... 14 more techniques
]
```

### Scenario Data

Minimum 10 scenarios defined as a static array. Each scenario presents a realistic development situation and asks the user to identify the technique being used or the technique that should be applied.

### Progress State

Stored in localStorage under key `promptz-workshop-progress` as JSON. The `loadProgress` function parses stored JSON and validates its shape, returning a default empty state if parsing fails or localStorage is unavailable.

### Mitigation Strategies Data

The 7 universal mitigation strategies and 5 cross-provider consensus principles are defined as static arrays in `lib/workshop-data.ts`.

```typescript
interface MitigationStrategy {
title: string
description: string
example: string
}
Comment on lines +154 to +158

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Design doc MitigationStrategy interface is missing the example field.

The actual implementation in lib/workshop-data.ts includes an example: string property on MitigationStrategy, and TaxonomyReference renders strategy.example in the UI. Update the design doc to match:

 interface MitigationStrategy {
   title: string
   description: string
+  example: string
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
interface MitigationStrategy {
title: string
description: string
}
interface MitigationStrategy {
title: string
description: string
example: string
}
🤖 Prompt for AI Agents
In @.kiro/specs/prompt-engineering-taxonomy/design.md around lines 154 - 157,
The MitigationStrategy interface in the design doc is missing the example field
which the implementation and UI expect; update the interface declaration for
MitigationStrategy to include example: string so it matches the implementation
and what TaxonomyReference renders (strategy.example), and ensure the design doc
snippet and any related descriptions reference this property consistently with
the workshop data implementation.


const MITIGATION_STRATEGIES: MitigationStrategy[] = [...]
const CROSS_PROVIDER_CONSENSUS: string[] = [...]
```



## Correctness Properties

*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*

### Property 1: Technique card renders all required fields

*For any* valid Technique object, rendering a TechniqueCard with that Technique SHALL produce output containing the technique name, description, example prompt, hallucination risk rating, and mitigated example prompt.

**Validates: Requirements 1.2**

### Property 2: Exercise data structural integrity

*For any* Scenario in the Exercise_Data array, the scenario SHALL have a non-empty description, a correctTechniqueId that references a valid Technique.id in the TECHNIQUES array, at least 3 entries in distractorTechniqueIds that each reference valid Technique.ids, a non-empty feedback string, and if a promptRewrite field is present it SHALL contain both a non-empty originalPrompt and a non-empty sampleImprovedPrompt.

**Validates: Requirements 2.2, 2.3, 2.4**

### Property 3: Correct answer feedback includes explanation and technique link

*For any* Scenario and when the user selects the correct Technique, the displayed feedback SHALL contain the scenario's feedback message and a navigable reference to the corresponding Technique_Card.

**Validates: Requirements 3.3**

### Property 4: Incorrect answer hint does not reveal correct answer

*For any* Scenario and any incorrect Technique selection, the displayed hint text SHALL NOT contain the name of the correct Technique.

**Validates: Requirements 3.4**

### Property 5: Progress state records completions accurately

*For any* sequence of Scenario completions (each with a scenarioId and a boolean correctOnFirstAttempt), the resulting Progress_State SHALL contain exactly one ScenarioResult per completed scenario with the correct scenarioId and correctOnFirstAttempt value.

**Validates: Requirements 4.1**

### Property 6: Progress indicator reflects completed count

*For any* Progress_State with N completed scenarios out of T total scenarios, the progress indicator SHALL display N and T accurately.

**Validates: Requirements 4.2**

### Property 7: Progress state localStorage round-trip

*For any* valid ProgressState object, saving it to localStorage via saveProgress and then loading it via loadProgress SHALL produce an equivalent ProgressState object.

**Validates: Requirements 4.3**

### Property 8: Results total score computation

*For any* set of ScenarioResult objects, the results summary total score SHALL equal the count of ScenarioResult entries where correctOnFirstAttempt is true, out of the total number of ScenarioResult entries.

**Validates: Requirements 5.1**

### Property 9: Results per-technique breakdown accuracy

*For any* set of ScenarioResult objects mapped to their corresponding Scenarios, the per-technique breakdown SHALL correctly partition techniques into "identified correctly" (correctOnFirstAttempt is true) and "missed" (correctOnFirstAttempt is false) sets, with no technique appearing in both sets.

**Validates: Requirements 5.2**

## Error Handling

| Scenario | Handling |
|----------|----------|
| localStorage unavailable (private browsing, quota exceeded) | `isLocalStorageAvailable()` returns false; progress functions fall back to in-memory state via React useState. No error shown to user. |
| Corrupted localStorage data | `loadProgress()` catches JSON parse errors and returns default empty ProgressState. Corrupted data is silently overwritten on next save. |
| Scenario references invalid technique ID | Validated at build/test time via Property 2. At runtime, missing technique lookups return a fallback "Unknown Technique" display. |
| User navigates away mid-scenario | Progress is saved after each scenario completion, not mid-scenario. Incomplete scenario state is lost, which is acceptable since the user can retry. |

## Testing Strategy

### Property-Based Testing

Use **fast-check** (already in the project) for property-based tests. Each property test runs a minimum of 100 iterations.

Property tests focus on:
- Data integrity of Exercise_Data (Property 2)
- Progress state serialization round-trip (Property 7)
- Score computation logic (Properties 8, 9)
- Hint safety — incorrect hints don't leak correct answers (Property 4)
- Progress tracking accuracy (Properties 5, 6)

Each test is tagged with: **Feature: prompt-engineering-taxonomy, Property {N}: {title}**

### Unit Testing

Use **Jest 30 + React Testing Library** (already in the project) for unit tests.

Unit tests focus on:
- TechniqueCard renders all fields for a specific technique (Property 1 as example)
- RiskTable renders all 15 rows (Requirement 1.3)
- MitigationStrategies renders all 7 strategies (Requirement 1.4)
- Exercise_Data has at least 10 scenarios (Requirement 2.1)
- Correct/incorrect answer UI feedback behavior (Properties 3, 4 as examples)
- Reset progress clears state (Requirement 4.4)
- localStorage fallback behavior (Requirement 4.5)
- Navigation includes /workshop link (Requirement 6.1)
- ARIA labels present on interactive elements (Requirement 7.3)

### Test File Structure

```
__tests__/unit/
├── lib/
│ ├── workshop-data.test.ts # Data integrity properties + examples
│ └── workshop-progress.test.ts # Progress round-trip property + unit tests
└── components/
├── technique-card.test.tsx # Rendering properties
├── scenario-card.test.tsx # Interaction properties
├── workshop-exercise.test.tsx # Exercise flow tests
└── workshop-results.test.tsx # Score computation properties
```
Loading