diff --git a/skills/openrouter-generations/README.md b/skills/openrouter-generations/README.md index b45163f..3df6af8 100644 --- a/skills/openrouter-generations/README.md +++ b/skills/openrouter-generations/README.md @@ -1,6 +1,6 @@ # openrouter-generations -Inspect individual OpenRouter generations — get request metadata (cost, latency, tokens, model, provider routing) and stored prompt/completion content. +Inspect individual OpenRouter generations — get request metadata (cost, latency, tokens, model, provider routing) and stored prompt/completion content or failure errors. ## Install @@ -34,7 +34,7 @@ rm -rf /tmp/or-skills See [SKILL.md](SKILL.md) for the full reference, including: - Fetching generation metadata (tokens, cost, latency, model, provider, routing) -- Retrieving stored prompt and completion content +- Retrieving stored prompt and completion content or failure errors - Debugging failed or unexpected generations - Understanding provider fallback chains - Tracing multi-generation sessions @@ -45,7 +45,7 @@ See [SKILL.md](SKILL.md) for the full reference, including: | Script | Purpose | |--------|---------| | `get-generation.ts` | Get metadata for a generation (cost, tokens, latency, provider) | -| `get-generation-content.ts` | Get stored prompt and completion text | +| `get-generation-content.ts` | Get stored prompt, completion, and failure error | ## Quick start diff --git a/skills/openrouter-generations/SKILL.md b/skills/openrouter-generations/SKILL.md index c127173..cd4be12 100644 --- a/skills/openrouter-generations/SKILL.md +++ b/skills/openrouter-generations/SKILL.md @@ -25,7 +25,7 @@ cd /scripts && npm install | Endpoint | Method | Purpose | |----------|--------|---------| | `/api/v1/generation` | GET | Request metadata and usage (tokens, cost, latency, model, provider) | -| `/api/v1/generation/content` | GET | Stored prompt and completion text | +| `/api/v1/generation/content` | GET | Stored prompt, completion, and failure error | Both take a single query parameter: `id` (the generation ID). @@ -65,8 +65,9 @@ npx tsx get-generation-content.ts --id gen-1234567890 --json - **Input**: `prompt` (raw text) and/or `messages` (array of `{role, content}`) - **Output**: `completion` (the model's response) and `reasoning` (chain-of-thought, if applicable) +- **Error**: `status`, `message`, `provider_name`, `raw`, and `previous_errors` for a stored failed generation, or `null` when the generation succeeded -**Note:** Content is only available if the generation was *not* made with Zero Data Retention (ZDR) enabled. If ZDR was on, this endpoint returns empty/null content. +**Note:** Content is only available if the generation was *not* made with Zero Data Retention (ZDR) enabled. If ZDR was on, this endpoint returns empty/null content. A failed generation may return a stored `error` even when `input` and `output` are empty. On a failed generation, `output.completion` is `null`. ## Direct API Usage (curl) @@ -131,6 +132,7 @@ curl -G https://openrouter.ai/api/v1/generation/content \ ```json { "data": { + "error": null, "input": { "prompt": "What is the meaning of life?", "messages": [ @@ -148,6 +150,34 @@ curl -G https://openrouter.ai/api/v1/generation/content \ } ``` +For a failed generation, `output.completion` is `null` and `error` contains the returned error plus any earlier failed provider attempts: + +```json +{ + "data": { + "input": {}, + "output": { + "completion": null, + "reasoning": null + }, + "error": { + "status": 504, + "message": "Timed out waiting for the provider", + "provider_name": "Vertex", + "raw": "{\"error\":{\"code\":504,\"message\":\"Deadline exceeded\"}}", + "previous_errors": [ + { + "code": 429, + "message": "Provider returned error", + "provider_name": "Google", + "raw": "{\"error\":{\"code\":429,\"message\":\"Resource exhausted\"}}" + } + ] + } + } +} +``` + ## Common Use Cases ### Debug a failed generation @@ -155,6 +185,8 @@ curl -G https://openrouter.ai/api/v1/generation/content \ ```bash # Check what happened — look at finish_reason, provider_responses, and cancelled cd /scripts && npx tsx get-generation.ts gen-abc123 --json +# Inspect the stored client/provider error and earlier failed attempts +npx tsx get-generation-content.ts gen-abc123 --json ``` Look for: @@ -162,6 +194,7 @@ Look for: - `finish_reason` = `"content_filter"` means content was filtered - `cancelled` = `true` means the request was cancelled by the client - `provider_responses` with multiple entries means fallbacks occurred +- Content response `data.error` for the status, message, returned provider, raw error body, and `previous_errors` ### Check cost of a specific request @@ -238,3 +271,12 @@ If you have a `request_id` or `session_id` from one generation, you can find rel | `data.input.messages` | array\|null | Messages array (`[{role, content}]`) | | `data.output.completion` | string\|null | Model's completion text | | `data.output.reasoning` | string\|null | Chain-of-thought reasoning | +| `data.error.status` | integer\|null | HTTP status returned to the client | +| `data.error.message` | string\|null | Error message returned to the client | +| `data.error.provider_name` | string\|null | Provider whose error was returned | +| `data.error.raw` | string\|null | Raw provider error body when stored | +| `data.error.previous_errors[]` | array | Earlier failed provider attempts, in attempt order | +| `data.error.previous_errors[].code` | integer | HTTP status returned by the attempt | +| `data.error.previous_errors[].message` | string | Error message returned by the attempt | +| `data.error.previous_errors[].provider_name` | string\|null | Provider that served the attempt | +| `data.error.previous_errors[].raw` | string\|null | Raw provider error body for the attempt | diff --git a/skills/openrouter-generations/scripts/get-generation-content.ts b/skills/openrouter-generations/scripts/get-generation-content.ts index d24ce6c..62529e6 100644 --- a/skills/openrouter-generations/scripts/get-generation-content.ts +++ b/skills/openrouter-generations/scripts/get-generation-content.ts @@ -1,7 +1,6 @@ /** * Retrieve the stored prompt and completion content for a generation. - * Returns input messages/prompt and output completion/reasoning text. - * Content is unavailable if Zero Data Retention (ZDR) was enabled. + * Returns input messages/prompt, output completion/reasoning text, and stored errors. */ import { requireApiKey, fetchGenerationContent, parseArgs } from "./lib.js"; @@ -18,6 +17,7 @@ Usage: npx tsx get-generation-content.ts [--json] Returns the stored prompt and completion content: - Input: original prompt text and messages array - Output: completion text and reasoning (if available) + - Error: stored client/provider error and previous failed attempts (if available) Note: Content is only available if the generation was not made with Zero Data Retention (ZDR) enabled. @@ -49,6 +49,19 @@ if (json) { const output = rawData.output as | { completion?: string; reasoning?: string } | undefined; + const error = rawData.error as + | { + status?: number | null; + message?: string | null; + provider_name?: string | null; + previous_errors?: Array<{ + code: number; + message: string; + provider_name: string | null; + }>; + } + | null + | undefined; console.log("Generation:", generationId); console.log(""); @@ -61,6 +74,7 @@ if (json) { const hasOutput = Boolean( output && (output.completion != null || output.reasoning != null) ); + const hasError = error != null; if (hasInput && input) { console.log("=== INPUT ==="); @@ -91,7 +105,23 @@ if (json) { } } - if (!hasInput && !hasOutput) { + if (hasError && error) { + console.log("=== ERROR ==="); + console.log("Status:", error.status ?? null); + console.log("Message:", error.message ?? null); + console.log("Provider:", error.provider_name ?? null); + if (error.previous_errors && error.previous_errors.length > 0) { + console.log("Previous attempts:"); + for (const attempt of error.previous_errors) { + console.log( + ` [${attempt.code}] ${attempt.message} (${attempt.provider_name ?? "unknown"})` + ); + } + } + console.log(""); + } + + if (!hasInput && !hasOutput && !hasError) { console.log("No content available for this generation."); console.log( "This may be because Zero Data Retention (ZDR) was enabled."