Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions skills/openrouter-generations/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# openrouter-generations

Inspect individual OpenRouter generations — get request metadata (cost, latency, tokens, model, provider routing) and stored prompt/completion content.
Inspect individual OpenRouter generations — get request metadata (cost, latency, tokens, model, provider routing) and stored prompt/completion content or failure errors.

## Install

Expand Down Expand Up @@ -34,7 +34,7 @@ rm -rf /tmp/or-skills
See [SKILL.md](SKILL.md) for the full reference, including:

- Fetching generation metadata (tokens, cost, latency, model, provider, routing)
- Retrieving stored prompt and completion content
- Retrieving stored prompt and completion content or failure errors
- Debugging failed or unexpected generations
- Understanding provider fallback chains
- Tracing multi-generation sessions
Expand All @@ -45,7 +45,7 @@ See [SKILL.md](SKILL.md) for the full reference, including:
| Script | Purpose |
|--------|---------|
| `get-generation.ts` | Get metadata for a generation (cost, tokens, latency, provider) |
| `get-generation-content.ts` | Get stored prompt and completion text |
| `get-generation-content.ts` | Get stored prompt, completion, and failure error |

## Quick start

Expand Down
46 changes: 44 additions & 2 deletions skills/openrouter-generations/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ cd <skill-path>/scripts && npm install
| Endpoint | Method | Purpose |
|----------|--------|---------|
| `/api/v1/generation` | GET | Request metadata and usage (tokens, cost, latency, model, provider) |
| `/api/v1/generation/content` | GET | Stored prompt and completion text |
| `/api/v1/generation/content` | GET | Stored prompt, completion, and failure error |

Both take a single query parameter: `id` (the generation ID).

Expand Down Expand Up @@ -65,8 +65,9 @@ npx tsx get-generation-content.ts --id gen-1234567890 --json

- **Input**: `prompt` (raw text) and/or `messages` (array of `{role, content}`)
- **Output**: `completion` (the model's response) and `reasoning` (chain-of-thought, if applicable)
- **Error**: `status`, `message`, `provider_name`, `raw`, and `previous_errors` for a stored failed generation, or `null` when the generation succeeded

**Note:** Content is only available if the generation was *not* made with Zero Data Retention (ZDR) enabled. If ZDR was on, this endpoint returns empty/null content.
**Note:** Content is only available if the generation was *not* made with Zero Data Retention (ZDR) enabled. If ZDR was on, this endpoint returns empty/null content. A failed generation may return a stored `error` even when `input` and `output` are empty. On a failed generation, `output.completion` is `null`.

## Direct API Usage (curl)

Expand Down Expand Up @@ -131,6 +132,7 @@ curl -G https://openrouter.ai/api/v1/generation/content \
```json
{
"data": {
"error": null,
"input": {
"prompt": "What is the meaning of life?",
"messages": [
Expand All @@ -148,20 +150,51 @@ curl -G https://openrouter.ai/api/v1/generation/content \
}
```

For a failed generation, `output.completion` is `null` and `error` contains the returned error plus any earlier failed provider attempts:

```json
{
"data": {
"input": {},
"output": {
"completion": null,
"reasoning": null
},
"error": {
"status": 504,
"message": "Timed out waiting for the provider",
"provider_name": "Vertex",
"raw": "{\"error\":{\"code\":504,\"message\":\"Deadline exceeded\"}}",
"previous_errors": [
{
"code": 429,
"message": "Provider returned error",
"provider_name": "Google",
"raw": "{\"error\":{\"code\":429,\"message\":\"Resource exhausted\"}}"
}
]
}
}
}
```

## Common Use Cases

### Debug a failed generation

```bash
# Check what happened — look at finish_reason, provider_responses, and cancelled
cd <skill-path>/scripts && npx tsx get-generation.ts gen-abc123 --json
# Inspect the stored client/provider error and earlier failed attempts
npx tsx get-generation-content.ts gen-abc123 --json
```

Look for:
- `finish_reason` = `"length"` means the model hit max tokens
- `finish_reason` = `"content_filter"` means content was filtered
- `cancelled` = `true` means the request was cancelled by the client
- `provider_responses` with multiple entries means fallbacks occurred
- Content response `data.error` for the status, message, returned provider, raw error body, and `previous_errors`

### Check cost of a specific request

Expand Down Expand Up @@ -238,3 +271,12 @@ If you have a `request_id` or `session_id` from one generation, you can find rel
| `data.input.messages` | array\|null | Messages array (`[{role, content}]`) |
| `data.output.completion` | string\|null | Model's completion text |
| `data.output.reasoning` | string\|null | Chain-of-thought reasoning |
| `data.error.status` | integer\|null | HTTP status returned to the client |
| `data.error.message` | string\|null | Error message returned to the client |
| `data.error.provider_name` | string\|null | Provider whose error was returned |
| `data.error.raw` | string\|null | Raw provider error body when stored |
| `data.error.previous_errors[]` | array | Earlier failed provider attempts, in attempt order |
| `data.error.previous_errors[].code` | integer | HTTP status returned by the attempt |
| `data.error.previous_errors[].message` | string | Error message returned by the attempt |
| `data.error.previous_errors[].provider_name` | string\|null | Provider that served the attempt |
| `data.error.previous_errors[].raw` | string\|null | Raw provider error body for the attempt |
36 changes: 33 additions & 3 deletions skills/openrouter-generations/scripts/get-generation-content.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
/**
* Retrieve the stored prompt and completion content for a generation.
* Returns input messages/prompt and output completion/reasoning text.
* Content is unavailable if Zero Data Retention (ZDR) was enabled.
* Returns input messages/prompt, output completion/reasoning text, and stored errors.
*/
import { requireApiKey, fetchGenerationContent, parseArgs } from "./lib.js";

Expand All @@ -18,6 +17,7 @@ Usage: npx tsx get-generation-content.ts <generation-id> [--json]
Returns the stored prompt and completion content:
- Input: original prompt text and messages array
- Output: completion text and reasoning (if available)
- Error: stored client/provider error and previous failed attempts (if available)

Note: Content is only available if the generation was not
made with Zero Data Retention (ZDR) enabled.
Expand Down Expand Up @@ -49,6 +49,19 @@ if (json) {
const output = rawData.output as
| { completion?: string; reasoning?: string }
| undefined;
const error = rawData.error as
| {
status?: number | null;
message?: string | null;
provider_name?: string | null;
previous_errors?: Array<{
code: number;
message: string;
provider_name: string | null;
}>;
}
| null
| undefined;

console.log("Generation:", generationId);
console.log("");
Expand All @@ -61,6 +74,7 @@ if (json) {
const hasOutput = Boolean(
output && (output.completion != null || output.reasoning != null)
);
const hasError = error != null;

if (hasInput && input) {
console.log("=== INPUT ===");
Expand Down Expand Up @@ -91,7 +105,23 @@ if (json) {
}
}

if (!hasInput && !hasOutput) {
if (hasError && error) {
console.log("=== ERROR ===");
console.log("Status:", error.status ?? null);
console.log("Message:", error.message ?? null);
console.log("Provider:", error.provider_name ?? null);
if (error.previous_errors && error.previous_errors.length > 0) {
console.log("Previous attempts:");
for (const attempt of error.previous_errors) {
console.log(
` [${attempt.code}] ${attempt.message} (${attempt.provider_name ?? "unknown"})`
);
}
}
console.log("");
}

if (!hasInput && !hasOutput && !hasError) {
console.log("No content available for this generation.");
console.log(
"This may be because Zero Data Retention (ZDR) was enabled."
Expand Down