Skip to content

fix: extract Google Gemini token usage and price cached tokens correctly - #8

Open
MukundaKatta wants to merge 1 commit into
mainfrom
chore/automated-improvements
Open

fix: extract Google Gemini token usage and price cached tokens correctly#8
MukundaKatta wants to merge 1 commit into
mainfrom
chore/automated-improvements

Conversation

@MukundaKatta

Copy link
Copy Markdown
Owner

Summary

The README, TypeScript types, and CHANGELOG all advertised that defaultExtractUsage (and therefore measureLLM) supported Google Gemini responses, but the extractor only ever inspected a usage key. The Gemini SDK (@google/genai) returns token usage under usageMetadata (promptTokenCount / candidatesTokenCount / cachedContentTokenCount), so a real Gemini response returned null and its tokens were silently recorded as zero cost — exactly the under-reporting this library exists to prevent.

Changes

  • Bug fix — Gemini usage extraction. defaultExtractUsage now recognizes the Gemini usageMetadata shape (including when nested under a response property) alongside the existing Anthropic / OpenAI usage shapes.
  • Hardening. When a usage/usageMetadata object is present but contains no recognized token fields, the extractor returns null instead of a misleading all-zero record.
  • Pricing accuracy. Added cacheRead rates to the Gemini 2.5/2.0 PRICING entries so cached-context tokens are priced at the (much lower) cached rate rather than falling back to the full input rate.
  • Flaky-test fix. The measure tracks latency test asserted elapsed >= 5 against a separately-sampled wall clock; Node timers can fire a hair early relative to performance.now(), so it failed intermittently. It now asserts on the library's own recorded step.latencyMs.
  • Tests. Added coverage for the Gemini shape, response-nested usage, and the no-known-fields guard. Test count 26 → 29.
  • Docs. README API section, src/index.d.ts doc comment, CHANGELOG, and a runnable Gemini call in examples/demo.js.

Validation

  • npm run lint — clean (syntax check of src + tests).
  • npm test29/29 passing, run 5 times consecutively with no flakes.
  • npm run test:examples — runs; the new rerank [gemini-2.5-flash] step is now priced ($0.000285) instead of $0.
  • Cost-under-report CI smoke check (pinned claude-sonnet-4) still passes.
  • Note: the coverage job uses c8 v11, which is incompatible with the very latest local Node (v26); it fails identically on unmodified main and works on CI's Node 22. Not affected by this change.

🤖 Generated with Claude Code

The README, TypeScript types, and CHANGELOG all advertised that
`defaultExtractUsage` (and therefore `measureLLM`) supported Google
Gemini responses, but the extractor only inspected a `usage` key. The
Gemini SDK (`@google/genai`) returns usage under `usageMetadata` with
`promptTokenCount` / `candidatesTokenCount` / `cachedContentTokenCount`,
so a Gemini call returned `null` and its tokens were silently recorded
as zero cost — exactly the under-reporting this library exists to catch.

Changes:
- `defaultExtractUsage` now recognizes the Gemini `usageMetadata` shape
  (including when nested under a `response` property) alongside the
  existing Anthropic / OpenAI `usage` shapes.
- When a `usage`/`usageMetadata` object is present but contains no
  recognized token fields, return `null` instead of a misleading
  all-zero record.
- Add `cacheRead` rates to the Gemini 2.5/2.0 PRICING entries so cached
  context tokens are priced at the (much lower) cached rate rather than
  falling back to the full input rate.
- Add a runnable Gemini call to examples/demo.js.
- Stabilize the previously flaky "measure tracks latency" test, which
  asserted `elapsed >= 5` against a separately-sampled wall clock; it
  now asserts on the library's own recorded `step.latencyMs`.
- Docs: README API section, `src/index.d.ts` doc comment, and CHANGELOG
  updated; test count 26 -> 29.

Validation: `npm run lint` clean; `npm test` 29/29 (run 5x, no flakes);
`npm run test:examples` runs and shows the Gemini step priced correctly;
cost-under-report CI smoke check still passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant