fix(billing): record charge lines for embeddings, images, moderations, and rerank - #542
Conversation
…, and rerank run_passthrough set usage_log.cost from compute_cost but never wrote billing_meters or pricing_breakdown, unlike the chat/messages/responses path (calculate_metered_cost) and the imported-usage repricing path, both of which set cost, meters, and breakdown together. Every embeddings/images/moderations/rerank usage row carried a bare cost with no auditable per-meter breakdown. Add a compute_meters callback to run_passthrough, mirroring compute_cost's signature and only invoked once a cost has been priced, and implement it per route: input tokens for embeddings/rerank, image count for images, a flat per-request charge for moderations. Refs mozilla-ai#432
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (6)
🚧 Files skipped from review as they are similar to previous changes (5)
WalkthroughPassthrough usage logging now supports billing meters and pricing breakdowns. Embeddings, images, moderations, and rerank routes provide endpoint-specific meter callbacks. Integration tests verify priced and unpriced billing data. ChangesPassthrough billing metering
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Moderations prices as a flat per-request rate and returns 0.0 rather than None when a model is unpriced, so the meters callback ran on the free path and wrote a `$0.00` line the dashboard renders as a "billed meters" block explaining a charge that never happened. Record nothing there instead, and take the per-request rate from the cost, which is the same number when one request is billed. Also name the embeddings and rerank input meter `total_input_tokens`: that is the meter the billed-token SQL and the dashboard's token bar read, so the row's own meter becomes their source instead of the raw-column fallback. Both routes log the same count as their prompt tokens, so the reported value is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
njbrake
left a comment
njbrake
left a comment
There was a problem hiding this comment.
Note: this review was drafted by Claude via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.
Approving. The mechanism is right, the rate math checks out on all four routes (per_image_cost stores raw USD per image, flat_request_cost divides by 1e6, both are reflected correctly in the lines), and the two line shapes match what the dashboard's breakdown renderer discriminates on. Full suite, lint, typecheck, and openapi-check all pass locally.
I pushed one commit to your branch rather than leaving it as review comments:
Unpriced moderations recorded a zero charge line. compute_cost there returns flat_request_cost(pricing), which is 0.0 and not None for an unpriced model, so compute_meters ran on the free path and wrote {"requests": 1} plus a $0.00 line. The dashboard renders that as a "Billed meters" block explaining a charge that never happened, which contradicts the route's own comment about unpriced moderations being intentionally silent, and diverges from calculate_metered_cost, which suppresses empty lines. Now returns None when nothing was charged, with a regression test.
Renamed the embeddings and rerank input meter to total_input_tokens. input_tokens had no readers: the billed-token SQL (usage.py:_billed_expr) and the dashboard's token bar (tokenComposition) both read total_input_tokens by name, so those rows were still falling through to the raw columns. Both routes log the same count as their prompt tokens, so no reported value changes; the meter just stops being decorative.
One thing I left alone deliberately. images and requests are genuinely new non-token units, which is the part of #432 this PR does not claim to solve, and they now land in the /v1/usage response before any reader exists for them, so a later rename is a wire change. I am fine shipping them as-is, but flagging that we are naming them now.
Minor, non-blocking: n_images is derived twice in images.py, once in each callback. Worth extracting so they cannot drift.
Codecov Report❌ Patch coverage is
🚀 New features to boost your workflow:
|
3 participants
Description
Passthrough usage rows for embeddings, images, moderations, and rerank set
costbut neverbilling_metersorpricing_breakdown, unlike the chat/messages/responses path (calculate_metered_cost) and the imported-usage repricing path, both of which set all three together. Every one of those rows carried a bare cost with no auditable per-meter breakdown.Root cause:
run_passthrough(_passthrough.py) only ever wroteusage_log.costfromcompute_cost; nothing in the shared scaffold or in the four routes wrote the two audit fields.Fix: added a
compute_meterscallback torun_passthrough, mirroringcompute_cost's signature and only invoked once a cost has been priced, and implemented it per route: input tokens for embeddings/rerank, image count for images, a flat per-request charge for moderations.Not covered: audio (transcriptions/speech) is one of the four surfaces this issue names, but it never calls
compute_costat all today (no per-second/per-minute pricing model exists yet for it), so there is no cost for a meters breakdown to attach to. Audio billing needs its own pricing unit first, henceRefs #432rather thanFixes #432.Relevant issues
Refs #432
Verification
test_embeddings_billing_meters_tracked_with_pricingand theimages/moderations/rerankequivalents) fail onmain(billing_metersisNone) and pass on this branch.make test(2596 unit + integration tests),make lint,make typecheck.PR Type
Checklist
tests/unit,tests/integration).make lint,make typecheck,make test).uv run python scripts/generate_openapi.py).AI Usage
AI Model/Tool used: Claude (Anthropic), in an agentic coding session directed and verified by the account owner.
Any additional AI details you'd like to share: None.
Summary
Benefits
Usage logs now show how passthrough charges are calculated. This improves billing visibility and auditability.