You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
/v1/responses reports usage.output_tokens_details.reasoning_tokens = 0 even when the reply has reasoning #717
With thinking on, /v1/responses returns a reasoning output item holding the full trace, but usage.output_tokens_details.reasoning_tokens is always 0, both non-streaming and in the streamed terminal events. A client that budgets or reports from this field is told no reasoning happened. When max_output_tokens cuts a reply off, the client cannot tell from usage how much of the budget the reasoning used. /v1/chat/completions has no completion_tokens_details at all, so it gives no count either (#307 and a comment on #302 mention this in passing).
Minimal reproduction
PYTHONPATH=python python -c "
from freetoken.server.generation import GenResult
from freetoken.server.responses_api import build_responses_response, ResponsesRequest
req = ResponsesRequest(model='m', input='hi')
r = GenResult(reasoning='Let me think about this carefully. '*20, content='4', tool_calls=[],
finish_reason='stop', prompt_tokens=12, completion_tokens=180)
u = build_responses_response(r, req, 'resp_1', 0).usage
print('output_tokens', u.output_tokens, 'reasoning_tokens', u.output_tokens_details.reasoning_tokens)
"
python/freetoken/server/responses_api.py:481-490: _usage() hard-codes OutputTokensDetails(reasoning_tokens=0). Every Responses usage object is built by it: the non-stream builder (:452) and the stream's response.incomplete / response.completed / response.failed events (:717, :726, :739, :754).
Nothing on the generation path counts reasoning tokens. GenDone (python/freetoken/server/generation.py:112) and GenResult (:124-132) carry only prompt_tokens/completion_tokens, and openai_api.py:637-646_usage() has no completion_tokens_details.
Proposed fix
Count reasoning tokens once in the generation path, carry the count on GenResult/GenDone, and report it as output_tokens_details.reasoning_tokens (Responses) and completion_tokens_details.reasoning_tokens (chat completions). The counter could start in the same state as the reasoning parser (force_reasoning, generation.py:389-422), so that a template that pre-opens <think> is counted from the first token. It would stop at the think-end token. If generation ends before that token, all completion tokens would count as reasoning, so a reply truncated mid-reasoning reports its real count rather than 0. With no reasoning parser, or thinking off, the count stays 0.
Environment
Seen serving nvidia/Qwen3.8-Flash-Next-NVFP4 on an RTX 5080 16 GB; the reproduction above is CPU-only on main 9b68e00.
Activity