Add NemotronParseBBox.to_original_coordinates for raw nemotron-parse output#1333
Closed
jamesbraza wants to merge 1 commit into
Closed
Add NemotronParseBBox.to_original_coordinates for raw nemotron-parse output#1333jamesbraza wants to merge 1 commit into
jamesbraza wants to merge 1 commit into
Conversation
…x mapping nemotron-parse emits bounding boxes normalized to its letterboxed 2048x1648 canvas (aspect-preserving resize + centered white pad). NVIDIA's hosted NIM maps these back to the input image server-side before returning, but raw vLLM/Hugging Face output does not. Port NVIDIA's postprocessing.py::transform_bbox_to_original as a module function (transform_canvas_bbox_to_original), a NemotronParseBBox.to_original_coordinates method, and the NEMOTRON_PARSE_TARGET_HEIGHT/WIDTH constants, so consumers of raw nemotron-parse output can recover original-image coordinates. Tested for parity against NVIDIA's reference, round-trip recovery, and the low-DPI heavy-padding case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a faithful inverse “letterbox + pad” transform for raw nemotron-parse outputs whose bounding boxes are normalized to the model’s fixed 2048×1648 canvas, enabling callers (e.g., self-hosted vLLM/HF deployments) to recover original-image pixel coordinates without affecting the existing NVIDIA NIM decode path.
Changes:
- Introduces
NEMOTRON_PARSE_TARGET_HEIGHT/WIDTHandtransform_canvas_bbox_to_original(...)to invert nemotron-parse preprocessing. - Adds
NemotronParseBBox.to_original_coordinates(...)as a companion toto_page_coordinates(...). - Extends
test_api.pywith parity tests vs. NVIDIA’s reference implementation plus round-trip and low-DPI padding regression coverage.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| packages/paper-qa-nemotron/src/paperqa_nemotron/api.py | Adds canvas-dimension constants, the inverse-transform helper, and a NemotronParseBBox method to map canvas-normalized bboxes back to original-image pixels. |
| packages/paper-qa-nemotron/tests/test_api.py | Adds reference-parity, round-trip, and heavy-padding regression tests for to_original_coordinates. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Member
Author
|
Claude why |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
nemotron-parseemits bounding boxes normalized to its letterboxed 2048×1648 canvas (aspect-preserving resize + centered white padding). NVIDIA's hosted NIM maps these back to the input image server-side — viapostprocessing.py::transform_bbox_to_original— before returning, so PaperQA's existing decode (coord × input_dims) is correct against NIM. Raw vLLM / Hugging Face output is not transformed, so a consumer of raw model output draws every box shifted and compressed — worst at low DPI, where the canvas padding is largest.This ports that inverse transform so raw-output consumers can recover original-image coordinates:
NEMOTRON_PARSE_TARGET_HEIGHT = 2048,NEMOTRON_PARSE_TARGET_WIDTH = 1648— sourced (with a link) from the model'spreprocessor_config.jsontransform_canvas_bbox_to_original(...)module functionNemotronParseBBox.to_original_coordinates(height, width)method, mirroring the existingto_page_coordinatesWhy it lives here
Our self-hosted vLLM Modal proxy returns raw canvas coordinates rather than replicating NIM's server-side mapping; this is the canonical, tested transform it will call. PaperQA's NIM decode path is intentionally unchanged — applying this on NIM coordinates would double-transform.
Tests
test_api.pyadds:transform_bbox_to_originalacross the no-resize, downsized, and width-resized regimes🤖 Generated with Claude Code