Skip to content

Add NemotronParseBBox.to_original_coordinates for raw nemotron-parse output#1333

Closed
jamesbraza wants to merge 1 commit into
mainfrom
nemotron-parse-original-coordinates
Closed

Add NemotronParseBBox.to_original_coordinates for raw nemotron-parse output#1333
jamesbraza wants to merge 1 commit into
mainfrom
nemotron-parse-original-coordinates

Conversation

@jamesbraza

Copy link
Copy Markdown
Member

What

nemotron-parse emits bounding boxes normalized to its letterboxed 2048×1648 canvas (aspect-preserving resize + centered white padding). NVIDIA's hosted NIM maps these back to the input image server-side — via postprocessing.py::transform_bbox_to_original — before returning, so PaperQA's existing decode (coord × input_dims) is correct against NIM. Raw vLLM / Hugging Face output is not transformed, so a consumer of raw model output draws every box shifted and compressed — worst at low DPI, where the canvas padding is largest.

This ports that inverse transform so raw-output consumers can recover original-image coordinates:

  • NEMOTRON_PARSE_TARGET_HEIGHT = 2048, NEMOTRON_PARSE_TARGET_WIDTH = 1648 — sourced (with a link) from the model's preprocessor_config.json
  • transform_canvas_bbox_to_original(...) module function
  • NemotronParseBBox.to_original_coordinates(height, width) method, mirroring the existing to_page_coordinates

Why it lives here

Our self-hosted vLLM Modal proxy returns raw canvas coordinates rather than replicating NIM's server-side mapping; this is the canonical, tested transform it will call. PaperQA's NIM decode path is intentionally unchanged — applying this on NIM coordinates would double-transform.

Tests

test_api.py adds:

  • Parity against NVIDIA's reference transform_bbox_to_original across the no-resize, downsized, and width-resized regimes
  • Round-trip recovery (forward-letterbox a page rectangle → invert → recover, 0 px error)
  • Low-DPI heavy-padding case, where the correction is large vs. naive page scaling

🤖 Generated with Claude Code

…x mapping

nemotron-parse emits bounding boxes normalized to its letterboxed 2048x1648
canvas (aspect-preserving resize + centered white pad). NVIDIA's hosted NIM maps
these back to the input image server-side before returning, but raw vLLM/Hugging
Face output does not. Port NVIDIA's postprocessing.py::transform_bbox_to_original
as a module function (transform_canvas_bbox_to_original), a
NemotronParseBBox.to_original_coordinates method, and the
NEMOTRON_PARSE_TARGET_HEIGHT/WIDTH constants, so consumers of raw nemotron-parse
output can recover original-image coordinates.

Tested for parity against NVIDIA's reference, round-trip recovery, and the
low-DPI heavy-padding case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings June 4, 2026 23:09
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request labels Jun 4, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a faithful inverse “letterbox + pad” transform for raw nemotron-parse outputs whose bounding boxes are normalized to the model’s fixed 2048×1648 canvas, enabling callers (e.g., self-hosted vLLM/HF deployments) to recover original-image pixel coordinates without affecting the existing NVIDIA NIM decode path.

Changes:

  • Introduces NEMOTRON_PARSE_TARGET_HEIGHT/WIDTH and transform_canvas_bbox_to_original(...) to invert nemotron-parse preprocessing.
  • Adds NemotronParseBBox.to_original_coordinates(...) as a companion to to_page_coordinates(...).
  • Extends test_api.py with parity tests vs. NVIDIA’s reference implementation plus round-trip and low-DPI padding regression coverage.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
packages/paper-qa-nemotron/src/paperqa_nemotron/api.py Adds canvas-dimension constants, the inverse-transform helper, and a NemotronParseBBox method to map canvas-normalized bboxes back to original-image pixels.
packages/paper-qa-nemotron/tests/test_api.py Adds reference-parity, round-trip, and heavy-padding regression tests for to_original_coordinates.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@jamesbraza

Copy link
Copy Markdown
Member Author

Claude why

@jamesbraza jamesbraza closed this Jun 4, 2026
@jamesbraza
jamesbraza deleted the nemotron-parse-original-coordinates branch June 4, 2026 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants