Skip to content

Add configurable timeouts for file downloads - #149

Open
sylvesterkaczmarek wants to merge 1 commit into
nasa:mainfrom
sylvesterkaczmarek:fix-122-download-timeout
Open

sylvesterkaczmarek wants to merge 1 commit into
nasa:mainfrom
sylvesterkaczmarek:fix-122-download-timeout

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Jira Issue ID

No Jira ID supplied. Fixes #122.

Description

Add optional Client(download_timeout=...) socket limits for file downloads. Accept a scalar or a (connect, read) pair and apply it to direct, bulk, intermediate-file, and OPeNDAP downloads. Default None preserves existing behavior; authentication and job polling are unchanged.

Document inactivity versus total-duration limits, retry behavior, Future exceptions, and partial-file retry handling. This does not add automatic partial-file cleanup or a total-download deadline.

Local Test Steps

macOS arm64, Python 3.12.11:

  • pytest --cov=harmony --cov-branch tests: 256 passed, including 17 new timeout cases.
  • Real localhost HTTP tests stall both response headers and bodies, verify timeout propagation through Future.result(), and confirm a single-worker executor becomes available again.
  • Removing the socket timeout argument makes both stalled-server regressions fail, independently of the new constructor API.
  • Ruff checks/formatting and git diff --check passed.

No live Earthdata downloads, multi-gigabyte transfers, or Windows/Linux execution were performed.

PR Acceptance Checklist

  • Requested timeout option implemented with backward-compatible defaults.
  • Tests added and passing.
  • Documentation updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Indefinite hangs on large downloads for ATL03 granules

1 participant