Skip to content

fix(storage): support bucket-scoped Spaces archives - #609

Merged
matteius merged 2 commits into
mainfrom
fix/spaces-scoped-archive
Sep 14, 2026
Merged

matteius merged 2 commits into
mainfrom
fix/spaces-scoped-archive

Conversation

@matteius

@matteius matteius commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

DigitalOcean Spaces rejects the current archive adapter in two places: bucket-scoped credentials receive 403 for lifecycle inspection, and escaping object-key path separators produces SignatureDoesNotMatch during uploads.

Preserve path separators while escaping literal percent signs and other key characters. For providers that deny lifecycle inspection, accept an optional operator verification file beside the scoped credentials. The file must have private permissions, bind the exact endpoint/region/bucket, contain safe lifecycle rules, and be less than one hour old. Readable unsafe rules, failed versioning checks, provider outages, and public object access still fail. Account-wide credentials stay in the control plane. The included operator helper validates bucket configuration and atomically publishes the file; deployment documentation explains refresh and the bounded configuration-check delay.

Validation:

  • Five relevant suites pass with ASan/UBSan, including archive, migration, target database/API, and seven operator helper tests.
  • The SigV4 fixture now rejects encoded path separators; coverage includes literal %2F, spaces, expired/future verification, mismatched bucket/region/endpoint, unsafe rules, file permissions, symlinks, and provider failures.
  • Live Spaces qualification using the actual C storage library and a bucket-scoped key passed private connection probing, a 9 MiB multipart archive, SHA-256 verification, hot eviction, byte-range reads, protected/indefinite metadata preservation, verified filesystem restore, and confirmed remote deletion. Synthetic objects and incomplete uploads were checked absent afterward.
  • Documentation link validation passes.

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The changes span storage signing, lifecycle security, deployment tooling, and CI, warranting final human review.

Pull request overview

Adds bucket-scoped DigitalOcean Spaces archive support by correcting SigV4 key encoding and introducing lifecycle attestations.

Changes:

  • Preserves object-key separators while escaping literal percent signs.
  • Adds lifecycle verification, atomic publication, and adapter validation.
  • Extends tests, CI, and deployment documentation.
File summaries
File Description
tests/unit/test_storage_archive.c Tests archive behavior, attestations, and key encoding.
tests/unit/test_archive_operator.py Tests lifecycle helper validation and publication.
tests/unit/s3_fixture.py Covers encoded separators and provider failures.
tests/unit/CMakeLists.txt Registers the operator test suite.
src/storage/storage_s3.c Implements corrected key encoding and attestation validation.
docs/STORAGE_ARCHIVE.md Documents scoped credentials and refresh behavior.
deployment/archive/verify_bucket_lifecycle.py Validates and atomically publishes lifecycle attestations.
.github/workflows/integration-test.yml Runs the expanded test suite.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@matteius
matteius merged commit 958cdec into main Sep 14, 2026
10 checks passed
@matteius
matteius deleted the fix/spaces-scoped-archive branch September 14, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants