Let a phone upload captures straight to a station's storage - #1409
Open
mihow wants to merge 7 commits into
Open
Conversation
Introduces get_presigned_put_url (per-file, uncached PUT URL; AWS flexible-checksum header gated to real AWS since Swift's s3api rejects it) and derive_upload_key, which builds the full object key directly rather than via key_with_prefix (whose split() dedup mangles filenames containing the subdir string). The derived key matches exactly what sync_captures stores as SourceImage.path, making re-upload idempotent. Unit tests (offline, local signing) cover key derivation edge cases and the checksum-gating behaviour. Part of #1379. Co-Authored-By: Claude <noreply@anthropic.com>
A1: POST /api/v2/deployments/{id}/upload-request/ mints presigned PUT URLs for
direct-to-storage capture uploads, with full per-file validation (parseable
timestamp, image extension, path-traversal, key length <= 255, regex, size)
returning rejected files in errors[] rather than minting a URL. Adds
Deployment.check_custom_permission mapping "upload_request" to SYNC_DEPLOYMENT
so the detail action's object-permission check passes (reuses an existing
guardian perm; no migration).
A5: DeploymentFilterSet exposes research_site_id/device_id (the exact param
names the mobile client sends); ProjectViewSet gains ?role=manager /
?writable=true via get_objects_for_user on update_project (covers managers and
owners). Sites/devices ?project_id scoping confirmed by test.
A2: @extend_schema on the existing sync action documents the {job_id} response,
the job-polling endpoint, and the three terminal states (SUCCESS, FAILURE,
REVOKED). The auto-sync cadence is a separate follow-up.
Tests: permission matrix, each validation error path, deterministic key, and an
E2E flagship (mint PUT -> requests.put -> sync_captures -> assert SourceImage
path) against live MinIO. Part of #1379.
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
A role group carries the model-level permission as well as the per-project one, so a user who manages any single project holds `update_project` globally. Guardian accepts a global permission by default, so `?role=manager` and `?writable=true` answered with the entire project list for exactly the users the filter exists to serve — anyone who manages at least one project. Asking guardian for object-level permissions only fixes it. Superusers are still short-circuited above and continue to see everything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Ej7dJGDozxeVg6AhfTkYo
Both lists live under the stations route — /api/v2/deployments/sites/ and /api/v2/deployments/devices/ — so the tests were asking for paths that answer 404 and asserting on a response that never arrived. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Ej7dJGDozxeVg6AhfTkYo
…romise The key-length test sent a 270-character filename, which the request serializer rejects with a 400 before the view ever builds a key, so the guard it meant to exercise never ran. The filename is now legal on its own and only exceeds the limit once the storage prefix and the station's subdirectory are in front of it, which is the case the guard exists for. The site and device scoping tests asserted an exact set, which fails as soon as the project carries any other site or device. They now assert what scoping actually promises: this project's rows are present and another project's are not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019Ej7dJGDozxeVg6AhfTkYo
✅ Deploy Preview for antenna-ssec canceled.
|
✅ Deploy Preview for antenna-preview canceled.
|
Contributor
|
Warning Review limit reachedNext included review available in 39 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
A couple of newly introduced API validation/messages are misleading or under-validated (docstring/error detail correctness and sha256 format validation), and should be corrected before approval.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a direct-to-storage mobile upload contract for deployments by minting short-lived presigned PUT URLs, plus related query filters and schema docs to support the mobile client’s upload → sync workflow.
Changes:
- Add
POST /api/v2/deployments/{id}/upload-request/to mint presigned PUT URLs with deterministic object keys that match subsequentsync_capturesingestion. - Add deployment list filters (
research_site_id,device_id) and project list “writable” filters (?role=manager/?writable=true) aligned with the mobile client’s needs. - Add unit + E2E tests for key derivation, presigned PUT signing behavior, endpoint validation/permissions, and filter scoping; add planning/index documentation.
File summaries
| File | Description |
|---|---|
| docs/claude/planning/2026-07-23-mobile-upload-api.md | Planning/notes for A1/A5 and sync-completion docs (A2 docs). |
| docs/claude/INDEX.md | Adds the planning doc to the Claude docs index. |
| ami/utils/s3.py | Introduces presigned PUT URL generation + deterministic upload key derivation helper. |
| ami/tests/test_storage.py | Adds unit tests for upload key derivation and checksum-header gating in presigned PUT URLs. |
| ami/main/tests.py | Adds API tests for upload-request permissions/validation and for the new deployment/project filters, plus an E2E MinIO flow. |
| ami/main/models.py | Maps upload_request action permission to existing SYNC_DEPLOYMENT permission on Deployment. |
| ami/main/api/views.py | Implements upload-request action, adds deployment filterset, and documents sync completion response shape. |
| ami/main/api/serializers.py | Adds request/response serializers for upload-request. |
Review details
- Files reviewed: 8/8 changed files
- Comments generated: 3
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+899
to
+900
| # base64-encoded raw SHA-256 digest; optional and only honoured against real AWS. | ||
| sha256 = serializers.CharField(required=False, allow_blank=True, default="") |
Comment on lines
+357
to
+359
| These mirror the constraints ``sync_captures`` later imposes so we never | ||
| mint a URL for a file the sync would silently drop or the DB would reject. | ||
| """ |
| if get_image_timestamp_from_filename(filename) is None: | ||
| return { | ||
| "code": "unparseable_timestamp", | ||
| "detail": "Filename has no parseable timestamp; sync would drop it.", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A phone in the field holds a night of captures and needs somewhere to put them. Routing that traffic through the platform would mean every image crossing the web server twice, so instead the client asks the platform for permission and uploads straight to the station's own storage.
A client calls
POST /api/v2/deployments/{id}/upload-request/with the files it wants to send, gets back a short-lived signed PUT URL for each one, uploads each file directly to the station's storage source, and then calls the existingsyncaction to ingest them. The key each URL is minted for is exactly the object key the subsequent sync stores, so re-requesting and re-uploading the same file is idempotent rather than producing a duplicate.Alongside that, the endpoints a mobile client needs in order to find where to upload now answer the questions it actually asks: which stations belong to a given research site or device, and which projects the signed-in user may write to.
This is the first of the three contracts in #1379 (A1 and A5, with A2 covered by documentation only). No migration: the new action reuses the permission that already governs syncing a station.
List of Changes
POST /api/v2/deployments/{id}/upload-request/mints short-lived presigned PUT URLserrorswith no URL mintedderive_upload_key()builds the object key the same waysync_captureswill store itkey_with_prefix, whose deduplication heuristic mangles filenames that contain the subdirectory nameupload_requestaction maps toSYNC_DEPLOYMENTresearch_site_idanddevice_idfilters on the station list?role=manager/?writable=trueon the project listsyncaction: the response, the polling endpoint, and the three terminal statesWorth a reviewer's attention
The checksum header is suppressed for non-AWS storage.
x-amz-checksum-sha256is only emitted when there is no custom endpoint, because Swift's S3 API rejects it. The consequence is that the end-to-end test against MinIO never exercises the checksum branch — that path is only live against real AWS.HEIC is not an accepted image extension.
IMAGE_FILE_EXTENSIONScovers jpg/jpeg/png/gif/webp/svg/bmp/ico/tiff/tif. A client uploading HEIC is rejected at the request stage, and a sync would drop it too. If phone uploads need HEIC, the extension list is the change to make and it is not made here.Fixed while bringing the branch up to date with main
Six weeks of drift, and running the suite against a live stack turned up three problems in the branch's own code and tests:
?role=managerand?writable=trueanswered with every project. A role group carries the model-level permission as well as the per-project one, so anyone who manages a single project holdsupdate_projectglobally, and guardian accepts a global permission by default. Asking for object-level permissions only fixes it — for exactly the users the filter exists to serve./api/v2/deployments/sites/,/deployments/devices/), so the tests were asserting on a 404. They now also assert what scoping promises — this project's rows present, another project's absent — rather than an exact set that breaks as soon as the project has any other site.One thing to watch at merge time
#1408 (station status) also adds
check_custom_permissiontoDeployment. The two definitions merge without a conflict, and Python keeps the last one — so whichever lands second silently removes the other's permission mapping and starts refusing every non-superuser. Whoever merges second should fold both actions into one method; the mapping is a one-line set membership.Testing
ami/tests/test_storage.py— key derivation, including the filename-matching-subdirectory regression, and the presigned URL headers with checksum gating. All offline: signing is local.ami/main/tests.py—TestDeploymentUploadRequestcovers the permission matrix, a station with no storage source, each per-file validation error, deterministic keys, and the file-count limit.TestDeploymentUploadRequestE2Emints a URL, uploads through it and asserts the synced capture's path matches the minted key; it needs MinIO.TestDeploymentAndProjectFiltersandTestProjectWritableFiltercover the filters.Refs #1379.
🤖 Generated with Claude Code
https://claude.ai/code/session_019Ej7dJGDozxeVg6AhfTkYo