Add support for S3 Multi-Region Access Point (MRAP) URLs#557
Add support for S3 Multi-Region Access Point (MRAP) URLs#557fedemengo wants to merge 7 commits intodrivendataorg:masterfrom
Conversation
|
the changes were simple enough, @pjbull gentle ping for review |
pjbull
left a comment
There was a problem hiding this comment.
A couple comments.Also is it possible to update the tests to use s3_rig following the pattern of the existing tests rather than using S3Path directly (as in some tests) or using monkeypatch as in the others.
| :type: :class:`str` | ||
| """ | ||
| if match := _MRAP_PATTERN.match(str(self)): | ||
| return match.group("arn") |
There was a problem hiding this comment.
Let's cache this since it shouldn't change.
| if ( | ||
| not (self.root / Key).exists() | ||
| or (self.root / Key).is_dir() | ||
| or Bucket != DEFAULT_S3_BUCKET_NAME |
There was a problem hiding this comment.
IIRC some of the tests use this check in some scenarios. Can we also just add a simple check that it doesn't have '.mrap' here
3106e54 to
b86732d
Compare
|
@pjbull I replaced S3Path with rig_ops when tests are not purly path parsing |
|
@fedemengo see linting errors in the action logs. |
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #557 +/- ##
========================================
- Coverage 94.0% 93.1% -0.9%
========================================
Files 28 28
Lines 2203 2211 +8
========================================
- Hits 2071 2060 -11
- Misses 132 151 +19
🚀 New features to boost your workflow:
|
|
@fedemengo tests are failing on windows because We're coming up against this separate issue: Maybe for now you can override the |
Closes #556
Changes
cloudpathlib/s3/s3path.py: added _MRAP_PATTERN regex and overridden bucket property on S3Pathtests/mock_clients/mock_s3.py: removed bucket name check from head_object (the mock never routes by bucket, only by key)tests/test_s3_specific.py: added 3 tests covering bucket/key parsing, path manipulation, and end-to-end file operations via themock backend
Note: using MRAP paths requires
botocore[crt]for SigV4a signing:pip install botocore[crt]Contributor checklist:
CONTRIBUTING.mdCloses #issueappears in the PR summary (e.g.,Closes #123).HISTORY.mdwith the issue that is addressed and the PR you are submitting. If the top section is not `## UNRELEASED``, then you need to add a new section to the top of the document for your change.