Add profile registry and URL-based profile selection - #9
Conversation
Support `s3://<profile>@bucket/key` to select an isolated, name-keyed profile directly in the URL, so pos3-powered CLIs can target custom S3-compatible endpoints with scoped credentials without touching global AWS state or threading a profile= argument through every tool. - URL userinfo profile is parsed in the _parse_s3_url chokepoint and takes precedence over the profile= argument. - Profiles auto-load from ~/.config/pos3/profiles.toml (overridable via POS3_PROFILES_FILE / XDG_CONFIG_HOME), with the secret credentials file kept separate from the non-secret endpoint/region config. - Credentialed profiles build their own isolated boto3.Session. - Unknown profiles are a hard error with no silent default-chain fallback. - Version bump to 0.3.0 so downstream tools can pin the new behavior. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
CI only ever runs 3.11 and release.yml already uses stdlib tomllib, so the >=3.9 claim was nominal and untested. Drop the conditional tomli dependency, import tomllib directly, and align requires-python and the ruff target-version with what is actually supported. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
Move the Profile dataclass, registry loading, resolution, and isolated client creation out of __init__.py into a self-contained pos3.profiles module (no dependency back on the mirror engine). __init__.py is now just the sync logic. Public API (pos3.Profile, pos3.register_profile) is unchanged via re-export. Also document the Python 3.11+ requirement in the release notes. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 13476dbcd5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Require both aws_access_key_id and aws_secret_access_key in a credentials_file. A half-populated section previously silently fell back to the ambient AWS credential chain in _create_s3_client, defeating the point of isolated per-profile credentials. - Build registry profiles atomically and set _REGISTRY_LOADED only after a successful load, so a malformed entry doesn't leave the process in a permanent "loaded with no profiles" state. Adds regression tests for both. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d8681cba2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A relative path like credentials_file = "acme.creds" previously resolved against the process CWD, so running pos3 from a different directory would fail with "not found or unreadable" even though the creds file sat next to profiles.toml. Resolve relative paths against the registry file's parent instead, which is the location users naturally expect. Absolute and ~-expanded paths are unchanged. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c0c8377252
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
s3://@bucket/key or s3://:token@bucket/key (which a CLI template variable can produce when the profile name expands to empty) previously slipped through _url_profile as "no URL profile" and silently fell back to the argument/default profile, violating the explicit no-silent- fallback contract for URL-based selection. Treat any '@' in the netloc without a profile name as a hard error. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be68460e27
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Falling back to parser.sections()[0] would silently bind a profile to an unrelated INI section, so a typo in the section header (e.g. [acne] instead of [acme]) could route operations to the wrong AWS account. Only [<profile_name>] and [default] are accepted now; anything else raises with the list of sections found. Same no-silent-fallback rule as the earlier credentials/empty-selector guards. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6ddc4d2a93
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Programmatic register_profile() must take precedence over a registry entry regardless of which loaded first. Previously, lazy registry load could populate _PROFILES before user code, causing a later register_profile() with different config to raise. Split storage: code registrations live in _PROFILES (consulted first); registry entries live in _REGISTRY_PROFILES (fallback). Both override the registry, neither overrides another code registration. - A Profile constructed with only one of access_key/secret_key previously slipped through _create_s3_client and fell back to the ambient credential chain. Validate the pair in Profile.__post_init__ so any wrong-shape Profile fails at construction, no matter the caller path. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a5f777e794
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A forced _load_profile_registry(..., force=True) previously merged new entries into _REGISTRY_PROFILES, so profiles that were removed or renamed in the updated profiles.toml stayed resolvable with their old endpoint/credential config. Treat a forced reload as "re-read the source of truth": clear and replace, including when the file is now absent. The clear runs only after the new mapping is successfully built, so a malformed reload leaves the previous snapshot intact. https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1
2 participants
Closes #8.
Summary
This PR adds support for named S3 profiles with a local registry file and URL-based profile selection, enabling users to work with S3-compatible endpoints without environment variables or code changes.
Key Changes
New
pos3.profilesmodule: Extracted profile management logic into a dedicated internal module with support for:~/.config/pos3/profiles.toml(configurable viaPOS3_PROFILES_FILEorXDG_CONFIG_HOME)boto3.Sessioncreation for profiles with explicit credentialsURL-based profile selection: S3 URLs can now specify a profile in the userinfo slot (
s3://<profile>@bucket/key):profile=argumentEnhanced
Profiledataclass: Added support for explicit AWS credentials (access_key,secret_key,session_token) that enable isolated session creation without reading ambient AWS configurationUpdated
_parse_s3_urland_normalize_s3_url: Now strip the userinfo slot (profile selector) from URLs before processing bucket/keyUpdated
_effective_profilemethod: Now resolves URL-based profile selection with proper precedence (URL > argument > context default)Public API re-exports:
Profile,register_profile, and related functions are re-exported frompos3/__init__.pyfor backward compatibilityImplementation Details
tomllib(Python 3.11+), requiring a minimum Python version bump from 3.9 to 3.11Profile.__repr__()to prevent accidental logging of secretsTesting
Added comprehensive test coverage for:
https://claude.ai/code/session_011QGQRbcxj9d47BRWkWAWM1