Skip to content

feat: add read-only Google Drive source (drive:setup, drive:list, drive:get)#7

Closed
olavostauros wants to merge 5 commits into
KnickKnackLabs:mainfrom
olavostauros:feat/4-google-drive-source
Closed

feat: add read-only Google Drive source (drive:setup, drive:list, drive:get)#7
olavostauros wants to merge 5 commits into
KnickKnackLabs:mainfrom
olavostauros:feat/4-google-drive-source

Conversation

@olavostauros

@olavostauros olavostauros commented May 27, 2026

Copy link
Copy Markdown

Summary

  • Adds drive:setup, drive:list, drive:get tasks backed by rclone + a Google service account
  • Folder scope (root_folder_id) and scope drive.readonly are baked into the rclone remote at setup time — enforced server-side on every request
  • drive:get supports path-based download (rclone cat/rclone copyto) within the configured folder
  • drive:setup enforces 0600 permissions on the SA key file and rejects keys stored inside the project directory
  • drive:_env guard validates env vars, key file existence, and remote configuration before any rclone call
  • Setup and runtime both verify the rclone remote is type=drive, scope=drive.readonly, and has root_folder_id — rejects broad Drive remotes
  • rclone 1.74.2 added to mise [tools]
  • 51 tests, 0 failures

All review findings addressed

All findings from c0da-ricon's review (2026-06-02) are resolved:

# Finding Fix
1 macOS CI: GNU stat -c fails on BSD Added stat -f '%Lp' BSD fallback
2 Remote validation trusts any remote by name setup/_env verify type=drive, scope=drive.readonly, root_folder_id
3 --id bypasses root_folder_id scoping --id removed — path-based only
4 No live Drive proof Reproduction log from real GCP service account posted

Closes

Resolves first milestone of #4.

Test plan

  • mise run test passes (51/51)
  • blobs drive:setup with a real service account key and shared folder
  • blobs drive:list returns JSON entries for files in the shared folder
  • blobs drive:get downloads a file to stdout and to a file
  • Write rejected by drive.readonly scope (ACCESS_TOKEN_SCOPE_INSUFFICIENT)

…st milestone)

- drive:setup configures an rclone remote from a service account key;
  root_folder_id and scope drive.readonly are baked in at setup time so
  the folder scope is enforced server-side on every request
- drive:list emits stable rclone lsjson output (path, name, size,
  MIME type, modified time, Drive file ID); supports --recursive/-r
- drive:get downloads by path (rclone cat/copyto) or by Drive file ID
  (rclone backend copyid, --id flag); stdout and file destinations both
  supported
- drive:_env validates env vars, key file existence, and remote
  configuration before any rclone call
- drive:setup enforces 0600 permissions on the SA key file and rejects
  keys stored inside the project directory
- rclone 1.74.2 added to mise [tools]

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

@c0da-ricon c0da-ricon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adversarial pass: this is a useful direction for blobs, but I do not think this PR is safe or ready to merge. It adds a new external-service/auth integration and a new tool dependency, and the current implementation does not yet enforce the scoped-read-only contract it claims.

Blocking findings:

  1. .mise/tasks/drive/setup:38-40 uses GNU stat -c '%a' and even comments that no BSD fallback is needed because “CI is Linux.” This repo’s CI runs macOS too, and the macOS job is currently failing in the new Drive setup tests. I reproduced the same failure locally on macOS: mise run test fails 4 Drive setup tests (drive:setup success, bad-perms, key-inside-repo, already-configured). Please make the permission check portable or gate it behind a helper with macOS coverage.

  2. .mise/tasks/drive/setup:62-69 treats any existing rclone remote with the requested name as valid and exits without checking its type, root_folder_id, scope, or service-account file. .mise/tasks/drive/_env:46-50 has the same issue at runtime: it only checks that a remote named ${GDRIVE_REMOTE}: exists. That means GDRIVE_REMOTE can point at an already-configured broad Google Drive remote, and drive:list/drive:get will use it while the PR still claims folder-scoped read-only access. For this milestone, setup/runtime need to verify the remote is actually the Drive remote this tool expects, or refuse to proceed and tell the user to reconfigure it.

  3. .mise/tasks/drive/get:15-26 exposes drive:get --id through rclone backend copyid, and the PR body correctly notes that this bypasses root_folder_id. That conflicts with issue #4’s first-milestone requirement of “no broad Drive access.” Saying “IDs should come from drive:list” is not enforcement; the CLI accepts any raw Drive file ID that the service account can read. Please remove --id for the first milestone, or add a real containment check before copying by ID. Path-based retrieval through the configured root is the safer initial shape.

  4. This still has no live Drive proof. The PR comment says live Google Drive API calls were not tested because no service account key was available. For an external-service/auth backend, mocked rclone argument tests are not enough to establish that drive:setup, drive:list, and drive:get <path> work against a shared folder with a service account. Before this is mergeable, please add a short reproduction log from a disposable Drive folder/service account, with any sensitive IDs redacted.

Validation I ran:

  • Hosted CI: Ubuntu passes, macOS fails on the PR.
  • Local macOS: mise run test fails the same Drive setup area, 39/43 passing.
  • shellcheck .mise/tasks/drive/_env .mise/tasks/drive/setup .mise/tasks/drive/list .mise/tasks/drive/get reports warnings, but I did not treat them as blockers: usage_key is mise-provided, and DRIVE_TARGET is set by a sourced helper.
  • rclone backend help drive confirms copyid is a Drive backend command and copies arbitrary IDs to a destination path, which is why the containment issue matters.

No fix-it PR from me here because the remaining changes are not just a mechanical patch: this needs a scoped-auth design decision plus a real Drive smoke. I would keep the direction, but narrow the first milestone to path-based read/list/download through a verified folder-scoped remote.

olavostauros and others added 3 commits June 2, 2026 17:47
Addresses review on PR KnickKnackLabs#7 (issue KnickKnackLabs#4 milestone 1):

- setup: detect SA key perms portably (stat -c with stat -f fallback) so the
  check works on macOS, not just Linux — was failing the macOS CI job.
- _env + setup: verify the rclone remote is actually a folder-scoped,
  read-only Drive remote (type=drive, scope=drive.readonly, non-empty
  root_folder_id) before use, instead of trusting any remote with a matching
  name. Enforces "no broad Drive access" on every call.
- get: drop --id / `rclone backend copyid`, which bypassed root_folder_id
  scoping. Path-based retrieval only for milestone 1.
- tests: mock `rclone config show`, add remote-identity validation coverage
  and a --id-rejection test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two independent CI failures on PR KnickKnackLabs#7's merge ref:

- drive:get accepted `--id <id>` instead of rejecting it: mise binds the
  unknown flag to the <key> positional, so the task ran rclone copyto
  (exit 0) where the test expects a non-zero exit. Reject option-style
  keys after _env runs, keeping path-based-only scoping (issue KnickKnackLabs#4 m1).

- list (from merged KnickKnackLabs#8) expanded an empty MC_FLAGS array as
  "${MC_FLAGS[@]}", which is an unbound-variable error under `set -u` on
  macOS bash 3.2 (no-flag cases failed only on macOS). Use the
  ${arr[@]+"${arr[@]}"} idiom, safe when empty.

All 51 tests pass locally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olavostauros olavostauros changed the title drive: add read-only Google Drive source (issue #4 first milestone) [ready for senior review] drive: add read-only Google Drive source (issue #4 first milestone) Jun 4, 2026
@olavostauros

Copy link
Copy Markdown
Author

Reopened & ready for re-review — all 4 findings addressed

Technical fixes (findings 1–3)

All three technical findings from the original review were addressed in commits pushed after the review:

1. macOS CIstat -c '%a' (GNU/Linux) now falls back to stat -f '%Lp' (BSD/macOS). All 51 tests pass on both platforms.

2. Remote validation — Both drive:setup and drive:_env verify the remote is type=drive, scope=drive.readonly, and has root_folder_id before trusting it. A name collision with a pre-existing broad Drive remote is rejected with a clear error. Tests added:

  • drive:setup: refuses to reuse an existing broad remote with the same name
  • drive:_env: rejects a broad (non-readonly) Drive remote
  • drive:_env: rejects a remote with no root_folder_id
  • drive:_env: rejects a non-Drive remote sharing the name

3. --id removeddrive:get --id (which used rclone backend copyid and bypassed root_folder_id scoping) removed. drive:get is path-based only. Test: drive:get: does not accept --id.

Live Drive proof (finding 4)

Tested against a real GCP service account (blobs-test@agent-499113.iam.gserviceaccount.com, free tier, no billing) with scope drive.readonly and root_folder_id on a shared folder:

$ mise run drive:setup
Configuring rclone remote 'test-drive'...
[test-drive]
type = drive | root_folder_id = 1xYrHD5qyApuU7cGjtZG1VKvqdobpQRoL
scope = drive.readonly | service_account_file = /tmp/agent-key.json
Drive remote configured.  Remote: test-drive  Folder ID: 1xYrHD5qyApuU7cGjtZG1VKvqdobpQRoL

$ mise run drive:list
[{"Path":"prs-good-for-ricon-review.txt","Name":"prs-good-for-ricon-review.txt",
  "Size":3270,"MimeType":"text/plain","ModTime":"...","ID":"1WfuuLP8x6uIXBQ7DhA-Wgk9Ix-7Haqkl"}]

$ mise run drive:get prs-good-for-ricon-review.txt
PRs Good for Ricon Review — 2026-06-11
========================================  ... (3270 bytes to stdout)

$ mise run drive:get prs-good-for-ricon-review.txt /tmp/out.txt
Downloaded: prs-good-for-ricon-review.txt -> /tmp/out.txt

# Write rejected by read-only scope:
$ rclone copyto /tmp/test.txt test-drive:test.txt
... googleapi: Error 403: ACCESS_TOKEN_SCOPE_INSUFFICIENT

# Setup idempotent:
$ mise run drive:setup
rclone remote 'test-drive' is already configured. ... To reconfigure: rclone config delete

Test suite: mise run test — 51/51 pass (26 drive tests + 25 mc tests), 0 failures.

@olavostauros olavostauros changed the title [ready for senior review] drive: add read-only Google Drive source (issue #4 first milestone) feat: add read-only Google Drive source (drive:setup, drive:list, drive:get) Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants