Skip to content

[Feature]: Download large folders as a single archive (auto-zip above a configurable file count) #24319

Open

Description

@aka-nez

Problem or use case

Download Folder in the File Explorer (#7780, #7793) works well for small folders but is impractical for folders with thousands of files (build output, datasets, logs, anything with node_modules or .venv inside):

  • Files are copied one by one over SFTP, sequentially, so per-file round-trips dominate and a folder of many small files takes a very long time.
  • One symlink or special file anywhere in the tree aborts the whole transfer (Cannot download symbolic link '…').
  • Because the transfer lands in a temp folder that is discarded on failure, a download that fails late leaves nothing behind.

The workaround today is to leave the explorer: run tar czf ./folder.tar.gz folder in a terminal, download the archive from the explorer, then delete it on the remote.

Proposed solution

When a folder is downloaded and it contains more files than a configurable threshold, archive it on the remote host and transfer a single file instead of copying the tree.

  1. Threshold check. Before transferring, count entries in the selected folder, stopping as soon as the count exceeds the threshold, so a huge tree costs only a few directory reads.
  2. Setting. For example File Explorer › Archive folders before download when they contain more than N files, default 10. 0 = always archive; a toggle (or empty value) = never archive, keeping today's behavior.
  3. Above the threshold: the save dialog suggests <folder>.zip, the remote side builds the archive, and the client downloads one file.
  4. At or below the threshold: unchanged recursive copy.
  5. Explicit action: a Download as .zip context-menu item on folder rows that always archives, regardless of the threshold.

Implementation notes / suggestions:

  • No host dependency. Build the archive in the relay (Node) and stream it to the client, rather than shelling out to zip, which is often not installed on Linux servers. If a temp file on the host is needed, put it in the host's temp directory and remove it after success, failure or cancel.
  • Symlinks are stored as links, not followed. They can no longer escape the selected tree or abort the download.
  • Format: .zip by default, since it opens natively on macOS, Windows and Linux. .tar.gz could be an option for preserving permissions.
  • Progress and cancel: reuse the transfer panel from feat(transfers): progress and cancel for SSH uploads and all remote downloads #23849. Cancel stops archiving on the host and removes the partial local file.
  • Limits: keep the budgets from fix(ssh): bound remote directory listing and folder transfer #10878 (depth, per-file bytes, total bytes) so archiving cannot be used to bypass them.
  • Runtime workspaces: because the result is a single file, this could also enable folder download for orca serve / runtime-hosted workspaces, which today support only single-file download.

Open question: should there be an optional "extract after download" so the user still ends up with a folder locally?

Alternatives or additional context

  • Parallel SFTP transfers would speed up the current copy, but would not fix the symlink aborts or the all-or-nothing failure on huge trees.
  • Manual workaround: tar/zip in the terminal, then download the archive. This works, but breaks the explorer flow and leaves cleanup to the user.

Related: #7780 (original request, which already mentioned "zipped automatically" as an option), #7793 (current recursive implementation), #10878 (transfer budgets), #23849 (transfer progress/cancel), #9031 (paired web Explorer transfers).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions