Skip to content

downloadBundle buffers entire bundle in memory — large bundles fail to open #4252

Description

@christierney

Problem statement

ConnectAPI.downloadBundle (packages/connect-api/src/client.ts) fetches a deployed bundle by reading the whole HTTP response into memory as an ArrayBuffer and returning a single Uint8Array:

async downloadBundle(contentId, bundleId, signal?): Promise<Uint8Array> {
  const { data } = await this.client.get<ArrayBuffer>(
    `/__api__/v1/content/${contentId}/bundles/${bundleId}/download`,
    { responseType: "arraybuffer", signal },
  );
  return new Uint8Array(data);
}

This is the same in-memory ceiling that #4249 fixed for the upload path, but on the download/browse path. The only caller is connect_content_fs.ts (extractBundleTree(bundleBytes)), which backs the "open Connect content bundle" file-system browsing feature. For a multi-GiB deployed bundle, buffering the full archive (plus the typed-array copy) will exhaust memory in the extension host and fail to open the content — and the tar extraction in extractBundleTree is also fully in-memory.

This was noted as a follow-up while fixing #4249 (streaming bundle upload). Upload now streams to/from a temp file; download was left out of scope.

Expected behavior

Opening/browsing a large deployed bundle should not be limited by available memory.

Suggested approach

Mirror the #4249 upload fix on the download side:

  • Stream the download response (responseType: "stream") to a temp file on disk instead of buffering it.
  • Have extractBundleTree (or the connect_content_fs.ts caller) read entries from the on-disk tar.gz via a streaming extractor rather than from an in-memory Uint8Array.
  • Clean up the temp file when the cached content root is evicted / no longer needed.

Notes

  • Affected: packages/connect-api/src/client.ts (downloadBundle), extensions/vscode/src/connect_content_fs.ts (extractBundleTree and the bundle fetch/cache path).
  • Related: Cannot upload 4.5 GiB bundle #4249 (streaming bundle upload).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions