Skip to content

feat: add Volcano Engine TOS provider entry - #60

Merged
Alex1990 merged 1 commit into
mainfrom
loop/59-tos
Sep 18, 2026
Merged

Alex1990 merged 1 commit into
mainfrom
loop/59-tos

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Closes #59

loop-task: #59

Why

issue #59 asks for first-class support for Volcano Engine Torch Object Storage (TOS). TOS is S3-shaped but not covered by tiny-oss/aws: it requires virtual-hosted addressing on a dedicated endpoint and rejects path-style, while the AWS entry's custom endpoint only puts the bucket in the path when pathStyle is true (src/aws/host.ts returns the endpoint verbatim).

What

A new, self-contained tiny-oss/tos entry:

  • src/tos/signature.ts — TOS4-HMAC-SHA256 (service tos): canonical request with host + every x-tos-* header signed, UNSIGNED-PAYLOAD body, standard signing-key chain.
  • src/tos/host.ts — virtual-hosted <bucket>.tos-<region>.volces.com; internal: true selects .ivolces.com; a custom endpoint is treated as the endpoint domain (the bucket is still prefixed), matching @volcengine/tos-sdk. region is always required because the credential scope embeds it.
  • src/tos/request.ts — signs and sends one request; lower-cases header names like the SDK and emits the query string sorted exactly as signed.
  • src/tos/signatureUrl.ts — X-Tos-* query-signed URLs (X-Tos-Expires is a TTL in seconds, host-only signed headers).
  • src/tos/index.ts — protocol binding and the full op set, including putSymlink (?symlink + x-tos-symlink-target); ObjectCallback is exported but callbacks are passed through headers (x-tos-callback), like COS.

Packaging: tsup entry, package.json exports/build:types:tos/check:types/keywords, @volcengine/tos-sdk@2.9.1 devDependency, pnpm-workspace.yaml tos-crc64-js: false (optional CRC build, unneeded for signing). Shared: optional Protocol.symlinkHeaders hook (OSS behavior unchanged) and callback comments/message now mention TOS. README.md/README_zh-CN.md document the provider.

Signer verification (oracle)

pnpm test:tos-oracle (test/tos-oracle.node.ts, also run by test:ci) drives the official @volcengine/tos-sdk@2.9.1 against a local capture server and recomputes every Authorization with our signer from the captured method/path/query/headers — byte-identical for putObject (metadata, non-ASCII key), createMultipartUpload, uploadPart, listParts, listMultipartUploads, completeMultipartUpload, abortMultipartUpload, uploadPartCopy, putSymlink, with and without an STS token. Pre-signed URLs are compared against getPreSignedUrl and the query-signing path/credential scope against the official Go SDK's published vector (ve-tos-golang-sdk tos/sign_v4_test.go, embedded as a fixed vector in test/tos-signature.spec.ts).

One deliberate deviation. The JS SDK's getPreSignedUrl puts the endpoint in the credential scope (src/methods/base.ts: region: this.opts.endpoint), which disagrees with the official Go SDK (SignV4.SignQuery uses the client region) and the standard V4 scope; the JS SDK has no test for getPreSignedUrl. tiny-oss signs the real region; the JS comparison neutralizes that single field with endpoint === region (every other byte is compared) and the Go vector pins the region choice. Both forms are accepted by the service.

Tests

  • test/tos-oracle.node.ts — official-SDK oracle (pnpm test:tos-oracle, added to test:ci/test:ci:coverage).
  • test/tos-signature.spec.ts — fixed vectors (JS header, Go query, JS signed URLs), host resolution, request building; 19 tests.
  • test/tos.spec.ts — entry-level put/putSymlink/uploadPartCopy/multipartUpload; 5 tests.
  • test/test-types-tos.ts — entry types.

Verification

  • pnpm lint — 0 errors (39 pre-existing warnings).
  • pnpm fmt:check — all files formatted.
  • pnpm build — exit 0; emits dist/tiny-oss.tos.es.js + .d.ts.
  • pnpm check:types — exit 0.
  • pnpm test:ci — azure oracle + TOS oracle + headless vitest: 24 files passed, 255 passed / 26 skipped.

Existing OSS/COS/OBS/AWS signers and their oracles are untouched. Tree-shaking confirmed: tiny-oss.tos.es.js carries no other provider's signing code.

Note

AGENTS.md's built-in-entry list and putSymlink wording are now slightly stale; that is the norms layer and should land as its own proposal PR rather than in this feature change.

Add a self-contained `tiny-oss/tos` entry implementing TOS4-HMAC-SHA256
(service `tos`, `host` + `x-tos-*` signed headers, UNSIGNED-PAYLOAD),
virtual-hosted `tos-<region>.volces.com` / `tos-<region>.ivolces.com`
hosts and the full operation set, including putSymlink (`x-tos-symlink-target`).
Wire the tsup entry, package.json export and type tests, and pin the signer
against the official @volcengine/tos-sdk in test/tos-oracle.node.ts
(`pnpm test:tos-oracle`, also run by test:ci). Closes #59.
@github-actions
github-actions Bot requested a review from Alex1990 as a code owner September 18, 2026 13:08
@Alex1990
Alex1990 merged commit d15d021 into main Sep 18, 2026
Alex1990 pushed a commit that referenced this pull request Sep 18, 2026
The tiny-oss/tos entry landed in #60 but AGENTS.md lagged in three
places:
- the built-in-entry list omitted Volcano Engine TOS;
- the entry-path example was src/cos|obs|aws|azure/index.ts;
- it claimed putSymlink is OSS-only, while the TOS entry also exports
  it (supportsSymlink: true, x-tos-symlink-target). The README is
  already correct.

Norms-layer change for #61; no product code, signer or workflow change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Torch Object Storage from VolcanoEngine

1 participant