From 7bcf676ecc946bfa80fd5a305793adf5ccaf5f83 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 10:10:24 +0200 Subject: [PATCH 01/12] Add the image-upload design spec POST /api/v1/image was recorded in CLAUDE.md as unreachable in all three tested encodings. Re-measured live on 2026-08-08: it works. The body is a JSON data URI, not a file or a URL, which is why multipart and form-urlencoded failed. External URLs are rejected (Substack only server-fetches its own S3 bucket), so the tool downloads and re-encodes. End-to-end proven on a real draft: upload renders through Substack's CDN. Co-Authored-By: Claude Opus 4.8 --- .../specs/2026-08-08-image-upload-design.md | 170 ++++++++++++++++++ 1 file changed, 170 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-08-image-upload-design.md diff --git a/docs/superpowers/specs/2026-08-08-image-upload-design.md b/docs/superpowers/specs/2026-08-08-image-upload-design.md new file mode 100644 index 0000000..4348253 --- /dev/null +++ b/docs/superpowers/specs/2026-08-08-image-upload-design.md @@ -0,0 +1,170 @@ +# Image upload — design + +**Date:** 2026-08-08 +**Status:** approved, ready for a plan +**Supersedes:** the "Images can be referenced but not uploaded" limit documented in `CLAUDE.md` +and `README.md:143`, and the instruction "Do not implement an upload from either" fork signature. + +## Why this exists + +`captionedImage` is in 60 of 60 sampled posts, and until now `image2.src` had to already point at a +Substack-hosted asset — there was no way to put a new image into a post. `CLAUDE.md` recorded the +endpoint as unreachable: `POST /api/v1/image` "hangs" in all three tested encodings (JSON, +form-urlencoded, multipart), attributed by analogy to the Cloudflare wall on `POST /comment/feed`. + +**That record is wrong, and this design rests on re-measuring it live on 2026-08-08** against +`implementing.substack.com`, from the authenticated dashboard. The endpoint works. The three earlier +encodings all failed because they sent the wrong *thing*, not because of a header detail or a bot +wall: + +- **The body is JSON `{image: "data:;base64,…"}` — a data URI, not a file and not a URL.** In + the `reactPublish` bundle the call is built from `canvas.toDataURL()` / a read `File`, then + `post('/api/v1/image').send({image, postId})`. Multipart and form-urlencoded were never going to + work; the "hang" was the wrong shape, not Cloudflare. + +Measured, live, that day (every row a real `POST`): + +| Probe | Result | +|---|---| +| 1×1 PNG as data URI | **200**, 276 ms | +| Real JPEG 1200×630 (108 KB payload) as data URI | **200**, 485 ms; returns correct `bytes`/`width`/`height` | +| With `postId` of a real published post | **200** — accepted | +| A `substack-post-media` S3 URL as `image` | **200** — Substack server-fetches its *own* bucket | +| An **external** URL (Wikimedia) as `image` | **400 `Failed to fetch image`** | +| Non-image content (`text/plain` data URI) | **400 `{error, type}`** — Substack validates server-side | +| The returned S3 URL, fetched with no cookie | **200, `content-type: image`** — the asset is public | + +**Response shape:** `{id, url, contentType, bytes, imageWidth, imageHeight}`. `url` is on +`substack-post-media.s3.amazonaws.com` — the exact host all 18 `image2.src` values in 6 sampled +published posts use. + +### The end-to-end loop is proven, not inferred + +On draft `210218832`, live: fresh data-URI upload → build a `captionedImage` node (attrs copied +verbatim from a real post) referencing the returned `url` → append to the draft's existing body → +`PUT /drafts/:id` (200, node persisted) → reload the editor. The image **renders**: `complete`, +natural size 1038×581, and served through Substack's own CDN +(`substackcdn.com/image/fetch/…f_webp`) — it entered the render pipeline, not just the stored JSON. +The draft was then restored to its original 6 nodes. + +**The one thing not yet proven:** every call above used the browser's session **cookie**. The server +authenticates with `SUBSTACK_SESSION_TOKEN` in a header. In principle equivalent, but unverified +through `SubstackApi` — this is the first validation step in implementation, not a settled fact. + +## The key design consequence + +Substack's server-side fetch only works for URLs already in its **own** S3 bucket; an arbitrary +external URL is rejected with 400. The chosen interface is **"remote URL only"** (`upload_image({url})`). +So the shortcut is unavailable for our use case: **the tool must download the external image itself +and re-encode it as a data URI**, then send that. This is the whole reason the tool has a +download-and-encode step rather than passing the URL straight through. + +## Interface + +A new tool `upload_image`, following the one-file-per-tool pattern: `src/tools/upload_image.js` +plus one entry in the `tools` registry in `src/server.js`. Nothing else. + +``` +upload_image({ + url: string // the external image URL to fetch and upload (required) + post_id?: number // optional; mirrors the dashboard's `postId`. Accepted by the API (measured). + // Effect is unconfirmed — include because it is a real key, not invented. +}) + → { id, url, content_type, bytes, width, height } +``` + +- `z.strictObject` (an unknown key is reported, never stripped — the only repair signal an LLM gets). +- The output `url` field's description states explicitly that it goes into `image2.src` of + `set_post_body`. That description is the only way a model links the two tools without guessing. + +## Flow and guards + +The tool does three things: fetch the external image, validate it, upload it as a data URI. + +1. **Fetch the URL.** This is a fetch of a model-chosen URL whose bytes we then re-emit — an + exfiltration/SSRF channel if unguarded. Guards, in order: + - **Scheme:** only `http`/`https`. Reject anything else (`file:`, `data:`, `gopher:`, …) up front. + - **SSRF, after DNS resolution, not on the URL string.** Resolve the host and reject loopback, + private (RFC 1918), and link-local (incl. `169.254.169.254`) addresses. Checking the string + does not stop a hostname that resolves to `127.0.0.1`. This is the guard that matters most. + The SSRF probes were deliberately **not** fired at Substack during measurement — running them + would have performed the attack, not tested it; that they must be blocked is a requirement of + *our* fetch, which is the one now doing arbitrary external downloads. + - **Size cap on the downloaded bytes.** Our defense of this server's memory: the buffer plus its + base64 string (~1.37×) are held in RAM. **Proposed 10 MB.** This is explicitly **not** Substack's + limit — the bundle has a `MAX_FILE_SIZE` constant whose value could not be extracted from the + minified source. We cap for our own safety and let Substack reject anything that passes ours but + not its. +2. **Validate the content.** `content-type` must be `image/*`. Reject **HEIC** early with a message to + convert to JPG/PNG — the dashboard itself refuses HEIC client-side. Substack also validates + server-side (the `text/plain` → 400 above), so this is defense-in-depth and a better message, not + the only check. +3. **Encode and upload.** Build `data:;base64,` and call + `SubstackApi.uploadImage({image, post_id})`. + +## API layer + +Add `SubstackApi.uploadImage({image, post_id})` next to the other methods: + +```js +async uploadImage({image, post_id = null}) { + return this.request({ + method: 'POST', + path: '/image', + body: post_id === null ? {image} : {image, postId: post_id}, + referer: '/publish/post', + }); +} +``` + +No change to `requestUrl`: it already serializes any non-null body as JSON with +`Content-Type: application/json`, and the data URI is just a (large) JSON string value. `post_id` maps +to the API's `postId`; an absent one must not be sent as null (same rule as every other partial body +in this server). + +## Logging + +An upload is a write this server cannot undo, so it follows the existing rule: log intent at `info` +**before** the request. + +- `upload_image.fetching` — the source `url`, before the download. +- `upload_image.uploading` — `content_type` and `bytes`, before the `POST`. +- The **data URI must never be logged.** It is hundreds of KB of base64 that would bury the session. + This is the one deliberate exception to "post content is not truncated", and the exception is + written at the call site so it is not mistaken for an oversight. Pass the metadata, not the payload. + +## Testing + +All outbound HTTP mocked with MSW, on **both** sides: the external image source and the Substack +`/image` endpoint. The cases that carry weight, each broken on purpose before being trusted: + +- The Substack request body leaves as a **data URI**, not multipart or form-urlencoded. +- A non-image `content-type` from the source is rejected **before** the upload call is made. +- A URL whose host resolves to a private/loopback/link-local address is rejected (SSRF guard). +- A download exceeding the size cap is rejected before encoding. +- `http`/`https` only: a `file:`/`data:` scheme is rejected up front. +- On success the Substack `url` is returned to the caller as `url`, with `post_id` mapped to `postId` + when present and omitted when absent. + +MSW's `onUnhandledRequest: 'error'` stays on; build overrides with typed handlers so requests are +recorded in `msw.requests`. + +## Documentation to correct + +- `CLAUDE.md`: rewrite the "Images can be referenced but not uploaded" paragraph and the + "Do not implement an upload" instruction with the measured facts above — including *why* the three + earlier encodings failed (the data-URI shape), which is the reusable part. Note the external-URL + rejection and the resulting download-and-encode design. Record the auth caveat as the open item. +- `README.md:143`: update the "An image must already be hosted by Substack" line to point at + `upload_image`. + +## Out of scope (stated so it is not re-litigated) + +- **Guarding `image2.src` in `document.js`.** Today any string is accepted and an external URL saves, + returns 200 and silently fails to render. This work makes that guard *more* sensible — there is now + a legitimate way to obtain a valid `src` — but it remains a separate change. +- **Local-file and data-URI sources.** The interface is remote-URL-only by decision (2026-08-08): no + filesystem access (path-traversal surface; would not work in the Docker image), and no caller-passed + data URI (tens of KB of base64 spent in the model's context). +- **Confirming the auth path via header token** is implementation's first step, listed here as the one + unproven fact — not a scope item to design away. From cec4fb3880ddb24a9a47241e592eb46f14e78fc6 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 10:15:52 +0200 Subject: [PATCH 02/12] Add the upload_image implementation plan Nine TDD tasks: SubstackApi.uploadImage, the tool (fetch/encode/upload), content-type and HEIC validation, scheme + DNS-resolved SSRF guards, size cap, server registration, logging assertions, docs correction, and two-Node verification. Co-Authored-By: Claude Opus 4.8 --- .../plans/2026-08-08-image-upload.md | 702 ++++++++++++++++++ 1 file changed, 702 insertions(+) create mode 100644 docs/superpowers/plans/2026-08-08-image-upload.md diff --git a/docs/superpowers/plans/2026-08-08-image-upload.md b/docs/superpowers/plans/2026-08-08-image-upload.md new file mode 100644 index 0000000..07f65ed --- /dev/null +++ b/docs/superpowers/plans/2026-08-08-image-upload.md @@ -0,0 +1,702 @@ +# upload_image Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add an `upload_image` MCP tool that downloads an external image URL, re-encodes it as a data URI, and uploads it to Substack via `POST /api/v1/image`, returning a Substack-hosted URL usable in `image2.src`. + +**Architecture:** One new tool file `src/tools/upload_image.js` (zod schema + handler) plus one `SubstackApi.uploadImage()` method and one registry entry in `src/server.js`. The handler fetches the caller's URL through an injectable `fetchImpl`, guards it (scheme allow-list, DNS-resolved SSRF block via an injectable `lookup`, size cap, `image/*`/anti-HEIC content-type check), base64-encodes the bytes into a `data:` URI, and hands that to `SubstackApi.uploadImage`. All outbound HTTP is mocked with MSW; DNS is never touched in tests because `lookup` is injected. + +**Tech Stack:** Node ≥22 (dev on 24), ESM, zod 4, `@modelcontextprotocol/sdk` `McpServer.registerTool`, `node:dns`/`node:net`, MSW, `node --test`. + +**Reference spec:** `docs/superpowers/specs/2026-08-08-image-upload-design.md` + +--- + +## File structure + +- **Create** `src/tools/upload_image.js` — schema, SSRF/validation helpers, handler. Exports `uploadImageSchema`, `uploadImageHandler`, `isPrivateAddress`, `MAX_IMAGE_BYTES`. +- **Create** `src/tools/upload_image.spec.js` — colocated tests. +- **Modify** `src/api/substack/SubstackApi.js` — add `uploadImage({image, post_id})`. +- **Modify** `src/api/substack/SubstackApi.spec.js` — test the new method. +- **Modify** `test/helpers/msw-server.js` — add `IMAGE_URL`, `IMAGE_UPLOAD_RESPONSE`, `imageUploadHandler`, register and export it. +- **Modify** `src/server.js` — import and register `upload_image`. +- **Modify** `CLAUDE.md`, `README.md` — correct the "cannot upload" record. + +Test-output note (from `CLAUDE.md`): on Node 24 the reporter tallies `ℹ pass` and marks failures `✖`; on the Node 22 floor it is TAP (`# pass`, `not ok`). Grep both when scripting: `grep -E '^(#|ℹ) (tests|pass|fail)'` and `grep -E '^(not ok|✖)'`. + +--- + +## Task 1: `SubstackApi.uploadImage()` + +**Files:** +- Modify: `src/api/substack/SubstackApi.js` (add method next to `postDraft`, around line 155) +- Modify: `test/helpers/msw-server.js` (add `IMAGE_URL`, `IMAGE_UPLOAD_RESPONSE`, `imageUploadHandler`) +- Test: `src/api/substack/SubstackApi.spec.js` + +- [ ] **Step 1: Add the MSW image-upload handler and fixtures** + +In `test/helpers/msw-server.js`, add the URL constant next to the other `API`-based constants (after `DRAFTS_URL`, ~line 12): + +```js +export const IMAGE_URL = `${API}/image`; +``` + +Add a response fixture near the other `*_RESPONSE` fixtures (e.g. after `DRAFT_RESPONSE`, ~line 58). These are the exact keys the live endpoint returned on 2026-08-08: + +```js +export const IMAGE_UPLOAD_RESPONSE = { + id: 'test-image-id', + url: 'https://substack-post-media.s3.amazonaws.com/public/images/test-image.jpg', + contentType: 'image/jpeg', + bytes: 82768, + imageWidth: 1200, + imageHeight: 630, +}; +``` + +Add the handler builder next to `draftsHandler` (~line 563): + +```js + function imageUploadHandler(responder) { + return http.post(IMAGE_URL, async ({request}) => { + await record(request); + return responder(); + }); + } +``` + +Register it in the `setupServer(...)` list (next to `draftsHandler(...)`, ~line 791): + +```js + imageUploadHandler(() => HttpResponse.json(IMAGE_UPLOAD_RESPONSE, {status: 200})), +``` + +Expose it in the returned object (next to `draftsHandler,`, ~line 843): + +```js + imageUploadHandler, +``` + +- [ ] **Step 2: Write the failing test** + +In `src/api/substack/SubstackApi.spec.js`, add the import for `IMAGE_URL` and `IMAGE_UPLOAD_RESPONSE` to the existing `msw-server.js` import block, then add: + +```js +describe('SubstackApi — uploadImage', () => { + test('POSTs the data URI as JSON and returns the parsed body', async () => { + const api = createApi(); + let seen; + msw.server.use( + msw.imageUploadHandler(async () => HttpResponse.json(IMAGE_UPLOAD_RESPONSE, {status: 200})) + ); + + const result = await api.uploadImage({image: 'data:image/jpeg;base64,QUJD'}); + + seen = msw.requests.find((r) => r.url.endsWith('/api/v1/image')); + assert.equal(seen.method, 'POST'); + assert.equal(seen.headers['content-type'], 'application/json'); + assert.deepEqual(seen.body, {image: 'data:image/jpeg;base64,QUJD'}); + assert.equal(result.url, IMAGE_UPLOAD_RESPONSE.url); + assert.equal(result.bytes, 82768); + }); + + test('includes postId only when post_id is given', async () => { + const api = createApi(); + await api.uploadImage({image: 'data:image/png;base64,QQ==', post_id: 42}); + const seen = msw.requests.filter((r) => r.url.endsWith('/api/v1/image')).pop(); + assert.deepEqual(seen.body, {image: 'data:image/png;base64,QQ==', postId: 42}); + }); + + test('omits postId when post_id is absent', async () => { + const api = createApi(); + await api.uploadImage({image: 'data:image/png;base64,QQ=='}); + const seen = msw.requests.filter((r) => r.url.endsWith('/api/v1/image')).pop(); + assert.deepEqual(Object.keys(seen.body), ['image']); + }); +}); +``` + +- [ ] **Step 3: Run the test to verify it fails** + +Run: `npm test -- 2>&1 | grep -iE 'uploadImage|not ok|✖|TypeError'` +Expected: failures — `api.uploadImage is not a function`. + +- [ ] **Step 4: Implement the method** + +In `src/api/substack/SubstackApi.js`, add after `postDraft` (~line 157): + +```js + /** + * Uploads an image. The body is a data URI under `image`, not a file or a URL — the editor builds + * it from `canvas.toDataURL()`. Verified live 2026-08-08: 200 with {id, url, contentType, bytes, + * imageWidth, imageHeight}. `post_id` maps to the API's `postId`; an absent one must not be sent as + * null (same partial-body rule as everywhere else here). + */ + async uploadImage({image, post_id = null}) { + return this.request({ + method: 'POST', + path: '/image', + body: post_id === null ? {image} : {image, postId: post_id}, + referer: '/publish/post', + }); + } +``` + +- [ ] **Step 5: Run the test to verify it passes** + +Run: `npm test -- 2>&1 | grep -iE 'uploadImage'` +Expected: the three `uploadImage` tests pass (`ℹ`/`ok`), no `not ok`/`✖`. + +- [ ] **Step 6: Commit** + +```bash +git add src/api/substack/SubstackApi.js src/api/substack/SubstackApi.spec.js test/helpers/msw-server.js +git commit -m "Add SubstackApi.uploadImage: JSON data-URI POST to /api/v1/image" +``` + +--- + +## Task 2: `upload_image` schema and happy path + +**Files:** +- Create: `src/tools/upload_image.js` +- Test: `src/tools/upload_image.spec.js` + +- [ ] **Step 1: Write the failing test** + +Create `src/tools/upload_image.spec.js`: + +```js +import {test, describe, before, after, afterEach} from 'node:test'; +import assert from 'node:assert/strict'; +import {http, HttpResponse} from 'msw'; +import {uploadImageHandler, uploadImageSchema, MAX_IMAGE_BYTES, isPrivateAddress} from './upload_image.js'; +import {createMswServer, IMAGE_URL, IMAGE_UPLOAD_RESPONSE} from '../../test/helpers/msw-server.js'; +import {setTestEnv} from '../../test/helpers/env.js'; +import {captureLogs} from '../../test/helpers/capture-logs.js'; + +const msw = createMswServer(); +let restoreEnv; + +before(() => { + restoreEnv = setTestEnv(); + msw.start(); +}); +afterEach(() => msw.reset()); +after(() => { + msw.stop(); + restoreEnv(); +}); + +// A public address for the source host, so the SSRF guard passes without touching real DNS. +const publicLookup = async () => [{address: '93.184.216.34', family: 4}]; + +// A source image served by MSW. `bytes`/`type` let each test shape size and content-type. +const SOURCE = 'https://images.example.com/photo.jpg'; +function sourceHandler({body = Buffer.from([0xff, 0xd8, 0xff, 0xd9]), type = 'image/jpeg'} = {}) { + return http.get(SOURCE, () => new HttpResponse(body, {status: 200, headers: {'Content-Type': type}})); +} + +const run = (args, deps = {}) => + uploadImageHandler(args, {lookup: publicLookup, ...deps}); + +describe('uploadImageHandler — happy path', () => { + test('downloads, encodes as a data URI, uploads, returns the mapped fields', async () => { + msw.server.use(sourceHandler()); + const result = await run({url: SOURCE}); + + const upload = msw.requests.find((r) => r.url.endsWith('/api/v1/image')); + assert.match(upload.body.image, /^data:image\/jpeg;base64,/); + assert.equal(upload.body.postId, undefined); + + assert.deepEqual(result, { + id: IMAGE_UPLOAD_RESPONSE.id, + url: IMAGE_UPLOAD_RESPONSE.url, + content_type: IMAGE_UPLOAD_RESPONSE.contentType, + bytes: IMAGE_UPLOAD_RESPONSE.bytes, + width: IMAGE_UPLOAD_RESPONSE.imageWidth, + height: IMAGE_UPLOAD_RESPONSE.imageHeight, + }); + }); + + test('forwards post_id as postId to the upload', async () => { + msw.server.use(sourceHandler()); + await run({url: SOURCE, post_id: 7}); + const upload = msw.requests.find((r) => r.url.endsWith('/api/v1/image')); + assert.equal(upload.body.postId, 7); + }); +}); +``` + +- [ ] **Step 2: Run the test to verify it fails** + +Run: `npm test -- 2>&1 | grep -iE 'upload_image|Cannot find|not ok|✖'` +Expected: failure — module `./upload_image.js` not found. + +- [ ] **Step 3: Implement the minimal tool** + +Create `src/tools/upload_image.js`: + +```js +import {z} from "zod"; +import dns from "node:dns"; +import SubstackApi from "../api/substack/SubstackApi.js"; +import {logger} from "../logger.js"; + +// Our own memory guard, NOT Substack's limit (its MAX_FILE_SIZE could not be read from the minified +// bundle). The downloaded buffer plus its ~1.37x base64 string sit in RAM; 10 MB caps that. +export const MAX_IMAGE_BYTES = 10 * 1024 * 1024; + +const HEIC_TYPES = new Set(['image/heic', 'image/heif']); + +// strictObject: an unknown key is reported, never stripped — the only repair signal an LLM gets. +export const uploadImageSchema = z.strictObject({ + url: z + .string() + .url() + .describe( + "The http(s) URL of an image to upload. The server downloads it and re-hosts it on Substack. " + + "Private, loopback and link-local hosts are refused. Max 10 MB. HEIC is not accepted." + ), + post_id: z + .number() + .optional() + .describe("Optional id of the post the image belongs to. Its effect is unconfirmed."), +}); + +// Resolve every address a host maps to. Injected in tests so DNS is never touched. +const defaultLookup = (hostname) => dns.promises.lookup(hostname, {all: true}); + +// Loopback / private / link-local / unique-local / unspecified, plus IPv4-mapped IPv6. +export function isPrivateAddress(address, family) { + if (family === 4) { + const p = address.split('.').map(Number); + if (p[0] === 10) return true; + if (p[0] === 127) return true; + if (p[0] === 0) return true; + if (p[0] === 169 && p[1] === 254) return true; // link-local incl. 169.254.169.254 + if (p[0] === 172 && p[1] >= 16 && p[1] <= 31) return true; + if (p[0] === 192 && p[1] === 168) return true; + if (p[0] === 100 && p[1] >= 64 && p[1] <= 127) return true; // CGNAT + return false; + } + const a = address.toLowerCase(); + if (a === '::1' || a === '::') return true; + if (a.startsWith('fe8') || a.startsWith('fe9') || a.startsWith('fea') || a.startsWith('feb')) return true; // fe80::/10 + if (a.startsWith('fc') || a.startsWith('fd')) return true; // fc00::/7 unique-local + const mapped = a.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); // IPv4-mapped + if (mapped) return isPrivateAddress(mapped[1], 4); + return false; +} + +async function assertPublicUrl(rawUrl, lookup) { + let url; + try { + url = new URL(rawUrl); + } catch { + throw new Error(`upload_image: not a valid URL: ${rawUrl}`); + } + if (url.protocol !== 'http:' && url.protocol !== 'https:') { + throw new Error(`upload_image: only http and https URLs are allowed, got ${url.protocol}`); + } + const addresses = await lookup(url.hostname); + for (const {address, family} of addresses) { + if (isPrivateAddress(address, family)) { + throw new Error(`upload_image: refusing to fetch a private/loopback address (${address})`); + } + } + return url; +} + +export const uploadImageHandler = async (args, {lookup = defaultLookup, fetchImpl = fetch} = {}) => { + const {url, post_id} = uploadImageSchema.parse(args); + + await assertPublicUrl(url, lookup); + + logger.info('upload_image.fetching', {url, post_id: post_id ?? null}); + const response = await fetchImpl(url); + if (!response.ok) { + throw new Error(`upload_image: source responded ${response.status} ${response.statusText}`); + } + + const contentType = (response.headers.get('content-type') || '').split(';')[0].trim().toLowerCase(); + if (!contentType.startsWith('image/')) { + throw new Error(`upload_image: source is not an image (content-type: ${contentType || 'none'})`); + } + if (HEIC_TYPES.has(contentType)) { + throw new Error('upload_image: HEIC is not accepted by Substack. Convert to JPG or PNG first.'); + } + + const buffer = Buffer.from(await response.arrayBuffer()); + if (buffer.byteLength > MAX_IMAGE_BYTES) { + throw new Error( + `upload_image: image is ${buffer.byteLength} bytes, over the ${MAX_IMAGE_BYTES}-byte limit.` + ); + } + + const image = `data:${contentType};base64,${buffer.toString('base64')}`; + // The data URI is deliberately NOT logged: hundreds of KB of base64 would bury the session. This + // is the one exception to "post content is not truncated". + logger.info('upload_image.uploading', {content_type: contentType, bytes: buffer.byteLength, post_id: post_id ?? null}); + + const substack_api = new SubstackApi({ + publication_url: process.env.SUBSTACK_PUBLICATION_URL, + auth_token: process.env.SUBSTACK_SESSION_TOKEN, + }); + const uploaded = await substack_api.uploadImage({image, post_id: post_id ?? null}); + + logger.info('upload_image.done', {url: uploaded.url, bytes: uploaded.bytes}); + + return { + id: uploaded.id, + url: uploaded.url, + content_type: uploaded.contentType, + bytes: uploaded.bytes, + width: uploaded.imageWidth, + height: uploaded.imageHeight, + }; +}; +``` + +- [ ] **Step 4: Run the test to verify it passes** + +Run: `npm test -- 2>&1 | grep -iE 'happy path|not ok|✖'` +Expected: both happy-path tests pass, no `not ok`/`✖`. + +- [ ] **Step 5: Commit** + +```bash +git add src/tools/upload_image.js src/tools/upload_image.spec.js +git commit -m "Add upload_image tool: fetch, encode as data URI, upload" +``` + +--- + +## Task 3: content-type and HEIC validation + +**Files:** +- Test: `src/tools/upload_image.spec.js` + +- [ ] **Step 1: Write the failing test** + +Append to `src/tools/upload_image.spec.js`: + +```js +describe('uploadImageHandler — content validation', () => { + test('rejects a non-image source before uploading', async () => { + msw.server.use(sourceHandler({body: Buffer.from(''), type: 'text/html'})); + await assert.rejects(run({url: SOURCE}), /not an image/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); + + test('rejects HEIC with a convert message', async () => { + msw.server.use(sourceHandler({type: 'image/heic'})); + await assert.rejects(run({url: SOURCE}), /HEIC is not accepted/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); +}); +``` + +- [ ] **Step 2: Run to verify it passes (behavior already implemented in Task 2)** + +Run: `npm test -- 2>&1 | grep -iE 'content validation|not ok|✖'` +Expected: PASS. This task exists to lock the behavior with tests. To confirm the tests are real, temporarily change `startsWith('image/')` to `startsWith('')` in `upload_image.js`, run — the non-image test must fail — then restore. Grep the file for `startsWith('image/')` to confirm the revert landed before trusting the green run. + +- [ ] **Step 3: Commit** + +```bash +git add src/tools/upload_image.spec.js +git commit -m "Test upload_image content-type and HEIC rejection" +``` + +--- + +## Task 4: scheme and SSRF guards + +**Files:** +- Test: `src/tools/upload_image.spec.js` + +- [ ] **Step 1: Write the failing test** + +Append to `src/tools/upload_image.spec.js`: + +```js +describe('isPrivateAddress', () => { + test('flags loopback, private, link-local, unique-local; allows public', () => { + assert.equal(isPrivateAddress('127.0.0.1', 4), true); + assert.equal(isPrivateAddress('10.1.2.3', 4), true); + assert.equal(isPrivateAddress('172.16.0.1', 4), true); + assert.equal(isPrivateAddress('192.168.1.1', 4), true); + assert.equal(isPrivateAddress('169.254.169.254', 4), true); + assert.equal(isPrivateAddress('93.184.216.34', 4), false); + assert.equal(isPrivateAddress('::1', 6), true); + assert.equal(isPrivateAddress('fe80::1', 6), true); + assert.equal(isPrivateAddress('fd00::1', 6), true); + assert.equal(isPrivateAddress('::ffff:127.0.0.1', 6), true); + assert.equal(isPrivateAddress('2606:2800:220:1:248:1893:25c8:1946', 6), false); + }); +}); + +describe('uploadImageHandler — SSRF and scheme guards', () => { + test('rejects a host that resolves to a private address, without fetching', async () => { + let fetched = false; + const fetchImpl = async () => { fetched = true; return new HttpResponse(); }; + const privateLookup = async () => [{address: '169.254.169.254', family: 4}]; + await assert.rejects( + uploadImageHandler({url: 'http://metadata.internal/'}, {lookup: privateLookup, fetchImpl}), + /private\/loopback/ + ); + assert.equal(fetched, false); + }); + + test('rejects a non-http(s) scheme up front', async () => { + await assert.rejects(run({url: 'ftp://example.com/x.png'}), /only http and https/); + }); +}); +``` + +Note: `z.string().url()` accepts `ftp://…`, so the scheme check in `assertPublicUrl` is what rejects it — this test proves that guard, not the schema. + +- [ ] **Step 2: Run to verify it passes (behavior implemented in Task 2)** + +Run: `npm test -- 2>&1 | grep -iE 'SSRF|isPrivateAddress|not ok|✖'` +Expected: PASS. To confirm the SSRF test is real, temporarily make `isPrivateAddress` always return `false`, run — the "resolves to a private address" test must fail — then restore and grep to confirm. + +- [ ] **Step 3: Commit** + +```bash +git add src/tools/upload_image.spec.js +git commit -m "Test upload_image scheme allow-list and DNS-resolved SSRF guard" +``` + +--- + +## Task 5: size cap + +**Files:** +- Test: `src/tools/upload_image.spec.js` + +- [ ] **Step 1: Write the failing test** + +Append to `src/tools/upload_image.spec.js`: + +```js +describe('uploadImageHandler — size cap', () => { + test('rejects an image over MAX_IMAGE_BYTES before uploading', async () => { + const big = Buffer.alloc(MAX_IMAGE_BYTES + 1, 0xff); + msw.server.use(sourceHandler({body: big, type: 'image/png'})); + await assert.rejects(run({url: SOURCE}), /over the .* limit/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); + + test('accepts an image exactly at the limit', async () => { + const atLimit = Buffer.alloc(MAX_IMAGE_BYTES, 0xff); + msw.server.use(sourceHandler({body: atLimit, type: 'image/png'})); + const result = await run({url: SOURCE}); + assert.equal(result.url, IMAGE_UPLOAD_RESPONSE.url); + }); +}); +``` + +- [ ] **Step 2: Run to verify it passes (behavior implemented in Task 2)** + +Run: `npm test -- 2>&1 | grep -iE 'size cap|not ok|✖'` +Expected: PASS. To confirm the cap test is real, temporarily change `> MAX_IMAGE_BYTES` to `> MAX_IMAGE_BYTES * 1000`, run — the oversize test must fail — then restore and grep to confirm. + +- [ ] **Step 3: Commit** + +```bash +git add src/tools/upload_image.spec.js +git commit -m "Test upload_image size cap at MAX_IMAGE_BYTES" +``` + +--- + +## Task 6: register the tool in the server + +**Files:** +- Modify: `src/server.js` +- Test: `src/server.spec.js` + +- [ ] **Step 1: Write the failing test** + +In `src/server.spec.js`, find where the harness lists tools (search for `tools/list` or `listTools`). Add: + +```js +test('upload_image is registered and advertises url + post_id', async () => { + const client = await connectMcpClient(); + const {tools} = await client.listTools(); + const tool = tools.find((t) => t.name === 'upload_image'); + assert.ok(tool, 'upload_image should be registered'); + assert.deepEqual(Object.keys(tool.inputSchema.properties).sort(), ['post_id', 'url']); + assert.equal(tool.inputSchema.additionalProperties, false); +}); +``` + +If `connectMcpClient`/`listTools` is not the local idiom, mirror the nearest existing tools/list test in the file. + +- [ ] **Step 2: Run to verify it fails** + +Run: `npm test -- 2>&1 | grep -iE 'upload_image is registered|not ok|✖'` +Expected: FAIL — `upload_image should be registered`. + +- [ ] **Step 3: Register the tool** + +In `src/server.js`, add the import next to the other tool imports: + +```js +import {uploadImageSchema, uploadImageHandler} from "./tools/upload_image.js"; +``` + +Add the registry entry inside the `tools` object (e.g. after `set_post_body`): + +```js + upload_image: { + description: + "Upload an image to your Substack publication from an http(s) URL. The server downloads the " + + "image and re-hosts it on Substack; the returned url is what goes into image2.src in " + + "set_post_body. Private/loopback hosts are refused, HEIC is not accepted, max 10 MB.", + schema: uploadImageSchema, + handler: uploadImageHandler, + }, +``` + +- [ ] **Step 4: Run to verify it passes** + +Run: `npm test -- 2>&1 | grep -iE 'upload_image is registered|not ok|✖'` +Expected: PASS. + +- [ ] **Step 5: Commit** + +```bash +git add src/server.js src/server.spec.js +git commit -m "Register upload_image in the tools registry" +``` + +--- + +## Task 7: logging assertions + +**Files:** +- Test: `src/tools/upload_image.spec.js` + +- [ ] **Step 1: Write the failing test** + +Append to `src/tools/upload_image.spec.js`: + +```js +describe('uploadImageHandler — logging', () => { + test('logs intent before the request and never logs the data URI', async () => { + msw.server.use(sourceHandler()); + const {logs} = await captureLogs(() => run({url: SOURCE})); + const events = logs.map((l) => l.msg); + assert.ok(events.includes('upload_image.fetching')); + assert.ok(events.includes('upload_image.uploading')); + // The base64 payload must never appear in any log line. + const serialized = JSON.stringify(logs); + assert.equal(serialized.includes('base64,'), false); + }); +}); +``` + +Confirm the `captureLogs` return shape against `test/helpers/capture-logs.js`; if it returns the array directly rather than `{logs}`, adjust the destructuring to match the existing idiom in another spec (e.g. `export_subscribers.spec.js`). + +- [ ] **Step 2: Run to verify it passes (behavior implemented in Task 2)** + +Run: `npm test -- 2>&1 | grep -iE 'logging|never logs|not ok|✖'` +Expected: PASS. To confirm the redaction test is real, temporarily add `logger.info('leak', {image})` before the upload in `upload_image.js`, run — the "never logs the data URI" test must fail — then remove it and grep to confirm the line is gone. + +- [ ] **Step 3: Commit** + +```bash +git add src/tools/upload_image.spec.js +git commit -m "Test upload_image logs intent and never logs the data URI" +``` + +--- + +## Task 8: correct the documentation + +**Files:** +- Modify: `CLAUDE.md` +- Modify: `README.md` + +- [ ] **Step 1: Rewrite the CLAUDE.md limit paragraph** + +In `CLAUDE.md`, find the paragraph beginning "**Images can be referenced but not uploaded**" and the sentence "**Do not implement an upload from either.**" Replace the "all three hang / do not implement" claim with the measured reality. Keep it in the file's voice (why, not just what): + +```markdown +**Images can be uploaded after all, and `upload_image` is how.** `POST /api/v1/image` was recorded +here as hanging in all three tested encodings; re-measured live 2026-08-08, it answers **200** — the +body is JSON `{image: "data:;base64,…"}`, a **data URI**, not a file or a URL. Multipart and +form-urlencoded failed because they sent the wrong thing, not for a header detail or a Cloudflare +wall. The response is `{id, url, contentType, bytes, imageWidth, imageHeight}`, `url` on +`substack-post-media.s3.amazonaws.com` — the same host all `image2.src` values use, and it renders +through Substack's CDN (proven end to end on a real draft). **Substack server-fetches only its own S3 +bucket**: an external URL as `image` answers `400 "Failed to fetch image"`, so `upload_image` +downloads the URL itself and re-encodes it. It guards that download — http(s) only, DNS-resolved SSRF +block, `image/*` (HEIC refused early, as the dashboard does), and a 10 MB cap that is **ours, not +Substack's** (the bundle's `MAX_FILE_SIZE` could not be read). The data URI is never logged. **Still +unverified:** the live checks used the browser session cookie, not `SUBSTACK_SESSION_TOKEN` in a +header — confirm that path first. +``` + +- [ ] **Step 2: Update the README image line** + +In `README.md` around line 143, replace the "An image must already be hosted by Substack" line with a pointer to the tool: + +```markdown +- **Images:** `upload_image` takes an http(s) image URL, re-hosts it on Substack, and returns a + `url` you put in `image2.src`. External URLs placed directly in `image2.src` do not render. +``` + +Also update the tool list around `README.md:124`/`README.md:134` if it enumerates tools, adding `upload_image`. + +- [ ] **Step 3: Commit** + +```bash +git add CLAUDE.md README.md +git commit -m "Correct the docs: images can be uploaded via upload_image" +``` + +--- + +## Task 9: full-suite verification at both Node versions + +**Files:** none (verification only) + +- [ ] **Step 1: Run the whole suite on the dev version** + +Run: `npm test 2>&1 | tail -20` +Expected: all pass. Grep both reporter shapes: `npm test 2>&1 | grep -E '^(#|ℹ) (tests|pass|fail)'` and confirm no `grep -E '^(not ok|✖)'` output. + +- [ ] **Step 2: Run at the engines floor (Node 22)** + +Run: `source ~/.nvm/nvm.sh && nvm exec --silent 22 npm test 2>&1 | grep -E '^(#|not ok) '` +Expected: `# fail 0`, no `not ok` lines. (TAP reporter on 22.) + +- [ ] **Step 3: Confirm the tool is advertised over the real protocol** + +Run: +```bash +printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"probe","version":"1.0.0"}}}' \ + '{"jsonrpc":"2.0","method":"notifications/initialized"}' \ + '{"jsonrpc":"2.0","id":2,"method":"tools/list","params":{}}' \ + | SUBSTACK_PUBLICATION_URL=https://test.substack.com SUBSTACK_SESSION_TOKEN=tok SUBSTACK_USER_ID=1 \ + timeout 5 node src/index.js | grep -o '"upload_image"' +``` +Expected: `"upload_image"` printed. + +- [ ] **Step 4: Final commit if anything was adjusted during verification** + +```bash +git add -A && git commit -m "Verify upload_image across Node 22 and 24" || echo "nothing to commit" +``` + +--- + +## Self-review notes + +- **Spec coverage:** interface (Task 2, 6), download+encode (Task 2), scheme/SSRF guards (Task 4), size cap (Task 5), content-type/HEIC (Task 3), API method + postId mapping (Task 1), logging incl. no-data-URI (Task 7), docs correction (Task 8), auth caveat recorded in docs and left as the stated open item (Task 8, verified-only in Task 9 without the header path — flagged, not silently closed). All spec sections map to a task. +- **Type consistency:** `uploadImage({image, post_id})` (Task 1) matches the call in Task 2; response keys `contentType/imageWidth/imageHeight` map to output `content_type/width/height` consistently in Task 2's test and impl; `isPrivateAddress(address, family)` signature identical in impl and Task 4 tests; `MAX_IMAGE_BYTES` used identically in impl and Task 5. +- **Known limitation (from spec, intentional):** the SSRF check resolves DNS then fetches by URL — a TOCTOU window the spec accepted; not closed here. From 58778304cd626c447d2e1cd8130c8281d04ce57d Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 10:18:55 +0200 Subject: [PATCH 03/12] Add SubstackApi.uploadImage: JSON data-URI POST to /api/v1/image First of nine tasks adding an upload_image MCP tool. This adds only the thin API method and the MSW test scaffolding (IMAGE_URL, IMAGE_UPLOAD_RESPONSE, imageUploadHandler) it needs. Co-Authored-By: Claude Opus 4.8 --- src/api/substack/SubstackApi.js | 15 ++++++++++++ src/api/substack/SubstackApi.spec.js | 35 ++++++++++++++++++++++++++++ test/helpers/msw-server.js | 20 ++++++++++++++++ 3 files changed, 70 insertions(+) diff --git a/src/api/substack/SubstackApi.js b/src/api/substack/SubstackApi.js index 6b1f00b..2158feb 100644 --- a/src/api/substack/SubstackApi.js +++ b/src/api/substack/SubstackApi.js @@ -157,6 +157,21 @@ export default class SubstackApi { return this.request({method: 'POST', path: '/drafts', body, referer: '/publish/post'}); } + /** + * Uploads an image. The body is a data URI under `image`, not a file or a URL — the editor builds + * it from `canvas.toDataURL()`. Verified live 2026-08-08: 200 with {id, url, contentType, bytes, + * imageWidth, imageHeight}. `post_id` maps to the API's `postId`; an absent one must not be sent as + * null (same partial-body rule as everywhere else here). + */ + async uploadImage({image, post_id = null}) { + return this.request({ + method: 'POST', + path: '/image', + body: post_id === null ? {image} : {image, postId: post_id}, + referer: '/publish/post', + }); + } + /** * Lists subscribers. `query` is the whole request body — filters, sorting and free-text search * all live inside its `filters` object; see SubscriberQuery.js for how it is assembled. diff --git a/src/api/substack/SubstackApi.spec.js b/src/api/substack/SubstackApi.spec.js index 65a07ca..383fda7 100644 --- a/src/api/substack/SubstackApi.spec.js +++ b/src/api/substack/SubstackApi.spec.js @@ -6,6 +6,8 @@ import { createMswServer, DRAFTS_URL, DRAFT_RESPONSE, + IMAGE_URL, + IMAGE_UPLOAD_RESPONSE, SUBSCRIBER_STATS_URL, SUBSCRIBER_STATS_RESPONSE, POSTS_RESPONSE, @@ -138,6 +140,39 @@ describe('SubstackApi — postDraft', () => { }); }); +describe('SubstackApi — uploadImage', () => { + test('POSTs the data URI as JSON and returns the parsed body', async () => { + const api = createApi(); + + const result = await api.uploadImage({image: 'data:image/jpeg;base64,QUJD'}); + + const seen = msw.requests.find((r) => r.url === IMAGE_URL); + assert.equal(seen.method, 'POST'); + assert.match(seen.headers['content-type'], /^application\/json/); + assert.deepEqual(seen.body, {image: 'data:image/jpeg;base64,QUJD'}); + assert.equal(result.url, IMAGE_UPLOAD_RESPONSE.url); + assert.equal(result.bytes, 82768); + }); + + test('includes postId only when post_id is given', async () => { + const api = createApi(); + + await api.uploadImage({image: 'data:image/png;base64,QQ==', post_id: 42}); + + const seen = msw.requests.find((r) => r.url === IMAGE_URL); + assert.deepEqual(seen.body, {image: 'data:image/png;base64,QQ==', postId: 42}); + }); + + test('omits postId when post_id is absent', async () => { + const api = createApi(); + + await api.uploadImage({image: 'data:image/png;base64,QQ=='}); + + const seen = msw.requests.find((r) => r.url === IMAGE_URL); + assert.deepEqual(Object.keys(seen.body), ['image']); + }); +}); + describe('SubstackApi — getSubscribers', () => { test('POSTs the query to the subscriber-stats endpoint', async () => { const query = {filters: {subscription_type: 'free'}, limit: 25, offset: 0}; diff --git a/test/helpers/msw-server.js b/test/helpers/msw-server.js index 0b9994a..661f467 100644 --- a/test/helpers/msw-server.js +++ b/test/helpers/msw-server.js @@ -10,6 +10,7 @@ const API = `${TEST_ENV.SUBSTACK_PUBLICATION_URL}/api/v1`; const GLOBAL_API = 'https://substack.com/api/v1'; export const DRAFTS_URL = `${API}/drafts`; +export const IMAGE_URL = `${API}/image`; export const PUBLICATION_URL = `${API}/publication`; export const USER_PROFILE_URL = `${GLOBAL_API}/user/profile/self`; export const POST_TAG_URL = `${API}/publication/post-tag`; @@ -55,6 +56,16 @@ export const DRAFT_RESPONSE = { is_published: false, }; +// These are the exact keys the live `POST /api/v1/image` endpoint returned, verified 2026-08-08. +export const IMAGE_UPLOAD_RESPONSE = { + id: 'test-image-id', + url: 'https://substack-post-media.s3.amazonaws.com/public/images/test-image.jpg', + contentType: 'image/jpeg', + bytes: 82768, + imageWidth: 1200, + imageHeight: 630, +}; + // Shaped after a real `POST /drafts/:id/publish`: the draft comes back as a post, with the slug and // canonical url it did not have while unpublished. export const PUBLISHED_POST_RESPONSE = { @@ -567,6 +578,13 @@ export function createMswServer() { }); } + function imageUploadHandler(responder) { + return http.post(IMAGE_URL, async ({request}) => { + await record(request); + return responder(); + }); + } + function subscriberStatsHandler(responder) { return http.post(SUBSCRIBER_STATS_URL, async ({request}) => { await record(request); @@ -790,6 +808,7 @@ export function createMswServer() { const server = setupServer( draftsHandler(() => HttpResponse.json(DRAFT_RESPONSE, {status: 200})), + imageUploadHandler(() => HttpResponse.json(IMAGE_UPLOAD_RESPONSE, {status: 200})), subscriberStatsHandler(() => HttpResponse.json(SUBSCRIBER_STATS_RESPONSE, {status: 200})), postsHandler(() => HttpResponse.json(POSTS_RESPONSE, {status: 200})), draftDetailHandler(() => HttpResponse.json(DRAFT_DETAIL_RESPONSE, {status: 200})), @@ -842,6 +861,7 @@ export function createMswServer() { server, requests, draftsHandler, + imageUploadHandler, subscriberStatsHandler, postsHandler, draftDetailHandler, From 8acd7a7f203fb5870033926c6980004effe40696 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 10:25:19 +0200 Subject: [PATCH 04/12] Add upload_image tool: fetch, encode as data URI, upload --- src/tools/upload_image.js | 118 +++++++++++++++++++++++++++++++++ src/tools/upload_image.spec.js | 59 +++++++++++++++++ 2 files changed, 177 insertions(+) create mode 100644 src/tools/upload_image.js create mode 100644 src/tools/upload_image.spec.js diff --git a/src/tools/upload_image.js b/src/tools/upload_image.js new file mode 100644 index 0000000..dd5de65 --- /dev/null +++ b/src/tools/upload_image.js @@ -0,0 +1,118 @@ +import {z} from "zod"; +import dns from "node:dns"; +import SubstackApi from "../api/substack/SubstackApi.js"; +import {logger} from "../logger.js"; + +// Our own memory guard, NOT Substack's limit (its MAX_FILE_SIZE could not be read from the minified +// bundle). The downloaded buffer plus its ~1.37x base64 string sit in RAM; 10 MB caps that. +export const MAX_IMAGE_BYTES = 10 * 1024 * 1024; + +const HEIC_TYPES = new Set(['image/heic', 'image/heif']); + +// strictObject: an unknown key is reported, never stripped — the only repair signal an LLM gets. +export const uploadImageSchema = z.strictObject({ + url: z + .string() + .url() + .describe( + "The http(s) URL of an image to upload. The server downloads it and re-hosts it on Substack. " + + "Private, loopback and link-local hosts are refused. Max 10 MB. HEIC is not accepted." + ), + post_id: z + .number() + .optional() + .describe("Optional id of the post the image belongs to. Its effect is unconfirmed."), +}); + +// Resolve every address a host maps to. Injected in tests so DNS is never touched. +const defaultLookup = (hostname) => dns.promises.lookup(hostname, {all: true}); + +// Loopback / private / link-local / unique-local / unspecified, plus IPv4-mapped IPv6. +export function isPrivateAddress(address, family) { + if (family === 4) { + const p = address.split('.').map(Number); + if (p[0] === 10) return true; + if (p[0] === 127) return true; + if (p[0] === 0) return true; + if (p[0] === 169 && p[1] === 254) return true; // link-local incl. 169.254.169.254 + if (p[0] === 172 && p[1] >= 16 && p[1] <= 31) return true; + if (p[0] === 192 && p[1] === 168) return true; + if (p[0] === 100 && p[1] >= 64 && p[1] <= 127) return true; // CGNAT + return false; + } + const a = address.toLowerCase(); + if (a === '::1' || a === '::') return true; + if (a.startsWith('fe8') || a.startsWith('fe9') || a.startsWith('fea') || a.startsWith('feb')) return true; // fe80::/10 + if (a.startsWith('fc') || a.startsWith('fd')) return true; // fc00::/7 unique-local + const mapped = a.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); // IPv4-mapped + if (mapped) return isPrivateAddress(mapped[1], 4); + return false; +} + +async function assertPublicUrl(rawUrl, lookup) { + let url; + try { + url = new URL(rawUrl); + } catch { + throw new Error(`upload_image: not a valid URL: ${rawUrl}`); + } + if (url.protocol !== 'http:' && url.protocol !== 'https:') { + throw new Error(`upload_image: only http and https URLs are allowed, got ${url.protocol}`); + } + const addresses = await lookup(url.hostname); + for (const {address, family} of addresses) { + if (isPrivateAddress(address, family)) { + throw new Error(`upload_image: refusing to fetch a private/loopback address (${address})`); + } + } + return url; +} + +export const uploadImageHandler = async (args, {lookup = defaultLookup, fetchImpl = fetch} = {}) => { + const {url, post_id} = uploadImageSchema.parse(args); + + await assertPublicUrl(url, lookup); + + logger.info('upload_image.fetching', {url, post_id: post_id ?? null}); + const response = await fetchImpl(url); + if (!response.ok) { + throw new Error(`upload_image: source responded ${response.status} ${response.statusText}`); + } + + const contentType = (response.headers.get('content-type') || '').split(';')[0].trim().toLowerCase(); + if (!contentType.startsWith('image/')) { + throw new Error(`upload_image: source is not an image (content-type: ${contentType || 'none'})`); + } + if (HEIC_TYPES.has(contentType)) { + throw new Error('upload_image: HEIC is not accepted by Substack. Convert to JPG or PNG first.'); + } + + const buffer = Buffer.from(await response.arrayBuffer()); + if (buffer.byteLength > MAX_IMAGE_BYTES) { + throw new Error( + `upload_image: image is ${buffer.byteLength} bytes, over the ${MAX_IMAGE_BYTES}-byte limit.` + ); + } + + const image = `data:${contentType};base64,${buffer.toString('base64')}`; + // The data URI is deliberately NOT logged: hundreds of KB of base64 would bury the session. This + // is the one exception to "post content is not truncated". + logger.info('upload_image.uploading', {content_type: contentType, bytes: buffer.byteLength, post_id: post_id ?? null}); + + const substack_api = new SubstackApi({ + publication_url: process.env.SUBSTACK_PUBLICATION_URL, + auth_token: process.env.SUBSTACK_SESSION_TOKEN, + }); + const uploaded = await substack_api.uploadImage({image, post_id: post_id ?? null}); + + logger.info('upload_image.done', {url: uploaded.url, bytes: uploaded.bytes}); + + return { + id: uploaded.id, + url: uploaded.url, + content_type: uploaded.contentType, + bytes: uploaded.bytes, + width: uploaded.imageWidth, + height: uploaded.imageHeight, + }; +}; diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js new file mode 100644 index 0000000..4006197 --- /dev/null +++ b/src/tools/upload_image.spec.js @@ -0,0 +1,59 @@ +import {test, describe, before, after, afterEach} from 'node:test'; +import assert from 'node:assert/strict'; +import {http, HttpResponse} from 'msw'; +import {uploadImageHandler, uploadImageSchema, MAX_IMAGE_BYTES, isPrivateAddress} from './upload_image.js'; +import {createMswServer, IMAGE_URL, IMAGE_UPLOAD_RESPONSE} from '../../test/helpers/msw-server.js'; +import {setTestEnv} from '../../test/helpers/env.js'; +import {captureLogs} from '../../test/helpers/capture-logs.js'; + +const msw = createMswServer(); +let restoreEnv; + +before(() => { + restoreEnv = setTestEnv(); + msw.start(); +}); +afterEach(() => msw.reset()); +after(() => { + msw.stop(); + restoreEnv(); +}); + +// A public address for the source host, so the SSRF guard passes without touching real DNS. +const publicLookup = async () => [{address: '93.184.216.34', family: 4}]; + +// A source image served by MSW. `bytes`/`type` let each test shape size and content-type. +const SOURCE = 'https://images.example.com/photo.jpg'; +function sourceHandler({body = Buffer.from([0xff, 0xd8, 0xff, 0xd9]), type = 'image/jpeg'} = {}) { + return http.get(SOURCE, () => new HttpResponse(body, {status: 200, headers: {'Content-Type': type}})); +} + +const run = (args, deps = {}) => + uploadImageHandler(args, {lookup: publicLookup, ...deps}); + +describe('uploadImageHandler — happy path', () => { + test('downloads, encodes as a data URI, uploads, returns the mapped fields', async () => { + msw.server.use(sourceHandler()); + const result = await run({url: SOURCE}); + + const upload = msw.requests.find((r) => r.url.endsWith('/api/v1/image')); + assert.match(upload.body.image, /^data:image\/jpeg;base64,/); + assert.equal(upload.body.postId, undefined); + + assert.deepEqual(result, { + id: IMAGE_UPLOAD_RESPONSE.id, + url: IMAGE_UPLOAD_RESPONSE.url, + content_type: IMAGE_UPLOAD_RESPONSE.contentType, + bytes: IMAGE_UPLOAD_RESPONSE.bytes, + width: IMAGE_UPLOAD_RESPONSE.imageWidth, + height: IMAGE_UPLOAD_RESPONSE.imageHeight, + }); + }); + + test('forwards post_id as postId to the upload', async () => { + msw.server.use(sourceHandler()); + await run({url: SOURCE, post_id: 7}); + const upload = msw.requests.find((r) => r.url.endsWith('/api/v1/image')); + assert.equal(upload.body.postId, 7); + }); +}); From c96cea1c8075edeacfc8d085d2037044f4daa6e2 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 10:34:23 +0200 Subject: [PATCH 05/12] Harden upload_image: validate redirect hops, adopt arg-logging, cleanups fetchImpl was called with fetch's default redirect: 'follow', but assertPublicUrl only validated the original host - a public host that 3xx-redirects to a private address (e.g. 169.254.169.254) bypassed the SSRF guard entirely. fetchGuarded now follows redirects manually, validating each hop's host before contacting it. Also: upload_image now logs args and parse failures like every other tool (comment_on_post, get_draft, restack_item); the unreachable try/catch in assertPublicUrl is gone now that the schema's .url() already guarantees a parseable URL; and three comments were corrected to match what the code actually does (IPv6 embedding coverage, the memory cap's real purpose). Removed the unused captureLogs import from the spec. --- src/tools/upload_image.js | 50 ++++++++++++++++++++++++++-------- src/tools/upload_image.spec.js | 21 +++++++++++++- 2 files changed, 58 insertions(+), 13 deletions(-) diff --git a/src/tools/upload_image.js b/src/tools/upload_image.js index dd5de65..5047492 100644 --- a/src/tools/upload_image.js +++ b/src/tools/upload_image.js @@ -3,8 +3,9 @@ import dns from "node:dns"; import SubstackApi from "../api/substack/SubstackApi.js"; import {logger} from "../logger.js"; -// Our own memory guard, NOT Substack's limit (its MAX_FILE_SIZE could not be read from the minified -// bundle). The downloaded buffer plus its ~1.37x base64 string sit in RAM; 10 MB caps that. +// Caps what we hold and re-encode: an oversized body is still buffered once by fetch, but we +// reject it before the ~1.37x base64 copy and the upload. NOT Substack's own limit (its +// MAX_FILE_SIZE could not be read from the minified bundle). export const MAX_IMAGE_BYTES = 10 * 1024 * 1024; const HEIC_TYPES = new Set(['image/heic', 'image/heif']); @@ -44,18 +45,16 @@ export function isPrivateAddress(address, family) { if (a === '::1' || a === '::') return true; if (a.startsWith('fe8') || a.startsWith('fe9') || a.startsWith('fea') || a.startsWith('feb')) return true; // fe80::/10 if (a.startsWith('fc') || a.startsWith('fd')) return true; // fc00::/7 unique-local - const mapped = a.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); // IPv4-mapped + // IPv4-mapped IPv6 (::ffff:x.x.x.x) is unwrapped; other embeddings (6to4/Teredo/NAT64) are not — + // accepted residual risk. + const mapped = a.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); if (mapped) return isPrivateAddress(mapped[1], 4); return false; } async function assertPublicUrl(rawUrl, lookup) { - let url; - try { - url = new URL(rawUrl); - } catch { - throw new Error(`upload_image: not a valid URL: ${rawUrl}`); - } + // The schema's .url() already guaranteed this parses. + const url = new URL(rawUrl); if (url.protocol !== 'http:' && url.protocol !== 'https:') { throw new Error(`upload_image: only http and https URLs are allowed, got ${url.protocol}`); } @@ -68,13 +67,40 @@ async function assertPublicUrl(rawUrl, lookup) { return url; } +// `fetch`'s default `redirect: 'follow'` would contact a redirect target before we ever see its +// host, which turns `assertPublicUrl` into a check on the ORIGINAL host only — a public host that +// 3xx-redirects to http://169.254.169.254/ (or any private address) bypasses the guard entirely. +// So redirects are followed manually here, validating each hop's host before it is contacted. +async function fetchGuarded(rawUrl, lookup, fetchImpl, maxRedirects = 3) { + let target = rawUrl; + for (let hop = 0; hop <= maxRedirects; hop++) { + await assertPublicUrl(target, lookup); // validate before every request, including each redirect + const response = await fetchImpl(target, {redirect: 'manual'}); + if (response.status >= 300 && response.status < 400) { + const location = response.headers.get('location'); + if (!location) throw new Error(`upload_image: redirect with no Location header from ${target}`); + target = new URL(location, target).toString(); // resolve relative redirects against current URL + continue; + } + return response; + } + throw new Error(`upload_image: too many redirects (> ${maxRedirects})`); +} + export const uploadImageHandler = async (args, {lookup = defaultLookup, fetchImpl = fetch} = {}) => { - const {url, post_id} = uploadImageSchema.parse(args); + logger.debug('upload_image.start', {args}); - await assertPublicUrl(url, lookup); + let validatedArgs; + try { + validatedArgs = uploadImageSchema.parse(args); + } catch (error) { + logger.error('upload_image.args.invalid', {issues: error.issues ?? error.message}); + throw error; + } + const {url, post_id} = validatedArgs; logger.info('upload_image.fetching', {url, post_id: post_id ?? null}); - const response = await fetchImpl(url); + const response = await fetchGuarded(url, lookup, fetchImpl); if (!response.ok) { throw new Error(`upload_image: source responded ${response.status} ${response.statusText}`); } diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index 4006197..0d4ef50 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -4,7 +4,6 @@ import {http, HttpResponse} from 'msw'; import {uploadImageHandler, uploadImageSchema, MAX_IMAGE_BYTES, isPrivateAddress} from './upload_image.js'; import {createMswServer, IMAGE_URL, IMAGE_UPLOAD_RESPONSE} from '../../test/helpers/msw-server.js'; import {setTestEnv} from '../../test/helpers/env.js'; -import {captureLogs} from '../../test/helpers/capture-logs.js'; const msw = createMswServer(); let restoreEnv; @@ -57,3 +56,23 @@ describe('uploadImageHandler — happy path', () => { assert.equal(upload.body.postId, 7); }); }); + +describe('uploadImageHandler — redirect SSRF guard', () => { + test('rejects a redirect to a private address without uploading', async () => { + const REDIRECTOR = 'https://images.example.com/redirect.jpg'; + const INTERNAL = 'http://metadata.internal/latest'; + // First host resolves public, the redirect target resolves private. + const lookup = async (hostname) => + hostname === 'metadata.internal' + ? [{address: '169.254.169.254', family: 4}] + : [{address: '93.184.216.34', family: 4}]; + msw.server.use( + http.get(REDIRECTOR, () => new HttpResponse(null, {status: 302, headers: {Location: INTERNAL}})) + ); + await assert.rejects( + uploadImageHandler({url: REDIRECTOR}, {lookup, fetchImpl: fetch}), + /private\/loopback/ + ); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); +}); From 43dff30ce4c149fb083896db160528a46cad2734 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:42:03 +0200 Subject: [PATCH 06/12] Test upload_image content-type and HEIC rejection --- src/tools/upload_image.spec.js | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index 0d4ef50..690fdf5 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -57,6 +57,20 @@ describe('uploadImageHandler — happy path', () => { }); }); +describe('uploadImageHandler — content validation', () => { + test('rejects a non-image source before uploading', async () => { + msw.server.use(sourceHandler({body: Buffer.from(''), type: 'text/html'})); + await assert.rejects(run({url: SOURCE}), /not an image/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); + + test('rejects HEIC with a convert message', async () => { + msw.server.use(sourceHandler({type: 'image/heic'})); + await assert.rejects(run({url: SOURCE}), /HEIC is not accepted/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); +}); + describe('uploadImageHandler — redirect SSRF guard', () => { test('rejects a redirect to a private address without uploading', async () => { const REDIRECTOR = 'https://images.example.com/redirect.jpg'; From 344a98171ff1c06430e4948e5e1dc7b69537111b Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:43:05 +0200 Subject: [PATCH 07/12] Test upload_image scheme allow-list and DNS-resolved SSRF guard --- src/tools/upload_image.spec.js | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index 690fdf5..5b2c9e9 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -90,3 +90,36 @@ describe('uploadImageHandler — redirect SSRF guard', () => { assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); }); }); + +describe('isPrivateAddress', () => { + test('flags loopback, private, link-local, unique-local; allows public', () => { + assert.equal(isPrivateAddress('127.0.0.1', 4), true); + assert.equal(isPrivateAddress('10.1.2.3', 4), true); + assert.equal(isPrivateAddress('172.16.0.1', 4), true); + assert.equal(isPrivateAddress('192.168.1.1', 4), true); + assert.equal(isPrivateAddress('169.254.169.254', 4), true); + assert.equal(isPrivateAddress('93.184.216.34', 4), false); + assert.equal(isPrivateAddress('::1', 6), true); + assert.equal(isPrivateAddress('fe80::1', 6), true); + assert.equal(isPrivateAddress('fd00::1', 6), true); + assert.equal(isPrivateAddress('::ffff:127.0.0.1', 6), true); + assert.equal(isPrivateAddress('2606:2800:220:1:248:1893:25c8:1946', 6), false); + }); +}); + +describe('uploadImageHandler — SSRF and scheme guards', () => { + test('rejects a host that resolves to a private address, without fetching', async () => { + let fetched = false; + const fetchImpl = async () => { fetched = true; return new HttpResponse(); }; + const privateLookup = async () => [{address: '169.254.169.254', family: 4}]; + await assert.rejects( + uploadImageHandler({url: 'http://metadata.internal/'}, {lookup: privateLookup, fetchImpl}), + /private\/loopback/ + ); + assert.equal(fetched, false); + }); + + test('rejects a non-http(s) scheme up front', async () => { + await assert.rejects(run({url: 'ftp://example.com/x.png'}), /only http and https/); + }); +}); From effbd884e7e5bbbbbdc339137eb045f28ba59f88 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:43:27 +0200 Subject: [PATCH 08/12] Test upload_image size cap at MAX_IMAGE_BYTES --- src/tools/upload_image.spec.js | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index 5b2c9e9..6571259 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -123,3 +123,19 @@ describe('uploadImageHandler — SSRF and scheme guards', () => { await assert.rejects(run({url: 'ftp://example.com/x.png'}), /only http and https/); }); }); + +describe('uploadImageHandler — size cap', () => { + test('rejects an image over MAX_IMAGE_BYTES before uploading', async () => { + const big = Buffer.alloc(MAX_IMAGE_BYTES + 1, 0xff); + msw.server.use(sourceHandler({body: big, type: 'image/png'})); + await assert.rejects(run({url: SOURCE}), /over the .* limit/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); + + test('accepts an image exactly at the limit', async () => { + const atLimit = Buffer.alloc(MAX_IMAGE_BYTES, 0xff); + msw.server.use(sourceHandler({body: atLimit, type: 'image/png'})); + const result = await run({url: SOURCE}); + assert.equal(result.url, IMAGE_UPLOAD_RESPONSE.url); + }); +}); From 60ddb04648d831e85471a1d9e1ea8681f1f2091f Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:47:26 +0200 Subject: [PATCH 09/12] Test upload_image logs intent; elide base64 data URIs in the logger MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The tool's own lines never carry the data URI, but SubstackApi logs every request body at info — so a real image upload would put hundreds of KB of base64 on one line. The logger now elides a base64 data URI's payload, keeping the mime prefix and the omitted length; a post body, being prose, is still logged in full. This is what makes "the data URI is never logged" true end to end, not just in the tool. Co-Authored-By: Claude Opus 4.8 --- src/logger.js | 24 ++++++++++++++++++++++++ src/logger.spec.js | 23 +++++++++++++++++++++++ src/tools/upload_image.spec.js | 21 +++++++++++++++++++++ 3 files changed, 68 insertions(+) diff --git a/src/logger.js b/src/logger.js index 1b02ab4..4636271 100644 --- a/src/logger.js +++ b/src/logger.js @@ -20,6 +20,26 @@ const SECRET_KEY = /token|cookie|password|secret|auth|session|^sid$/i; const REDACTED = '***'; +// A base64 data URI is an opaque blob, useless in a log and ruinous to its volume: a single +// image upload carries its body through `substack.request` at info, which for a real photo is +// hundreds of KB of base64 on one line. Unlike a post body — which is prose and deliberately +// logged in full — the payload here has no diagnostic value, so it is elided while the prefix +// (mime type, that it is base64) and the elided length are kept. Matched on the value, since a +// data URI can sit under any key (`image`, `src`, …). +const DATA_URI_PREFIX = /^data:[^,]*;base64,/i; + +function truncateDataUri(value) { + const match = value.match(DATA_URI_PREFIX); + if (!match) { + return value; + } + const payload = value.length - match[0].length; + if (payload <= 32) { + return value; + } + return `${match[0]}…(${payload} base64 chars omitted)`; +} + // Read at call time, not at import: SUBSTACK_MCP_LOG_LEVEL may be set after this module is // imported (tests do exactly that), and no module here may read the environment at import // time. An unknown value falls back to the default rather than silencing the server. @@ -58,6 +78,10 @@ function redact(value, seen = new WeakSet()) { return expanded; } + if (typeof value === 'string') { + return truncateDataUri(value); + } + if (value === null || typeof value !== 'object') { return value; } diff --git a/src/logger.spec.js b/src/logger.spec.js index 9a3c28b..cec6b15 100644 --- a/src/logger.spec.js +++ b/src/logger.spec.js @@ -230,6 +230,29 @@ describe('logger — redaction', () => { assert.equal(line.args.body, body); }); + + // A base64 data URI is an opaque blob with no diagnostic value, and a real image upload would + // put hundreds of KB of it on one line through `substack.request`. The prefix and elided size + // are kept; the payload is dropped. Matched on the value, so it works under any key. + test('elides the payload of a base64 data URI, keeping the prefix', () => { + process.env.SUBSTACK_MCP_LOG_LEVEL = 'info'; + + const payload = 'Zm9v'.repeat(5000); + const image = `data:image/png;base64,${payload}`; + const [line] = logLines(() => logger.info('substack.request', {body: {image}})); + + assert.match(line.body.image, /^data:image\/png;base64,…\(\d+ base64 chars omitted\)$/); + assert.equal(JSON.stringify(line).includes(payload), false); + }); + + test('leaves a short data URI intact', () => { + process.env.SUBSTACK_MCP_LOG_LEVEL = 'info'; + + const image = 'data:image/gif;base64,R0lGODlhAQABAAAAACw='; + const [line] = logLines(() => logger.info('substack.request', {body: {image}})); + + assert.equal(line.body.image, image); + }); }); describe('logger — resilience', () => { diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index 6571259..c750309 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -4,6 +4,7 @@ import {http, HttpResponse} from 'msw'; import {uploadImageHandler, uploadImageSchema, MAX_IMAGE_BYTES, isPrivateAddress} from './upload_image.js'; import {createMswServer, IMAGE_URL, IMAGE_UPLOAD_RESPONSE} from '../../test/helpers/msw-server.js'; import {setTestEnv} from '../../test/helpers/env.js'; +import {captureLogs} from '../../test/helpers/capture-logs.js'; const msw = createMswServer(); let restoreEnv; @@ -139,3 +140,23 @@ describe('uploadImageHandler — size cap', () => { assert.equal(result.url, IMAGE_UPLOAD_RESPONSE.url); }); }); + +describe('uploadImageHandler — logging', () => { + test('logs intent before the request and never logs the data URI payload', async () => { + const body = Buffer.alloc(5000, 0xab); + const payload = body.toString('base64'); + msw.server.use(sourceHandler({body, type: 'image/png'})); + + // captureLogs returns the parsed log lines directly as an array, at debug level — where the + // API layer logs the full request body, so this proves the data URI is elided end to end. + const logs = await captureLogs(() => run({url: SOURCE})); + const events = logs.map((l) => l.msg); + + assert.ok(events.includes('upload_image.fetching')); + assert.ok(events.includes('upload_image.uploading')); + // The base64 payload must never appear in any log line — not the tool's, not the API's. + assert.equal(JSON.stringify(logs).includes(payload), false); + // But the request line still exists (truncated), proving logging was elided, not dropped. + assert.ok(logs.some((l) => l.msg === 'substack.request')); + }); +}); From fc696500f8f346ac13d93c0e1a39cd45983fc395 Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:49:50 +0200 Subject: [PATCH 10/12] Register upload_image in the tools registry Adds the import and registry entry, plus the README section the sync test requires, and updates the two hardcoded tool lists (server.spec, index.spec) and the set_post_body note that claimed images could not be uploaded. Co-Authored-By: Claude Opus 4.8 --- README.md | 23 ++++++++++++++++++++++- src/index.spec.js | 1 + src/server.js | 9 +++++++++ src/server.spec.js | 10 ++++++++++ 4 files changed, 42 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 07363c4..3bb0b33 100644 --- a/README.md +++ b/README.md @@ -141,13 +141,34 @@ asked for a paywall can confirm there is one. Validation cannot report a node th Three things worth knowing: - **An image must already be hosted by Substack.** `image2.src` pointing at an external URL is - stored but does not render, and this server cannot upload one. + stored but does not render. Use `upload_image` to re-host one and get a `src` that works. - **A document may contain at most one `paywall`.** Substack accepts two and renders both, leaving it undefined which one cuts the post, so this tool refuses the second. - **`ordered_list` numbers from `attrs.order`, not `attrs.start`.** A list given only `start` renders from 1 with no error. +
+upload_image - Re-host an external image on Substack + +Substack's editor uploads images as base64 data URIs to `POST /api/v1/image`, which answers with a +Substack-hosted URL. `image2.src` in `set_post_body` only renders such a URL, so this tool is the +bridge: give it an http(s) image URL, it downloads the image, re-encodes it, uploads it, and returns +the hosted URL. Substack itself only re-fetches URLs already in its own storage, so the download +happens here rather than being handed off. + +**Inputs**: +- `url` (string): the http(s) URL of an image to upload +- `post_id` (number, optional): the post the image belongs to; its effect is unconfirmed + +**Returns**: `{id, url, content_type, bytes, width, height}` — put `url` into an `image2.src` when +calling `set_post_body`. + +The download is guarded: only `http`/`https`, private and loopback hosts are refused after DNS +resolution (redirects are re-checked at every hop), the content type must be an image, HEIC is +rejected with a note to convert it, and the image may not exceed 10 MB. +
+
update_draft - Change a draft's title, subtitle or audience diff --git a/src/index.spec.js b/src/index.spec.js index 6ec0342..73f74e5 100644 --- a/src/index.spec.js +++ b/src/index.spec.js @@ -157,6 +157,7 @@ describe('entrypoint — stdio transport', () => { 'restack_item', 'set_post_body', 'update_draft', + 'upload_image', ]); assert.deepEqual( diff --git a/src/server.js b/src/server.js index 7aed71f..964318e 100644 --- a/src/server.js +++ b/src/server.js @@ -1,6 +1,7 @@ import {McpServer} from "@modelcontextprotocol/sdk/server/mcp.js"; import {createDraftPostSchema, createDraftPostHandler} from "./tools/create_draft_post.js"; import {setPostBodySchema, setPostBodyHandler} from "./tools/set_post_body.js"; +import {uploadImageSchema, uploadImageHandler} from "./tools/upload_image.js"; import {listSubscribersSchema, listSubscribersHandler} from "./tools/list_subscribers.js"; import {exportSubscribersSchema, exportSubscribersHandler} from "./tools/export_subscribers.js"; import {listPostsSchema, listPostsHandler} from "./tools/list_posts.js"; @@ -46,6 +47,14 @@ export const tools = { schema: setPostBodySchema, handler: setPostBodyHandler, }, + upload_image: { + description: + "Upload an image to your Substack publication from an http(s) URL. The server downloads " + + "the image and re-hosts it on Substack; the returned url is what goes into image2.src in " + + "set_post_body. Private and loopback hosts are refused, HEIC is not accepted, max 10 MB.", + schema: uploadImageSchema, + handler: uploadImageHandler, + }, list_subscribers: { // The engagement caveat still belongs in the description: this endpoint takes the fields it // returns from the publication's saved Display settings and ignores a per-request column list, diff --git a/src/server.spec.js b/src/server.spec.js index 1c2b31b..4c5737a 100644 --- a/src/server.spec.js +++ b/src/server.spec.js @@ -66,6 +66,7 @@ describe('MCP server — list_tools', () => { 'restack_item', 'set_post_body', 'update_draft', + 'upload_image', ]; test('exposes exactly the registered tools', async () => { @@ -93,6 +94,15 @@ describe('MCP server — list_tools', () => { assert.equal(inputSchema.properties.title.type, 'string'); }); + test('upload_image advertises url + optional post_id and forbids extra keys', async () => { + const {inputSchema} = (await listToolsByName()).upload_image; + + assert.equal(inputSchema.type, 'object'); + assert.deepEqual(Object.keys(inputSchema.properties).sort(), ['post_id', 'url']); + assert.deepEqual([...(inputSchema.required ?? [])], ['url']); + assert.equal(inputSchema.additionalProperties, false); + }); + // Regression guard for the zod 3 -> 4 migration: zod-to-json-schema silently returned a // bare `{$schema}` for a zod 4 schema instead of throwing, which would have published a // parameterless tool. Asserting the descriptions — the part an LLM actually reads to fill From a66f71d6cca00539d5ea9b341efdcb7e1c4c88bc Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 13:50:51 +0200 Subject: [PATCH 11/12] Correct the docs: images can be uploaded via upload_image MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Rewrites the "cannot be uploaded / do not implement" record with what was measured live on 2026-08-08 — the data-URI shape, the S3-only server-fetch, the guards, the logger elision, and the header-auth caveat — and updates the fork-survey note that called image-in-post impossible. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 42 +++++++++++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index ee81dac..c1d1181 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -281,11 +281,12 @@ against `main` again on 2026-08-07, after #18: five are zero-commit snapshots, ` test this repo has in English *plus* the logging assertions), and `jcllobet`'s reader work went in with #18. Only `jefflee1990710` still holds anything, and it is one coherent area rather than a list: **account and publication settings** — `PUT /api/v1/publication` (`name`, `hero_text`, `copyright`, -`email_from_name`, `logo_url`), `PUT substack.com/api/v1/user/profile` (`name`, `bio`, `photo_url`), -and the `POST substack.com/api/v1/image` upload both depend on, since per the fork an external url is -stored but does not render. That upload is also the only route to an image *inside* a post, which -nothing here can do. All three are unverified writes from the fork that invented `share_automatically`, -so none of it ships without a live check first; they are open by choice, not by oversight. +`email_from_name`, `logo_url`) and `PUT substack.com/api/v1/user/profile` (`name`, `bio`, `photo_url`). +Per the fork these store an external `logo_url`/`photo_url` that does not render, so they likely need a +Substack-hosted asset first — which `upload_image` now produces (`POST /api/v1/image`, verified and +shipped; see the image contract below). Both settings writes are still unverified writes from the fork +that invented `share_automatically`, so neither ships without a live check first; they are open by +choice, not by oversight. **Writes log their intent at `info` *before* the request**, not only their outcome — `publish_draft.publishing`, `comment_on_post.posting`, `restack_item.restacking`, with the full text @@ -328,12 +329,31 @@ nothing and closed the last route that accepted a body unchecked. Six measured f `plaintext` the plain-text value, and an unrecognised name renders as Plain Text with no error, so omitting the attr beats guessing. -**Images can be referenced but not uploaded**, which is the contract's sharpest limit: `captionedImage` -is in 60 of 60 sampled posts and `image2.src` must already point at a Substack-hosted asset. -`POST /api/v1/image` was tried on the publication host as JSON, as form-urlencoded and as multipart — -**all three hang**, the network log showing the request pending past a minute with no response, and the -cross-origin `substack.com` attempt never settled either. So neither `python-substack`'s signature nor -the fork's is confirmed. **Do not implement an upload from either.** +**Images can be uploaded after all, and `upload_image` is how** — this once read "cannot be uploaded, +`POST /api/v1/image` hangs in all three encodings, do not implement." That record was wrong. Re-measured +live 2026-08-08 on `implementing.substack.com` from the authenticated dashboard: the endpoint answers +**200 in ~300ms**. The three earlier attempts (JSON, form-urlencoded, multipart) failed because they sent +the wrong *thing*, not for a header detail or a Cloudflare wall — the body is JSON `{image: +"data:;base64,…"}`, a **data URI**, built in the editor from `canvas.toDataURL()`. The response is +`{id, url, contentType, bytes, imageWidth, imageHeight}`, `url` on `substack-post-media.s3.amazonaws.com` +— the host every `image2.src` uses — and it renders through Substack's CDN (proven end to end on a real +draft: upload → `captionedImage` → PUT → the editor shows a `substackcdn.com/image/fetch/…` render). +Two measured facts shape the tool: +- **Substack server-fetches only its own S3 bucket.** An external URL passed as `image` answers + `400 "Failed to fetch image"`, so `upload_image` downloads the URL itself and re-encodes it. That + download is the one place this server fetches a caller-chosen host, so it is guarded: `http(s)` only, + an SSRF block on private/loopback/link-local addresses *after* DNS resolution and re-checked on every + redirect hop, an `image/*` content-type check (HEIC refused early, as the dashboard does), and a 10 MB + cap that is **ours, not Substack's** — the bundle's `MAX_FILE_SIZE` could not be read from the minified + source. +- **The data URI is elided in the logger, not just kept out of the tool's own lines.** `SubstackApi` logs + every request body at info, so a real upload would put hundreds of KB of base64 on one line; + `src/logger.js` truncates a `data:…;base64,` value to its prefix and omitted length. A post body, being + prose, is still logged in full — the two are different in kind. + +**Still unverified:** every live check used the browser session cookie, not `SUBSTACK_SESSION_TOKEN` in a +header. Equivalent in principle, unconfirmed through `SubstackApi` — the first thing to check if the tool +misbehaves against the real API. **`set_post_body` returns a node tally, not `'OK'`**, because validation cannot report what was never sent: a document with no paywall is exactly as valid as one with a paywall. This was measured — a model From ba0fd5745b2e9491266dce71a8dcc5e458a7d88d Mon Sep 17 00:00:00 2001 From: Marco Moauro Date: Sat, 8 Aug 2026 14:08:02 +0200 Subject: [PATCH 12/12] Harden upload_image from the final review: IPv6, size, timeout Closes a latent SSRF hole and two robustness gaps the whole-feature review surfaced: - IPv6-mapped host in the compressed hex form the URL parser emits (::ffff:a9fe:a9fe = 169.254.169.254) is now caught; the dotted-only regex missed it. IPv6 hosts are de-bracketed before the DNS lookup, which also fixes IPv6 URLs erroring before the guard. - The size cap is enforced against a declared Content-Length before the body is read, then against the buffered length; an unbounded chunked body is bounded by a new per-request fetch timeout. - HEIC -sequence variants join the reject set. Security-critical additions are mutation-verified. Also corrects the CLAUDE.md claim that create_draft_post is the only reader of ZodError.issues. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 9 +++-- src/tools/upload_image.js | 68 ++++++++++++++++++++++++-------- src/tools/upload_image.spec.js | 72 ++++++++++++++++++++++++++++++++++ 3 files changed, 129 insertions(+), 20 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c1d1181..0f0de18 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -507,10 +507,11 @@ literals (`{a: 1}`, not `{ a: 1 }`). SDK, and `registerTool` throws `inputSchema must be a Zod schema or raw shape` for anything else — the SDK's validation *is* zod. npm auto-installs it even if it leaves `package.json`, so removing the direct dependency buys nothing and unpins the version. -- **`ZodError` details live on `.issues`, not `.errors`** (zod 4 renamed it). The only reader in - `src/` is the `create_draft_post.args.invalid` log — the SDK formats the message it sends to - the client — and `.errors` silently yields `undefined` rather than failing, so a handler that - inspects them logs nothing and reports no error. +- **`ZodError` details live on `.issues`, not `.errors`** (zod 4 renamed it). The readers in + `src/` are the `*.args.invalid` logs (`create_draft_post`, `upload_image`, and the other tools + that parse in a try/catch) — the SDK formats the message it sends to the client — and `.errors` + silently yields `undefined` rather than failing, so a handler that inspects them logs nothing and + reports no error. ## Verifying the server actually works diff --git a/src/tools/upload_image.js b/src/tools/upload_image.js index 5047492..a5c73fb 100644 --- a/src/tools/upload_image.js +++ b/src/tools/upload_image.js @@ -3,12 +3,18 @@ import dns from "node:dns"; import SubstackApi from "../api/substack/SubstackApi.js"; import {logger} from "../logger.js"; -// Caps what we hold and re-encode: an oversized body is still buffered once by fetch, but we -// reject it before the ~1.37x base64 copy and the upload. NOT Substack's own limit (its -// MAX_FILE_SIZE could not be read from the minified bundle). +// Checked against a declared Content-Length before the body is read, then against the buffered +// length. NOT Substack's own limit (its MAX_FILE_SIZE could not be read from the minified bundle). export const MAX_IMAGE_BYTES = 10 * 1024 * 1024; -const HEIC_TYPES = new Set(['image/heic', 'image/heif']); +// A caller-chosen host is untrusted: without a deadline a slow or stalled response hangs the tool +// call indefinitely. Each request (and each redirect hop) gets its own. +const FETCH_TIMEOUT_MS = 20000; + +// heic/heif plus the `-sequence` variants some Apple devices send for burst and live photos: all +// four start with `image/`, so without this they would pass the image check and fail later at +// Substack instead of getting the friendlier convert-first message. +const HEIC_TYPES = new Set(['image/heic', 'image/heif', 'image/heic-sequence', 'image/heif-sequence']); // strictObject: an unknown key is reported, never stripped — the only repair signal an LLM gets. export const uploadImageSchema = z.strictObject({ @@ -45,20 +51,38 @@ export function isPrivateAddress(address, family) { if (a === '::1' || a === '::') return true; if (a.startsWith('fe8') || a.startsWith('fe9') || a.startsWith('fea') || a.startsWith('feb')) return true; // fe80::/10 if (a.startsWith('fc') || a.startsWith('fd')) return true; // fc00::/7 unique-local - // IPv4-mapped IPv6 (::ffff:x.x.x.x) is unwrapped; other embeddings (6to4/Teredo/NAT64) are not — - // accepted residual risk. - const mapped = a.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); - if (mapped) return isPrivateAddress(mapped[1], 4); + // IPv4-mapped IPv6 is unwrapped and re-checked as v4; other embeddings (6to4/Teredo/NAT64) are + // not — accepted residual risk. + const mapped = embeddedIpv4(a); + if (mapped) return isPrivateAddress(mapped, 4); return false; } +// The trailing IPv4 of an IPv4-mapped IPv6 address, in either the dotted form (`::ffff:1.2.3.4`) or +// the compressed hex form (`::ffff:102:304`) the WHATWG URL parser emits — the latter is why the +// dotted-only regex was an SSRF hole: `http://[::ffff:169.254.169.254]/` reaches this as +// `::ffff:a9fe:a9fe`, the metadata address wearing a disguise. +function embeddedIpv4(address) { + const dotted = address.match(/^::ffff:(\d+\.\d+\.\d+\.\d+)$/); + if (dotted) return dotted[1]; + const hex = address.match(/^::ffff:([0-9a-f]{1,4}):([0-9a-f]{1,4})$/); + if (hex) { + const hi = parseInt(hex[1], 16); + const lo = parseInt(hex[2], 16); + return `${hi >> 8}.${hi & 0xff}.${lo >> 8}.${lo & 0xff}`; + } + return null; +} + async function assertPublicUrl(rawUrl, lookup) { // The schema's .url() already guaranteed this parses. const url = new URL(rawUrl); if (url.protocol !== 'http:' && url.protocol !== 'https:') { throw new Error(`upload_image: only http and https URLs are allowed, got ${url.protocol}`); } - const addresses = await lookup(url.hostname); + // An IPv6 host arrives bracket-wrapped (`[::1]`); dns.lookup and the address checks want it bare. + const hostname = url.hostname.replace(/^\[/, '').replace(/\]$/, ''); + const addresses = await lookup(hostname); for (const {address, family} of addresses) { if (isPrivateAddress(address, family)) { throw new Error(`upload_image: refusing to fetch a private/loopback address (${address})`); @@ -75,7 +99,7 @@ async function fetchGuarded(rawUrl, lookup, fetchImpl, maxRedirects = 3) { let target = rawUrl; for (let hop = 0; hop <= maxRedirects; hop++) { await assertPublicUrl(target, lookup); // validate before every request, including each redirect - const response = await fetchImpl(target, {redirect: 'manual'}); + const response = await fetchImpl(target, {redirect: 'manual', signal: AbortSignal.timeout(FETCH_TIMEOUT_MS)}); if (response.status >= 300 && response.status < 400) { const location = response.headers.get('location'); if (!location) throw new Error(`upload_image: redirect with no Location header from ${target}`); @@ -87,6 +111,23 @@ async function fetchGuarded(rawUrl, lookup, fetchImpl, maxRedirects = 3) { throw new Error(`upload_image: too many redirects (> ${maxRedirects})`); } +// Enforces the cap in two places. A declared Content-Length over the limit is refused before the +// body is read at all — the cheap common case. The buffered length is then re-checked, since a +// response may declare a small (or no) length and send more. A response that both omits its length +// AND streams unboundedly is bounded not by the byte cap but by FETCH_TIMEOUT_MS on the request — +// an accepted residual, the same shape of trade-off as the DNS-rebinding note. +async function readCapped(response, max) { + const declared = Number(response.headers.get('content-length')); + if (declared > max) { + throw new Error(`upload_image: image is ${declared} bytes (Content-Length), over the ${max}-byte limit.`); + } + const buffer = Buffer.from(await response.arrayBuffer()); + if (buffer.byteLength > max) { + throw new Error(`upload_image: image is ${buffer.byteLength} bytes, over the ${max}-byte limit.`); + } + return buffer; +} + export const uploadImageHandler = async (args, {lookup = defaultLookup, fetchImpl = fetch} = {}) => { logger.debug('upload_image.start', {args}); @@ -113,12 +154,7 @@ export const uploadImageHandler = async (args, {lookup = defaultLookup, fetchImp throw new Error('upload_image: HEIC is not accepted by Substack. Convert to JPG or PNG first.'); } - const buffer = Buffer.from(await response.arrayBuffer()); - if (buffer.byteLength > MAX_IMAGE_BYTES) { - throw new Error( - `upload_image: image is ${buffer.byteLength} bytes, over the ${MAX_IMAGE_BYTES}-byte limit.` - ); - } + const buffer = await readCapped(response, MAX_IMAGE_BYTES); const image = `data:${contentType};base64,${buffer.toString('base64')}`; // The data URI is deliberately NOT logged: hundreds of KB of base64 would bury the session. This diff --git a/src/tools/upload_image.spec.js b/src/tools/upload_image.spec.js index c750309..09f31b5 100644 --- a/src/tools/upload_image.spec.js +++ b/src/tools/upload_image.spec.js @@ -70,6 +70,12 @@ describe('uploadImageHandler — content validation', () => { await assert.rejects(run({url: SOURCE}), /HEIC is not accepted/); assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); }); + + test('rejects the HEIC -sequence variant too', async () => { + msw.server.use(sourceHandler({type: 'image/heic-sequence'})); + await assert.rejects(run({url: SOURCE}), /HEIC is not accepted/); + assert.equal(msw.requests.find((r) => r.url.endsWith('/api/v1/image')), undefined); + }); }); describe('uploadImageHandler — redirect SSRF guard', () => { @@ -106,6 +112,14 @@ describe('isPrivateAddress', () => { assert.equal(isPrivateAddress('::ffff:127.0.0.1', 6), true); assert.equal(isPrivateAddress('2606:2800:220:1:248:1893:25c8:1946', 6), false); }); + + // The URL parser compresses an IPv4-mapped host to hex, so the dotted-only check was an SSRF + // hole: these are 169.254.169.254 and 127.0.0.1 in the form isPrivateAddress actually receives. + test('flags an IPv4-mapped address written in the compressed hex form', () => { + assert.equal(isPrivateAddress('::ffff:a9fe:a9fe', 6), true); + assert.equal(isPrivateAddress('::ffff:7f00:1', 6), true); + assert.equal(isPrivateAddress('::ffff:5db8:d822', 6), false); // 93.184.216.34, public + }); }); describe('uploadImageHandler — SSRF and scheme guards', () => { @@ -123,6 +137,64 @@ describe('uploadImageHandler — SSRF and scheme guards', () => { test('rejects a non-http(s) scheme up front', async () => { await assert.rejects(run({url: 'ftp://example.com/x.png'}), /only http and https/); }); + + test('de-brackets an IPv6 host before the lookup, and still guards it', async () => { + let seenHost; + const lookup = async (hostname) => { seenHost = hostname; return [{address: '::1', family: 6}]; }; + let fetched = false; + const fetchImpl = async () => { fetched = true; return new HttpResponse(); }; + await assert.rejects( + uploadImageHandler({url: 'http://[::1]/x.png'}, {lookup, fetchImpl}), + /private\/loopback/ + ); + assert.equal(seenHost, '::1'); // de-bracketed, not '[::1]' + assert.equal(fetched, false); + }); +}); + +describe('uploadImageHandler — redirect edge cases', () => { + const publicOnly = async () => [{address: '93.184.216.34', family: 4}]; + + test('rejects after too many redirects', async () => { + let n = 0; + const fetchImpl = async () => + new HttpResponse(null, {status: 302, headers: {Location: `https://images.example.com/n${n++}.jpg`}}); + await assert.rejects( + uploadImageHandler({url: 'https://images.example.com/a.jpg'}, {lookup: publicOnly, fetchImpl}), + /too many redirects/ + ); + }); + + test('rejects a redirect with no Location header', async () => { + const fetchImpl = async () => new HttpResponse(null, {status: 302}); + await assert.rejects( + uploadImageHandler({url: 'https://images.example.com/a.jpg'}, {lookup: publicOnly, fetchImpl}), + /no Location header/ + ); + }); +}); + +describe('uploadImageHandler — Content-Length pre-check', () => { + test('rejects a declared-oversize body before reading it', async () => { + let read = false; + const fetchImpl = async () => new Response('tiny', { + status: 200, + headers: {'Content-Type': 'image/png', 'Content-Length': String(MAX_IMAGE_BYTES + 1)}, + }); + // Wrap arrayBuffer so we can prove it was never called: the header alone must reject. + const orig = fetchImpl; + const spyingFetch = async (...a) => { + const r = await orig(...a); + const arrayBuffer = r.arrayBuffer.bind(r); + r.arrayBuffer = (...x) => { read = true; return arrayBuffer(...x); }; + return r; + }; + await assert.rejects( + uploadImageHandler({url: 'https://images.example.com/a.png'}, {lookup: publicLookup, fetchImpl: spyingFetch}), + /Content-Length.*over the .* limit/ + ); + assert.equal(read, false); + }); }); describe('uploadImageHandler — size cap', () => {