fix(remote): pull a team larger than a command line, and say what it is doing (#882)

[fujibee]

Declared reviewers: 1

Closes #882 — both halves of it.

The one thing the reporter actually asked for, twice, was progress output. They
lost 79 minutes to a command that printed nothing, misdiagnosed it as a hang,
and carried that misdiagnosis into the report. Landing the size fix alone would
answer the failure they hit and not the reason they could not investigate it, so
the progress lines (previously #883, now closed) ride here.

Change class: user-affecting. A Windows machine cannot pull a team past
about four hundred messages today. This is the fix for that, not a CI change.

What was wrong

storage_sync_apply_pull builds outcome_ids — one '<wire-id>', per message
in the page — and embeds it twice in the SQL that reads the outcomes back,
which went to sqlite3 as an argument:

sqlite-sync.sh:903   outcome_ids="${outcome_ids}${outcome_ids:+,}'$wire'"
sqlite-sync.sh:1102  AND   wire_id IN (${outcome_ids:-''})
sqlite-sync.sh:1108  AND c.wire_id IN (${outcome_ids:-''})
sqlite.sh:34         _sqlite_data() { … agmsg_sqlite "$(_sqlite_db "$1")" "$2" … }
                                                                        ^^^^ argv

Windows caps a command line at 32,767 characters. 78 characters per message
— 39 for the id, twice — so the page stops fitting at a few hundred:

rendered argv SQL = 1537 + 78 x page size      (measured by building the string)

    100 messages     9,335
    400 messages    32,735
   1000 messages    79,535

Measured on a Windows machine by pm, against the real sqlite3:

413 messages   32,340 chars   pulls
418 messages                  does not

Nothing in the product chose that number. The team in the report had 2,079
messages and a default page of 1,000, so it failed on the first page — while
the same data on Linux (≈2MB limit) completed in 312 seconds, which is why
"same path, only the volume differs" had two explanations that fit.

The fix

Stdin has no such limit, and storage_sync_apply_pull already applies its page
that way (sqlite-sync.sh:1082) — that line is correct and is untouched. The
three statements whose length grows with the data move to stdin:

site	grows by
pull outcomes (_sqlite_data_stdin)	78 chars per message in the page
push acknowledgements	55 chars per acked message, 1000 in catch-up
roster read members	59 chars per member

Only the reported one was failing in the field. Fixing it alone would leave the
same defect in the two beside it, on paths that are one busy team away from the
same size.

Measured

tests/test_remote_sync.bats           32/32   (1 new)
tests/test_remote_status_liveness.bats 31/31
check-enforced-assertions              639 — unchanged from origin/main *

* main is one over its baseline today; that is #887, not this branch.

The new case measures the length of the command line, not whether an
operating system refuses it:

page of   5 messages -> longest sqlite3 argv: N
page of 120 messages -> longest sqlite3 argv: N + <200

A test that waited for the limit to be hit would be green on every machine that
runs CI, because the limit that broke this is Windows'. What must not happen is
growth per message, and that is observable anywhere. It also asserts the
larger page really imported (120 rows), so the comparison cannot pass by both
runs having applied nothing.

Reverting the driver turns it red:

without the fix:  [ "$long" -lt "$((short + 200))" ] failed

The other half: it says what it is doing

agmsg: [0s] pulling myteam from https://... -- this can take several minutes
agmsg: [1s] fetching messages after 0 (0 pulled so far)
agmsg: [9s] applying 1000 messages
agmsg: [140s] fetching messages after 1000 (1000 pulled so far)

stderr only. cmd_pull captures this process's stdout as the result channel
and greps it for pull_bootstrap_result, so a line written there would ride in
the stream the caller parses.

Two separate claims, with what backs each, because an earlier revision of this
body ran them together as "verified end-to-end":

• the fd wiring — remote-sync.sh:47 execs node with no redirection, and
  cmd_pull redirects only stdout. That is read from the source, not executed
  by a test.
• what the process writes — the committed cases call the exported
  pullBootstrap with injected dependencies and capture both streams. That is a
  unit measurement of the output, not of the pipeline it travels down.

I did also run the real CLI once with the two streams captured to separate
files, and it agreed. That was a one-off measurement and is not what any of the
green here rests on.

Fetching and applying are named separately because applying spawns a child
process per batch and fetching does not — when the output stops moving, the line
it stopped on says which half to look at. Elapsed seconds are stamped because a
terminal does not timestamp its own scrollback: without them, "it printed this
and then stopped" cannot say whether the stop was ten seconds or ten minutes.

progress written to stdout instead of stderr -> red (stdout 4 lines, expected 1)
the `applying N messages` line deleted       -> red
none (control)                               -> green, 96/96

What the progress line must not carry

A hosted endpoint is https://host/t/<token> and the token IS the capability.
The first version of this line printed the whole URL — into the output someone
pastes into an issue when a pull is taking too long, which is the entire reason
the line exists. It prints hostOf() now: new URL(...).host, already this
file's rule for the refusal record status shows, and already remote.sh's
rule in shell (_remote_endpoint_display). Nothing new was written to redact.

The cursor on the second line is the server's, and publicSnapshot validates
only the team id and the server instance id — min_available_seq reaches the
first progress line exactly as sent. Only a canonical sequence prints as itself.

hostOf() reverted to the full endpoint   -> red (capability, credential, query, fragment)
the cursor guard removed                 -> red (the escape sequence reaches stderr)
the guard replacing everything           -> red (a real sequence stops printing as itself)
none (control)                           -> green, 96/96

The malformed cursor is planted as min_available_seq rather than as a second
page's next_after: the real sqlite driver refuses a non-numeric
sync_pull_cursor (sqlite-sync.sh:891-897) before the loop comes round again,
so a fixture built there exercises a state only a stubbed driver produces.

The test captures both streams, because the regression worth catching is a
progress line landing in stdout, and that is invisible if only stderr is
measured. It asserts stdout is exactly one line and parses as
pull_bootstrap_result.

Still open

Field verification on a real Windows machine — pm has the hardware, and the
reproduction is now a size rather than an environment.