chore(release): promote rc-2026.8.3 - #180
Open
jacderida wants to merge 11 commits into
Open
Conversation
…t-cli 0.3.4-beta.1
Beta participants need their node logs in the beta-channel Elasticsearch so the release gates have evidence to judge a build on, and the only way to collect them today is for each user to hand-roll a Vector config against their node log directory. That does not scale past the first few technical users and it is not something we can ask a wider beta cohort to do. The daemon already supervises every node on the machine and already knows each one's log directory, so it is the natural place to do this: no Windows service, no MSI, no admin prompt, no systemd unit, no separate install, and cross-platform for free. `ant node logs forward enable --token <token>` is the consent act; `disable` stops the flow and changes nothing else about any node. Two properties drive the design. Forwarding must never slow a node down, so it only ever reads log files, on its own task, with a bounded drop-oldest queue and bounded retries — a lost batch is acceptable, a stalled or memory-hungry daemon is not. And a daemon restart must neither duplicate nor lose events, which is why tail offsets are persisted and every document carries a deterministic `_id` derived from node, file and byte offset. That `_id` is coupled to the index name, not independent of it: `_id` uniqueness is per index, so each document is filed under an index derived from its own `@timestamp` rather than the wall clock. Deriving the index from the wall clock would send a batch replayed after midnight to a different daily index, where the duplicate would be silently accepted instead of rejected with a 409. Notable contract details from V2-1016, all covered by tests: - the bulk action must be `create`, never `index` — the write key grants `create_doc`, and `index` comes back as a per-item 403 - a `_bulk` response is HTTP 200 even when documents failed; success is per position in `items[].status`, so trusting the HTTP status alone silently discards failures - at a position, 201 is created, 200 is dropped by the server-side level filter, and 409 is a document our own earlier attempt already landed — all three are successes and none is retried - `host` and `beta_user` are never sent: the ingest pipeline strips the first (hostnames routinely contain personal names) and stamps the second from the authenticated API key Node file logging remains off by default. `enable` forwards only nodes that already have a log directory and reports the ones it is skipping, pointing at `--log-dir-path`, rather than appearing to succeed while shipping nothing. - Add ant-core/src/node/daemon/forward/: config (0600 token file), line parsing for both the text and JSON log layouts, rotation-aware tailing with persisted offsets, document tagging, a bounded batching sink with per-position retry, the Elasticsearch bulk sink, and the background task - Add GET /api/v1/logs/forward and POST .../enable|disable, with OpenAPI paths and schemas - Add `ant node logs forward enable|disable|status`, dual-path so the opt-in is still recorded when the daemon is down - Update CLAUDE.md and the e2e node management skill The status API returns a token fingerprint, never the token itself. Test results: - cargo test -p ant-core --lib: 562 passed (105 new) - cargo test -p ant-core --test log_forward_integration: 8 passed, driven against a real HTTP endpoint speaking the bulk contract, covering restart resume, replay idempotency, daily rotation, multi-line events and index-by-event-date - daemon_integration 6, node_add_integration 3, datamap_file 20, merkle_unit 8, unit_self_encrypt 16, ant-cli 18 — all passed - cargo clippy --all-targets --all-features -- -D warnings: clean - cargo fmt --all -- --check: clean `data::client::adaptive::tests::controller_perf_overhead_is_bounded` fails under a fully parallel run on a loaded machine. It is pre-existing and unrelated: it fails identically with these tests excluded, passes in isolation, and adaptive.rs is untouched here. The beta endpoint is still being provisioned, so a smoke test against the real logs.autonomi.com is outstanding. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Whether forwarding is enabled before or after a node starts changes what gets shipped, and the difference is not obvious from the code: the join-at-end rule applies only to files that already existed when forwarding was switched on, so a node that has never run is read from its first byte while one already running is picked up from wherever it had got to. That matters more than it looks. ant-node reports its version, commit and peer id on its startup line, so enabling after the node is up costs those fields on every document in the batch, along with the bootstrap and listen-address lines that show whether the node actually joined. Both directions were verified by hand before being written down here; these tests stop a later change to the priming logic silently reversing either one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Beta participants have nowhere to learn two things they need to know, and both are the kind of detail that only announces itself once it has already cost someone a run. The first is that log forwarding is optional. It is opt-in telemetry from people's own machines, so the docs need to say plainly that running beta builds is the whole requirement and that declining to forward costs nothing — if it reads as expected, the consent that `enable` is supposed to represent stops meaning very much. The section is marked optional in its heading and its walkthrough is conditional on having chosen to turn it on. It still makes the case for saying yes, because an informed choice needs one, but leaves it a choice. The second is ordering. Enabling forwarding is forward-looking consent, so for a log file that already exists the daemon starts reading at the end of it. A node that has not started yet has no file, so the one it creates is read from the first line — meaning enable-then-start captures the node's startup line and start-then-enable silently skips it, losing the version, commit and peer id that say which build produced everything that follows. The same applies to --log-dir-path, which has to be set when a node is added: node file logging is off by default, and a node added without it writes nothing to forward. `enable` reports those nodes rather than failing quietly, but the fix is to re-add the node, so it is much better to get it right first time. Adds a Beta Programme section covering both, with the working order as a single copyable block and the reasoning below it, plus what is and is not sent, how to tell if delivery is failing, and how to stop — keeping turning forwarding off separate from leaving the beta channel, since they are unrelated actions. The existing beta channel content was filed under `ant update` although it is not an `ant update` subcommand; it moves here rather than being duplicated. Also adds the `ant node logs forward` command reference alongside the other subcommands, the three new REST endpoints to the API table, and the new modules to the project structure. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Installing the beta client meant downloading an archive from the releases page by hand, because the
installers could only ever fetch the newest stable build. That is a poor first step for a programme
whose whole point is getting more people onto the pre-release build, and it is the one part of beta
onboarding with no way around it.
Both installers now take ANT_CHANNEL=stable|beta, defaulting to stable so existing invocations are
unaffected. ANT_VERSION still overrides it.
The version resolution is the substance of this. /releases/latest cannot serve the beta channel at
all: GitHub excludes pre-releases from that endpoint, so it would always return the newest stable
build. Beta scans the release list instead, while stable keeps using /releases/latest, which already
means exactly the right thing.
Picking the highest pre-release from that list would be worse than not implementing this. Semver
ranks -rc above -beta, and both 0.3.4-beta.1 and 0.3.4-rc.1 exist right now, so the naive choice
installs a release candidate — code published before the release gates have reported. Both scripts
therefore mirror version_matches_channel from ant-core/src/channel.rs: final releases on either
channel, -beta.N additionally on beta, everything else rejected, matching the whole first identifier
so `betamax.1` is not caught by a prefix test.
Semver comparison is hand-rolled in both rather than delegated. `sort -V` does not implement the
rule that a pre-release ranks below the release it was cut from, and BSD and GNU builds disagree,
which matters because install.sh runs on macOS as well as Linux. PowerShell's [version] cannot parse
a pre-release suffix at all.
Download URLs and asset names needed no change: ant-cli-v0.3.4-beta.1 already publishes its assets
as ant-{version}-{target}.{tar.gz,zip}, so only resolution differed.
Note that the channel rule now lives in three places — these two scripts and channel.rs — with only
comments binding them together. Worth collapsing if it grows a fourth.
Test evidence:
- 19 logic cases in bash and 14 in PowerShell, both agreeing with channel.rs: -rc rejected on beta,
betamax rejected, 0.3.4 ranked above 0.3.4-beta.1, 0.10.0 above 0.9.0
- live resolution from both scripts: stable -> 0.3.3, beta -> 0.3.4-beta.1, i.e. beta correctly
preferred over the higher-ranked 0.3.4-rc.1
- install.sh run end to end against real GitHub releases into a temp prefix: ANT_CHANNEL=beta
installed a working ant 0.3.4-beta.1, and the default installed ant 0.3.3
- install.ps1 parses clean under pwsh and its helpers were exercised directly; its Windows-only
install body was not run
Also documents ANT_CHANNEL, ANT_VERSION and INSTALL_DIR in the Installation section, and replaces
the manual-download step in the beta walkthrough.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Addresses both blocking findings from review of 8658e16. Document ids could collide across installations. The id was built from node id, filename and byte offset, all of which are local values: every participant has a node 1, writing the same daily filename, whose first line starts at byte 0. Since the whole cohort writes into one shared beta-nodes-YYYY.MM.DD index and the sink counts a 409 as delivered, the second machine to send a given position had its event silently discarded. This was worse than a possibility. Offset 0 of each day's file is reached by every node on every machine, so on any given day the first event from node 1 collided across the entire cohort and exactly one won — and that first event is the startup line carrying version, commit and peer id, which is precisely the record the forwarding exists to collect. Ids are now prefixed with a random 64-bit installation namespace, minted on first enable and persisted alongside the opt-in. It is generated from random bytes rather than derived from hostname, MAC or username, so it separates installations without describing them, and it is deliberately stable: regenerating it would make a replayed batch look like new documents and duplicate them, which is the property the deterministic id exists to provide. Disabling did not stop delivery that was already under way. Cancellation was only observed between poll cycles, so a disable issued mid-flush kept uploading through the rest of the retry ladder — with the default policy, up to three 30s request timeouts plus backoff per batch, repeated for every batch left in the queue — while the CLI had already told the user forwarding had stopped. For a feature whose entire basis is opt-in consent, revocation has to mean something more definite. Cancellation now reaches the delivery loop: checked before taking each batch, and raced against the delivery itself, so the in-flight future is dropped and the HTTP request cancelled with it. The handle retains its JoinHandle and `stop_and_wait` awaits the task, so the disable endpoint returns only once the sender has actually stopped rather than merely having been signalled. Starting a replacement forwarder awaits the old one for the same reason, so two never overlap. Tests: - identical node/file/offset tuples from two installations produce different ids, and an integration test drives two forwarders through the shared mock endpoint to show both events are stored rather than one being swallowed as a conflict - the installation id is minted once, survives save/reload, and is unchanged by re-enabling or rotating the token - a blocking sink holds a request open across a disable: stop_and_wait returns promptly, the blocked send is confirmed never to have completed, and no request starts afterwards. Reverting the mid-delivery cancellation makes this test fail, so it pins the behaviour rather than describing it Full run: 570 lib tests, 9 log-forwarding integration tests, 6 daemon integration, 18 ant-cli, all passing; clippy -D warnings and fmt --check clean. Also documents the installation identifier and the disable guarantee in the README's beta section, since both are things a participant deciding whether to opt in should be told. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
feat: forward beta node logs from the daemon
…t-cli 0.3.4-beta.2
Resolves the merge-back conflict for the 0.3.4 / 0.7.0 promotion. Only README.md conflicted. Both sides added text at the same point, after the two install commands: `main` gained a note that the `curl | bash` installer is Linux/macOS only and WSL users want the PowerShell one (#169), and the rc branch gained the table of environment variables shared by both installers (#175). They are complementary, so both are kept — the note first, since it decides which installer to run, then the table that applies to either. Everything else merged cleanly, including both manifests and Cargo.lock. Verified after the merge that the promotion survived it: ant-core 0.7.0 and ant-cli 0.3.4, ant-protocol pinned to 2.3.3, both ant-node pins (optional devnet and dev-dependency) to 0.17.2, no `-rc.`/`-beta.` suffixes and no git+branch sources left in the lock. `cargo check --all-targets --all-features` passes against the published crates and left the lock unchanged. The release tags are unaffected: ant-core-v0.7.0 and ant-cli-v0.3.4 point at 7145a0d, which this merge builds on rather than rewrites.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotes
rc-2026.8.3to release version(s): 0.3.4,0.7.0.-rc.*from[package].versionCargo.lockOnce merged, the release tag will be pushed to fire the publish workflow.