From 8e0415c12f5627ceadfd06b14d337c35f12120ea Mon Sep 17 00:00:00 2001 From: Harry Beckwith Date: Mon, 12 Oct 2026 14:13:10 -0700 Subject: [PATCH] Recognise a release's node as the reference; install programs as new files Trying the published v31.99-7 files the way a new Mac install uses them found two faults that need a real release to show. - The console did not recognise the release's ready-made node as its unmodified reference. A release build names itself by its tag ("v31.99-7"), and the console only understood a commit suffix. With no reference, every ready-made experiment asked for a compile first. The console now resolves either, and a ready-made node is installed with a note of the commit it was built from. Both are believed only while the build folder has never been shared with a build of changed source, and compiling there removes the note. - Installing a ready-made node over an earlier one overwrote the program in place. macOS on Apple silicon silently kills a program changed that way after it has been run. Programs are now put in place as new files, in the console and in the installer. The installer no longer says the first experiment compiles the node: the ready-made experiments do not. Verified against the published release files: checksum and commit match, the node is adopted as the reference, and the three changed nodes give their documented outcomes. Co-Authored-By: Claude Opus 5.5 --- contrib/vibes/QA.md | 44 +++++++++++++++++--- contrib/vibes/bitcoin-vibes | 64 +++++++++++++++++++++++------ contrib/vibes/install.sh | 11 +++-- contrib/vibes/tests/test_console.py | 61 +++++++++++++++++++++++++++ contrib/vibes/tests/test_install.py | 3 ++ 5 files changed, 162 insertions(+), 21 deletions(-) diff --git a/contrib/vibes/QA.md b/contrib/vibes/QA.md index 25f1d4d06686..f15330b5d596 100644 --- a/contrib/vibes/QA.md +++ b/contrib/vibes/QA.md @@ -69,7 +69,7 @@ checked. ## Automated suites -- 106 console tests and 17 installer tests (`python3 -m unittest discover -s +- 108 console tests and 17 installer tests (`python3 -m unittest discover -s contrib/vibes/tests`), all passing, including the tests that need a real node. See `tests/README.md` for what each group covers. - Browser checks of the landing page and demo in Chrome via Playwright: the @@ -189,6 +189,37 @@ own rules; tools from Homebrew on Linux may not run inside the sandbox; a shared request that needs invisible joiners (some scripts, some emoji) is refused and has to be retyped. +## The release, tried as a new install would use it + +The release workflow ran for the first time on `v31.99-7`. All three platforms +(macOS on Apple silicon, Linux on x86_64 and on ARM64) compiled the node, +passed Bitcoin Core's unit tests and passed this project's tests against the +node just built; on the Mac that includes the tests that probe the sandbox +from inside. It built the three changed nodes, checked that each disagrees +with the unmodified node in the documented direction, and published the +release only after every file was attached. + +The published files were then used on an Apple silicon Mac the way a new +install uses them: the installer's ready-made node step, and the console's +download of the node and of the three changed nodes. That found two faults +that no test had caught, because both need a real release to show: + +- The console did not recognise the release's node as its unmodified + reference. A release build names itself by its tag, and the console only + understood a commit. With no reference, every ready-made experiment asked + for a compile first. The console now understands both, and the installer + leaves a note of the commit beside the node. +- Installing a ready-made node over an earlier one overwrote the program in + place, and macOS on Apple silicon silently refuses to run a program changed + that way. Programs are now put in place as new files. + +`v31.99-7` was marked a pre-release so that it is not offered as the latest. +With both faults fixed, the same check against the same published files +passed: the checksum and commit matched, the node was adopted as the +reference, and the doubled-reward, halved-reward and earlier-halving nodes +gave "loosened", "tightened" and "tightened", with the rule-breaking-miner +scene refusing all thirty blocks. + ## Not checked - **A live AI session.** The agent's command line was accepted by Claude Code @@ -205,10 +236,13 @@ refused and has to be retyped. routes are closed. They are not an attempt by a specialist to find others. - **Docker.** CI builds the image and checks that the console answers. The review-first behaviour inside it is covered by a unit test only. -- **The Mac app**, the release workflow, signing and notarisation. None can run - outside GitHub Actions or without credentials. That includes the new step - that builds and ships the three changed nodes, and the console's download of - them, which is covered by tests against a local folder only. +- **The Mac app**, signing and notarisation. The release workflow builds the + app, but nobody has opened the one it built, and signing and notarisation + need credentials the project does not have yet. +- **A complete install from nothing.** The installer's ready-made node step + and the console's downloads were run against a real release. The rest of + the installer (installing build tools, fetching the source) was not run on + a clean machine, and the ready-made files were tried on a Mac only. - **Whether any of this teaches.** No learner has used it. The lessons have not been reviewed by a Bitcoin Core contributor or an educator. Both are things only people can do; `doc/vibes-study.md` and the "Reviewing the diff --git a/contrib/vibes/bitcoin-vibes b/contrib/vibes/bitcoin-vibes index c6565f97493f..73bc39e62a9a 100755 --- a/contrib/vibes/bitcoin-vibes +++ b/contrib/vibes/bitcoin-vibes @@ -404,6 +404,18 @@ SETTINGS = ( ) +def install_program(source, target): + """Copy a program into place as a new file, never over the old one. + + macOS on Apple silicon remembers the signature of a program it has run. + One that is overwritten in place no longer matches what was remembered + and is killed the moment it is started, with no message.""" + target = Path(target) + fresh = target.with_name(f".{target.name}.{os.getpid()}.new") + shutil.copy2(source, fresh) + os.replace(fresh, target) + + def run(cmd, cwd=None, timeout=60, env=None): """Run a command, return (returncode, stdout, stderr).""" try: @@ -577,7 +589,7 @@ class Console: for name in ("bitcoind", "bitcoin-cli"): binary = self.bin_path(name) if binary: - shutil.copy2(binary, bootstrap / name) + install_program(binary, bootstrap / name) self.build_state = {"path": str(bootstrap), "sha": None, "verified": False} self._write_json(self.state_dir / "build.json", self.build_state) @@ -1645,7 +1657,7 @@ class Console: if current.get("tree") == tree and (self.reference_dir / "bitcoind").exists(): return True self.reference_dir.mkdir(parents=True, exist_ok=True) - shutil.copy2(Path(release_dir) / "bitcoind", self.reference_dir / "bitcoind") + install_program(Path(release_dir) / "bitcoind", self.reference_dir / "bitcoind") self._write_json(self.reference_dir / "reference.json", {"tree": tree, "sha": sha, "time": time.time()}) if job: @@ -1653,18 +1665,35 @@ class Console: return True def _adopt_installed_reference(self): - """On first start, accept the installer's build as the reference if the - program itself reports that it was built from unmodified source.""" + """On first start, accept the installed node program as the reference + if it was built from unmodified source. + + Which source it was built from is read from the note a ready-made + node is installed with (the commit its release recorded, already + checked against this source when it was installed), or else from what + the program says about itself: a commit, or the release tag it was + built at. Either is believed only while the build folder has never + been shared with a build of changed source.""" if self.reference_state()["ready"] or self._build_folder_was_shared(): return bitcoind = self.bin_path("bitcoind") if not bitcoind: return - code, out, _ = self.ask_node_program(bitcoind, "-version") - found = re.search(r"version v[0-9][^\s-]*-([0-9a-f]{7,40})(-dirty)?\b", out) if code == 0 else None - if not found or found.group(2): - return - code, sha, _ = self.git("rev-parse", "--verify", "--quiet", found.group(1) + "^{commit}") + note, name = bitcoind.parent / "COMMIT", "" + try: + if note.is_file() and not note.is_symlink(): + name = note.read_text()[:100].strip() + except OSError: + name = "" + if not re.fullmatch(r"[0-9a-f]{40}", name): + code, out, _ = self.ask_node_program(bitcoind, "-version") + found = re.search(r"version (v[0-9][0-9A-Za-z._-]*)", out) if code == 0 else None + if not found or found.group(1).endswith("-dirty"): + return + # "v31.99.0-1a2b3c4d5e" names a commit; "v31.99-7" is a release tag. + commit = re.search(r"-([0-9a-f]{7,40})$", found.group(1)) + name = commit.group(1) if commit else "refs/tags/" + found.group(1) + code, sha, _ = self.git("rev-parse", "--verify", "--quiet", name + "^{commit}") if code == 0: self._capture_reference(sha.strip(), bitcoind.parent) @@ -2437,6 +2466,12 @@ class Console: raise RuntimeError("Your build folder is not an ordinary folder (it is a link or a file), so " "nothing is compiled there. If you did not put it there, a program built " "from changed source did. Remove it, then build again.") + # The note a ready-made node came with says nothing about what is + # compiled here from now on. + try: + (build_dir / "bin" / "COMMIT").unlink() + except OSError: + pass code = self._stream(job, ["cmake", "-B", "build", "-DBUILD_TESTS=ON", "-DBUILD_BENCH=OFF", "-DBUILD_FUZZ_BINARY=OFF", "-DBUILD_GUI=OFF"], write=[build_dir], on_line=on_line) @@ -2495,7 +2530,7 @@ class Console: release = self.state_dir / "releases" / (sha + "-" + job.id) release.mkdir(parents=True, exist_ok=True) for name in ("bitcoind", "bitcoin-cli"): - shutil.copy2(self.bin_path(name), release / name) + install_program(self.bin_path(name), release / name) if identical and self._build_folder_was_shared(): # Unmodified source, but compiled where builds of changed source # have been able to write. The unmodified node is never taken @@ -2508,7 +2543,7 @@ class Console: job.meta["test_status"] = "cancelled" return None if self.reference_state()["ready"]: - shutil.copy2(self.reference_dir / "bitcoind", release / "bitcoind") + install_program(self.reference_dir / "bitcoind", release / "bitcoind") job.emit(t="log", line="Source is unmodified: the node kept for it is the reference node itself") item = lessons.curated(job.meta.get("curated", "")) @@ -2833,7 +2868,7 @@ class Console: def _keep_variant(self, item, release_dir, source): folder = self.variant_dir(item["id"]) folder.mkdir(parents=True, exist_ok=True) - shutil.copy2(Path(release_dir) / "bitcoind", folder / "bitcoind") + install_program(Path(release_dir) / "bitcoind", folder / "bitcoind") self._write_json(folder / "variant.json", {"tree": self.node_key(self.stock_rev()), "patch": self._patch_digest(item), "source": source, "time": time.time()}) @@ -3194,8 +3229,11 @@ class Console: dest = self.repo / "build" / "bin" dest.mkdir(parents=True, exist_ok=True) for name in ("bitcoind", "bitcoin-cli"): - shutil.copy2(src / name, dest / name) + install_program(src / name, dest / name) (dest / name).chmod(0o755) + # A note of the commit this program was built from, for adopting + # it as the unmodified reference node. + (dest / "COMMIT").write_text(built_from + "\n") if self.ask_node_program(dest / "bitcoind", "-version")[0] != 0: job.emit(t="log", line="It would not run on this machine; compiling instead") return False diff --git a/contrib/vibes/install.sh b/contrib/vibes/install.sh index ca3be74ae09e..8fa94e0a3536 100755 --- a/contrib/vibes/install.sh +++ b/contrib/vibes/install.sh @@ -389,7 +389,12 @@ try_prebuilt() { rm -rf "$tmp"; return 1 fi mkdir -p "$DEST/build/bin" - cp "$dir/bitcoind" "$dir/bitcoin-cli" "$DEST/build/bin/" || { rm -rf "$tmp"; return 1; } + # Remove any earlier programs first, so the new ones are new files: macOS + # on Apple silicon kills a program that was overwritten in place after it + # had been run. COMMIT goes along as a note of what the program was built + # from; the console reads it when it adopts this node as its reference. + rm -f "$DEST/build/bin/bitcoind" "$DEST/build/bin/bitcoin-cli" "$DEST/build/bin/COMMIT" + cp "$dir/bitcoind" "$dir/bitcoin-cli" "$dir/COMMIT" "$DEST/build/bin/" || { rm -rf "$tmp"; return 1; } chmod +x "$DEST/build/bin/bitcoind" "$DEST/build/bin/bitcoin-cli" if ! "$DEST/build/bin/bitcoind" -version >/dev/null 2>&1; then note "the prebuilt node would not run here — building instead" @@ -397,8 +402,8 @@ try_prebuilt() { fi rm -rf "$tmp" note "using a prebuilt node built from this exact source" - note "your first experiment compiles the node from source: that wait has been" - note "postponed, not avoided" + note "the ready-made experiments need no compiling. A request in your own" + note "words compiles the node from source: that wait is postponed, not avoided" return 0 } diff --git a/contrib/vibes/tests/test_console.py b/contrib/vibes/tests/test_console.py index 6626c4bf7c80..e57deb7923eb 100644 --- a/contrib/vibes/tests/test_console.py +++ b/contrib/vibes/tests/test_console.py @@ -824,6 +824,67 @@ def compile_apart(job): self.assertEqual(record['compare']['verdict'], 'identical') self.assertEqual((Path(record['path']) / 'bitcoind').read_text(), 'compiled apart') + def test_programs_are_put_in_place_as_new_files(self): + """Overwriting a program that has been run gets it killed on Apple + silicon. Replacing it gives the new program a file of its own.""" + folder = self.repo / 'build/bin' + folder.mkdir(parents=True) + (folder / 'new').write_text('new program') + target = folder / 'bitcoind' + target.write_text('old program') + before = target.stat().st_ino + held = open(target) # as if the old program were still running + self.addCleanup(held.close) + vibes.install_program(folder / 'new', target) + self.assertEqual(target.read_text(), 'new program') + self.assertNotEqual(target.stat().st_ino, before) + self.assertEqual(held.read(), 'old program') + self.assertEqual(sorted(p.name for p in folder.iterdir()), ['bitcoind', 'new']) + self.programs('from a build') + self.console._capture_reference(self.console.stock_rev(), folder) + first = (self.console.reference_dir / 'bitcoind').stat().st_ino + (self.console.reference_dir / 'reference.json').unlink() + self.console._capture_reference(self.console.stock_rev(), folder) + self.assertNotEqual((self.console.reference_dir / 'bitcoind').stat().st_ino, first) + + def test_a_release_node_is_adopted_as_the_reference(self): + """A node installed from a release says "v31.99-7", a tag, where a + local build says "v31.99.0-". Both, and the note a ready-made + node is installed with, must be recognised: otherwise a new install + has no reference and every ready-made experiment asks for a compile.""" + head = self.git('rev-parse', 'HEAD') + self.git('tag', 'v31.99-7') + says = {'version': ''} + + def program(bitcoind, flag, timeout=30): + return 0, f"Bitcoin Vibes daemon version {says['version']} bitcoind\nCopyright\n", '' + + def adopted(version, note=None): + shutil.rmtree(self.console.reference_dir, ignore_errors=True) + self.programs('from the release') + if note: + (self.repo / 'build/bin/COMMIT').write_text(note + '\n') + elif (self.repo / 'build/bin/COMMIT').exists(): + (self.repo / 'build/bin/COMMIT').unlink() + says['version'] = version + with patch.object(self.console, 'ask_node_program', program): + self.console._adopt_installed_reference() + return self.reference() == 'from the release' + self.assertTrue(adopted('v31.99-7')) # a release tag + self.assertTrue(adopted('v31.99.0-' + head[:12])) # a commit + self.assertTrue(adopted('nonsense', note=head)) # the installer's note + self.assertFalse(adopted('v31.99-8')) # a tag this source does not have + self.assertFalse(adopted('v31.99.0-' + head[:12] + '-dirty')) + self.assertFalse(adopted('v31.99-7', note='0' * 40)) # a note naming no commit here + # Compiling here ends what the note says. + (self.repo / 'build/bin/COMMIT').write_text(head + '\n') + with self.compiling(): + self.console._compile(vibes.Job('build')) + self.assertFalse((self.repo / 'build/bin/COMMIT').exists()) + # And nothing in a shared folder is believed. + self.console._note_build_folder_shared('abc') + self.assertFalse(adopted('v31.99-7', note=head)) + def test_current_source_is_kept_as_the_reference_before_a_first_proposal(self): self.programs() diff --git a/contrib/vibes/tests/test_install.py b/contrib/vibes/tests/test_install.py index a29127022fd1..bf77bd9c582b 100644 --- a/contrib/vibes/tests/test_install.py +++ b/contrib/vibes/tests/test_install.py @@ -198,6 +198,9 @@ def test_matching_archive_is_used(self): self.assertEqual(code, 0, out) self.assertTrue((self.dest / 'build/bin/bitcoind').is_file()) self.assertIn('not an independent signature', out) + # The commit it was built from goes along, so the console can adopt it + # as the unmodified reference node. + self.assertEqual((self.dest / 'build/bin/COMMIT').read_text().strip(), self.commit) def test_tampered_archive_is_refused(self): self.publish(checksum='0' * 64)