diff --git a/README.md b/README.md index 874544c..3ef590f 100644 --- a/README.md +++ b/README.md @@ -81,7 +81,11 @@ show as ranges instead of one tile per height. **Insights** has system health (status, uptime, node version, chain, initial-download state), inbound against outbound peers, how much disk the chain uses next to the free space on its volume, and how many blocks arrived in each of the last -12 minutes. +12 minutes. With a managed node the chain size is **measured**: what the network's folder takes on disk (refreshed about +once a minute), and the free space is that of the volume holding the data root. The node's own `size_on_disk` is not used +there: the official `blvm` release computes it as an estimate of about 1 MB per block, which was off by a factor of 2 to 8 +on real chains (102 GB reported against 12 GB on disk at 99 000 blocks). With a standalone node the console measures the +node's data folder when it can find it, and otherwise shows the node's figure, labelled "reported by the node". ![Insights](docs/screenshots/insights.png) @@ -114,7 +118,9 @@ What you see depends on the mode. one does not come up. On Linux only (see below). When updates are turned off, or left to an app store (`BLVM_UI_UPDATES`), the card says so. * **Modules.** Add-ons that run next to the node. Installing them is not available yet; the screenshot of this card is - mock data. + mock data. Mining appears in the sidebar only once a mining module is installed and on. +* **Console.** *Show the Logs page* (off by default; `show_logs` in `console.json`). Part of the sidebar rules: Home, Network, + Settings are always there, Logs when the switch is on, and a module's page (Mining) when that module is installed and on. * **About.** Chain, node RPC, versions, uptime, and a per-network **Storage** list with free disk and the data folder. *Run setup again* is here. @@ -134,18 +140,24 @@ computer (Unix only), and version and uptime details. ### Logs -With a managed node the Logs page **tails the node's log live**, next to a short guide to the lines worth watching. The +The Logs page is **off by default**: *Show the Logs page* in Settings (the *Console* card) puts it in the sidebar, and a +typed `#logs` while it is off shows a note with a button to turn it on. With a managed node the page **tails the node's +log live**, starting with the last 500 lines of the file (also those written before the console started), next to a short guide to the lines worth watching. The full file is `networks//logs/node.log` in the data folder (copy-truncated at 50 MB, one older file kept), and the page has a button to copy its path. With a standalone node the console cannot see the node's output, so the page shows the commands to run your node with verbose output, save it and follow it. A line such as "RPC authentication is required but no tokens or certificates are configured" in the node's log is an upstream wording issue: with the login this console gives the node, Basic authentication is in effect (a call without it gets `401`). +The node does not inherit the console's own `RUST_LOG` (the container image sets one that names only the console, which +silenced every line the node wrote); give the node one with *Child environment* (`child_env`) if you want a different level. + ![Logs](docs/screenshots/logs.png) ### Mining -An information page about the Stratum V2 endpoint and the Commons pool: commands to run, no live numbers. Nothing live is +Shown in the sidebar only once a mining module is installed and on (the sidebar is otherwise Home, Network, Logs, Settings; +a typed `#mining` without a module shows a note pointing to Settings). An information page about the Stratum V2 endpoint and the Commons pool: commands to run, no live numbers. Nothing live is wired to it yet. ### Status lights @@ -160,10 +172,10 @@ field. ## Umbrel Commons Node View is in the **Bitcoin Commons community app store**: -[`BTCDecoded/blvm-umbrel-store`](https://github.com/BTCDecoded/blvm-umbrel-store). +[`BTCDecoded/commons-umbrel-store`](https://github.com/BTCDecoded/commons-umbrel-store). 1. In umbrelOS open the **App Store**, then **⋯** (top right), then **Community App Stores**. -2. Add `https://github.com/BTCDecoded/blvm-umbrel-store`. +2. Add `https://github.com/BTCDecoded/commons-umbrel-store`. 3. Open **Bitcoin Commons App Store** and install **Commons Node View**. What you get: @@ -707,11 +719,11 @@ The page only uses these; they are handy for scripts too. | Route | Does | |---|---| -| `GET /api/status` | the full snapshot the page renders (sync, peers, disk, feed, health) | +| `GET /api/status` | the full snapshot the page renders (sync, peers, disk, feed, health). `disk_used_*` is the chain size and `disk_used_source` says where it comes from: `"folder"` (measured, always in managed mode) or `"node"` (the node's own figure, an estimate in the official release) | | `POST /api/connect` `{"rpc":"host:port"}` | point the console at another node | | `POST /api/node` `{"action":"toggle"}` | start or stop the node process | | `POST /api/rpc` `{"method":…,"params":[…]}` | pass-through for peer settings: `addnode`, `setban`, `clearbanned`, `disconnectnode`, `setnetworkactive` | -| `GET /api/status` (more) | adds `mode`, `setup_done`, `managed` (process state, pid, network or `null` before one is chosen, data root, free disk, `networks[]` with `bytes`, `has_data`, `last_height`, `last_used`, `switch`, node version, crash log; `null` when standalone), `modules_pages`, `bundle_available`, `blvm_version_seen` | +| `GET /api/status` (more) | adds `mode`, `setup_done`, `managed` (process state, pid, network or `null` before one is chosen, data root, free disk, `networks[]` with `bytes`, `has_data`, `last_height`, `last_used`, `switch`, node version, crash log; `null` when standalone), `modules_pages` (page ids of installed and enabled modules; `[]` while nothing can be installed), `show_logs`, `bundle_available`, `blvm_version_seen` | | `POST /api/node` `{"action":"start"\|"stop"\|"restart"\|"toggle"}` | managed: returns at once, watch `managed.process.state`. Also `POST /api/node/start`, `/stop`, `/restart` | | `GET /api/node/logs?after=&limit=200` | managed: `{lines:[{seq,text}],file}` from the in-memory tail of `node.log` (`lines=` also works); 404 when standalone | | `GET /api/settings` | `console.json` as the console sees it (effective values, `env_locked`) | @@ -719,7 +731,7 @@ The page only uses these; they are handy for scripts too. | `POST /api/node/install` `{"tag"?:"v0.1.60","version"?:"0.1.60","downgrade"?:false}` | start the first install (newest release when no tag): `202 {ok:true,job}`; `409 {ok:false,error:"busy"}` while a job runs; `400 {ok:false,error}` in plain words (unsupported platform, nothing to install, already installed, `BLVM_UI_NODE_BIN` set, bad tag, offline, a tag GitHub does not know: `code:"no_such_release"`, an older tag than the installed one without `downgrade:true`: `code:"downgrade"`) | | `POST /api/node/update` same body | update an installed release; also `400` when nothing was installed by the console, or the channel is `off`/`store` | | `GET /api/node/install` (also `/api/node/update`) | the last job until a new one starts: `{phase:"idle"\|"download"\|"verify"\|"test"\|"install"\|"restart"\|"done"\|"failed", tag, bytes, total, error, rolled_back}` (also under `job`, with `kind`, `started`, `finished`). The same fields are in `managed.update` (with `available`, `latest`, `channel`) while a job runs and for 30 s after | -| `PUT` or `POST /api/settings` | patch: `mode`, `network`, `data_root`, `node_bin`, `rpc`, `rpc_overrides`, `start_on_boot` (alias `start_on_launch`; default on in managed mode), `stop_node_on_exit` (`keep_node_running_on_exit` is its inverse), `modules_disabled`, `updates`, `assume_valid`, `child_env`. `setup_done: true` alone just marks the setup as done (what *Skip* sends on Umbrel). Mode, network, data root and binary can only change while the node is stopped (409), and not while an install or update runs and not when an environment variable pins them (409) | +| `PUT` or `POST /api/settings` | patch: `mode`, `network`, `data_root`, `node_bin`, `rpc`, `rpc_overrides`, `start_on_boot` (alias `start_on_launch`; default on in managed mode), `stop_node_on_exit` (`keep_node_running_on_exit` is its inverse), `show_logs` (the Logs page switch, default `false`), `modules_disabled`, `updates`, `assume_valid`, `child_env`. `setup_done: true` alone just marks the setup as done (what *Skip* sends on Umbrel). Mode, network, data root and binary can only change while the node is stopped (409), and not while an install or update runs and not when an environment variable pins them (409) | | `POST /api/setup` `{"mode":"managed","network":"regtest","data_root"?,"node_bin"?,"start":bool,"stop_on_exit":bool}` | the wizard's result: validates (managed needs a network; a binary only when `start` is true, so a setup that only records the choice works before a program is downloaded; standalone may carry `rpc`), writes `console.json`, creates the folders, starts the node only when `start` is true, and remembers whether the node was to run. Refused with 409 while an install or update runs | | `POST /api/setup/test` `{"rpc":"host:port","user"?,"pass"?}` | one RPC check: `{ok,chain,height,version,error}` | | `GET /api/network/preview?to=signet` (or `POST` with `{"to":"signet"}` / `{"network":"signet"}`) | managed: what the switch dialog shows: `{from,to,from_bytes,to_bytes,from_height,to_height,to_last_used,from_ibd_pct,free_bytes,estimate_bytes,blocked,warnings[],allowed,reason}`. `allowed:false` carries the plain reason (same network, a switch running, under 5 GB free, no network chosen). 404 when standalone, 400 for an unknown network | diff --git a/src/agent.rs b/src/agent.rs index 7c0eb46..27d7175 100644 --- a/src/agent.rs +++ b/src/agent.rs @@ -36,8 +36,10 @@ mod external; const RING_CAP: usize = 2000; const CRASH_LOG_LINES: usize = 50; -/// First read of a log file takes at most this many trailing bytes. -const PRELOAD_BYTES: u64 = 64 * 1024; +/// The first read of a log file starts this many lines before its end (the Logs page shows them first). +const PRELOAD_LINES: usize = 500; +/// ...but looks back at most this many bytes for them. +const PRELOAD_MAX_BYTES: u64 = 2 * 1024 * 1024; /// One tail pass reads at most this much; a bigger backlog skips to the end. const TAIL_CHUNK: u64 = 4 * 1024 * 1024; const TAIL_MAX_BACKLOG: u64 = 8 * 1024 * 1024; @@ -1128,6 +1130,10 @@ impl Inner { cmd.env_remove(&k); } } + // `RUST_LOG` is the console's own log filter (the Docker image sets `blvm_ui_next=info`). The node reads + // the same variable, and a filter that names only the console's crate silences every line the node + // writes: the Logs page stayed empty. The node gets one only through the Child environment setting. + cmd.env_remove("RUST_LOG"); for (k, v) in child_env(&cfg) { match v { Some(v) => { @@ -1696,8 +1702,9 @@ impl Inner { let len = meta.len(); if !r.preloaded { r.preloaded = true; - r.offset = len.saturating_sub(PRELOAD_BYTES); - r.skip_to_newline = r.offset > 0; + let (offset, at_line_start) = preload_offset(&mut f, len); + r.offset = offset; + r.skip_to_newline = offset > 0 && !at_line_start; } if len < r.offset { // Rotated (copy-truncate) or cleared. @@ -1854,6 +1861,43 @@ async fn port_conflict(cfg: &AgentConfig, net: &'static Network) -> Option (u64, bool) { + let floor = len.saturating_sub(PRELOAD_MAX_BYTES); + let mut end = len; + let mut newlines = 0usize; + let mut buf = vec![0u8; 64 * 1024]; + // A final newline ends the last line; it does not start another. + let mut first = true; + while end > floor { + let start = end.saturating_sub(buf.len() as u64).max(floor); + let n = (end - start) as usize; + if f.seek(SeekFrom::Start(start)).is_err() || f.read_exact(&mut buf[..n]).is_err() { + return (floor, floor == 0); + } + for i in (0..n).rev() { + if buf[i] != b'\n' { + continue; + } + if first && start + i as u64 + 1 == len { + first = false; + continue; + } + first = false; + newlines += 1; + if newlines == PRELOAD_LINES { + // The line after this newline is the first of the last PRELOAD_LINES lines. + return (start + i as u64 + 1, true); + } + } + end = start; + } + (floor, floor == 0) +} + // ---- log rotation --------------------------------------------------------------------------- /// Copy-truncate: `node.log` -> `node.log.1`, then empty `node.log` in place. The node keeps its @@ -2069,6 +2113,42 @@ pub(crate) mod testkit { script } + /// A free loopback port, picked by the system (bind to port 0, read it back, let go). Every test that + /// starts a node or binds a listener takes its ports from here: the networks' own ports may be held by a + /// node of the person running the tests, and a test must not depend on them being free. + pub(crate) fn free_port() -> u16 { + std::net::TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap().port() + } + + /// Two different free loopback ports (the RPC and the peer-to-peer port of one node). + pub(crate) fn free_port_pair() -> (u16, u16) { + // Hold both listeners until both are known, so the two cannot be the same port. + let a = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + let b = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); + (a.local_addr().unwrap().port(), b.local_addr().unwrap().port()) + } + + /// Point a test console at free ports only: every network's RPC address is an override on a free loopback + /// port (an override already there stays), and the node's peer-to-peer port is a free one. Without this a + /// test would start a node on the network's own ports (regtest: 18443 and 18444). + pub(crate) fn isolate(console: &mut crate::managed::Console) { + for n in crate::networks::ALL { + console + .settings + .rpc_overrides + .entry(n.id.to_string()) + .or_insert_with(|| format!("127.0.0.1:{}", free_port())); + } + console.p2p_port = Some(free_port()); + console.reresolve(); + } + + /// Where a test console polls `net` (its override, which `isolate` put there). + pub(crate) async fn rpc_of(state: &crate::state::Shared, net: &str) -> String { + let g = state.read().await; + g.console.rpc_for(crate::networks::by_id(net).unwrap()) + } + /// Put one in the fixture of a test: kills the nodes started under `dir` when it is dropped. pub(crate) struct ReapNodes(PathBuf); @@ -2703,16 +2783,16 @@ while kill -0 $PPID 2>/dev/null; do sleep 0.1; done #[test] fn exposed_addresses_are_everything_but_loopback() { - for open in ["0.0.0.0:8332", "10.0.0.7:8332", "node.internal:8332", "[::]:8332", "btc-commons-node-view_agent_1:48332"] { + for open in ["0.0.0.0:4242", "10.0.0.7:4242", "node.internal:4242", "[::]:4242", "btc-commons-node-view_agent_1:4243"] { assert!(rpc_exposed(open), "{open}"); } - for local in ["127.0.0.1:8332", "127.5.5.5:1", "localhost:8332", "[::1]:8332"] { + for local in ["127.0.0.1:4242", "127.5.5.5:1", "localhost:4242", "[::1]:4242"] { assert!(!rpc_exposed(local), "{local}"); } - assert_eq!(probe_addr("0.0.0.0:48332"), "127.0.0.1:48332"); - assert_eq!(probe_addr("[::]:48332"), "127.0.0.1:48332"); - assert_eq!(probe_addr("10.0.0.7:48332"), "10.0.0.7:48332", "a specific bind address is dialled as it is"); - assert_eq!(probe_addr("127.0.0.1:18443"), "127.0.0.1:18443"); + assert_eq!(probe_addr("0.0.0.0:4243"), "127.0.0.1:4243"); + assert_eq!(probe_addr("[::]:4243"), "127.0.0.1:4243"); + assert_eq!(probe_addr("10.0.0.7:4243"), "10.0.0.7:4243", "a specific bind address is dialled as it is"); + assert_eq!(probe_addr("127.0.0.1:4244"), "127.0.0.1:4244"); } #[tokio::test] @@ -2820,14 +2900,10 @@ while kill -0 $PPID 2>/dev/null; do sleep 0.1; done c } - fn free_addr() -> String { - let l = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); - l.local_addr().unwrap().to_string() - } + use testkit::free_port; - fn free_port() -> u16 { - let l = std::net::TcpListener::bind("127.0.0.1:0").unwrap(); - l.local_addr().unwrap().port() + fn free_addr() -> String { + format!("127.0.0.1:{}", free_port()) } #[tokio::test] @@ -3281,6 +3357,118 @@ while kill -0 $PPID 2>/dev/null; do sleep 0.1; done cleanup(&env); } + fn numbered_lines(from: usize, to: usize) -> String { + (from..=to).map(|i| format!("line {i}\n")).collect() + } + + #[tokio::test] + async fn the_tailer_starts_with_the_last_500_lines_then_follows_appends_and_truncation() { + for external in [false, true] { + let env = setup(if external { "tail-ext" } else { "tail" }); + let net = networks::by_id("regtest").unwrap(); + let log = networks::log_file(&env.root, net); + std::fs::create_dir_all(log.parent().unwrap()).unwrap(); + // The file is there before the console starts. + std::fs::write(&log, numbered_lines(1, 1000)).unwrap(); + let agent = if external { Agent::external(cfg(&env, "run"), fast()) } else { Agent::new(cfg(&env, "run"), fast()) }; + // The very first request already has them (the external client reads on demand). + agent.inner.ensure_tailer(); + wait_for("first 500", || agent.logs(None, 1000).len() == 500).await; + let first = agent.logs(None, 1000); + assert_eq!((first[0].seq, first[0].text.as_str()), (1, "line 501")); + assert_eq!((first[499].seq, first[499].text.as_str()), (500, "line 1000")); + assert!(first.windows(2).all(|w| w[1].seq == w[0].seq + 1), "sequence numbers are consecutive"); + + // Appended lines follow, after the last sequence number the page saw. + let mut f = std::fs::OpenOptions::new().append(true).open(&log).unwrap(); + f.write_all(numbered_lines(1001, 1003).as_bytes()).unwrap(); + wait_for("appended lines", || agent.logs(Some(500), 1000).len() == 3).await; + let more = agent.logs(Some(500), 1000); + assert_eq!(more.iter().map(|l| (l.seq, l.text.as_str())).collect::>(), [(501, "line 1001"), (502, "line 1002"), (503, "line 1003")]); + // A line is not shown before it is finished. + f.write_all(b"half a li").unwrap(); + tokio::time::sleep(Duration::from_millis(150)).await; + assert_eq!(agent.logs(Some(503), 10).len(), 0); + f.write_all(b"ne\n").unwrap(); + wait_for("finished line", || agent.logs(Some(503), 10).iter().any(|l| l.text == "half a line")).await; + + // Truncation (rotation by copy-truncate, or a cleared file): reading starts again from the top. + f.set_len(0).unwrap(); + drop(f); + std::fs::write(&log, "fresh 1\nfresh 2\n").unwrap(); + wait_for("after truncation", || agent.logs(Some(504), 10).iter().any(|l| l.text == "fresh 2")).await; + let after = agent.logs(Some(504), 10); + assert_eq!(after.iter().map(|l| l.text.as_str()).collect::>(), ["fresh 1", "fresh 2"]); + assert!(after[0].seq > 504, "sequence numbers keep counting up"); + cleanup(&env); + } + } + + #[tokio::test] + async fn the_first_read_of_a_short_or_exact_file_loses_no_line() { + for (n, tail_newline) in [(3usize, true), (500, true), (501, true), (500, false), (1, false)] { + let env = setup("tail-edge"); + let net = networks::by_id("regtest").unwrap(); + let log = networks::log_file(&env.root, net); + std::fs::create_dir_all(log.parent().unwrap()).unwrap(); + let mut text = numbered_lines(1, n); + if !tail_newline { + text.pop(); + } + std::fs::write(&log, text).unwrap(); + let agent = Agent::new(cfg(&env, "run"), fast()); + agent.inner.ensure_tailer(); + let want = if tail_newline { n.min(500) } else { n.min(500) - 1 }; + wait_for("preload", || agent.logs(None, 1000).len() >= want).await; + tokio::time::sleep(Duration::from_millis(100)).await; + let got = agent.logs(None, 1000); + assert_eq!(got.len(), want, "{n} lines, final newline {tail_newline}"); + if tail_newline { + assert_eq!(got.last().unwrap().text, format!("line {n}")); + assert_eq!(got[0].text, format!("line {}", n + 1 - want)); + } + cleanup(&env); + } + } + + #[tokio::test] + async fn the_nodes_stdout_and_stderr_reach_node_log_at_once_and_without_the_consoles_log_filter() { + // The console's own filter (the Docker image sets one that names only the console's crate) must not reach + // the node: it would silence every line the node writes. The Child environment setting still can. + std::env::set_var("RUST_LOG", "blvm_ui_next=info"); + for (child_filter, expect) in [(None, "filter=unset"), (Some("debug"), "filter=debug")] { + let env = setup("node-log"); + // A node that writes to both streams and keeps running; it also says which log filter it got. + std::fs::write( + &env.bin, + "#!/bin/sh\nif [ \"$1\" = version ]; then echo 'blvm 9.9.9'; exit 0; fi\ntrap 'exit 0' TERM\n\ + echo \"filter=${RUST_LOG-unset}\"\necho 'to stdout INFO'\necho 'to stderr WARN' >&2\n\ + while kill -0 $PPID 2>/dev/null; do sleep 0.1; done\n", + ) + .unwrap(); + let mut c = cfg(&env, "run"); + if let Some(f) = child_filter { + c.child_env.insert("RUST_LOG".into(), f.into()); + } + let agent = Agent::new(c, fast()); + agent.start().await.unwrap(); + let _reap = Reaper(agent.clone()); + let log = networks::log_file(&env.root, networks::by_id("regtest").unwrap()); + // Straight from the file, not through the ring: the node's own descriptor, no buffering in between. + wait_for("both streams in node.log", || { + let t = std::fs::read_to_string(&log).unwrap_or_default(); + t.contains("to stdout INFO") && t.contains("to stderr WARN") + }) + .await; + let t = std::fs::read_to_string(&log).unwrap(); + assert!(t.lines().any(|l| l == expect), "expected {expect} in {t}"); + wait_for("the page sees them", || agent.logs(None, 50).iter().any(|l| l.text == "to stderr WARN")).await; + agent.stop_and_wait().await; + cleanup(&env); + } + std::env::remove_var("RUST_LOG"); + } + #[tokio::test] async fn b3_a_foreign_listener_on_the_rpc_port_is_named_and_refused() { let env = setup("b3-name"); diff --git a/src/agent_daemon.rs b/src/agent_daemon.rs index 2bd63c6..b0b1010 100644 --- a/src/agent_daemon.rs +++ b/src/agent_daemon.rs @@ -505,9 +505,7 @@ while kill -0 $PPID 2>/dev/null; do sleep 0.1; done _reap: crate::agent::testkit::ReapNodes, } - fn free_port() -> u16 { - std::net::TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap().port() - } + use crate::agent::testkit::free_port; fn env(tag: &str) -> Env { let dir = std::env::temp_dir().join(format!("blvm-daemon-{tag}-{}", std::process::id())); diff --git a/src/http.rs b/src/http.rs index 7866800..81c6160 100644 --- a/src/http.rs +++ b/src/http.rs @@ -694,7 +694,8 @@ mod tests { data_root: Some(dir.join("root").display().to_string()), ..Default::default() }; - let console = Console::load(Some(dir.join("state")), env, None); + let mut console = Console::load(Some(dir.join("state")), env, None); + crate::agent::testkit::isolate(&mut console); // free ports, never the networks' own let mut live = LiveState::new(console.initial_rpc()); live.console = console; crate::state::stamp_power(&mut live); @@ -938,7 +939,7 @@ mod tests { } let (_, st) = call(s.addr, "GET", "/api/status", &[], None).await; assert_eq!(st["managed"]["network"], "signet"); - assert_eq!(st["rpc_addr"], "127.0.0.1:38332"); + assert_eq!(st["rpc_addr"], crate::agent::testkit::rpc_of(&s.state, "signet").await.as_str()); agent.stop_and_wait().await; let _ = std::fs::remove_dir_all(&s.dir); } @@ -1081,8 +1082,9 @@ mod tests { let (c, _) = call(s.addr, "POST", "/api/connect", &[JSON, UI], Some("")).await; assert_eq!(c, 400); assert_eq!(s.state.read().await.rpc_addr, before, "a bad body changes nothing"); - let (c, v) = call(s.addr, "POST", "/api/connect", &[JSON, UI], Some(r#"{"rpc":"127.0.0.1:38332"}"#)).await; - assert_eq!((c, v["rpc_addr"].as_str()), (200, Some("127.0.0.1:38332")), "a good body still works"); + let target = format!("127.0.0.1:{}", crate::agent::testkit::free_port()); + let (c, v) = call(s.addr, "POST", "/api/connect", &[JSON, UI], Some(&format!(r#"{{"rpc":"{target}"}}"#))).await; + assert_eq!((c, v["rpc_addr"].as_str()), (200, Some(target.as_str())), "a good body still works"); // /api/rpc. let (c, _) = call(s.addr, "POST", "/api/rpc", &[JSON, UI], Some("{oops")).await; assert_eq!(c, 400); diff --git a/src/installer.rs b/src/installer.rs index 72fe57c..43b0e72 100644 --- a/src/installer.rs +++ b/src/installer.rs @@ -1188,7 +1188,7 @@ mod tests { let ready = Arc::new(AtomicBool::new(true)); let r2 = Arc::clone(&ready); console.agent = None; - console.reresolve(); + crate::agent::testkit::isolate(&mut console); // free ports, never the networks' own (also resolves) let agent = Agent::with_ready(console.agent_config(), fast(), Arc::new(move || r2.load(Ordering::Relaxed))); agent.skip_login_check(); // the fake node has no RPC server to ask console.agent = Some(agent); diff --git a/src/managed.rs b/src/managed.rs index bb5e7e7..d29e624 100644 --- a/src/managed.rs +++ b/src/managed.rs @@ -547,6 +547,24 @@ fn managed_view(c: &Console, view: ProcessView, agent: &Agent, live_height: Opti } } +/// The sidebar pages that installed and enabled modules add (design L). `enabled_modules` are module ids, with or +/// without the `blvm-` prefix. A module that adds no page, or one this console does not know, adds nothing. +pub fn modules_pages_for>(enabled_modules: &[S]) -> Vec { + let mut out: Vec = Vec::new(); + for m in enabled_modules { + let id = m.as_ref(); + let id = id.strip_prefix("blvm-").unwrap_or(id); + let page = match id { + "stratum-v2" | "datum" | "gridpool" => "mining", + _ => continue, + }; + if !out.iter().any(|p| p == page) { + out.push(page.to_string()); + } + } + out +} + fn stamp_console_fields(snap: &mut Snapshot, c: &Console) { snap.mode = c.eff.mode.as_str(); snap.setup_done = c.eff.setup_done; @@ -554,7 +572,35 @@ fn stamp_console_fields(snap: &mut Snapshot, c: &Console) { // it has, for instance the one baked into its image. snap.bundle_available = c.eff.node_bin.as_deref().is_some_and(|p| p.is_file()) || (c.is_external() && c.agent.as_ref().is_some_and(|a| a.binary_found() == Some(true))); - snap.modules_pages = vec!["mining".to_string()]; + // This host has no module store yet, so nothing can be installed: no module pages, in either mode. + snap.modules_pages = modules_pages_for::(&[]); + snap.show_logs = c.settings.show_logs; + stamp_disk(snap, c); +} + +/// Managed mode: the chain size is what the network's folder takes on disk (the cached measurement behind +/// `managed.networks[].bytes`), never the node's `size_on_disk`: the official `blvm` computes that as about 1 MB +/// per block, an estimate that was off by a factor of 2 to 8 on real chains. The free space is that of the +/// volume holding the data root. Before the first measurement the figure is empty rather than the estimate. +fn stamp_disk(snap: &mut Snapshot, c: &Console) { + if !c.is_managed() { + return; + } + let bytes = c.network().map_or(0, |n| c.disk.lock().unwrap().net(n.id).bytes); + let (num, unit, label) = crate::state::format_bytes_parts(bytes); + snap.disk_used_num = num; + snap.disk_used_unit = unit; + snap.disk_used_label = label; + snap.disk_used_bytes = bytes; + snap.disk_used_source = "folder"; + let (total, free) = crate::state::volume_space_at(&c.eff.data_root); + let (free_num, free_unit, free_label) = crate::state::format_bytes_parts(free); + snap.disk_free_num = free_num; + snap.disk_free_unit = free_unit; + snap.disk_free_label = free_label; + snap.disk_free_bytes = free; + snap.disk_total_bytes = total; + snap.disk_vol_used_label = crate::state::format_bytes_parts(total.saturating_sub(free)).2; } /// Fill the console-level fields of the snapshot and, in managed mode, let the agent's state decide @@ -883,6 +929,7 @@ pub fn settings_json(c: &Console) -> Value { "modules_selected": s.modules_selected, "stop_node_on_exit": s.stop_node_on_exit, "keep_node_running_on_exit": !s.stop_node_on_exit, + "show_logs": s.show_logs, "modules_disabled": s.modules_disabled, "updates": c.updates_channel(), "rpc_overrides": s.rpc_overrides, @@ -991,6 +1038,10 @@ pub async fn patch_settings(state: &Shared, body: &Value) -> (u16, Value) { Some(b) => s.stop_node_on_exit = b, None => return err(400, "stop_node_on_exit must be true or false."), }, + "show_logs" => match v.as_bool() { + Some(b) => s.show_logs = b, + None => return err(400, "show_logs must be true or false."), + }, "keep_node_running_on_exit" => match v.as_bool() { Some(b) => s.stop_node_on_exit = !b, None => return err(400, "keep_node_running_on_exit must be true or false."), @@ -1458,7 +1509,8 @@ mod tests { if env.data_root.is_none() { env.data_root = Some(dir.join("root").display().to_string()); } - let console = Console::load(Some(dir.join("state")), env, None); + let mut console = Console::load(Some(dir.join("state")), env, None); + crate::agent::testkit::isolate(&mut console); // free ports, never the networks' own let mut live = LiveState::new(console.initial_rpc()); live.console = console; let _reap = crate::agent::testkit::ReapNodes::under(&dir); @@ -1485,7 +1537,7 @@ mod tests { // Once a network is chosen, polling follows it. let (code, _) = patch_settings(&f.state, &json!({"network":"regtest"})).await; assert_eq!(code, 200); - assert_eq!(f.state.read().await.rpc_addr, "127.0.0.1:18443"); + assert_eq!(f.state.read().await.rpc_addr, crate::agent::testkit::rpc_of(&f.state, "regtest").await); let _ = std::fs::remove_dir_all(&f.dir); } @@ -1543,7 +1595,7 @@ mod tests { assert_eq!(v["status"]["managed"]["process"]["state"], "stopped"); assert_eq!(v["status"]["managed"]["network"], "regtest"); assert_eq!(v["status"]["setup_done"], true); - assert_eq!(v["status"]["rpc_addr"], "127.0.0.1:18443"); + assert_eq!(v["status"]["rpc_addr"], crate::agent::testkit::rpc_of(&f.state, "regtest").await); assert!(f.dir.join("root/networks/regtest/logs").is_dir()); let saved = settings::load_console(&f.dir.join("state")); assert!(saved.setup_done); @@ -1568,12 +1620,13 @@ mod tests { let f = fixture("sa", EnvOverrides::default(), false); let (code, v) = setup(&f.state, &json!({"mode":"standalone","rpc":"nonsense"})).await; assert_eq!(code, 400, "{v}"); - let (code, v) = setup(&f.state, &json!({"mode":"standalone","rpc":"127.0.0.1:8332"})).await; + let target = format!("127.0.0.1:{}", crate::agent::testkit::free_port()); + let (code, v) = setup(&f.state, &json!({"mode":"standalone","rpc":target})).await; assert_eq!(code, 200, "{v}"); assert_eq!(v["status"]["mode"], "standalone"); assert_eq!(v["status"]["managed"], Value::Null); - assert_eq!(v["status"]["rpc_addr"], "127.0.0.1:8332"); - assert_eq!(settings::load_console(&f.dir.join("state")).rpc.as_deref(), Some("127.0.0.1:8332")); + assert_eq!(v["status"]["rpc_addr"], target.as_str()); + assert_eq!(settings::load_console(&f.dir.join("state")).rpc.as_deref(), Some(target.as_str())); // Skip: standalone with the current target. let (code, v) = setup(&f.state, &json!({"mode":"standalone"})).await; assert_eq!(code, 200, "{v}"); @@ -1643,7 +1696,11 @@ mod tests { agent.stop_and_wait().await; let (code, v) = patch_settings(&f.state, &json!({"network":"signet"})).await; assert_eq!(code, 200, "{v}"); - assert_eq!(f.state.read().await.rpc_addr, "127.0.0.1:38332", "polling follows the network"); + assert_eq!( + f.state.read().await.rpc_addr, + crate::agent::testkit::rpc_of(&f.state, "signet").await, + "polling follows the network" + ); let _ = std::fs::remove_dir_all(&f.dir); } @@ -1723,6 +1780,14 @@ mod tests { let _ = std::fs::remove_dir_all(&f2.dir); } + #[test] + fn modules_pages_follow_enabled_modules() { + assert!(modules_pages_for::(&[]).is_empty()); + assert_eq!(modules_pages_for(&["blvm-stratum-v2"]), vec!["mining"]); + assert_eq!(modules_pages_for(&["blvm-datum", "blvm-stratum-v2", "blvm-gridpool"]), vec!["mining"], "one page, once"); + assert!(modules_pages_for(&["blvm-zmq", "blvm-fibre", "something-else"]).is_empty()); + } + #[tokio::test] async fn status_carries_mode_setup_and_managed_fields() { // Standalone: managed is null, setup_done follows the rules, everything else stays. @@ -1735,7 +1800,7 @@ mod tests { assert_eq!(v["mode"], "standalone"); assert_eq!(v["setup_done"], true, "no node program to offer: nothing to set up beyond the RPC target"); assert_eq!(v["managed"], Value::Null); - assert_eq!(v["modules_pages"], json!(["mining"])); + assert_eq!(v["modules_pages"], json!([])); assert_eq!(v["bundle_available"], false); assert!(v.get("connected").is_some() && v.get("blvm_version").is_some(), "old fields still there"); @@ -1878,10 +1943,10 @@ mod tests { #[test] fn rpc_address_validation() { - for ok in ["127.0.0.1:8332", "localhost:48332", "my-node.local:1", "[::1]:8332", "host_name:65535"] { + for ok in ["127.0.0.1:4242", "localhost:4243", "my-node.local:1", "[::1]:4242", "host_name:65535"] { assert!(valid_rpc_addr(ok), "{ok}"); } - for bad in ["", "8332", "host", "host:", "host:0", "host:70000", ":8332", "http://h:1", "a b:1", "h:1/x", "[::1:8332"] { + for bad in ["", "4242", "host", "host:", "host:0", "host:70000", ":4242", "http://h:1", "a b:1", "h:1/x", "[::1:4242"] { assert!(!valid_rpc_addr(bad), "{bad}"); } } @@ -1950,7 +2015,7 @@ mod tests { let (code, _) = patch_settings(&f.state, &json!({"network":"regtest"})).await; assert_eq!(code, 200); { - // Something answers on 127.0.0.1:18443 (another node), so the last poll was live. + // Something answers on regtest's RPC address (another node), so the last poll was live. let mut g = f.state.write().await; g.snapshot.connected = true; g.snapshot.blvm_version = "v0.1.13".into(); @@ -2173,10 +2238,22 @@ mod tests { }; settings::save_console(&dir.join("state"), &saved).unwrap(); // No env mode: the saved file alone says "managed", as after a restart of the app. - let c = Console::load(Some(dir.join("state")), env, None); + // An external agent (another container) is not started from here: its tests keep the network's own port. + let c = load_in(&dir, env); (dir, c) } + /// A console on `dir`'s saved state, as the next run of the app loads it: on free ports (the networks' own + /// stay free for the person running the tests), unless its agent is another container's. + fn load_in(dir: &Path, env: EnvOverrides) -> Console { + let external = env.agent_external; + let mut c = Console::load(Some(dir.join("state")), env, None); + if !external { + crate::agent::testkit::isolate(&mut c); + } + c + } + fn shared(c: Console) -> Shared { let mut live = LiveState::new(c.initial_rpc()); live.console = c; @@ -2191,7 +2268,8 @@ mod tests { assert!(c.eff.setup_done, "no wizard on a configured console"); assert!(c.settings.start_on_boot_effective(true), "start on boot is the managed default"); assert_eq!(c.network().map(|n| n.id), Some("regtest")); - assert_eq!(c.initial_rpc(), "127.0.0.1:18443"); + assert_eq!(c.initial_rpc(), c.rpc_for(networks::by_id("regtest").unwrap())); + assert!(c.initial_rpc().starts_with("127.0.0.1:")); let state = shared(c); { let mut g = state.write().await; @@ -2218,7 +2296,7 @@ mod tests { node_bin: Some(dir.join("fake-blvm.sh").display().to_string()), ..Default::default() }; - let state2 = shared(Console::load(Some(dir.join("state")), env, None)); + let state2 = shared(load_in(&dir, env)); boot(&state2).await; let agent2 = state2.read().await.console.agent.clone().unwrap(); assert_eq!(agent2.state(), ProcState::Stopped); @@ -2236,7 +2314,7 @@ mod tests { data_root: Some(dir.join("root").display().to_string()), ..Default::default() }; - let state = shared(Console::load(Some(dir.join("state")), env, None)); + let state = shared(load_in(&dir, env)); boot(&state).await; let agent = state.read().await.console.agent.clone().unwrap(); assert_eq!(agent.state(), ProcState::Stopped, "no crash banner for a node that was never installed"); @@ -2250,7 +2328,7 @@ mod tests { let (dir, c) = saved_console("ext", env); assert!(c.is_external()); // The poll target is the other container, on the network's own RPC port. - assert_eq!(c.initial_rpc(), "node.internal:18443"); + assert_eq!(c.initial_rpc(), format!("node.internal:{}", networks::by_id("regtest").unwrap().rpc_port)); let state = shared(c); let root = dir.join("root"); // The agent already runs the node (what it wrote before the console restarted): adopt, write nothing. @@ -2294,11 +2372,12 @@ mod tests { let base = EnvOverrides { mode: Some("managed".into()), ..Default::default() }; let regtest = networks::by_id("regtest").unwrap(); let c = Console::load(None, base.clone(), None); - assert_eq!(c.rpc_for(regtest), "127.0.0.1:18443", "in-process: loopback"); + let port = regtest.rpc_port; + assert_eq!(c.rpc_for(regtest), format!("127.0.0.1:{port}"), "in-process: loopback"); let c = Console::load(None, EnvOverrides { agent_external: true, ..base.clone() }, None); - assert_eq!(c.rpc_for(regtest), "btc-commons-node-view_agent_1:18443"); + assert_eq!(c.rpc_for(regtest), format!("btc-commons-node-view_agent_1:{port}")); let c = Console::load(None, EnvOverrides { agent_external: true, node_host: Some("agent.local".into()), ..base.clone() }, None); - assert_eq!(c.rpc_for(regtest), "agent.local:18443"); + assert_eq!(c.rpc_for(regtest), format!("agent.local:{port}")); // An override in the settings still wins. let mut c = Console::load(None, EnvOverrides { agent_external: true, ..base.clone() }, None); c.settings.rpc_overrides.insert("regtest".into(), "10.0.0.7:1234".into()); @@ -2334,6 +2413,92 @@ mod tests { let _ = std::fs::remove_dir_all(&g.dir); } + #[tokio::test] + async fn the_logs_page_is_off_until_the_user_turns_it_on_and_that_is_remembered() { + assert!(!ConsoleSettings::default().show_logs, "off by default"); + let old: ConsoleSettings = serde_json::from_str(r#"{"setup_done":true}"#).unwrap(); + assert!(!old.show_logs, "an older file without the field: off"); + let f = fixture("show-logs", managed_env(), true); + assert_eq!(settings_json(&f.state.read().await.console)["show_logs"], false); + { + let mut g = f.state.write().await; + stamp_power(&mut g); + } + let v = serde_json::to_value(&f.state.read().await.snapshot).unwrap(); + assert_eq!(v["show_logs"], false, "status carries it, off"); + let (c, v) = patch_settings(&f.state, &json!({"show_logs": "yes"})).await; + assert_eq!(c, 400, "{v}"); + let (c, v) = patch_settings(&f.state, &json!({"show_logs": true})).await; + assert_eq!((c, v["show_logs"].clone()), (200, json!(true))); + assert!(settings::load_console(&f.dir.join("state")).show_logs, "saved to console.json"); + let v = serde_json::to_value(&f.state.read().await.snapshot).unwrap(); + assert_eq!(v["show_logs"], true, "status follows the switch at once"); + let (_, v) = patch_settings(&f.state, &json!({"show_logs": false})).await; + assert_eq!(v["show_logs"], false); + assert!(!settings::load_console(&f.dir.join("state")).show_logs); + let _ = std::fs::remove_dir_all(&f.dir); + } + + #[tokio::test] + async fn managed_chain_size_is_the_measured_folder_not_the_nodes_estimate() { + let f = fixture("disk-measured", managed_env(), true); + let (code, _) = patch_settings(&f.state, &json!({"network":"regtest"})).await; + assert_eq!(code, 200); + // Real bytes in the network's folder: 3 MB of chain data. + let data = f.dir.join("root/networks/regtest/data"); + std::fs::create_dir_all(&data).unwrap(); + std::fs::write(data.join("blocks.bin"), vec![7u8; 3 * 1024 * 1024]).unwrap(); + let (cache, root) = f.state.read().await.console.disk_job().expect("a measurement is due"); + run_disk_job(cache, root); + let measured = f.state.read().await.console.disk_snapshot().lock().unwrap().net("regtest").bytes; + assert!(measured >= 3 * 1024 * 1024, "{measured}"); + // The node says 27 GB (about 1 MB a block at 27 000 blocks), a made-up figure. + { + let mut g = f.state.write().await; + g.snapshot.disk_used_bytes = 27_000_000_000; + g.snapshot.disk_used_num = "27.0".into(); + g.snapshot.disk_used_unit = "GB".into(); + g.snapshot.disk_used_label = "27.0 GB".into(); + stamp_power(&mut g); + } + let s = f.state.read().await.snapshot.clone(); + assert_eq!(s.disk_used_bytes, measured, "the folder, not the node's number"); + assert_eq!(s.disk_used_source, "folder"); + assert_ne!(s.disk_used_label, "27.0 GB"); + let (total, free) = crate::state::volume_space_at(&f.dir.join("root")); + assert!(total > 0); + assert_eq!(s.disk_total_bytes, total, "the volume that holds the data root"); + assert!(s.disk_free_bytes.abs_diff(free) < 1 << 30); + let v = serde_json::to_value(&s).unwrap(); + assert_eq!(v["disk_used_source"], "folder"); + let _ = std::fs::remove_dir_all(&f.dir); + + // Before the first measurement: no figure, and never the node's. + let g = fixture("disk-unmeasured", managed_env(), true); + let (code, _) = patch_settings(&g.state, &json!({"network":"regtest"})).await; + assert_eq!(code, 200); + { + let mut w = g.state.write().await; + w.snapshot.disk_used_bytes = 27_000_000_000; + stamp_power(&mut w); + } + assert_eq!(g.state.read().await.snapshot.disk_used_bytes, 0); + let _ = std::fs::remove_dir_all(&g.dir); + } + + #[tokio::test] + async fn standalone_keeps_the_nodes_figure_and_says_so() { + let f = fixture("disk-standalone", EnvOverrides::default(), false); + { + let mut g = f.state.write().await; + g.snapshot.disk_used_bytes = 5_000_000_000; + stamp_power(&mut g); + } + let s = f.state.read().await.snapshot.clone(); + assert_eq!((s.disk_used_bytes, s.disk_used_source), (5_000_000_000, "node")); + let _ = std::fs::remove_dir_all(&f.dir); + } + #[tokio::test] async fn the_updates_setting_obeys_the_environment() { let env = EnvOverrides { updates: Some("off".into()), ..managed_env() }; @@ -2374,7 +2539,7 @@ mod tests { let dir = std::env::temp_dir().join(format!("blvm-review-skip-{}", std::process::id())); let _ = std::fs::remove_dir_all(&dir); let env = EnvOverrides { mode: Some("managed".into()), agent_external: true, data_root: Some(dir.join("root").display().to_string()), ..Default::default() }; - let state = shared(Console::load(Some(dir.join("state")), env, None)); + let state = shared(load_in(&dir, env)); let (code, v) = setup(&state, &json!({"mode": "standalone", "start": false, "stop_on_exit": true, "modules": []})).await; let _ = std::fs::remove_dir_all(&dir); assert_eq!(code, 409); @@ -2410,7 +2575,7 @@ mod tests { assert_eq!(console_json(&dir).await.node_wanted, Some(false), "saved at once"); // The app starts again: "start when the console opens" is on (the default) but the node stays off. - let state2 = shared(Console::load(Some(dir.join("state")), reload_env(&dir), None)); + let state2 = shared(load_in(&dir, reload_env(&dir))); assert!(state2.read().await.console.settings.start_on_boot_effective(true)); boot(&state2).await; let agent2 = state2.read().await.console.agent.clone().unwrap(); @@ -2421,7 +2586,7 @@ mod tests { assert!(out.ok, "{out:?}"); assert_eq!(console_json(&dir).await.node_wanted, Some(true)); agent2.stop_and_wait().await; // (the process goes, the wish stays: a quit is not a Stop) - let state3 = shared(Console::load(Some(dir.join("state")), reload_env(&dir), None)); + let state3 = shared(load_in(&dir, reload_env(&dir))); boot(&state3).await; let agent3 = state3.read().await.console.agent.clone().unwrap(); assert_eq!(agent3.state(), ProcState::Starting, "start -> restart -> running"); @@ -2433,7 +2598,7 @@ mod tests { g.console.settings.start_on_boot = Some(false); g.console.save().unwrap(); } - let state4 = shared(Console::load(Some(dir.join("state")), reload_env(&dir), None)); + let state4 = shared(load_in(&dir, reload_env(&dir))); boot(&state4).await; assert_eq!(state4.read().await.console.agent.clone().unwrap().state(), ProcState::Stopped); let _ = std::fs::remove_dir_all(dir); @@ -2506,8 +2671,8 @@ mod tests { /// `console.json` holds no login; the new console generates one at load. #[tokio::test] async fn an_old_install_gets_its_open_node_restarted_with_a_password_and_the_login_saved() { - let free = || std::net::TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap(); - let rpc = free().to_string(); + let (rpc_port, p2p) = crate::agent::testkit::free_port_pair(); + let rpc = format!("127.0.0.1:{rpc_port}"); let dir = std::env::temp_dir().join(format!("blvm-managed-r2-{}", std::process::id())); let _ = std::fs::remove_dir_all(&dir); std::fs::create_dir_all(&dir).unwrap(); @@ -2530,7 +2695,6 @@ mod tests { settings::save_console(&dir.join("state"), &old).unwrap(); assert_eq!(settings::load_console(&dir.join("state")).rpc_pass, None); // The node the old console started (it has no login) and that outlived it. - let p2p = free().port(); let old_cfg = AgentConfig { bin: Some(bin.clone()), root: root.clone(), @@ -2632,7 +2796,7 @@ mod tests { let dir = std::env::temp_dir().join(format!("blvm-skip-umbrel-{}", std::process::id())); let _ = std::fs::remove_dir_all(&dir); let env = EnvOverrides { mode: Some("managed".into()), agent_external: true, data_root: Some(dir.join("root").display().to_string()), ..Default::default() }; - let state = shared(Console::load(Some(dir.join("state")), env, None)); + let state = shared(load_in(&dir, env)); assert!(!state.read().await.console.eff.setup_done, "fresh Umbrel install: the wizard shows"); let (code, v) = patch_settings(&state, &json!({"setup_done": true})).await; assert_eq!(code, 200, "{v}"); @@ -2684,7 +2848,7 @@ mod tests { let mode = std::fs::metadata(dir.join("state/console.json")).unwrap().permissions().mode() & 0o777; assert_eq!(mode, 0o600, "the file holds a password"); // ... and the next run of the console finds the same one (a node it adopts keeps working). - let again = Console::load(Some(dir.join("state")), reload_env(&dir), None); + let again = load_in(&dir, reload_env(&dir)); assert_eq!(again.rpc_auth(), Some(auth.clone())); // The page never sees it. let shown = settings_json(&again).to_string(); diff --git a/src/rpc.rs b/src/rpc.rs index 9c093c9..aa1fa92 100644 --- a/src/rpc.rs +++ b/src/rpc.rs @@ -405,11 +405,11 @@ mod tests { #[test] fn frozen_vs_connect_errors() { - assert!(is_frozen_error("RPC 127.0.0.1:38332 frozen")); - assert!(is_frozen_error("RPC 127.0.0.1:38332 timed out")); - assert!(!is_frozen_error("connect 127.0.0.1:38332")); + assert!(is_frozen_error("RPC 127.0.0.1:4242 frozen")); + assert!(is_frozen_error("RPC 127.0.0.1:4242 timed out")); + assert!(!is_frozen_error("connect 127.0.0.1:4242")); assert!(!is_frozen_error("Connection refused")); - assert!(is_connect_refused("connect 127.0.0.1:38332")); + assert!(is_connect_refused("connect 127.0.0.1:4242")); assert!(is_connect_refused("Connection refused")); assert!(!is_connect_refused("Connection reset by peer")); assert!(!is_connect_refused("Connection rate limit exceeded")); diff --git a/src/settings.rs b/src/settings.rs index 8acf781..fcf583f 100644 --- a/src/settings.rs +++ b/src/settings.rs @@ -90,6 +90,8 @@ pub struct ConsoleSettings { pub modules_selected: Vec, /// Stop the node when the console quits (default). `false` leaves it running. pub stop_node_on_exit: bool, + /// Show the Logs page in the sidebar. Off by default: the page is the node's raw log output. + pub show_logs: bool, pub modules_disabled: Vec, /// `self` | `store` | `off`. pub updates: String, @@ -132,6 +134,7 @@ impl Default for ConsoleSettings { rpc_pass: None, modules_selected: Vec::new(), stop_node_on_exit: true, + show_logs: false, modules_disabled: Vec::new(), updates: "self".into(), rpc_overrides: BTreeMap::new(), @@ -489,7 +492,7 @@ mod console_tests { #[test] fn setup_done_rules() { // Env RPC or mode skips the wizard for standalone installs. - let env = EnvOverrides { rpc: Some("127.0.0.1:8332".into()), ..Default::default() }; + let env = EnvOverrides { rpc: Some("127.0.0.1:4242".into()), ..Default::default() }; assert!(resolve(&ConsoleSettings::default(), &env, None).setup_done); let env = EnvOverrides { mode: Some("standalone".into()), ..Default::default() }; assert!(resolve(&ConsoleSettings::default(), &env, None).setup_done); @@ -574,6 +577,7 @@ mod console_tests { data_root: Some("/data/x".into()), node_bin: Some("/bin/blvm".into()), stop_node_on_exit: false, + show_logs: true, ..Default::default() }; s.rpc_overrides.insert("regtest".into(), "127.0.0.1:19999".into()); diff --git a/src/state.rs b/src/state.rs index e07687f..7380d9c 100644 --- a/src/state.rs +++ b/src/state.rs @@ -57,6 +57,9 @@ pub struct Snapshot { pub disk_used_unit: String, pub disk_used_label: String, pub disk_used_bytes: u64, + /// Where the chain size comes from: `"folder"` (measured on disk) or `"node"` (what the node reports; the + /// official `blvm` computes about 1 MB per block, an estimate). + pub disk_used_source: &'static str, pub disk_free_num: String, pub disk_free_unit: String, pub disk_free_label: String, @@ -88,6 +91,8 @@ pub struct Snapshot { pub managed: Option, /// Page ids that installed modules add to the sidebar. pub modules_pages: Vec, + /// The user turned the Logs page on in Settings (`show_logs`, off by default). + pub show_logs: bool, /// Last version the node itself reported, while `blvm_version` is `"—"` (design M.1). pub blvm_version_seen: Option, /// A node binary is known to this console, so "run a node for me" is possible. @@ -133,6 +138,7 @@ impl Snapshot { disk_used_unit: "".into(), disk_used_label: "—".into(), disk_used_bytes: 0, + disk_used_source: "node", disk_free_num: vol.free_num, disk_free_unit: vol.free_unit, disk_free_label: vol.free_label, @@ -156,7 +162,8 @@ impl Snapshot { mode: "standalone", setup_done: true, managed: None, - modules_pages: vec!["mining".to_string()], + modules_pages: Vec::new(), + show_logs: false, blvm_version_seen: None, bundle_available: false, } @@ -675,7 +682,7 @@ async fn refresh(rpc_addr: &str, state: &Shared, full: bool) -> anyhow::Result<( }; let blvm_version_now = version.clone(); node_ctl::remember_if_running(rpc_addr, &mut g.launch); - let size_on_disk = chain_bytes_on_disk(rpc_addr, &g.launch, size_on_disk, blocks, headers); + let (size_on_disk, disk_source) = chain_bytes_on_disk(rpc_addr, &g.launch, size_on_disk, blocks, headers); let (disk_num, disk_unit, disk_label) = format_bytes_parts(size_on_disk); let vol = disk_volume(); @@ -714,6 +721,7 @@ async fn refresh(rpc_addr: &str, state: &Shared, full: bool) -> anyhow::Result<( disk_used_unit: disk_unit, disk_used_label: disk_label, disk_used_bytes: size_on_disk, + disk_used_source: disk_source, disk_free_num: vol.free_num, disk_free_unit: vol.free_unit, disk_free_label: vol.free_label, @@ -738,6 +746,7 @@ async fn refresh(rpc_addr: &str, state: &Shared, full: bool) -> anyhow::Result<( setup_done: prev.setup_done, managed: None, modules_pages: prev.modules_pages.clone(), + show_logs: prev.show_logs, blvm_version_seen: if blvm_version_now == "—" { version_seen.map(|(version, at)| VersionSeen { version, at }) } else { @@ -1118,16 +1127,16 @@ fn chain_bytes_on_disk( rpc_bytes: u64, blocks: u64, headers: u64, -) -> u64 { +) -> (u64, &'static str) { if let Some(n) = node_ctl::datadir_usage(rpc_addr, launch) { if n > 0 { - return n; + return (n, "folder"); } } if rpc_size_is_placeholder(rpc_bytes, blocks, headers) { - 0 + (0, "node") } else { - rpc_bytes + (rpc_bytes, "node") } } @@ -1314,7 +1323,7 @@ pub fn volume_space_at(_path: &std::path::Path) -> (u64, u64) { (0, 0) } -fn format_bytes_parts(n: u64) -> (String, String, String) { +pub(crate) fn format_bytes_parts(n: u64) -> (String, String, String) { const TB: f64 = 1_000_000_000_000.0; const GB: f64 = 1_000_000_000.0; const MB: f64 = 1_000_000.0; @@ -2000,7 +2009,7 @@ mod tests { ] { assert!(v.get(k).is_some(), "status lost field {k}"); } - for k in ["mode", "setup_done", "managed", "modules_pages", "blvm_version_seen", "bundle_available"] { + for k in ["mode", "setup_done", "managed", "modules_pages", "show_logs", "blvm_version_seen", "bundle_available"] { assert!(v.get(k).is_some(), "status is missing new field {k}"); } } diff --git a/src/switch.rs b/src/switch.rs index 7685e8e..ea67716 100644 --- a/src/switch.rs +++ b/src/switch.rs @@ -495,9 +495,7 @@ while kill -0 $PPID 2>/dev/null; do sleep 0.1; done } } - fn free_port() -> u16 { - std::net::TcpListener::bind("127.0.0.1:0").unwrap().local_addr().unwrap().port() - } + use crate::agent::testkit::free_port; fn timings() -> Timings { Timings { diff --git a/web/mock/managed.mjs b/web/mock/managed.mjs index 78f4c35..7551546 100644 --- a/web/mock/managed.mjs +++ b/web/mock/managed.mjs @@ -70,7 +70,7 @@ export function createManaged(chain) { installed: [], // {name, version, enabled} canInstall: true, platform: "x86_64-linux", - settings: { version: 1, start_on_boot: true, start_on_launch: true, stop_node_on_exit: true, updates: "self" }, + settings: { version: 1, start_on_boot: true, start_on_launch: true, stop_node_on_exit: true, show_logs: false, updates: "self" }, lastRunning: secs(), logs: [], seq: 0, @@ -330,8 +330,8 @@ export function createManaged(chain) { // ---- modules (design E, L) -------------------------------------------- const entry = (name) => INDEX.find((e) => e.name === name); function modulePages() { - // The host sends a constant ["mining"] until the module manager exists (U5). - if (!mg.future) return ["mining"]; + // The host has no module store yet (U5), so it sends an empty list: nothing installed, no module pages. + if (!mg.future) return []; const out = []; for (const m of mg.installed) { if (!m.enabled) continue; @@ -406,8 +406,8 @@ export function createManaged(chain) { /** Adds (managed/standalone-with-new-host) fields and the "node is not running" overrides to a snapshot. */ function decorate(snap) { - if (mg.mode === undefined) return snap; // old host shape - const out = { ...snap, mode: mg.mode, setup_done: mg.setupDone, bundle_available: mg.mode === "managed" ? !!mg.nodeVersion : mg.bundle, modules_pages: modulePages() }; + if (mg.mode === undefined) return { ...snap, show_logs: mg.settings.show_logs === true, disk_used_source: "node" }; // old host shape: Logs switch only + const out = { ...snap, mode: mg.mode, setup_done: mg.setupDone, bundle_available: mg.mode === "managed" ? !!mg.nodeVersion : mg.bundle, modules_pages: modulePages(), show_logs: mg.settings.show_logs === true, disk_used_source: mg.mode === "managed" ? "folder" : "node" }; if (mg.mode === "managed") { out.managed = managedBody(); const st = mg.proc.state; @@ -476,7 +476,7 @@ export function createManaged(chain) { freeBytes: 1_042_500_000_000, nodeVersion: "0.1.60", custom: false, latest: "0.1.60", channel: "self", platformSupported: true, sw: null, installed: [], canInstall: true, platform: "x86_64-linux", lastRunning: secs(), logs: [], seq: 0, failNextStart: false, stopStuck: false, - settings: { version: 1, start_on_boot: true, start_on_launch: true, stop_node_on_exit: true, updates: "self" }, + settings: { version: 1, start_on_boot: true, start_on_launch: true, stop_node_on_exit: true, show_logs: false, updates: "self" }, job: idleJob(), jobEnd: 0, failVerify: false, failTest: false, }); } @@ -712,8 +712,13 @@ export function createManaged(chain) { return true; } if (p === "/api/settings") { - if (mg.mode === undefined) return json(res, { error: "not found" }, 404), true; + if (mg.mode === undefined) { + // The old-host shape has only the page switch. + if (mutating && "show_logs" in (body ?? {})) mg.settings.show_logs = body.show_logs === true; + return json(res, { version: 1, show_logs: mg.settings.show_logs === true }), true; + } if (mutating) { + if ("show_logs" in (body ?? {})) mg.settings.show_logs = body.show_logs === true; for (const k of ["start_on_launch", "stop_node_on_exit", "updates", "data_root"]) if (k in (body ?? {})) mg.settings[k] = body[k]; // Skip on Umbrel: only marks setup as done, nothing else moves. if ("setup_done" in (body ?? {})) mg.setupDone = !!body.setup_done; @@ -792,6 +797,7 @@ export function createManaged(chain) { if (g("fail_verify") !== null) mg.failVerify = g("fail_verify") !== "0"; if (g("fail_test") !== null) mg.failTest = g("fail_test") !== "0"; if (g("stop_stuck") !== null) mg.stopStuck = g("stop_stuck") !== "0"; + if (g("show_logs") !== null) mg.settings.show_logs = g("show_logs") !== "0"; if (g("future") !== null) mg.future = g("future") !== "0"; if (g("port_taken") !== null) mg.portTaken = g("port_taken") !== "0" ? { port: NETS[mg.network ?? "testnet4"].rpc, pid: 4242, name: "blvm" } : null; if (g("agent_alive") !== null) mg.agentAlive = g("agent_alive") !== "0"; diff --git a/web/mock/screenshots.mjs b/web/mock/screenshots.mjs index 023f2fa..b489630 100644 --- a/web/mock/screenshots.mjs +++ b/web/mock/screenshots.mjs @@ -53,6 +53,11 @@ async function startMock() { } const mock = (q) => fetch(`${BASE}/__mock?${q}`).then((r) => r.json()); +// The Logs page is off by default; the shot of it turns the switch on first. +const logsOn = async (page) => { + await mock("show_logs=1"); + await page.waitForTimeout(1600); +}; const card = (title) => `article.panel:has(h2:text-is("${title}"))`; // name → [page hash, what to capture]. `null` target = the whole window. @@ -61,8 +66,7 @@ const SYNCED = [ ["insights", "insights", null], ["network", "network", null], ["settings", "settings", null], - ["mining", "mining", null], - ["logs", "logs", null], + ["logs", "logs", null, logsOn], ["home-network-sync", "home", ".sync-card"], ["home-p2p-connections", "home", ".p2p-float"], @@ -78,7 +82,6 @@ const SYNCED = [ ["settings-console-target", "settings", card("Console Target")], ["settings-node-process", "settings", card("Node Process")], ["settings-about", "settings", card("About")], - ["mining-stratum-v2", "mining", card("Stratum V2")], ]; // Managed-node screens (design G, K, L). The optional 4th item prepares the page (clicks) before the shot. @@ -106,7 +109,7 @@ const MANAGED_RUNNING = [ ["managed-node", "settings", card("Node")], ["managed-network", "settings", card("Network")], // Modules and Updates cards are shown from the MOCK-ONLY scenarios below (the host has no such routes yet). - ["managed-logs", "logs", null], + ["managed-logs", "logs", null, logsOn], ["managed-switch-confirm", "settings", "dialog[open]", pickNet("Signet")], ]; const SWITCHING = [["managed-switching", "settings", card("Network")]]; @@ -124,7 +127,7 @@ const NO_BINARY = [ ["setup-get-node", "home", ".setup", steps(click("Set up"), pickChoice("Signet"), click("Next"))], ["setup-get-node-downloading", "home", ".setup", steps(click("Set up"), pickChoice("Signet"), click("Next"), click("Download"))], ]; -const MODULES = [["managed-modules-installed", "settings", card("Modules")], ["managed-mining", "mining", null]]; +const MODULES = [["managed-modules-installed", "settings", card("Modules")], ["managed-mining", "mining", null], ["mining-stratum-v2", "mining", card("Stratum V2")]]; const SYNCING = [ ["ibd-home", "home", null], diff --git a/web/mock/server.mjs b/web/mock/server.mjs index 9bda252..bb1fede 100644 --- a/web/mock/server.mjs +++ b/web/mock/server.mjs @@ -29,13 +29,13 @@ // Host parity. /api/node/releases, /api/node/install (GET job, POST start) and /api/node/update exist in every // managed scenario. Routes the Rust host does not have yet (/api/modules*; U5) exist in the mock ONLY in the scenarios // marked MOCK-ONLY above (or with &future=1). In every other scenario they answer a JSON 404 like the real host, and -// modules_pages is the host's constant ["mining"]. +// modules_pages is the host's empty list (nothing can be installed yet). // // Control (used by the screenshot script and for hand-testing): // GET /__mock?scenario=ibd&live=0 switch scenario, freeze or unfreeze // extra knobs: fail_verify=1 / fail_test=1 (the next install or update job fails its checksum check / its test run; // an update then reports rolled_back) channel=self|store|off platform_supported=0|1 can_install=0|1 platform=aarch64-macos -// future=1|0 (turn the not-yet-in-the-host routes on or off) port_taken=0|1 (the port-conflict program) +// show_logs=1|0 (the Logs page switch; off by default) future=1|0 (turn the not-yet-in-the-host routes on or off) port_taken=0|1 (the port-conflict program) // slow=20 (stretch every delay) free_gb=3 custom=1 fail_start=1 (next start crashes) stop_stuck=1 (next switch cannot stop the old node) external=1 (with first-run: Umbrel-style) // agent_alive=0 (external only: the node agent's heartbeat is gone) offline=1 (GitHub cannot be reached; Try again // asks GET /api/node/releases?force=1, counted in force_checks of this answer) diff --git a/web/src/App.svelte b/web/src/App.svelte index 74b3d1f..0aea615 100644 --- a/web/src/App.svelte +++ b/web/src/App.svelte @@ -9,6 +9,7 @@ import SettingsPanel from "./lib/SettingsPanel.svelte"; import LogsPanel from "./lib/LogsPanel.svelte"; import MiningPanel from "./lib/MiningPanel.svelte"; + import LogsGate from "./lib/LogsGate.svelte"; import ModuleGate from "./lib/ModuleGate.svelte"; import { MODULE_PAGES } from "./lib/modules"; import Setup from "./lib/Setup.svelte"; @@ -18,28 +19,27 @@ type Item = { id: Page; label: string; icon: string }; const home: Item = { id: "home", label: "Home", icon: "home" }; - const mining: Item = { id: "mining", label: "Mining", icon: "mining" }; - const core: Item[] = [ - { id: "network", label: "Network", icon: "network" }, - { id: "logs", label: "Logs", icon: "logs" }, - { id: "settings", label: "Settings", icon: "settings" }, - ]; + const network: Item = { id: "network", label: "Network", icon: "network" }; + const logs: Item = { id: "logs", label: "Logs", icon: "logs" }; + const settings: Item = { id: "settings", label: "Settings", icon: "settings" }; // Design L: core pages plus the pages installed modules add. A host that does not send `modules_pages` - // (every host before the managed-node work) keeps the sidebar exactly as it was, Mining included. + // (and the moment before the first /api/status answers) shows the core pages only: a module page is never + // shown unless the host says its module is installed and on. const side = $derived.by(() => { const mp = ui.ready ? modulePages(ui.snap) : null; - if (mp === null) return [home, mining, ...core]; - const extra = mp.filter((id) => id in MODULE_PAGES).map((id) => ({ id: id as Page, ...MODULE_PAGES[id] })); - return [home, ...extra, ...core]; + // Logs is off until the user turns it on in Settings (`show_logs`). + const showLogs = ui.ready && ui.snap.show_logs === true; + const extra = (mp ?? []).filter((id) => id in MODULE_PAGES).map((id) => ({ id: id as Page, ...MODULE_PAGES[id] })); + return [home, ...extra, network, ...(showLogs ? [logs] : []), settings]; }); // A module page whose module is not there (typed URL, or the module was just removed). const gated = $derived.by(() => { if (!ui.ready || !(ui.page in MODULE_PAGES)) return false; - const mp = modulePages(ui.snap); - return mp !== null && !mp.includes(ui.page); + return !(modulePages(ui.snap) ?? []).includes(ui.page); }); + const logsOff = $derived(ui.ready && ui.page === "logs" && ui.snap.show_logs !== true); const wizard = $derived(wizardActive()); // Umbrel: the node runs under an agent in another container; when its heartbeat is gone the page says so // next to the "Console Live" light (the console itself is fine, the node is not being looked after). @@ -127,6 +127,8 @@ {/if} {#if gated} + {:else if logsOff} + {:else if ui.page === "home"} {:else if ui.page === "insights"} diff --git a/web/src/lib/ConsoleCard.svelte b/web/src/lib/ConsoleCard.svelte new file mode 100644 index 0000000..e39c5b0 --- /dev/null +++ b/web/src/lib/ConsoleCard.svelte @@ -0,0 +1,62 @@ + + +
+
+

Console

+
+
+ Show the Logs page + +
+

Shows the node's raw log output. Off by default.

+ {#if err}

{err}

{/if} +
+ + diff --git a/web/src/lib/LogTail.svelte b/web/src/lib/LogTail.svelte index e244db6..2746b82 100644 --- a/web/src/lib/LogTail.svelte +++ b/web/src/lib/LogTail.svelte @@ -21,7 +21,9 @@ async function poll() { try { - const d = await fetchLogs(lastSeq); + // The first request asks for the last 500 lines (what was written before the page opened); later ones for + // everything new since the last line seen. + const d = await fetchLogs(lastSeq, lastSeq === undefined ? 500 : 1000); err = ""; file = d.file || file; if (d.lines.length) { diff --git a/web/src/lib/LogsGate.svelte b/web/src/lib/LogsGate.svelte new file mode 100644 index 0000000..ab77a14 --- /dev/null +++ b/web/src/lib/LogsGate.svelte @@ -0,0 +1,36 @@ + + +
+
+
+

Logs

+
+

The Logs page is off. It shows the node's raw log output. You can turn it on in Settings, or here.

+
+ +
+ {#if err}

{err}

{/if} +
+
diff --git a/web/src/lib/NodePanel.svelte b/web/src/lib/NodePanel.svelte index ce57d7e..089fad1 100644 --- a/web/src/lib/NodePanel.svelte +++ b/web/src/lib/NodePanel.svelte @@ -82,6 +82,9 @@ Chain Size + {#if s.disk_used_source === "node"} + reported by the node + {/if}
{/if} + {#if s.show_logs !== undefined} + + {/if} +

About

diff --git a/web/src/lib/types.ts b/web/src/lib/types.ts index 88ad941..400f42f 100644 --- a/web/src/lib/types.ts +++ b/web/src/lib/types.ts @@ -59,6 +59,9 @@ export type Snapshot = { disk_used_unit: string; disk_used_label: string; disk_used_bytes: number; + /** Where the chain size comes from: "folder" (measured on disk) or "node" (what the node reports, an estimate + * of about 1 MB a block in the official `blvm`). Absent on an older host. */ + disk_used_source?: "folder" | "node"; disk_free_num: string; disk_free_unit: string; disk_free_label: string; @@ -89,8 +92,10 @@ export type Snapshot = { /** Design F. Null or absent in standalone mode. */ managed?: Managed | null; /** Design L. Page ids that installed + enabled modules add to the sidebar (e.g. ["mining"]). - * Absent => the sidebar stays as before (static list, Mining included). */ + * Absent => core pages only (no module page without its module). */ modules_pages?: string[]; + /** The user turned the Logs page on (Settings > Console). Absent or false => no Logs page in the sidebar. */ + show_logs?: boolean; /** Design M.1. Last version the node itself reported, shown only as muted "last seen". */ blvm_version_seen?: VersionSeen | null; /** Design K step 2 / G: a node binary shipped next to the console exists ("Run a node for me" is possible). @@ -378,6 +383,8 @@ export type ConsoleSettings = { stop_node_on_exit?: boolean; /** Host: same fact as `stop_node_on_exit`, inverted (accepted on write, sent on read). */ keep_node_running_on_exit?: boolean; + /** Show the Logs page in the sidebar. Off by default. */ + show_logs?: boolean; modules_disabled?: string[]; updates?: UpdateChannel; /** Host only. Network id to the node's RPC `host:port` when not the network's default. */