Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,11 @@ make install-runtime # installs the daemon + smoothnas-runtime.service
```

The runtime wraps [LXC2Docker](https://github.com/games-on-whales/LXC2Docker)
(defaulting to upstream `main`; pin with `LXC2DOCKER_REF`). Build hosts need
(pinned to a tested upstream commit; override with `LXC2DOCKER_REF`). Build hosts need
`golang-go`, `pkg-config`, `build-essential`, and `lxc-dev`; runtime hosts
need `lxc`, `lxc-templates`, `skopeo`, `umoci`, `rsync`, `nftables`,
`iptables`, `iproute2`, and `uidmap`. SmoothNAS carries no local LXC2Docker
patches — fixes land upstream and are picked up on the next runtime build.
`iptables`, `iproute2`, and `uidmap`. Runtime fixes are pinned upstream; reviewed downstream corrections in
`runtime/patches/` are applied by the same build path used for releases.
See [../runtime/README.md](../runtime/README.md) for detail.

### Kernel and OpenZFS
Expand Down
23 changes: 23 additions & 0 deletions docs/compose-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,26 @@ services:
SmoothNAS-specific concerns compose doesn't: tiered-volume bind resolution + pin,
the cross-project host-port guard, and treating `docker compose ps` as the source
of truth for state (cached, with a periodic reconcile sweep).

## Direct web interfaces

For an application that needs its own HTTPS origin, declare a direct link
instead of `x-smoothnas.ui` (the iframe/proxy target):

```yaml
x-smoothnas:
webUI:
scheme: https
port: 8007
path: /
services:
server:
image: example/server:1.0
ports: ["8007:8007"]
```

`webUI.port` must match a fixed published TCP port; `scheme` is `http` or
`https`, and `path` is an absolute path. The Open action uses the hostname of
the current appliance page and opens a new tab. It does not route through
SmoothNAS nginx or inject authentication. The application owns its certificate
and login. See [the PBS plugin guide](pbs-plugin.md) for a complete example.
11 changes: 11 additions & 0 deletions docs/patches/pbs-manifest-fsync.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
--- a/pbs-datastore/src/backup_info.rs
+++ b/pbs-datastore/src/backup_info.rs
@@ -1008,7 +1008,7 @@
path.push(MANIFEST_BLOB_NAME.as_ref());

// atomic replace invalidates flock - no other writes past this point!
- replace_file(&path, raw_data, CreateOptions::new(), false)?;
+ replace_file(&path, raw_data, CreateOptions::new(), true)?;
Ok(())
}

539 changes: 539 additions & 0 deletions docs/pbs-plugin.md

Large diffs are not rendered by default.

12 changes: 6 additions & 6 deletions runtime/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,19 @@ The build target writes `bin/docker-lxc-daemon`. Appliance installs
place it at `/usr/lib/smoothnas/docker-lxc-daemon` and install
`runtime/smoothnas-runtime.service`.

The upstream source is pinned to the reviewed LXC2Docker commit:
The upstream source is pinned to the tested LXC2Docker commit that removes
container request and failure-config logging of credentials:

```text
https://github.com/games-on-whales/LXC2Docker.git
7171568494541a1c9a776670a1623f3db64d7e46
ff6d6d8f9f6f62ea4b2fe7592e88a10ed20569bb
```

Set `LXC2DOCKER_REF=<branch-or-commit-or-tag>` to evaluate a newer revision.
Downstream corrections are applied by the same build path used for releases.


SmoothNAS carries runtime corrections in `runtime/patches/`. The build script
applies them after checking out LXC2Docker; a patch that no longer applies fails
the build. The bridge correction replaces only the owned postrouting chain in
one nft transaction, preserving published port DNAT rules and removing old
duplicate masquerade rules. Bridge and port-forward mutations share `nftMu`.
the build. The pinned upstream already bounds masquerade rules in one nft transaction.
The bridge correction serializes bridge and port-forward mutations with
`nftMu` and checks that published port DNAT rules survive repeated reconciliation.
28 changes: 0 additions & 28 deletions runtime/patches/0001-idempotent-bridge-masquerade.patch
Original file line number Diff line number Diff line change
Expand Up @@ -9,34 +9,6 @@
iface, err := net.InterfaceByName(BridgeName)
if err != nil || iface == nil {
// Bridge doesn't exist — create it and assign the gateway IP.
@@ -69,18 +71,16 @@
}
}

- // Ensure the nftables table exists with the masquerade rule.
- // Using "nft -f" with a table block is idempotent — it merges into any
- // existing table rather than replacing it.
+ // Reconcile only our postrouting chain in one atomic transaction.
+ // A table block appends rules on every call; flush removes old duplicates
+ // while retaining prerouting/output DNAT rules used by published ports.
nftRule := fmt.Sprintf(`
-table ip %s {
- chain postrouting {
- type nat hook postrouting priority srcnat; policy accept;
- ip saddr 10.100.0.0/24 oifname != "%s" masquerade
- ct status dnat masquerade
- }
-}
-`, NATTableName, BridgeName)
+add table ip %s
+add chain ip %s postrouting { type nat hook postrouting priority srcnat; policy accept; }
+flush chain ip %s postrouting
+add rule ip %s postrouting ip saddr 10.100.0.0/24 oifname != "%s" masquerade
+add rule ip %s postrouting ct status dnat masquerade
+`, NATTableName, NATTableName, NATTableName, NATTableName, BridgeName, NATTableName)
cmd := exec.Command("nft", "-f", "-")
cmd.Stdin = strings.NewReader(nftRule)
if out, err := cmd.CombinedOutput(); err != nil {
--- /dev/null
+++ b/internal/lxc/network_nat_integration_test.go
@@ -0,0 +1,51 @@
Expand Down
4 changes: 2 additions & 2 deletions scripts/build-smoothnas-runtime.sh
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
#!/usr/bin/env bash
# Build the SmoothNAS plugin runtime daemon from a reviewed upstream commit.
# Build the SmoothNAS plugin runtime daemon from a tested upstream LXC2Docker commit.
set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"

LXC2DOCKER_REPO="${LXC2DOCKER_REPO:-https://github.com/games-on-whales/LXC2Docker.git}"
LXC2DOCKER_REF="${LXC2DOCKER_REF:-7171568494541a1c9a776670a1623f3db64d7e46}"
LXC2DOCKER_REF="${LXC2DOCKER_REF:-ff6d6d8f9f6f62ea4b2fe7592e88a10ed20569bb}"
BUILD_DIR="${PROJECT_DIR}/runtime/build/LXC2Docker"
OUT="${PROJECT_DIR}/bin/docker-lxc-daemon"

Expand Down
2 changes: 1 addition & 1 deletion scripts/ci/smoothfs-ref
Original file line number Diff line number Diff line change
@@ -1 +1 @@
13b4491ba43bca6f6b67f35add7026083505d228
e85ea3a251262e4bea95efa8862d91258e76f880
47 changes: 47 additions & 0 deletions scripts/pbs-image/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Experimental manifest-durability candidate; build from the repository root:
# docker build -f scripts/pbs-image/Dockerfile -t pbs:4.2.0-manifest-fsync .
# The live validation used these commands in a disposable Debian 13 guest,
# then inserted the resulting binaries into this same clean runtime image.
FROM debian:13 AS build
ARG DEBIAN_FRONTEND=noninteractive
ARG BUILD_JOBS=1
RUN test "$(dpkg --print-architecture)" = amd64 \
&& apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl git dpkg-dev \
&& curl -fsSL https://enterprise.proxmox.com/debian/proxmox-release-trixie.gpg \
-o /usr/share/keyrings/proxmox.gpg \
&& printf '%s\n' \
'deb [signed-by=/usr/share/keyrings/proxmox.gpg] http://download.proxmox.com/debian/pbs trixie pbs-no-subscription' \
'deb [signed-by=/usr/share/keyrings/proxmox.gpg] http://download.proxmox.com/debian/devel trixie main' \
> /etc/apt/sources.list.d/proxmox.list
# PBS 4.2.0 needs the older apt API; newer devel packages use incompatible majors.
RUN printf '%s\n' \
'Package: librust-proxmox-apt-dev librust-proxmox-apt+*' 'Pin: version 0.99.7-1' 'Pin-Priority: 1001' '' \
'Package: librust-proxmox-apt-api-types*' 'Pin: version 2.0.5-1' 'Pin-Priority: 1001' '' \
'Package: librust-pbs-api-types*' 'Pin: version 1.0.15-1' 'Pin-Priority: 1001' '' \
'Package: librust-proxmox-upgrade-checks*' 'Pin: version 1.0.1-1' 'Pin-Priority: 1001' '' \
'Package: librust-proxmox-openid*' 'Pin: version 1.0.2-1' 'Pin-Priority: 1001' '' \
'Package: librust-proxmox-subscription*' 'Pin: version 1.0.1-1' 'Pin-Priority: 1001' \
> /etc/apt/preferences.d/pbs-build \
&& git clone --branch v4.2.0 --single-branch https://git.proxmox.com/git/proxmox-backup.git /build
WORKDIR /build
RUN test "$(git rev-parse HEAD)" = 035c449897fafc228c8bbf3a5b5ba38564478ac7 \
&& apt-get update \
&& apt-get --solver 3.0 -Pnodoc --no-install-recommends -y build-dep .
COPY docs/patches/pbs-manifest-fsync.patch /manifest-fsync.patch
RUN git apply --check /manifest-fsync.patch && git apply /manifest-fsync.patch \
&& export DEB_VERSION="$(dpkg-parsechangelog -SVersion)" \
&& export DEB_VERSION_UPSTREAM="${DEB_VERSION%-*}" \
&& export REPOID=035c449897fafc228c8bbf3a5b5ba38564478ac7-manifest-fsync \
&& CARGO_BUILD_JOBS="$BUILD_JOBS" CARGO_PROFILE_RELEASE_DEBUG=0 \
cargo build --offline --release --package proxmox-backup \
--bin proxmox-backup-api --bin proxmox-backup-proxy --bin proxmox-backup-manager \
&& mkdir /out \
&& cp target/release/proxmox-backup-api target/release/proxmox-backup-proxy \
target/release/proxmox-backup-manager /out/
FROM ayufan/proxmox-backup-server@sha256:01f685a010025233b649417a317cc175b27920cd64f0429abfdc46a0d8d2e88a
COPY --from=build /out/proxmox-backup-api /out/proxmox-backup-proxy /usr/lib/x86_64-linux-gnu/proxmox-backup/
COPY --from=build /out/proxmox-backup-manager /usr/sbin/proxmox-backup-manager
LABEL pbs.candidate="manifest-fsync" \
pbs.source="035c449897fafc228c8bbf3a5b5ba38564478ac7" \
pbs.base="sha256:01f685a010025233b649417a317cc175b27920cd64f0429abfdc46a0d8d2e88a"
140 changes: 140 additions & 0 deletions scripts/pbs-plugin-smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
#!/usr/bin/env python3
"""Validate a PBS plugin datastore and exercise a disposable backup group.

Credentials use the normal PBS_* environment variables. Run --restore-only
again after PBS verification/GC, tier movement, or plugin reinstall.
"""

import argparse
import datetime
import fcntl
import hashlib
import json
import os
from pathlib import Path
import subprocess
import tempfile
import time
import uuid


def filesystem_probe(path):
# Use a new sibling directory, never modify actual PBS chunks or indexes.
with tempfile.TemporaryDirectory(prefix=".smoothnas-pbs-probe-", dir=path) as tmp:
root = Path(tmp)
source = root / "chunk.tmp"
with source.open("wb") as handle:
handle.write(b"PBS filesystem probe\n")
handle.flush()
os.fsync(handle.fileno())
chunk = root / "chunk"
source.rename(chunk)
old = time.time_ns() - 3 * 24 * 3600 * 10**9
os.utime(chunk, ns=(old, old))
current = time.time_ns()
os.utime(chunk, ns=(current, old))
st = chunk.stat()
if abs(st.st_atime_ns - current) > 10**9 or abs(st.st_mtime_ns - old) > 10**9:
raise RuntimeError("Explicit chunk atime update did not preserve mtime")
with chunk.open("rb") as first, chunk.open("rb") as second:
fcntl.flock(first, fcntl.LOCK_EX | fcntl.LOCK_NB)
try:
fcntl.flock(second, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
pass
else:
raise RuntimeError("Conflicting datastore locks were both granted")
fd = os.open(root, os.O_RDONLY | os.O_DIRECTORY)
try:
os.fsync(fd)
finally:
os.close(fd)
print("Filesystem probe passed: timestamp updates, locking, rename and fsync")


def client(args, *command):
cmd = [args.client, *command]
if args.namespace:
cmd += ["--ns", args.namespace]
subprocess.run(cmd, check=True)


def contents(root):
result = {}
for path in sorted(root.rglob("*")):
name = str(path.relative_to(root))
if path.is_symlink():
result[name] = {"link": os.readlink(path)}
elif path.is_file():
result[name] = {
"sha256": hashlib.sha256(path.read_bytes()).hexdigest(),
"mode": path.stat().st_mode & 0o777,
}
return result


def restore(args, record):
# Keep the source/evidence between invocations; restored trees are disposable.
with tempfile.TemporaryDirectory(prefix="restore-", dir=args.workdir) as tmp:
target = Path(tmp) / "files"
client(args, "restore", record["snapshot"], "data.pxar", str(target))
if contents(target) != record["contents"]:
raise RuntimeError("Restored hashes, symlinks or permissions differ")
print("Restore passed: hashes, symlinks and permissions match")


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--filesystem", type=Path, help="Probe a mounted datastore path locally")
parser.add_argument("--workdir", type=Path, help="New evidence directory for network backup/restore")
parser.add_argument("--restore-only", action="store_true", help="Restore an existing workdir after maintenance/reinstall")
parser.add_argument("--client", default="proxmox-backup-client")
parser.add_argument("--namespace", default="")
args = parser.parse_args()
if not args.filesystem and not args.workdir:
parser.error("provide --filesystem and/or --workdir")
if args.filesystem:
filesystem_probe(args.filesystem)
if not args.workdir:
return
if not os.environ.get("PBS_REPOSITORY"):
parser.error("set PBS_REPOSITORY and PBS credentials in the environment")
evidence = args.workdir / "validation.json"
if args.restore_only:
record = json.loads(evidence.read_text())
if record["repository"] != os.environ["PBS_REPOSITORY"] or record["namespace"] != args.namespace:
parser.error("repository and namespace must match the saved evidence")
restore(args, record)
return
args.workdir.mkdir(mode=0o700, parents=True, exist_ok=False)
source = args.workdir / "source"
source.mkdir()
(source / "payload").write_bytes(os.urandom(2 * 1024 * 1024))
(source / "note").write_text("first version\n")
(source / "note").chmod(0o640)
(source / "link").symlink_to("note")
group = "host/smoothnas-validation-" + uuid.uuid4().hex
snapshots = []
for version in (1, 2):
stamp = int(time.time())
(source / "note").write_text(f"version {version}\n")
client(args, "backup", f"data.pxar:{source.resolve()}", "--backup-id", group.split("/")[1],
"--backup-type", "host", "--backup-time", str(stamp), "--crypt-mode", "none")
iso = datetime.datetime.fromtimestamp(stamp, datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
snapshots.append(f"{group}/{iso}")
if version == 1:
time.sleep(1.1)
record = {"snapshot": snapshots[-1], "contents": contents(source),
"repository": os.environ["PBS_REPOSITORY"], "namespace": args.namespace}
evidence.write_text(json.dumps(record, indent=2) + "\n")
restore(args, record)
# Retention applies ONLY to this new UUID group, never existing backups.
client(args, "prune", group, "--keep-last", "1", "--dry-run", "false")
restore(args, record)
print(f"Backup/prune/restore passed for {group}")
print("Run PBS verification and garbage collection, then rerun with --restore-only.")
print("Repeat the restore after tier movement and uninstall/reinstall on the same tiers.")


if __name__ == "__main__":
main()
7 changes: 7 additions & 0 deletions tierd-ui/src/pages/Plugins/PluginDetail.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { api } from '../../api/api';
import { extractError } from '../../utils/errors';
import Spinner from '../../components/Spinner/Spinner';
import ConfirmDialog from '../../components/ConfirmDialog/ConfirmDialog';
import { pluginWebURL } from '../../utils/pluginWebUI';

// Detail mirrors the JSON shape POST /api/plugins/<name> returns.
// Loose typing — the backend's PluginRecord adds fields over time;
Expand Down Expand Up @@ -150,6 +151,7 @@ export default function PluginDetail() {
</div>
);
}
const webURL = detail.plugin.state === 'running' ? pluginWebURL(detail.plugin.webUI) : null;

return (
<div className="page">
Expand All @@ -166,6 +168,11 @@ export default function PluginDetail() {
</span>
</div>
</div>
{webURL && (
<a className="btn" href={webURL} target="_blank" rel="noopener noreferrer">
{t('plugins.action.open')}
</a>
)}
<Link to="/plugins" className="btn">
{t('plugins.detail.backToList')}
</Link>
Expand Down
Loading
Loading