Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 48 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,20 @@ jobs:
GOTOOLCHAIN=auto go test -c -o /tmp/runtime-network.test ./internal/lxc
sudo env LXC2DOCKER_NETWORK_TEST=1 unshare --net /tmp/runtime-network.test -test.run '^TestEnsureBridgeNATRemainsBounded$' -test.v

test-boot-regressions:
# Boot regressions from the Aoostar WTR Max outage: the bond rename that
# took it off the network, the DHCP fallback for when the configured
# network doesn't come up, and kernels left without DKMS modules.
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@v6
- name: DHCP fallback script
run: sh packaging/tests/network-fallback-test.sh
- name: DKMS modules for kernels configured before their headers
run: sh packaging/tests/dkms-autoinstall-test.sh
- name: NIC naming link files never rename a bond (real udev)
run: sudo bash scripts/ci/test-link-bond-rename.sh

package:
# The smoothnas package is otherwise only built by the release pipeline.
# Build it, and install it the way the installer does: with apt, from the
Expand Down Expand Up @@ -197,14 +211,35 @@ jobs:
test -x /usr/sbin/tierd
test -f /usr/share/tierd-ui/index.html
test -x /usr/lib/smoothnas/docker-lxc-daemon
for unit in tierd tierd-host-init smoothnas-runtime smoothnas-tls-init smoothnas-gpu-init; do
for unit in tierd tierd-host-init smoothnas-runtime smoothnas-tls-init smoothnas-gpu-init smoothnas-network-fallback; do
systemctl is-enabled "$unit.service"
done
# DKMS built both storage modules for SmoothKernel.
kver=$(dpkg-query -W -f="\${Depends}" linux-image-smoothkernel | sed -n "s/^linux-image-\([^ ,]*\).*/\1/p")
dkms status
dkms status -k "$kver" | grep -E "^zfs/.*: installed"
dkms status -k "$kver" | grep -E "^smoothfs/.*: installed"
# A kernel configured before its headers boots without ZFS: its
# DKMS hook skipped it, and the SmoothKernel headers package never
# reruns the hook. Recreate that state, prove it took (so the
# checks below cannot pass vacuously), and check that both repair
# paths rebuild the module.
unbuilt() {
dkms unbuild -m "$1" -v "$(dkms status -k "$kver" | sed -n "s|^$1/\([^,]*\),.*|\1|p")" -k "$kver"
if dkms status -k "$kver" | grep -E "^$1/.*: installed"; then echo "$1 is still installed for $kver" >&2; exit 1; fi
if [ -n "$(find "/lib/modules/$kver" -name "$1.ko*")" ]; then echo "$1.ko is still in /lib/modules/$kver" >&2; exit 1; fi
}
# Path 1: any later dpkg run (apt.conf.d/53smoothnas-dkms).
unbuilt zfs
apt-get install -y -qq --reinstall hdparm >/dev/null
dkms status -k "$kver" | grep -E "^zfs/.*: installed"
modinfo -k "$kver" zfs >/dev/null
# Path 2: the smoothnas postinst, which an upgrade runs. Not via apt,
# so the Post-Invoke hook cannot be what rebuilt it.
unbuilt smoothfs
DEBIAN_FRONTEND=noninteractive dpkg-reconfigure smoothnas
dkms status -k "$kver" | grep -E "^smoothfs/.*: installed"
modinfo -k "$kver" smoothfs >/dev/null
# The package keeps its own apt source; apt reads it cleanly.
apt-get update > /tmp/update.log 2>&1 || { cat /tmp/update.log; exit 1; }
if grep -E "^(Err|E):" /tmp/update.log; then exit 1; fi
Expand All @@ -215,6 +250,18 @@ jobs:
with:
repository: RakuenSoftware/smoothiso
path: deps/smoothiso
- name: Installer pins NICs by their permanent MAC
run: |
# A MACAddress= selector also matches a bond carrying that NIC's
# MAC, and udev renames the bond at boot.
tmp=$(mktemp -d)
cp -a deps/smoothiso/. "$tmp/"
for p in iso/smoothiso-patches/*.patch; do patch -d "$tmp" -p1 --batch --silent < "$p"; done
grep -qxF 'PermanentMACAddress=$mac' "$tmp/installer.sh"
if grep -nE '^[[:space:]]*MACAddress=' "$tmp/installer.sh"; then
echo "::error::the installer writes a MACAddress= link selector"
exit 1
fi
- name: Build the ISO
env:
SMOOTHISO_DIR: ${{ github.workspace }}/deps/smoothiso
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ If you know Linux and you want an appliance that still feels like Linux, this is
- Create named storage tiers with slot-based array assignment for `NVME`, `SSD`, and `HDD`.
- Run ZFS pools, datasets, zvols, and snapshots alongside the mdadm/LVM path.
- Publish storage over SMB, NFS, and iSCSI.
- Schedule backups of your storage with live progress, throughput, and cancel control.
- Schedule backups over NFS, SMB, SSH or Google Drive, with live progress, throughput, and cancel control. Google Drive jobs support optional encryption and an independent recovery kit.
- Run co-located apps as managed plugins (local LLM inference, runners, media tools) in isolated containers.
- Benchmark local and remote targets with live fio-driven telemetry.
- Monitor disk health and system alerts, and spin idle disks down to save power.
Expand Down
77 changes: 74 additions & 3 deletions docs/OPERATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,11 @@ make install-runtime # installs the daemon + smoothnas-runtime.service
```

The runtime wraps [LXC2Docker](https://github.com/games-on-whales/LXC2Docker)
(defaulting to upstream `main`; pin with `LXC2DOCKER_REF`). Build hosts need
(pinned to a tested upstream commit; override with `LXC2DOCKER_REF`). Build hosts need
`golang-go`, `pkg-config`, `build-essential`, and `lxc-dev`; runtime hosts
need `lxc`, `lxc-templates`, `skopeo`, `umoci`, `rsync`, `nftables`,
`iptables`, `iproute2`, and `uidmap`. SmoothNAS carries no local LXC2Docker
patches — fixes land upstream and are picked up on the next runtime build.
`iptables`, `iproute2`, and `uidmap`. Runtime fixes are pinned upstream; reviewed downstream corrections in
`runtime/patches/` are applied by the same build path used for releases.
See [../runtime/README.md](../runtime/README.md) for detail.

### Kernel and OpenZFS
Expand Down Expand Up @@ -343,3 +343,74 @@ Two specific cleanup tracks are still open and should be treated as engineering
The source deep dive calls both out in more detail:

- [../src/README.md](../src/README.md)

## Google Drive backups

The **Backups** page supports Google Drive alongside NFS, SMB and rsync. The
appliance needs `rclone >= 1.60`; the SmoothNAS package and installer include it.
For a source checkout on an existing machine, install rclone before creating an
encrypted job or starting a Drive run.

1. Open [Google Cloud Console](https://console.cloud.google.com/), create or
select a project, and enable the **Google Drive API**.
2. Configure the OAuth consent screen as **External** and publish it to
**Production**. Tokens issued while the app is in Testing expire after seven
days. Personal-use apps can proceed through Google's unverified-app warning.
3. Create an OAuth client of type **TVs and Limited Input devices**. Copy its
client ID and client secret.
4. In **Backups → Google accounts → Connect Google account**, enter a label and
the client credentials. Open the Google link and approve the displayed code.
The NAS polls Google; no public callback URL or HTTPS reverse proxy is needed.
5. Add a backup job with method **Google Drive**, choose the account, an absolute
local path and a relative Drive folder such as `smoothnas/storage`. Choose
**Push** to back up or **Pull** to restore. Restores always copy and never
delete local files. Push deletion mode mirrors the selected folder.
6. Enter any exclude patterns, one per line (for example `/video/**` and
`/models/**`). These are rclone exclusion patterns, not `+`/`-` filter rules.
7. Optionally enable encryption. **Prepare recovery kit** produces a private
`smoothnas-recovery.conf`; download it and confirm it is safely stored before
saving. Preparation expires after 30 minutes and is lost on daemon restart.
If it expires, cancel and reopen the form to prepare another kit.
8. Enable a schedule, select weekdays and set a time in the **NAS's local time
zone**. Existing jobs remain manual until scheduling is enabled.

Encryption covers file contents, filenames and directory names. Encryption
cannot change after the first run. An encrypted job keeps the Google account
and folder recorded in its recovery kit; create a new job for another target.
The kit includes the OAuth credentials, refresh token and encryption keys, so
keep it private and store it somewhere other than the NAS. The API never
exports saved encryption keys. Losing both the NAS and its kit makes encrypted
backups unrecoverable.

To restore an encrypted job through SmoothNAS, edit the original job and change
its direction to **Pull**, retaining its encryption setting. A newly created
encrypted job generates new keys and cannot decrypt a different job's backup.

To restore without SmoothNAS, install stock rclone on another machine and run:

```bash
rclone --config ./smoothnas-recovery.conf copy sncrypt: /path/to/restore
```

Keep the original Google Cloud project and OAuth client. SmoothNAS requests
`drive.file`, so it can only see files created through that project. Connecting
through a different project cannot discover the old backup. Disconnecting an
unused account removes its credentials from SmoothNAS and leaves Drive files
intact. Google access can be revoked separately in Google Account settings;
revocation may also affect other clients in the same project.

Schedules check every 30 seconds. A missed slot in the last 24 hours is caught
up once, and an already running job causes that slot to be skipped. Repeated
DST hours run once; nonexistent times run at the first valid minute afterward.
A daily Google upload cap ends the run with a continuation summary; the next
scheduled run copies remaining changes. Full Drive storage and other errors
remain failures in run history.

Credentials and encryption keys use the existing SQLite secrets posture. They
are omitted from JSON, command arguments and run errors. Runs use environment
configuration rather than a persistent rclone configuration file.

Implementation checks include mocked Google device-flow tests and real rclone
local-backend backup, restore and independent recovery-kit tests. A real Google
account connection, encrypted upload and recovery on a second machine still
need operator validation before a release is described as live-validated.
23 changes: 23 additions & 0 deletions docs/compose-plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,26 @@ services:
SmoothNAS-specific concerns compose doesn't: tiered-volume bind resolution + pin,
the cross-project host-port guard, and treating `docker compose ps` as the source
of truth for state (cached, with a periodic reconcile sweep).

## Direct web interfaces

For an application that needs its own HTTPS origin, declare a direct link
instead of `x-smoothnas.ui` (the iframe/proxy target):

```yaml
x-smoothnas:
webUI:
scheme: https
port: 8007
path: /
services:
server:
image: example/server:1.0
ports: ["8007:8007"]
```

`webUI.port` must match a fixed published TCP port; `scheme` is `http` or
`https`, and `path` is an absolute path. The Open action uses the hostname of
the current appliance page and opens a new tab. It does not route through
SmoothNAS nginx or inject authentication. The application owns its certificate
and login. See [the PBS plugin guide](pbs-plugin.md) for a complete example.
11 changes: 11 additions & 0 deletions docs/patches/pbs-manifest-fsync.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
--- a/pbs-datastore/src/backup_info.rs
+++ b/pbs-datastore/src/backup_info.rs
@@ -1008,7 +1008,7 @@
path.push(MANIFEST_BLOB_NAME.as_ref());

// atomic replace invalidates flock - no other writes past this point!
- replace_file(&path, raw_data, CreateOptions::new(), false)?;
+ replace_file(&path, raw_data, CreateOptions::new(), true)?;
Ok(())
}

Loading
Loading