Conversation
… believes, and which bytes it will stage The first piece of the update channel (docs/design/update-channel.md), and the part that is pure: no socket, no file, no signature. A pointer's text is parsed strictly, with an unknown key refused rather than skipped. A verified pointer is accepted for this image's role and never backwards: an `issued` earlier than the newest already accepted is a replay, however valid its signature. Its age says whether it is stale. A relative base is resolved against the channel address from the verified root, and only a development image may be pointed at plain http. For staging: nothing but the manifest and its signature, whole and small, until they have verified; after that only a name the signed manifest lists, at exactly the offset already held, never past the signed size. `still_needed` is the answer to a poll: what is missing, and from which byte. Versions compare as `sort -V` orders them for kryptik-update. Nothing is wired to the broker yet; nine unit tests cover the rules.
…, with the code that checks a payload
Two read-only subcommands, which zone 0 runs before it believes anything
the net zone hands it:
check-manifest DIR the signature, role and version steps of `apply`
on a manifest that has no payload beside it yet;
prints the version, the hash of the copy it
verified, and each listed file with its size
check-pointer FILE SIG the signature over a statement of what is
current, in a namespace of its own
The manifest steps were the first half of verify_payload; they are now
verify_manifest, which verify_payload calls, so `apply` and the channel
cannot come to disagree about what a valid manifest is. A downgrade is
refused as in `apply`: nothing that arrives over the network is a
recovery. The signature check is one function with the namespace as an
argument: kryptik-release for a manifest, kryptik-latest for a pointer,
so neither signature can be presented as the other.
Neither check needs root or this installation's disks, so the tool asks
for those only for the subcommands that write.
The suite that lifts verify_payload out of the tool now covers both
checks with the same real ssh-keygen: the other namespace, a stranger's
key, another role, an older release, a listed path that climbs, an
edited pointer, a manifest presented as a pointer, and the tool itself
run without root. It had been registered nowhere; it is now
`make test-update-verify` and runs with the offline suites. 18 rows.
…verbs does with it The stateful half of update.rs, still not wired to the broker. Under /var/lib/kryptik/update, root's alone: the newest statement accepted, when one was last looked at, the version the person asked for, the verified manifest's file list, and the staging directory `apply` will be given. latest: one statement is looked at per hour, whoever sent it; its signature is checked before a word of it is parsed, and it replaces the stored one only if it is for this image's role and not older. want: the person asks; nothing is fetched that was not asked for. poll: what is wanted, the base address from the verified statement, and what is still missing and from which byte, or idle. put: the manifest and its signature first; when both are there they are verified, held to the hash the statement announced and measured against the room there is, and a refusal throws both away. Only then is a listed file taken, in pieces of at most 1 MiB, each appended at exactly the offset held. The two signature checks are passed in as functions, so the tests stand in for kryptik-update and run against a directory of their own: a statement that does not verify, the interval, a replay, the whole staging order with a dropped connection in the middle, a manifest with another hash, another version, a size nothing could hold, and no signature. The staged directory ends up holding exactly what `apply` accepts. Thirteen tests.
…e person
Three verbs on the zone-facing socket, taken from the zone that holds the
network and from no other, like time-offset:
update-latest <plen> <slen> a statement of what is current and its
signature, at most 8 KiB each
update-poll idle, or what is wanted, from where, and
what is still missing and from which byte
update-put <name> <off> <len> one piece of the release, at most 1 MiB
Who is asking is settled before a byte of payload is read. A release
crosses in pieces, each one request answered between two looks at the
zone, so supervision is never further away than one piece; the log gets a
line for what ends something, not for each of some thousands of pieces.
What is believed and what is stored is update.rs's decision, and every
signature is kryptik-update's.
For the person, through the launch service like `kryptik wifi`:
`kryptik update status` (the release running, the newest known and how
old that news is, what is staged), `fetch` (the asking without which the
net zone is told idle) and `apply`, which hands the staged directory to
kryptik-update apply: it verifies all of it again before it writes a
slot, and the trial boot and the fallback are the ones that exist.
Tests: the verbs' grammar and bounds and the refusal by identity as unit
tests (202 pass); five rows in the boundary suite, against a real zone,
for the three verbs from a zone that does not hold the network and two
requests outside the grammar. The serve and cli suites pass unchanged.
…er, and nothing more tools/net/update-fetch.py is the net zone's half. `latest` brings the signed statement of what is current and its signature to zone 0; `poll` asks whether a release is wanted and streams what zone 0 says is still missing, from the byte zone 0 names, in pieces of at most 1 MiB. It decides nothing and holds nothing: the address comes from zone 0's configuration and then from the statement zone 0 verified, a piece zone 0 refuses ends the run and is not sent again, and a server that ignores Range has the bytes before the offset dropped rather than sent. netzone-init.sh runs it in the background, one at a time: the statement every half hour until zone 0 has taken one and daily after that, the question "is a release wanted?" every minute, which is one line on a local socket and is how `kryptik update fetch` is noticed. With no /etc/kryptik/update.conf there is no channel and nobody is asked; the nic zone now sees that file read-only, like the time sources. Stage 04 installs the fetcher and compiles it under the target python. `make test-update-fetch`: a real HTTP server on loopback and a stand-in for zone 0's broker. A whole release byte for byte, the manifest first, no piece over 1 MiB, a download cut at a byte resumed from that byte with and without Range, a refusal, no channel, a dead host. Eleven rows.
… what is left The design's status, the paragraph on how a release crosses the broker (pieces of at most 1 MiB, one request each, rather than a copy handed to a child of the launcher: no second process and no state between pieces but the staged file's length), where the channel address lives and what its absence means, and the list of files and suites. The roadmap item stays unticked: what is left is the row that needs the installed system (a release fetched over the test network, staged, applied and committed) and release tooling that publishes a signed pointer.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…ot the caller Code scanning reads a peer's uid in a log line as sensitive data, which it is not, and raised one alert on the new line. The other verbs' lines carry the same finding, dismissed as a false positive. This one does not need the argument: the launch service answers the session's user and root and nobody else, so the uid on an update line says nothing the service's own rule does not, and the line goes without it.
…rom the environment
Found in review. The directory the manifest is copied into was
`${SNAP:-$(mktemp -d ...)}`, with nothing above it clearing SNAP, and the
exit trap is `rm -rf -- "$SNAP"`. So `SNAP=/some/dir kryptik-update ...`
as root changed that directory's mode, copied a manifest into it and
removed it recursively on the way out, whatever it was. It needs someone
who already controls root's environment, so it is a trap for the unwary
rather than a boundary; but the update channel adds callers, and a
script with rm -rf in its exit trap should not take the path from
outside. kryptikd clears the environment when it runs the tool; the next
caller might not.
One line: SNAP is set empty beside the other settings. The suite is not
affected, since it lifts the functions out and sets SNAP in a shell of
its own; it gains a row that runs the real script with SNAP naming a
directory that holds a marker, and requires the marker to survive. The
row fails against the script without the line. 19 rows.
…second key, for statements only Found while reading the build for the installed-system test. The trust anchor stage 04 installs has always restricted the release key to `namespaces="kryptik-release"`. check-pointer verifies in kryptik-latest against the same file, so on a real image no pointer could ever verify: the one key enrolled is not honoured in that namespace. It failed closed, and the offline suite did not see it because its anchor was one line with no namespaces, which accepts what the installed system refuses. The anchor now has two lines. The release key, honoured for manifests and nothing else, as before; and a second development key made beside it, enrolled as kryptik-latest and honoured for statements of what is current and nothing else. That is the design's "separate freshness key", enforced by the anchor rather than by convention: the key that has to be at hand on a schedule cannot sign a release, and the key that signs releases never has to be at hand. An owner who wants one key lists the release key on the second line; the design says so. Stage 04 proves both keys both ways round on every build, and the probe fails against an anchor without the namespaces. The workflow keeps the new private half out of the acceptance artifact, like the other two. verify_signed tries every principal a key is enrolled under, so one key on two lines works as well as two keys. The suite's anchor is now shaped like the real one and gains two rows: a pointer signed by the release key and a manifest signed by the statement key are both refused, whatever namespace they sign in. 21 rows.
…nd stage 06 publishes one beside each payload `release-manifest.sh pointer --key K --manifest M --base B --out FILE` writes the update channel's KRYPTIK-LATEST-1 for a manifest that is already signed (its version and role, its SHA-256, the base, the date) and signs it in kryptik-latest. Re-running it with a later date for an unchanged release is how a channel shows nothing is being withheld. It is the first piece of the release tooling the roadmap asks for, and it is here now because the acceptance job is never given a private key, so a statement for the update suite has to be made where the manifest is signed. Stage 06 writes images/channel-<version>/latest and latest.sig beside the payload, outside it because `apply` refuses a payload holding anything unlisted; verifies it through the anchor the image carries; and writes a control, the same statement signed by the release key in the manifest's namespace, which the image must refuse. Stage 04, from review: the probe's negative half requires the signature to exist before it counts a refusal, so it cannot pass because signing failed. And a rule for the keys like the one for the kernel tree: they live outside the sysroot and outside any cache of it, so a restored tree with release-trust stamped and a key missing loses the stamp and the step makes both keys and writes the anchor again. It has been rerunning on cached builds only because an earlier step happened to go stale. Suite: six rows, 46 in all. What the tool writes is what kryptik-update's own check-pointer accepts under an anchor shaped like the image's; one signed with the release key is refused; a re-issue changes only the date; no statement is written about an unsigned manifest.
… to end, on the installed system Not yet run: it needs a Distro run, and none is free. Written to be read. Step 3 gains two lines that need no network. The signed statement that B is current, which stage 06 now writes beside B's payload, rides on the refusals disk with its control; the real updater judges both against the image's real anchor. The statement verifies, signed by kryptik-latest; the same text signed by the release key does not. Step 8, last, from the state step 7 leaves (slot b, B committed). Nothing newer exists to fetch there, so first a rollback to slot a, and the copy step 7 applied from is removed, so the staged release is the second payload's worth on kryptik-state and not the third. Then a release host on this side of QEMU's user network, bound to loopback, serving B's statement and links to B's payload; the guest with a network for this step and no other. Zone 0 names the channel and the net zone's service is restarted, because the zone sees that file only from its next launch. In order: status names B once the net zone has brought the statement; seventy seconds later nothing has been staged, because nobody asked; the person asks; the release arrives and is complete, and the staged directory holds exactly the six files; `kryptik update apply`; slot b trial-boots and commits; the home file is intact. The release host's log must show the statement, then the manifest and its signature, before any image. Each wait is a loop in the guest that gives up before the driver would, in a subshell so that giving up is not the login shell's exit, and as a failed command, because the driver judges the exit status and the word it then expects is also in the command line the console echoes. The statement is made in stage 06 and not here because the acceptance job is never given a private key.
…it that fails on a stall, a release host that cannot outlive the suite From the review of the step, by the suite's author. The net zone is restarted the way the guest checks already do it and have proven: down, a pause, up, then a NEW "netzone: READY" line in the catch-all log, and only then is `status` read. `s6-svc -r` is proven nowhere, and before the new line the old zone could still answer. The wait for the release is no longer a clock. Gigabytes through QEMU's user network on a nested-KVM runner may take longer than any one driver step allows, so each wait fails when the staged line has not changed for 100 seconds, succeeds at "complete", and also succeeds at the end of its six minutes if bytes were still arriving, for the next wait to take over; four of them, then "complete" is required. A stall fails in under two minutes and a slow link costs waits, not the run. The release host is tools/image/release-host.py, shared with the fetch suite, which had its own copy: static files on loopback and a port the kernel picks, Range honoured with a 206 as the design asks of a release host (python's http.server ignores it, and a resumed fetch would re-read gigabytes), streamed rather than read whole, a name that leaves the root refused, one log line per request. It is killed from an EXIT trap, so no way out of the suite leaves it behind. And two commands given to the guest as root had single quotes in them, which the driver wraps its `su -c` argument in; they use double quotes now, and the restart command ends in a subshell rather than an exit, or the driver's own marker after it would never print. Checked offline: the fetch suite on the shared host, 11 rows; a 3 GiB sparse file streamed whole and from a Range near its end; the arrival wait against stand-ins for complete, stalled and still arriving; both suites parse and pass shellcheck. The step itself still needs a run.
The seed file came in with the parser tests, which said the new verbs were worth a line each once they existed. Six lines: the poll, a statement inside and just past its size limit, a piece of the manifest, a piece of an image at an offset, and a name that climbs. The test's zone holds no network, so every one of them is refused by who is asking before a byte of payload is read, and nothing the fuzz sends can reach the staging area. 203 unit tests.
… touches the filesystem with it Code scanning raised three path alerts on tools/image/release-host.py: the requested path reached isfile, getsize and open. The check that the normalised name stays under the root was there, but it shared one condition with the isfile call, so the name was used before it had been judged. The check now stands alone and first, and only a name that passed it reaches any filesystem call. Behaviour is the same, probed both ways: a file and a link inside the root are served (the suites link to a payload rather than copy it), /../ and /sub/../../ outside it are 404, as are the root itself and a directory. The fetch suite passes on it, 11 rows. It binds loopback only and exists for the suites; the alerts were still right about the order.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the update channel design. It builds on the clock (#16, now on main): it reuses the clock's date parser and its "only the zone that holds the network" rule, and freshness means nothing without a clock the net zone cannot set. Main is merged in, so this diff is the channel alone.
What it does
A release reaches a machine today on a disk someone carries to it. With this, the net zone fetches one and zone 0 stages it, without zone 0 getting a network and without the net zone being trusted with anything.
The pieces
kryptikd/src/update.rsissuednever goes backwards), staleness after 30 days, a relativebaseresolved against zone 0's channel address only, plain http for a development image only,sort -Vversion order. Then the state under/var/lib/kryptik/updateand what each verb does with it.broker.rsupdate-latest,update-poll,update-put. Who is asking is settled before a byte of payload is read.kryptik-updatecheck-manifest DIRandcheck-pointer FILE SIG. The manifest steps were the first half ofverify_payload; they are nowverify_manifest, whichapplyalso calls, so the channel andapplycannot disagree about what a valid manifest is. Pointers are signed in their own namespace (kryptik-latest), so neither signature can be presented as the other.serve.rs,kryptik-launch.c,tools/kryptikkryptik update status | fetch | apply, through the launch service likekryptik wifi. Nothing is fetched that the person did not ask for.tools/net/update-fetch.py,netzone-init.shrootfs.rs, stage 04/etc/kryptik/update.confread-only; the image installs the fetcher.Where it is simpler than the design
The design handed each copy to a child of the launcher. A release instead crosses in pieces of at most 1 MiB, each an ordinary request under the broker's existing five-second deadline, like
clipboard-set. Supervision is never further away than one piece, and there is no second process, no long-lived connection and no state between pieces other than the staged file's length. The design document says so now.The most a hostile net zone can make zone 0 store is unchanged: the declared size of one release the release key signed, once, checked against free space before the first byte.
Tests
update.rs: 13 unit tests. The whole staging order against a temporary directory with stand-in signature checks, including a dropped connection, a replay, the hourly interval, and a manifest with another hash, another version, a size nothing could hold, or no signature. The staged directory ends up holding exactly whatapplyaccepts.broker.rs: the verbs' grammar and bounds; the refusal by identity.make test-update-verify(21 rows, realssh-keygen, an anchor shaped like the image's): the suite that liftsverify_payloadout of the tool now covers both checks: the other namespace, a stranger's key, another role, an older release, a listed path that climbs, an edited pointer, a manifest presented as a pointer, the tool run without root, and aSNAPin the environment that the tool must neither use nor remove (found in review: the exit trap isrm -rf "$SNAP", and nothing cleared it; one line fixes it, and the row fails without the line). This suite existed but was registered nowhere; it now runs with the offline suites.make test-update-fetch(11 rows): a real HTTP server on loopback and a stand-in broker. A whole release byte for byte, manifest first, no piece over 1 MiB, resume from a byte with and without Range, a refusal, no channel, a dead host.On the installed system
Written, and not yet run: it needs a Distro run, and none has been free.
tools/release-manifest.sh pointerwrites and signs the statement of what is current for a signed manifest; stage 06 publishes one beside each payload (images/channel-<version>/latest, outside the payload becauseapplyrefuses anything unlisted), verifies it through the image's anchor, and writes a control signed by the release key that the anchor must refuse. It is made in the build because the acceptance job is never given a private key. Six rows intest-release-manifest.sh(46 in all), including thatkryptik-update's owncheck-pointeraccepts what the tool writes.update-test.shstep 3: the real updater judges the statement and its control against the real anchor, no network.update-test.shstep 8: back to slot a by rollback, a release host bound to loopback on the host side of QEMU's user network, the net zone brings the statement, nothing is staged untilkryptik update fetch, the release arrives whole,kryptik update apply, slot b trial-boots and commits, data intact; the host's log must show the statement, then the manifest and its signature, before any image.Not in this PR
/etc/kryptik/update.confis shipped, so on an image built from this the channel is off until someone names one: the net zone asks nobody andupdate-pollanswersidle.Which key signs the pointer
The design leaves this to the owner and lists three options. An earlier version of this text said
check-pointerverifies against the release key meanwhile. That was wrong, and it was a defect: the trust anchor stage 04 installs restricts the release key tonamespaces="kryptik-release", so on a real image no pointer could ever have verified. It failed closed, and the offline suite missed it because its anchor had no namespaces. Fixed here: the development build makes a second key beside the release key and enrols it askryptik-latest namespaces="kryptik-latest", which is the design's separate freshness key, enforced by the anchor. Stage 04 proves on every build that each key verifies in its own namespace and is refused in the other's; the workflow keeps the new private half out of the acceptance artifact; the suite's anchor is shaped like the real one and has a row each way (21 rows). An owner who wants one key lists the release key on the second line; nothing else changes./etc/kryptik/update.confis shipped, so on an image built from this the channel is off: the net zone asks nobody andupdate-pollanswersidle.Merge order
It touches
compartments/,tools/and stage 04, so it starts a Distro run; not while one is in flight.