Repository navigation
feat!: enforce unresolved facets at run time with an object table - #41
Merged
Merged
Conversation
Implements RFC 0032 (runtime enforcement). Every enforcing weavec-cc build now links a runtime whose allocator keeps each heap object's exact extent and liveness findable from a pointer, and which tracks escaping stack objects and globals. A spatial or temporal facet that the analysis leaves unresolved and that has a pointer operand becomes `guarded`: a guard inserted at the operation asks the object table and traps on an access outside the object, on a use of a released object, and on an invalid release. Temporal bugs are caught at run time for the first time. - runtime/: libweavec_rt.a (size-class arena allocator with slot metadata, quarantine, huge blocks, Darwin malloc zone, stack and global object tables, guards, reports) and libweavec_alloc.a (malloc, calloc, realloc, free for the image; weak shims for the rest), with a C test that runs on Darwin and Linux. - Core: the sixth outcome `guarded` (rank between checked and unresolved), the check templates object, live and release, the require level `guarded`, ledger schema version 2, unit record format 30. - Analysis: the planner turns unresolved facets into guards after the engine has published; library-call requirements get object witnesses. - Frontend: guard helpers in the prelude with an inline arena lookup, per-function range caches and quiet loops, registration of escaping locals and of globals, the driver's link line and its fallbacks (sanitizers, -nostdlib, a program that defines the allocator, a link without the runtime), -fweavec-runtime, -fno-weavec-stack-objects, -fno-weavec-global-objects, -Wweavec-possible, --require=guarded. - Possible temporal findings on guarded facets are no longer printed by default in enforcing builds; -Wweavec-possible prints them. - Verify mode guards proven temporal facets too. - Fixes a false proof: the place class of an array-element cell at a boundary (a dangling pointer stored into g[1] or r->slot[i]). - Fixes a second false proof: a cleanup-attribute function's call at scope exit is now an unknown callee handed the variable. - Fixes RFC 0030's span check for *(p - k), which checked p. - The corpus gate times three builds and peak memory (G14), checks the unresolved shares (rfc0032.R4), reads runtime reports from a log, and has eight injections that exercise the guards; lz4's randomised tests take a fixed seed. BREAKING CHANGE: the ledger JSON is schema version 2 and unit records are format 30; the summary line prints a guarded count; default builds link libweavec_rt.a and libweavec_alloc.a and replace the image's allocator (-fno-weavec-runtime restores the previous behaviour, with guardable facets left unresolved). Measured cost in the default mode against the reference compiler: cJSON 1.66x, zlib 1.85x, Lua 5.94x user CPU (1.14x, 1.00x, 1.11x with -fno-weavec-runtime). The RFC's bounds for zlib (1.5x) and Lua (2.0x) are not met; RFC 0032's implementation amendment 3 records the measurement and carries the gate forward to static guard elimination.
… dumps - CheckPlanner, CheckEmitter, Driver, ObjectRegistration, Ledger: the clang-tidy findings in the new code (precedence parentheses, const references, forwarding references, a parameter name, noexcept on a function that may throw, a CRTP hook). - Many tests trap on purpose. On Ubuntu each crash goes to apport, which handles them one at a time, so parallel trapping runs waited on each other: the runtime test took 258 s and one case run passed its 10 s timeout. The case runner and the runtime test's trap children now set RLIMIT_CORE to 0, and CI's test step runs with `ulimit -c 0`.
…n report - weavec_objects.c: a grown stack list's mapping is a struct with its size ahead of the entries, so no pointer is scaled by the wrong type. - weavec_report.c: WEAVEC_RT_REPORT_LOG is ignored in a set-user-ID or set-group-ID process (AT_SECURE, issetugid) and opened with O_NOFOLLOW and O_CLOEXEC. - CI and the dev-asan test preset run with allow_user_poisoning=0: WeaveC is ASan-instrumented while the system libclang-cpp is not, and LLVM's inline BumpPtrAllocator poisons slab tails in one copy that the other writes without unpoisoning (a false use-after-poison in Emission/ rfc0030-trap-runtime.c, where WeaveC now interns new identifiers).
The four configs CI measures on Linux (log.c, cJSON-program, linenoise-program, jansson) were still at their RFC 0031 numbers: the new guarded counts were missing and the unresolved counts had improved. Recorded with --update-from the corpus-gate-quick-ubuntu-24.04 artifact of run 37100674548.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements RFC 0032, Runtime enforcement. Every enforcing
weavec-ccbuild now links a small runtime: an allocator that keeps each heap block's exact extent and liveness findable from a pointer (with a quarantine for freed blocks), plus tables of escaping stack locals and globals. A spatial or temporal facet that the analysis leaves unresolved, and that has a pointer operand, becomes a sixth outcome,guarded. An inserted guard asks the object table and traps on an access outside the object, on a use of a released object, and on an invalid release. Temporal bugs are caught at run time for the first time.On the corpus, unresolved shares fall to 5.6% spatial / 14.3% temporal for the original projects and 9.5% / 18.7% for the held-out ones (gate R4: at most 12% / 20%). Before this change they were roughly 36–42% spatial and 35–37% temporal. All 39 corpus injections are detected.
Design notes
runtime/).libweavec_rt.aholds the size-class arena with slot metadata words, the quarantine, huge blocks, the Darwin malloc zone, the stack and global tables, the guards and reports.libweavec_alloc.adefinesmalloc,calloc,reallocandfreefor the image, with weak definitions for the other allocation entry points.runtime/test/rt_test.cruns as theruntimeCTest.guardedoutcome ranks between checked and unresolved. There are three new check templates (object,live,release) and a new require level,guarded. Ledger JSON is now schema version 2 and unit records are format 30.CheckPlanner::planGuardsturns unresolved facets into guards after the engine has published. The prelude's guard helpers look up arena pointers inline, with per-function range caches and "quiet" loops that skip state reads. Escaping locals and globals are registered. The driver adds the runtime to the link line, and falls back with a note under sanitizers,-nostdlib, a program that defines its own allocator, or a link without the runtime.-fweavec-runtime,-fno-weavec-runtime,-fno-weavec-stack-objects,-fno-weavec-global-objects,-Wweavec-possible,-fweavec-require=guarded, and--no-runtimefor theweavectool. New environment variables:WEAVEC_RT_QUARANTINE,WEAVEC_RT_STATS,WEAVEC_RT_REPORT_LOG.__attribute__((cleanup))functions, which the engine did not see. Their call at scope exit is now an unknown callee handed the variable.spancheck for*(p - k), which checkedpinstead of the accessed address.Cost, gate R6, amended. These are the default mode's user CPU over the reference compiler:
-fno-weavec-runtimeThe RFC's original bounds for zlib (1.5×) and Lua (2.0×) are not met. The Lua benchmark executes about 4.5 × 10⁹ guards at roughly 3 cycles each, so closing the gap needs guards removed at compile time; that is carried forward to a follow-up RFC. The manifest's G14 limits are set to what was measured (amendment 3).
Other limits raised to measured values (amendments 3, 12 and 20):
Known limits, documented in the guarantees reference:
Breaking changes:
-fno-weavec-runtimerestores the previous behaviour, with guardable facets left unresolved.Test plan
test/Driver/runtime-{link,flags}.cpin the link line, the fallbacks, the notes, the ledger fields and the require levels.test/Emission/runtime-oracle-*.crewrite oracles cover the guards, range caches, and stack and global registration.test/cases/semantics/runtime/, plus the alias probes, the span-difference cases, andTRAPmarkers on 16 soundness probes.ctest --preset devpasses 705 of 705, andscripts/run-cases.pypasses 532 of 532 in trap,--asanand--checks verifymodes.ci-releasepreset passes 703 of 705. The two failures come from the container itself (noccinstalled; tests run as root).runtime/testalso passes on Linux x86-64 under emulation. Linux x86-64 as a whole is CI's to confirm.scripts/corpus-gate.py --fullpasses G9–G13, rfc0032.R4, RFC 0031's G5, G6 and G12, and G14/G15 with their amended limits. Every config builds and passes its tests with no trap, except two known cases: sqlite stops at its three triaged-true errors, as before; jansson has four triaged-true guard failures. With--checks verifythe corpus has noweavec.proventrap.expected.jsonis re-recorded from that run.scripts/check-hygiene.py,scripts/check-format.sh, andnpm test/npm run buildindocs/pass.Checklist
scripts/format.sh). clang-tidy was not run locally; theci-tidyjob will check it.docs/architecture.md,docs/development.md,AGENTS.md, and the test and corpus READMEs.lib/Core.