feat: add ProbeELF, RequireMinKernel, and check --from-elf#56
Merged
Conversation
Generated, pure-Go tables mapping every BPF helper, program type, and map type to the minimum kernel version that introduced it. Sourced from BCC's kernel-versions.md and Linux UAPI bpf.h at pinned commits via internal/kernelversions/cmd/kvgen, which cross-validates UAPI against BCC and refuses to emit a partial snapshot. Audited cross-validation gaps live in known_gaps.go with one-line rationales. Consumers go through HelperKernelVersion / MapTypeKernelVersion / ProgramTypeKernelVersion accessors, never import tables.go directly. Co-authored-by: Ona <no-reply@ona.com>
Adds ELFProbes / ELFProgram / ELFMap / ELFHelperRequirement / ELFProgramTypeRequirement / ELFMapTypeRequirement / KernelVersion / MemoryAccessSummary / ELFWarning value types and the Requirements() method that projects them to a FeatureGroup consumable by Check(...). Requirements() is the only path callers should use to gate on an ELF snapshot; the per-row metadata (versions, warnings, memory-access summaries) is for inspection and reporting. nil-receiver guard returns an empty FeatureGroup so callers can chain through Probe() failures. Co-authored-by: Ona <no-reply@ona.com>
ProbeELF(path) and ProbeELFWith(path, opts...) parse a compiled eBPF object via cilium/ebpf and populate an *ELFProbes snapshot. probesFromCollectionSpec is the inner helper that operates on an already-parsed *ebpf.CollectionSpec, so unit tests can drive every branch programmatically without clang or a real .o on disk. Output is deterministic (program / map names sorted, helpers deduped per program and globally, helpers sorted by version desc with helper id as secondary key). Unknown program / map / helper kinds fail closed. ProbeELFWith carries // coverage:ignore: its disk-load wrapper needs clang to exercise; the per-branch behaviour is covered through programmatic CollectionSpec fixtures. Co-authored-by: Ona <no-reply@ona.com>
Flags calls to bpf_probe_read / bpf_probe_read_str (split into bpf_probe_read_kernel* / bpf_probe_read_user* in 5.5) and bpf_get_current_task (subsumed by bpf_get_current_task_btf in 5.11). init() binds the rule table to supersededHelperWarnings so the extractor in probe_elf_extract.go picks it up without a circular import. Advisory only: warnings populate ELFProbes.Warnings but never gate Check(...) verdicts. Co-authored-by: Ona <no-reply@ona.com>
ProbeELFWith(path, kfeatures.WithCOREChecks()) opts in to a per- program memory-access classifier. Walks instructions once tracking per-register provenance (Context / MapValue / KernelDirect / COREProtected), classifies each load by source, and clobbers R1-R5 after every helper call. Produces a MemoryAccessSummary per program (Context / MapValue / CORE Protected / KernelDirect / Uncategorized / Total) plus per-program ELFWarning entries for unprotected kernel-direct loads. The accessNotLoad zero-value sentinel ensures non-load instructions are not counted in Total. cilium/ebpf v0.20 does not expose btf.LineInfoMetadata, so lineInfo() is currently a stub and warnings drop file/line attribution. To be revisited when cilium/ebpf re-exposes the metadata. Co-authored-by: Ona <no-reply@ona.com>
RequireMinKernel(major, minor) is a new Requirement item gated by
Check(...) against uname -r. Composes with the kernel-version
snapshot under internal/kernelversions, so ELFProbes.Requirements()
can automatically reject an object whose helpers, program types, or
map types were introduced after the running kernel.
parseKernelRelease handles the common release-string formats
("6.1.0-generic", "6.1.0-1.el9.x86_64", etc.). Constructor panics
on negative version components.
Adds the requirementSet plumbing (minKernels slice, seenMinKernels
dedup map, MinKernelRequirement case in add) and the gating loop in
Check that emits a FeatureError when uname -r is below the required
version.
Co-authored-by: Ona <no-reply@ona.com>
`kfeatures probe` is now a parent command. `probe host` is the canonical name for the live-kernel probe and exposes the same `--json` flag as the v0.5.x `probe`. Bare `kfeatures probe` keeps working byte-for-byte by reusing the host leaf's RunE and mirroring the leaf's flag set onto the parent (cmd.Flags().AddFlagSet); the two share a process-global `probeHostOpts` pointer so flag values land in the same memory regardless of which surface parsed them. `probe bpf <path.o>` runs ProbeELF on a compiled eBPF object and prints the resulting *ELFProbes snapshot. Flags: `--with-core` opts in to the CO-RE memory-access classifier, `--requirements` prints only the FeatureGroup projection consumed by Check(...), `--json` switches both to JSON. MCP exposure: structcli's MCP registry only registers runnable leaves, so the parent `probe` is auto-excluded by shouldIncludeMCPCommand and tools/list returns `probe-host` / `probe-bpf` (alongside check / config). The bare `probe` alias is a CLI ergonomic affordance only; agents see the canonical names. Bats coverage: cli_common updated for the new help text and host/bpf help leaves; cli_linux gains parity, --json, missing-arg, and nonexistent-file tests for the new commands; cli_mcp asserts the new tools/list shape and uses `probe-host` for live-kernel tools/call cases. Co-authored-by: Ona <no-reply@ona.com>
`kfeatures check --from-elf <path.o>` derives the FeatureGroup from a compiled eBPF object via kfeatures.FromELF and gates on it. Composes with --require: the union of both is gated. The two flags are now both optional and at-least-one-required (drops `flagrequired:"true"` from --require; the runtime check returns the same "no features specified" message wrapped in the structcli envelope). Extracted assembleCheckRequirements out of the RunE so the union logic can be unit-tested without going through cobra. Bats coverage: cli_common gains a missing-file test for --from-elf and a bare-invocation test asserting the new error message; the old "missing required flag exit 10" test is removed (the flag is no longer flagrequired). Co-authored-by: Ona <no-reply@ona.com>
Weekly cron (Mon 06:00 UTC) plus workflow_dispatch with optional SHA overrides. Resolves iovisor/bcc and torvalds/linux HEAD via `git ls-remote`, rewrites the defaultBCCCommit / defaultKernelCommit constants in cmd/kvgen/main.go, regenerates the snapshot, and verifies the result with `go vet` plus the kernelversions test package. Opens a PR via peter-evans/create-pull-request@v6.1.0 only when internal/kernelversions actually drifted; the `deps(kernelversions): refresh BCC + UAPI snapshot` title prefix routes the PR into the dependencies bucket of release notes via the release.yml configuration, and the `dependencies` label is also applied explicitly as a belt-and-braces measure. The PR body surfaces both pinned SHAs and references internal/kernelversions/cmd/kvgen/known_gaps.go as the escape hatch when UAPI/BCC drift causes cross-validation failures. Co-authored-by: Ona <no-reply@ona.com>
…on snapshot README: adds runnable examples for ProbeELFWith(WithCOREChecks()), RequireMinKernel, the new `probe host` / `probe bpf` / `check --from-elf` invocations; updates the MCP tools list to `probe-host` / `probe-bpf`; refreshes the detection table and comparison table to mention CO-RE memory-access classification and min-kernel derivation. CONTRIBUTING: adds RequireMinKernel and ProbeELF / ProbeELFWith to the requirement-items table; clarifies that FromELF stays frozen against its 4-point contract while new extraction surface (warnings, CO-RE) belongs on ProbeELF; new section on the kernel-version snapshot lifecycle (auto-refresh workflow, manual go generate, the known_gaps.go allow-list policy); documents the make cover-check gate. AGENTS: adds the agent-actionable rules for the kernel-version snapshot (don't hand-edit, use the bot, when to grow known_gaps.go, go through the accessor functions) and for the coverage gate (when to extend COVER_FILES, when a // coverage:ignore marker is acceptable, the rule against bumping COVER_THRESHOLD per-file). CHANGELOG: five new `[Unreleased] Added` entries covering ProbeELF, RequireMinKernel, the kernel-version snapshot + auto-refresh workflow, superseded-helper warnings, and the new CLI surface. Co-authored-by: Ona <no-reply@ona.com>
`make cover-check` runs the test suite with -coverprofile=coverage.out and then runs internal/tools/covercheck against that profile, failing if any gated source file falls below COVER_THRESHOLD (default 90). The gated set lives in the COVER_FILES makefile variable; it covers the public/library files added in this branch only. Internal tools (kvgen, covercheck itself) are intentionally excluded — their happy paths are exercised by the scheduled refresh workflow against live data. The checker honours a single-line `// coverage:ignore` marker placed in the doc comment of a function declaration or of a `var foo = func(...)` declaration. The marker excludes every statement attributed to that function from both numerator and denominator. Used for the disk-bound ProbeELFWith wrapper (branches exercised through programmatic CollectionSpec fixtures against the inner probesFromCollectionSpec) and for the test-time stub variables overwritten by package init functions. Wired into ci.yml as a new "Coverage gate" step that runs after the test step. `golang.org/x/tools` promoted from indirect to direct because covercheck imports `golang.org/x/tools/cover`. Co-authored-by: Ona <no-reply@ona.com>
leodido
force-pushed
the
feat/from-elf-deepen
branch
from
f51fdc5 to
407a0d5
Compare
Clear stale register provenance after destination overwrites unless an instruction explicitly propagates provenance. Abort kvgen table emission when cilium name discovery fails so generated tables cannot silently omit rows.
Keep tables.go as the committed generated snapshot and stop writing source.json from kvgen. Update refresh docs and add a generator test that asserts source.json is not emitted.
leodido
changed the title
FromELF into ProbeELF + RequireMinKernel + check --from-elf
leodido
changed the title
FromELF into ProbeELF + RequireMinKernel + check --from-elf
leodido
changed the title
ProbeELF, RequireMinKernel, and check --from-elf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
FromELFreturns the requirement items thatCheck(...)consumes. That is enough to gate an object, but not enough to explain why an object will or will not load on a target kernel: which helpers are superseded, which memory loads are not CO-RE protected, and what minimum kernel version the object needs.This PR keeps
FromELFfrozen against its existing contract and adds richer ELF inspection throughProbeELF/ProbeELFWith, plusRequireMinKernel(major, minor)and CLI support for using ELF objects as requirement sources.What This Changes
Library
ProbeELF(path)andProbeELFWith(path, opts...), returning an*ELFProbessnapshot with programs, maps, helper-per-program requirements, derived minimum kernel version, and warnings.Requirements()projection from*ELFProbesback to aFeatureGroup, soProbeELFremains a strict superset of theFromELFrequirement surface.RequireMinKernel(major, minor), gated againstuname -r, and wires ELF-derived helper/program/map requirements into the minimum-kernel snapshot.WithCOREChecks(), including context, map-value, CO-RE-protected, kernel-direct, and uncategorized access buckets.FromELFitself unchanged: parser-only, cross-platform, deterministic, fail-closed, and still returningFeatureGroup.Kernel-Version Snapshot
internal/kernelversionswith generated pure-Gotables.godata for helper, program type, and map type minimum kernel versions.internal/kernelversions/cmd/kvgen, sourced from BCCkernel-versions.mdand Linux UAPIbpf.hat pinned commits.kvgenfail closed on UAPI/BCC validation gaps, cilium name-discovery failures, and missing kernel-version rows instead of emitting partial or zero-valued tables.source.jsonis intentionally not versioned or emitted.CLI
kfeatures probeinto a parent command while preserving bareprobeas the host-probe alias.kfeatures probe hostas the canonical live-kernel probe.kfeatures probe bpf <path.o>with--with-core,--requirements, and--json.kfeatures check --from-elf <path.o>, allowing--requireand--from-elfto satisfy the at-least-one-requirement rule.check,config,probe-bpf, andprobe-host.CI, Coverage, And Docs
tables.go, and opens a dependencies-labelled PR when the snapshot drifts.make cover-checkplusinternal/tools/covercheckto enforce per-file coverage on gated files.Stability
FromELFremains unchanged and additive callers can migrate toProbeELFonly when they need richer diagnostics.RequireMinKernelcomposes with the existing requirement model.probeCLI path remains available as a compatibility alias.check --require=...continues to work;--from-elfadds a second requirement source.Verification
go test ./... -count=1make cover-checkgit diff --checkinternal/kernelversions/tables.gois included;internal/kernelversions/source.jsonis not included.