Repository navigation
CLI: tracekit help shows every command, grouped, and the main features - #619
Merged
Merged
Conversation
`tracekit --help` printed every subcommand as one flat list in the order they were registered, and `tracekit` alone was an argparse error. - `tracekit help` and a bare `tracekit` print an overview: each command under what a person runs it for, one entry each from its own help text (up to where its usage starts), experimental ones marked as their help marks them, then the features that are not commands (witnesses and anchoring, Tracekit Box, the SDKs and integrations) with their pages. - `tracekit help NAME` is `tracekit NAME --help`; an unknown name lists the commands and exits 2. - The parser is built in `_parser()` so the overview and the tests read the registered commands; tests/test_cli.py checks every command is in exactly one group. - docs/quickstart-v2.md names `tracekit help`.
…elp's own description From #619's review: - The overview cut each help at the first `; ` or `: \``, which dropped audit's "read-only, not a signed record". It now cuts only where usage starts: a `; ` or `: ` before a backquote, `--` or a URL. - daemon, observe, ingest, otel and proxy (the v1 path) move to a "Legacy (v1)" group, so they don't read as current server tools. - `tracekit help causeway|sql|proofpack` gives the "moved to a separate package" answer `tracekit causeway` gives. - `tracekit help help` has a description and its argument a help line.
LoopGlitch26
added a commit
that referenced
this pull request
Nov 6, 2026
#619's help test needs every command in exactly one group.
LoopGlitch26
added a commit
that referenced
this pull request
Nov 6, 2026
… (counts only) (#636) * Adoption: a week summary at the end of audit, and `tracekit feedback` (counts only) `tracekit audit` text now ends with the week in three lines: the calls, how many Tracekit would have held and blocked, the rules that stopped them most, and `tracekit init --dev --v2`. It reuses the audit's own counts and rule sections. `tracekit feedback` prints a pre-filled link to the field-report issue form and exactly what it holds: the Tracekit version, OS, Python, which coding agents' hooks are installed, and the dev store's `report --share` week when there is one (for #385). It sends nothing; `--open` hands the link to the browser. `--harmless <approval-id> [--why]` files one false alarm with the approval's rule ids (others as "custom rule"), tool name, policy hash and verdict, and reads no other field, so the command, arguments and paths never reach the link. `tracekit approvals show` prints the line that runs it. * Field reports as pull requests, and each rule's holds with what people did with them `tracekit feedback --file PATH` writes a field report to a new file: the Tracekit version, OS, Python, which agents' hooks are installed, the dev store's `report --share` week, a "Run by" line and one optional line. It prints where the file goes: docs/field-reports/<github-login>-<date>.md in a fork, as a pull request with `git commit -s`. docs/field-reports/README.md says what a report is and how to file one; the folder is data, kept out of the wiki. tests/test_feedback.py checks every report there for its lines, a length cap, and no absolute or home path, email or secret-looking string, and its failure says what to remove. `tracekit report --share` now ends with each rule's holds: how many calls it held, how many a person approved and how many were rejected or expired, read from the signed approval records (view.review's outcome for each held call, the agent's retry counted with its hold). The report's JSON gains these rules as "ask" rows; its text page is unchanged. * Field report form: one optional notes field, the machine in "agent", and the install line under uvx The field-report form asked for false alarms, misses and what broke; people share counts and we find the issues, so those three fields become one optional `notes` ("Anything you'd like to tell us (optional)"), which `tracekit feedback --harmless` fills. `tracekit feedback` puts the version, OS, Python and hooks in `agent` and leaves "How long, and on what" to the person. The audit's last line says to install Tracekit first when `tracekit` isn't on PATH (a uvx run). * Feedback review fixes: holds counted by first answer, honest label, safe tool names, submittable links - Holds by rule counts a hold as approved from its first answer (an approval the signer later let expire unused stays approved), and rejected or expired only for holds never approved. Its label says approved by a person or the run's own user: on a dev signer every answer is the run's own user's. - view.review stays counts-only; the report passes it a dict for each hold's outcome, so the viewer's rows don't grow with the number of holds. - --harmless keeps a tool name only when the default policy names it (Bash, Edit, WebFetch, ...): an MCP tool is "an MCP tool", any other "another tool". It also fills the form's required share and agent fields, as the field report does, so it can be submitted; without a dev store, share says so in a line. - The OS is its name and major.minor release on Linux too. - The field-report check reads each counts line against what report --share and audit --share print, the values of the lines above, and rejects URLs, host and file names, paths and backticked commands anywhere; free text is marked lean, with the pull request's review as the backstop. * Feedback delta review: share vocabularies in the report check, no store read for --harmless - The field-report check builds its counts patterns from what --share can name: Tracekit's agents, decisions, hold outcomes, gap kinds and rule ids, each other as `other` or `custom rule`. A script's name, a command or a policy's own rule id in a count line is refused. - --harmless fills the form's share field with "not included: a single false alarm" instead of verifying the local dev store, which may not be --signer's. - Its help, the module docstring and docs/field-reports.md say it also holds the Tracekit, OS, Python and hook names, and that all of it is printed first. - The OS line takes any shape feedback._os gives (FreeBSD-14.0-RELEASE); the host-name check stays strict and says to write names without dots. - _os reads Linux's major.minor with a regex. * Report: each hold's first answer is the answer to its own approval request _tally keyed the first answer by tool_use_id, so two holds that reuse a call id both read the first one's answer: rejected then approved counted the approval nowhere, approved then expired counted two approvals. It now files each answer under the decision its approval request was for (why.import_v2.held_calls' owners, as view.review does; with no request in the window, the latest decision under its call id and attempt). * Help: tracekit feedback in the Start here group #619's help test needs every command in exactly one group.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tracekit --helpprinted 47 subcommands as one flat list in the order they were registered (103 lines).tracekitalone,
tracekit helpandtracekit commandswere errors.What changed
tracekit helpand a baretracekitprint the overview below and exit 0. Each command sits under what a person runsit for, with one entry from its own help text, cut where its usage starts (a
;or:before a backquote,--ora URL), so the overview can't go stale. Commands marked experimental today (
_EXPERIMENTAL, or "(experimental)" in their help) say so. After thegroups come the features that aren't commands, each with its page.
tracekit help NAMEis exactlytracekit NAME --help. For a moved command (causeway,sql,proofpack) it givesthe same "moved to a separate package" answer as
tracekit causeway. An unknown name prints the valid names on stderrand exits 2.
helpis a registered subcommand with its own description, sotracekit --helplists it first._parser()(the same lines at the same indentation), so the overview and the tests read theregistered commands.
tests/test_cli.pychecks five things:tracekitprints the same astracekit help, with exit 0;help NAMEmatchesNAME --helpfor all 48 commands, andhelp helphas a description;helpon a moved command gives the moved answer, exit 2;docs/quickstart-v2.mdmentionstracekit helpwhere it introduces thetracekitcommand.How the groups differ from the plan:
pendingis under Approvals, andverify-model(missing from the plan's list) isunder Evidence. The 21-command server group is split into "Run it on a server", "Record model, tool and network traffic"
and "Connect other tools". The v1 path (
daemon,observe,ingest,otel,proxy; see docs/support.md) is inits own "Legacy (v1)" group, so those don't read as current server tools.
TIPS isn't listed yet. It's in #609, not on main, and
tests/test_wiki.pyrefuses a link toViewer#tipsuntil thatanchor exists. Once #609 merges, it's one entry at the top of
_FEATURES:tracekit help(andtracekit)Not in this PR
ingest,otelanddaemonare experimental (they need--experimental, or print the experimental line when theystart), but their help texts don't say so, so the overview doesn't mark them. First-day fixes: plainer messages, a Claude Code quickstart, and an uninstall that says what it keeps #628 adds "(experimental)" to those
help strings.
ingest's help doesn't say it's deprecated (docs/remote-ingest.md does), so its overview line doesn't either.policy's help starts with "policy v2:", so its overview line reads only "policy v2".All three are small fixes to those commands' help strings in
tracekit/cli.py. They are left for a follow-up.Checks
The gates ran in the worktree (
make test-ts lint build test):twine checkpassed.tests/test_wiki.py,tests/test_cli.py,tests/test_docs.pyandtests/test_experimental.pypassed again.