From a6c3a067cc59796848e3c7f0f46af335841559c0 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Sun, 20 Sep 2026 19:16:25 +0000 Subject: [PATCH 01/11] feat(mdcode)!: trim the action command lines to what curation needs kcmd is for curating a semantic model, not for dispatching its actions, and the action commands had drifted into the second job. `--judge` hired a Gemini model from a debugging CLI, `--judge-reads-store` let it issue SQL, `--judge-model` and `--judge-location` configured it, `--skip-guards` existed to undo it, and `--json` was a second output format nobody consumed. Every listing then had to say which combination a reader was looking at. All of them are gone. What is left is four flags with no combinations between them: `action-list` (--profile, --store), `action-run` (--arg, --profile), `agent-tools` (--profile). `kcmd action-run` binds the arguments, opens one transaction and applies the statements. Before it writes, it names the guards it is passing over, so nobody reads a committed write as a checked one: $ kcmd action-run IssueCredit --arg order=12346 --arg amount=3.00 \ --arg memo="Coupon applied late" Running 'IssueCredit' on .../databases/semantic_agent_demo... NOT CHECKED: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this command settles no guard, and the write still happens Committed at 2026-09-20T18:51:11.964483Z. That is the useful half for curation: whether an action binds its arguments, writes the line it says it writes and leaves the store consistent is a question about SQL, and one command against your own database answers it with no judge to stand up first. `kcmd agent-tools` derives the same set with `skipGuards`, so a guarded action is offered rather than marked `[NOT RUNNABLE]`: who settles a rule belongs to whoever dispatches the call, and this command cannot know what that will be. What still marks a tool unrunnable is what no judge would repair -- no executor under the binding, an executor no handler runs, a guard naming a rule the model never declares, or one that quotes nothing. `runFlags` no longer emits a judge flag, so the suggested line in `action-list` and in a generated Agent Skill (#451) carries the arguments and nothing else. Where the action states a rule, the skill now says that the line settles none of them, so an agent that tries the call and watches it commit does not read that as the rules having held. The verbs are also flat now, so the skill and its golden fixture say `kcmd action-run` rather than `kcmd action run`. Guards reach a runtime unchanged. `Judge`, `GeminiJudge` and `modelJudgeStore` are untouched, the demo agent in demo/semantic-model/agent/ settles all four of its rules against Gemini, and `runAction`'s `judge` option is still how an application passes one in. Only the command lines stopped pretending to be that application. A later change can put a judge back behind a command if one turns out to be wanted. One defect the live run found: under `skipGuards` the unsettled-guard loop still warned once per rule, burying the outcome of the write under a list the caller had just written. The loop is gated, with a test. Docs: actions.md section 7 and the agent README's steps 3 and 4 rewritten. Every CLI listing in both was regenerated from a live run -- the README's against the demo's Spanner store, actions.md's against the `payments` model rebuilt from the page's own snippets until `action-list` matched it byte for byte. --- .../EntryGroups/commerce_demo/commerce.yaml | 6 +- toolbox/mdcode/docs/semantic-model/README.md | 2 +- toolbox/mdcode/docs/semantic-model/actions.md | 304 ++++++------- .../mdcode/docs/semantic-model/fidelity.md | 6 +- .../mdcode/docs/semantic-model/model_spec.md | 7 +- .../mdcode/docs/semantic-model/reference.md | 81 ++-- toolbox/mdcode/docs/semantic-model/skills.md | 20 +- toolbox/mdcode/src/libts/gcp/gemini.ts | 17 +- .../src/libts/semantic/runtime/agent_tools.ts | 15 +- .../src/libts/semantic/runtime/judge_store.ts | 16 +- .../src/libts/semantic/runtime/run_action.ts | 87 ++-- .../src/libts/semantic/runtime/runtime.ts | 2 +- toolbox/mdcode/src/libts/semantic/skills.ts | 33 +- toolbox/mdcode/src/libts/semantic/validate.ts | 2 +- toolbox/mdcode/src/tool/commands.ts | 406 ++++++++---------- toolbox/mdcode/src/tool/main.ts | 51 ++- .../tests/libts/semantic/actions.test.ts | 2 +- ...ions_place_order.sql_bound.skill.golden.md | 6 +- .../semantic/runtime/agent_tools.test.ts | 2 +- .../libts/semantic/runtime/run_action.test.ts | 54 ++- .../tests/libts/semantic/skills.test.ts | 72 ++-- toolbox/mdcode/tests/tool/action.test.ts | 266 +++++++----- toolbox/mdcode/tests/tool/main_cli.test.ts | 15 +- 23 files changed, 787 insertions(+), 685 deletions(-) diff --git a/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml b/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml index e661f056..6bb047fb 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml +++ b/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml @@ -21,8 +21,10 @@ # # One of the judgments compares the call against a row in the database, which # a judge can settle only if it can read the database. Running this demo -# therefore takes a judge that has been given one -- `--judge-reads-store` on -# the command line, or a `store` on the judge the agent hires. +# therefore takes a judge that has been given one -- a `store` on the judge the +# agent hires. Giving a judge your tables belongs to whoever embeds the +# runtime, so no kcmd flag offers it, and `kcmd action-run --judge` stops on +# that rule rather than guessing at the number it names. # # What this costs: every gate is a model call, and a model can answer two # identical calls differently. Don't copy the `$25` ceiling below -- a threshold diff --git a/toolbox/mdcode/docs/semantic-model/README.md b/toolbox/mdcode/docs/semantic-model/README.md index 752b79f5..59daca07 100644 --- a/toolbox/mdcode/docs/semantic-model/README.md +++ b/toolbox/mdcode/docs/semantic-model/README.md @@ -136,7 +136,7 @@ concepts the call changes — an executor in another system is an opaque handle, so its blast radius is declared or it is unknown. Knowledge Catalog is the only system an action is published to, and the only place it is governed; an action that carries its own DML is also the one kind `kcmd` runs itself, with -`kcmd action run`. See +`kcmd action-run`. See [Modeling write operations](actions.md). A model can also state **constraints**: named invariants over the ontology that diff --git a/toolbox/mdcode/docs/semantic-model/actions.md b/toolbox/mdcode/docs/semantic-model/actions.md index fb0e2fbb..6f2aa05d 100644 --- a/toolbox/mdcode/docs/semantic-model/actions.md +++ b/toolbox/mdcode/docs/semantic-model/actions.md @@ -473,9 +473,8 @@ Put that rule inside the transaction, or in your schema, where the store enforces it. A judgment that reads stored data also needs a judge that has been given the -store to read. Whoever dispatches the call decides that, and -[when the judge needs a fact](#when-the-judge-needs-a-fact) does it from the -command line. +store to read. Whoever dispatches the call decides that — see +[when the judge needs a fact](#when-the-judge-needs-a-fact). **Status: `judgment` is the only body a constraint has.** Whatever checks a guard puts the sentence to a language model and routes the verdict by @@ -522,11 +521,12 @@ around the arguments the call carries, and try every guard against a case it ought to refuse. A rule that does need a stored row — comparing a credit against the order total, -say — is settled only by a judge that has been given the store to read. Word the -rule to say the value is on record and has to be read, because the judge decides -for itself whether to look. Put to a judge with no store, the same rule refuses -every call. [When the judge needs a fact](#when-the-judge-needs-a-fact) runs one -of these from the command line. +say — is settled only by a judge that can read your tables, which is something +an agent embedding the runtime gives it rather than something `kcmd` does. Word +the rule to say the value is on record and has to be read, because the judge +decides for itself whether to look. The same rule refuses every call when it +goes to a judge that can't read. See +[when the judge needs a fact](#when-the-judge-needs-a-fact). ## A credit policy, worked through @@ -671,13 +671,13 @@ The bottom row of table 2 applies the strictest-outcome rule from section 2 — an `escalate` with nothing stricter beside it holds the first call, and rule 4's `reject` decides the second. -**Status: a run doesn't compute the strictest outcome.** `--judge` puts the +**Status: a run doesn't compute the strictest outcome.** The runtime puts the guards to the judge in the order your model declares them and stops at the first one that fails without being advisory. What comes back is that guard's outcome rather than the strictest of them, and a `warn` collected on the way -there doesn't travel with the refusal. And [`kcmd action run`](#7-run-it) won't -perform `IssueCredit` as declared here, so the two calls above are what the -published policy says should happen rather than what kcmd does with this action +there doesn't travel with the refusal. And [`kcmd action-run`](#7-run-it) +settles no guard at all, so the two calls above are what the published policy +says should happen rather than what that command does with this action today. ## 3. Say what it changes @@ -832,7 +832,7 @@ resolves your model. Resolving drops entities and relationships the profile can't bind, so holding `affects` to the ontology there would fail your deploy over a concept the profile removed rather than one you mistyped. An undeclared concept and an undeclared field fall back to the warning the loader already -gave. A catalog-only push and `kcmd action run` read the author's model +gave. A catalog-only push and `kcmd action-run` read the author's model whole, so both treat the same two as hard errors. Fields beside a `delete` read only the entry, so that one fails everywhere. @@ -952,20 +952,21 @@ Pasting the statements into a SQL console would tell you the DML is valid; a run is what exercises everything wrapped around it. Everything below happens at a command line, and that is a way of watching the -model work rather than the place it is meant to work. `kcmd action run` -performs the steps any runtime dispatching these calls has to perform — bind the -arguments, check the guards, open one transaction — and narrates each of them. -The behaviour is a property of the model you published rather than of -this tool: where a flag here hires a judge or lets it read, a service -dispatching the same action decides the same thing in its own configuration, -and reaches the same verdicts from the same sentences. - -`kcmd action list` prints the actions your model declares, each with its -parameters, executor, guards and blast radius, plus the command line that -calls it -- flags and all, so a guarded action's line arrives ready to run: +model work rather than the place it is meant to work. `kcmd action-run` performs +most of what any runtime dispatching these calls has to perform — bind the +arguments, open one transaction, apply the statements — and narrates each step. +What it leaves out is the guards: it settles none of them, names the ones it +passed over, and writes. Who settles a rule, and what that judge may read while +it does, are decided by whoever dispatches the call in earnest — and from the +same sentences, so a service running this action reaches verdicts this command +never asks for. + +`kcmd action-list` prints the actions your model declares, each with its +parameters, executor, guards and blast radius, plus the command line that calls +it, filled in with the parameters that line has to carry: ```bash -kcmd action list +kcmd action-list ``` ``` @@ -976,7 +977,7 @@ Model 'payments' (payments_eg), profile 'operational': executor: sql guards: TransferWithinAvailableBalance affects: Account (modify), Transfer (create), TransferDebits (create) - run: kcmd action run TransferFunds --judge --judge-reads-store --arg source= --arg target= --arg amount= + run: kcmd action-run TransferFunds --arg source= --arg target= --arg amount= ``` Where a run would be refused before it opened a transaction, that line says so @@ -1002,18 +1003,19 @@ asks, rather than working it out again here — so the two cannot disagree about what will happen. An action executed over MCP is marked the same way, since this command holds no handler for one and could not roll it back. -`kcmd action run` performs one of those actions, against the database your +`kcmd action-run` performs one of those actions, against the database your model's deployment target names under the selected profile. ### What a run does -`kcmd action run` binds every argument as a typed query parameter, then applies -the action's statements in one transaction. `TransferFunds` is guarded, so the -line carries `--judge` too -- [when the rule is a -sentence](#when-the-rule-is-a-sentence) covers what that hires: +`kcmd action-run` binds every argument as a typed query parameter, then applies +the action's statements in one transaction. `TransferFunds` is guarded, and this +command settles no guard -- it names the ones it passed over and writes anyway. +[When the rule is a sentence](#when-the-rule-is-a-sentence) covers who does +settle them: ``` - kcmd action run TransferFunds --judge --arg source=7 --arg target=8 --arg amount=250 + kcmd action-run TransferFunds --arg source=7 --arg target=8 --arg amount=250 │ │ bind @source = 7 as Integer, from Account.accountId │ @target = 8 as Integer, from Account.accountId @@ -1076,10 +1078,14 @@ that performs the write as DML, or declare the action with a 'sql' executor. ### When a rule stops the call +This is what a runtime does with a guard, and `kcmd action-run` is not that +runtime: it settles none of them, so none of the outcomes below come out of the +command line above. They come out of whatever dispatches the call in earnest. + Only a constraint the action names in `guards` has a say in a call, which is [section 2](#2-gate-it-with-a-constraint)'s rule reaching the runtime. A constraint your model declares and your action doesn't name has no bearing on -the write, and kcmd never goes looking for one. +the write, and nothing goes looking for one. A guard is a sentence, and settling a sentence needs something that reads one. A run given nothing to read with refuses a call that a non-advisory guard covers @@ -1105,47 +1111,44 @@ same way, before any judge is asked, because there is nothing to ask about. ### When the rule is a sentence A guard stated as a `judgment` needs something that can read a sentence, and -whatever checks the guard has to be given one. At a command line that is -`--judge`: Gemini on Vertex AI, reached with the project and the credentials -kcmd already holds. - -```bash -kcmd action run IssueCredit --arg order=12347 --arg amount=5 \ - --arg memo="customer asked for a credit" --judge --judge-reads-store -``` +whatever dispatches the call has to be holding one. In the runtime shipped here +that is Gemini on Vertex AI, hired by the application that embeds the runtime +and handed to it once, at construction: -One of the demo's rules is about a number on record rather than a number in the -call — the order's total — so the judge has to be able to read the store, which -`--judge-reads-store` is how this command arranges. Given no store the rule -cannot be settled, and it stops the call before any of the others is reached. - -That call runs against the commerce model under `demo/semantic-model/skill` — -the [credit policy worked through earlier](#a-credit-policy-worked-through), -rebuilt around what kcmd can settle today. A profile binds `IssueCredit` to a -`sql` executor, and the action names four rules in `guards`. One of the four is -the 25-dollar ceiling, written there as a judgment rather than left to the desk; -the demo keeps it to show what settling arithmetic with a model call costs. - -Leave `--judge` off and the guards stop the call, because you supplied nothing -to settle them: - -``` -Error: Action 'IssueCredit' is guarded by 'CreditWithinOrderTotal', -'CreditUnderReviewThreshold' and 'CreditIsNotSplitToAvoidReview', which are -settled by reading the call, and this runtime was given no judge to ask. -Running it would apply a write the model says must be checked first, so it is -refused rather than run unchecked. +```ts +const judge = new GeminiJudge(ctx, {model: 'gemini-2.5-flash'}); ``` -Three names, though the action guards on four. The fourth declares `warn`, and -an advisory rule reports rather than refuses, so having nobody to ask is not a -reason to stop. - -Add the flag and each rule's own sentence goes to the model with the attempted -call. A verdict comes back with a reason, and `on_violation` decides what -follows. The demo declares `warn` on the memo rule; the three outputs below come -from setting that one field to each of its values in turn, so a single rule -shows all three branches. With `reject`, the call stops: +**No kcmd command line hires one.** `kcmd action-run` performs the write and +names the guards it did not check; it is for finding out whether your statements +do what you meant, not for finding out whether your rules hold. The two demands +pull apart: a judge costs a model call per guard and credentials to reach one, +and an author checking a `WHERE` clause should not have to stand either up. The +[commerce demo](../../demo/semantic-model/skill/README.md) is where the guards +are actually settled, against the same model, live. + +The rules below run against the commerce model under `demo/semantic-model/skill` +— the [credit policy worked through earlier](#a-credit-policy-worked-through), +rebuilt around what a runtime can settle today. A profile binds `IssueCredit` to +a `sql` executor, and the action names four rules in `guards`. One of the four +is the 25-dollar ceiling, written there as a judgment rather than left to the +desk; the demo keeps it to show what settling arithmetic with a model call +costs. + +Each rule's own sentence goes to the model with the attempted call. A verdict +comes back with a reason, and `on_violation` decides what follows. The demo +declares `warn` on the memo rule; the three outputs below come from setting that +one field to each of its values in turn, so a single rule shows all three +branches. + +> These three were recorded through `kcmd action-run`, back when it took a judge +> and could give that judge the store to read — which is why each of them shows +> the judge reading `Orders`. Neither is a command-line flag any more, for the +> reason above. They are +> kept because what they show — one rule, all three values of `on_violation`, +> one run each — is not shown anywhere else. + +With `reject`, the call stops: ``` Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... @@ -1211,25 +1214,32 @@ name and description, and the arguments as the caller stated them — `order=12347`, the value itself, and not the `Order` row it identifies. That's the whole of what it has, unless it was also given the store to read. -**A rule that never reached a judge is reported as unchecked.** You supplied no -judge, or the model call failed. Either way `on_violation` routes that like any -other breach: an advisory guard lets the write through and warns, and a guard -declaring `reject` or `escalate` stops the call. Committing in silence would -tell you every rule passed when one was never put to anybody. +**A rule that never reached a judge is reported as unchecked.** The runtime +holds no judge, or the model call failed. Either way `on_violation` routes that +like any other breach: an advisory guard lets the write through and warns, and a +guard declaring `reject` or `escalate` stops the call. Committing in silence +would tell you every rule passed when one was never put to anybody. ### When the judge needs a fact Some rules can't be settled from the call alone. *The credit must not exceed the total of the order it is applied to* compares an argument against a number in -your database, and the caller is under no obligation to state it correctly. -Settling it takes a judge that has been given the store to read, which at a -command line is `--judge-reads-store`. - -Two things have to be in place. `--judge-reads-store` says what a judge may do -without hiring one, so pass `--judge` alongside it. And your profile has to bind -the entities the rule talks about to tables, because that binding is the whole -of what the judge is told about your database; with nothing bound, the run stops -before it starts. +your database, and the caller is under no obligation to state it correctly. A +judge that can read your tables goes and gets the number. + +`kcmd action-run` hires no judge at all, so it certainly does not hire that one. +Letting a model compose and send queries against your data is a property of the +runtime an application embeds, decided by whoever builds the application; a +command line for curating a model is the wrong place to turn it on. The runtime +supplies it — +`modelJudgeStore()` in `src/libts/semantic/runtime/judge_store.ts` — and an +application that embeds the runtime is what calls it. Everything below describes +that judge; the transcripts are in +[the demo's README](../../demo/semantic-model/skill/README.md). + +One thing has to be in place either way: your profile has to bind the entities +the rule talks about to tables, because that binding is the whole of what the +judge is told about your database. With nothing bound there is nothing to read. Then write the rule so the judge goes and looks — it decides that for itself, from the sentence you give it. This is the rule the demo under @@ -1248,44 +1258,14 @@ from the sentence you give it. This is the rule the demo under credit amount, or split it across the orders it actually covers. ``` -A run carrying the flag says the judge may read, and prints every statement the -judge sends: - -```bash -kcmd action run IssueCredit --judge --judge-reads-store \ - --arg order=12345 --arg amount=3.00 \ - --arg memo="Shipping charge applied in error" -``` - -``` -Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... - rules stated in words go to gemini-2.5-flash (us-central1) - it may read commerce's tables to settle them - the judge reads: SELECT total FROM Orders WHERE order_id = 12345 - order: '12345' -> Order 12345 -Committed at 2026-09-15T03:39:42.804901Z. -``` - -The judge composed that statement itself, from the rule's sentence and the -tables your profile binds. kcmd prints every one, because a read made on your -behalf is yours to check. Here's the same order again, with a credit of $200 and -a memo asserting the order is worth $900: - -``` - the judge reads: SELECT total FROM Orders WHERE order_id = 12345 -Error: Action 'IssueCredit' is guarded by 'CreditWithinOrderTotal' ("The -credit amount requested must not exceed the total of the order it is applied -to. ..."), and gemini-2.5-flash (us-central1) judged that it does not hold for -this call: The credit amount of 200.00 exceeds the order total of 162.85. The -model marks this rule 'escalate', so an approver may allow it; nothing here -can. A credit cannot exceed the total of the order it credits. Lower the -credit amount, or split it across the orders it actually covers. No -transaction was opened, so nothing was written. -``` - -The judge read the row, compared the argument against $162.85, and paid no -attention to the $900 in the memo. Drop the flag and the same call is refused -for a different reason: the judge says it can't get the total. +The judge composes its statement itself, from the rule's sentence and the tables +your profile binds — nobody writes that SQL. The runtime hands every statement +back to the caller as it is sent, because a read made on your behalf is yours to +check, and an application that embeds the runtime is expected to show them. A +credit against order 12345 sends one the caller never asked for: `SELECT total +FROM Orders WHERE order_id = 12345`. The rule named the order's total in words, +and the judge went and got it. Put the same call to a judge that cannot read and +it is refused for a different reason: the judge says it can't get the total. **The judge sees what your model declares.** The entities, tables and columns in its instructions come from your binding profile, so a column your model doesn't @@ -1374,13 +1354,13 @@ nothing totals anything on its behalf. ### The set an agent is handed -`kcmd agent tools` prints every tool the derivation produces, with the +`kcmd agent-tools` prints every tool the derivation produces, with the instruction they arrive with. It reads your model under the profile you name and needs the store that profile binds, because what an agent can call depends on it. The command opens no connection and runs nothing: ```bash -kcmd agent tools +kcmd agent-tools ``` For the model built up on this page, that set is: @@ -1389,7 +1369,7 @@ For the model built up on this page, that set is: Model 'payments' (payments_eg), profile 'operational': store: my-project/my-instance/semantic_skill_demo - action transfer_funds (TransferFunds) [NOT RUNNABLE] + action transfer_funds (TransferFunds) Move money from one account to another. Resolve both accounts before calling. @@ -1400,12 +1380,6 @@ Model 'payments' (payments_eg), profile 'operational': record rather than stated in the arguments, so read it before answering. A transfer cannot move more than the source account holds. Lower the amount, or choose another account. - - Calling this will not work: Action 'TransferFunds' is guarded by - 'TransferWithinAvailableBalance', which is settled by reading the call, - and this runtime was given no judge to ask. Running it would apply a - write the model says must be checked first, so it is refused rather than - run unchecked. Report that rather than retrying. source: integer -- The account the money leaves. target: integer -- The account the money goes to. amount: number -- How much money to move. @@ -1448,14 +1422,20 @@ Model 'payments' (payments_eg), profile 'operational': you changed. ``` -`transfer_funds` is listed and marked `[NOT RUNNABLE]`. `TransferFunds` names a -guard, this derivation holds no judge to settle it, and so the -[refusal from section 7](#when-a-rule-stops-the-call) arrives here instead -— before any agent exists, rather than inside a transaction. - -A tool marked `[NOT RUNNABLE]` is **withheld**, and the listing keeps it, -printed named and described, with what it's waiting on underneath, because an -action your model declares shouldn't vanish from the set your model offers. +`transfer_funds` is offered, guard and all. The rule it is gated by is in the +tool's own description, wording and all, so the agent argues its call against +the rule before making it rather than learning it from a refusal. + +A guard is not a reason to withhold a tool here. Who settles a rule belongs to +whoever dispatches the call, and this listing cannot know what that will be, so +marking the action unrunnable would describe a caller rather than your model. +What does get marked `[NOT RUNNABLE]` is what supplying a judge would not +repair: no executor under this binding, an executor this runtime holds no +handler for, a guard naming a rule your model never declares, or a guard naming +one that states no rule to put to a judge. Such a tool is **withheld**, and the +listing keeps it, printed named +and described, with what it's waiting on underneath, because an action your +model declares shouldn't vanish from the set your model offers. Nothing in the listing was written for a particular agent. It reads the same whether your caller is ADK, LangChain, or a person deciding whether the model @@ -1518,9 +1498,10 @@ line of output per key: agent listing.* Two things come from neither file. The derivation appends a paragraph to the -instruction, its own text about using the tools, identical for every model. The -runtime adds `[NOT RUNNABLE]` and the paragraph under it, which say whether this -call could succeed. +instruction, its own text about using the tools, identical for every model. And +where a call could not succeed, the runtime adds `[NOT RUNNABLE]` and the +paragraph under it saying what stands in the way — absent above, because this +call can. The instruction at the foot of the listing has two parts, because two different people own them. @@ -1543,8 +1524,10 @@ not. Put it in the model. ### What a write tool and a lookup tool do A **write tool** runs the action. Calling `transfer_funds` does the same bind -and transact as [`kcmd action run TransferFunds`](#7-run-it) — the same typed -parameters, the same single transaction, the same three outcomes. +and transact as [`kcmd action-run TransferFunds`](#7-run-it) — the same typed +parameters, the same single transaction, the same three outcomes. The guards are +where the two part: the tool puts each one to whatever judge the runtime behind +it holds, and the command line settles none. A **lookup tool** reads one entity: exact match on any bound field, combined with AND, capped at 50 rows. It can't join, compare ranges, aggregate or order. @@ -1574,27 +1557,24 @@ How an entity is keyed is not among them. Every parameter is a scalar, so an action taking the three key fields of a three-part key is as callable as one taking a single id. -An action guarded by a judgment is withheld when the derivation holds no judge, -because the listing reports what the runtime would do with what it's holding. -Supply a judge and the same action is callable, with the same description and -the same parameters: +A guard is not one of the reasons a tool is withheld. Who settles a rule belongs +to the application that embeds the runtime, and this command cannot know what +that will be, so marking a guarded action unrunnable here would describe a +caller rather than the model. What the listing does print, in the tool's own +description, is which rules the agent's calls will be held to: ```console -$ kcmd agent tools -... - action issue_credit (IssueCredit) [NOT RUNNABLE] - -$ kcmd agent tools --judge -Rules stated in words go to gemini-2.5-flash (us-central1). +$ kcmd agent-tools ... action issue_credit (IssueCredit) + ... + This call is gated by CreditWithinOrderTotal, CreditUnderReviewThreshold + and CreditIsNotSplitToAvoidReview: ``` -`--judge` takes an optional model name, the same way [`kcmd action run ---judge`](#when-the-rule-is-a-sentence) does. Naming a model doesn't call one: -a judge settles a rule when an action runs, and printing what an agent is -offered runs no action, so this listing costs you nothing however many guarded -actions it names. +No model is called. A judge settles a rule when an action runs, and printing +what an agent is offered runs no action, so this listing costs you nothing +however many guarded actions it names. A **lookup** is withheld for reasons of its own: @@ -1610,7 +1590,7 @@ withheld that would have worked is never tried. ### Calling it from code -`kcmd agent tools` prints these tools; `modelTools` returns them. Both take a +`kcmd agent-tools` prints these tools; `modelTools` returns them. Both take a **semantic runtime**: one model paired with the store your profile binds it to. `createSemanticRuntimes` assembles them the way `kcmd action` does, so your agent reads the model the CLI reads, under the same profile, with the same merge @@ -1740,7 +1720,7 @@ This is a prototype. Four things you might reasonably expect are absent. - **kcmd calls no executor but its own.** A `sql` action runs; an `mcp`, `rest` or `grpc` one is published for whoever dispatches it, which is why those three name coordinates instead of a statement. -- **The store is Spanner or AlloyDB.** `kcmd action run` binds and +- **The store is Spanner or AlloyDB.** `kcmd action-run` binds and transacts against the database your profile's deployment target names, which may be either of those. A model bound to BigQuery publishes its actions and runs none of them. diff --git a/toolbox/mdcode/docs/semantic-model/fidelity.md b/toolbox/mdcode/docs/semantic-model/fidelity.md index 25045af7..ca4eacd4 100644 --- a/toolbox/mdcode/docs/semantic-model/fidelity.md +++ b/toolbox/mdcode/docs/semantic-model/fidelity.md @@ -114,9 +114,9 @@ agree on every structural row and differ only where a Spanner target has no 13. **Constraints.** A constraint reaches Knowledge Catalog only, as one `semantic-constraint` entry under the model entry, and `pull` reads it back. Every other push target deploys nothing for it and warns once. Publishing is - all that push does with a constraint; what settles one is a run, where - `kcmd action run --judge` puts the guards an action names to a judge before - the write. + all that push does with a constraint; what settles one is a run, and the + runtime an application embeds is what puts each guard to a judge before the + write. `kcmd action-run` is not that runtime and checks no guard. 14. **Binding profiles.** A model may define several physical realizations, one per binding profile. `--all-profiles` deploys a graph for every profile that declares a deployment target, each to the backend its own target names; a diff --git a/toolbox/mdcode/docs/semantic-model/model_spec.md b/toolbox/mdcode/docs/semantic-model/model_spec.md index bdbb5f77..a9619e60 100644 --- a/toolbox/mdcode/docs/semantic-model/model_spec.md +++ b/toolbox/mdcode/docs/semantic-model/model_spec.md @@ -537,8 +537,11 @@ reads the document ([§6](#6-the-extension-mechanism)). Status: authored, validated, published, and settled at run time. Whatever dispatches a call puts each guard to a language model before the transaction opens and routes the verdict by `on_violation`; given no judge, the action is - refused rather than run past its rules. `kcmd action run --judge` does this - from a command line. Rules in + refused rather than run past its rules. No command line here does that: + `kcmd action-run` checks no guard, because who settles one belongs to whoever + dispatches the call in earnest. The [commerce + demo](../../demo/semantic-model/skill/README.md) is where it is shown. Rules + in [Reference → Validation](reference.md#validation). A constraint says two things about a violation, under two separate keys. diff --git a/toolbox/mdcode/docs/semantic-model/reference.md b/toolbox/mdcode/docs/semantic-model/reference.md index 477dde8d..629bbcc9 100644 --- a/toolbox/mdcode/docs/semantic-model/reference.md +++ b/toolbox/mdcode/docs/semantic-model/reference.md @@ -63,35 +63,59 @@ scope you authored under. See [Pull](README.md#pull) for behavior. | `--dry-run` | Reconstruct from the catalog and report what would be written, but write no files. | | `--force-remove` | Replace a differently-named local model with the catalog's (see [Pull](README.md#pull)); without it, a pull that would leave the entry group holding two models fails. | -### action +### action-list ```bash -kcmd action list -kcmd action run --arg = ... +kcmd action-list [name] ``` -`list` prints every action the models in the scope declare, with the store a run -would reach and the command line that runs each one. `run` executes one against -the Spanner or AlloyDB database the selected profile's deployment target names; -only a `sql` executor runs. A guard states its rule as a `judgment`, and is -settled by `--judge` before the transaction opens, and by a judge that can query -the model's tables when `--judge-reads-store` is passed as well. Without a judge -the action is refused rather than run unchecked. See -[Run it](actions.md#7-run-it). +Prints every action the models in the scope declare, with the store a run would +reach and the command line that runs each one. Opens no store and calls no +model. See [Run it](actions.md#7-run-it). | Flag | Effect | |------|--------| -| `--arg =` | Bind one action parameter. Repeat the flag for each one; the value is text, parsed against the parameter's declared ontology type. `run` only. | | `--profile [name]` | Read the model under this binding profile. Its deployment target names the database the action runs against, so this is how you change stores. Defaults to `default_profile`, else the model's inline bindings. | -| `--store` | Print only where a run would land, on one line and nothing else, for a script to read rather than parse back out of the listing: `project/instance/database` for a Spanner store, `bigquery:project/dataset` for a BigQuery one. Errors when the scope holds more than one model, since those may name different databases. `list` only. | -| `--judge [model]` | Settle each guard stated as a `judgment` by asking Gemini on Vertex AI, using the project and credentials `kcmd` already holds. Takes a model id, defaulting to `gemini-2.5-flash`. Without the flag, an action guarded by such a rule is refused rather than run unchecked, unless the rule declares `warn`, in which case the run commits and reports that the rule went unchecked. `run` only. | -| `--judge-location ` | Ask the judge in this Vertex AI region. The region is where the argument values are sent, so a project that has to keep them somewhere in particular names that region here. Defaults to `us-central1`. The environment's `compute/region` is deliberately not read, because a region chosen for Compute Engine is often one Vertex AI does not serve. `global` is accepted and reaches the host that serves it. `run` only. | -| `--judge-reads-store` | Let the judge query the model's own tables while it settles a rule, so a guard can compare the call against what is recorded rather than only against what the caller stated. The judge is shown the entities, tables and columns the selected profile binds, writes its own statement in that profile's dialect, and every statement it sends is printed. Each one is checked to be a single read beginning with `SELECT` or `WITH` and wrapped in a subquery, so that a data-modifying CTE cannot run; at most 20 rows come back and each value is clipped. Costs one model call more per guard, plus one for each round of reading. Says what a judge may do rather than hiring one, so pass `--judge` as well. Errors when the profile binds no table to read. `run` only. See [When the judge needs a fact](actions.md#when-the-judge-needs-a-fact). | +| `--store` | Print only where a run would land, on one line and nothing else, for a script to read rather than parse back out of the listing: `project/instance/database` for a Spanner store, `bigquery:project/dataset` for a BigQuery one. Errors when the scope holds more than one model, since those may name different databases. | -### agent +### action-run ```bash -kcmd agent tools +kcmd action-run --arg = ... +``` + +Executes one action against the Spanner or AlloyDB database the selected +profile's deployment target names; only a `sql` executor runs. + +**This command does not check the action's guards.** A guard states its rule as +a `judgment`, and settling one means putting it to a judge — which is a piece of +the runtime an application embeds, not of a command line for curating a model. +So the write happens and every rule the model states goes unenforced. The run +names the guards it passed over, before it opens the transaction, so a reader +watching one land sees what did not stand between them and it. + +Running an action is not what `kcmd` is for — the command exists so that an +author can exercise a model they are curating, and find out whether the +statements do what they meant, without first standing up an agent. To see the +guards actually settled, run the model through something that embeds the +runtime: the [commerce demo](../../demo/semantic-model/skill/README.md) hires +a judge, gives it the model's tables to read, and refuses the call when a rule +does not hold. + +What this command still refuses is a model that is wrong about its own rules: a +guard naming a constraint the model does not declare, or one whose `judgment` +states nothing. Neither is a check standing down — they are the same two things +a push refuses, and no judge would have repaired either. + +| Flag | Effect | +|------|--------| +| `--arg =` | Bind one action parameter. Repeat the flag for each one; the value is text, parsed against the parameter's declared ontology type. | +| `--profile [name]` | Read the model under this binding profile. Its deployment target names the database the action runs against, so this is how you change stores. Defaults to `default_profile`, else the model's inline bindings. | + +### agent-tools + +```bash +kcmd agent-tools ``` Prints what an agent would be handed for the models in the scope: one lookup @@ -102,15 +126,15 @@ runs nothing. A tool the runtime cannot call is listed and marked `[NOT RUNNABLE]` rather than dropped, with the reason in its description, so a refusal is visible before any -agent exists. To see what a guard costs today, add a constraint to an action's -`guards` and run this again. What is offerable depends on what the caller holds, -so an action guarded by a `judgment` is marked unrunnable without `--judge` and -callable with it. +agent exists. A guard is not one of those reasons: who settles a rule belongs to +the application that embeds the runtime, and this command cannot know what that +will be, so withholding a guarded tool here would describe a caller rather than +the model. What it does print, in the tool's own description, is which rules the +agent's calls will be held to. | Flag | Effect | |------|--------| | `--profile [name]` | Read the model under this binding profile. Defaults to `default_profile`, else the model's inline bindings. | -| `--judge [model]` | List what an agent holding a judge is offered: an action guarded by a rule stated in words is callable rather than `[NOT RUNNABLE]`. Takes a Gemini model id, defaulting to `gemini-2.5-flash`, on the same region rule as [`action run --judge`](#action). No model is called either way — a judge settles a rule when an action runs, and this listing runs none. | A model whose profile names no Spanner database offers no tools, because calling one needs a store. That model is reported as offering none and the rest of the @@ -135,16 +159,17 @@ the caller typed, and a name the format does not allow fails before anything is written. Everything the binding decides lives in `SKILL.md`: the store, the executor -kinds, which actions this deployment cannot run and why, and the `kcmd action -run` line to try one with, all under one heading, plus the snippet for reading +kinds, which actions this deployment cannot run and why, and the `kcmd +action-run` line to try one with, all under one heading, plus the snippet for reading the store directly. A reference page is the same bytes under any profile. A guarded action is always described as runnable, because a rule stated in words is settled by the runtime before the transaction opens and a guarded action only ever runs against a runtime that has a judge. The command line printed for one -says `--judge`. There is no flag here to say otherwise: whether the caller of -`skills-generate` had a judge configured is a fact about that invocation, not -about the deployment the document is read against. +is `kcmd`'s, and `kcmd` is not that runtime: it settles no guard, and the +paragraph under the line says so. There is no flag here to say otherwise: +whether the caller of `skills-generate` had a judge configured is a fact about +that invocation, not about the deployment the document is read against. | Flag | Effect | |------|--------| diff --git a/toolbox/mdcode/docs/semantic-model/skills.md b/toolbox/mdcode/docs/semantic-model/skills.md index e1fc229f..dbfa2121 100644 --- a/toolbox/mdcode/docs/semantic-model/skills.md +++ b/toolbox/mdcode/docs/semantic-model/skills.md @@ -33,7 +33,7 @@ at startup as a model with one. ## Generate one Run it in a semantic-model scope, the same directory `kcmd push` and -`kcmd action run` work in: +`kcmd action-run` work in: ```bash kcmd skills-generate --out skills @@ -122,7 +122,7 @@ it, which lives in the model rather than in whoever wrote the agent: ``` The row names the action the way it was authored, because that's the string -`kcmd action run` takes and the string a refusal quotes back. The snake_case tool +`kcmd action-run` takes and the string a refusal quotes back. The snake_case tool name a framework would register it under is on the reference page, stated once. Then **Finding a record**, which exists because a skill of writes has a hole in @@ -275,17 +275,19 @@ one. A rule stated in words is settled by asking a judge, and the runtime asks it before the transaction opens — not the agent making the call. An agent that judged its own call would be the constrained thing certifying itself, which is no guard at all. So a guarded action only ever runs against a runtime that has a -judge, and that's the runtime every generated skill is written for: the command -line says `--judge`, and the paragraph under it says what the flag settles. +judge, and that's the runtime every generated skill is written for. The command +line in the skill is `kcmd`'s, and `kcmd` isn't that runtime: it settles no +guard, and the paragraph under the line says so, so an agent that tries the call +and watches it commit doesn't read that as the rules having held. Whether you had a judge configured when you ran `skills-generate` is a fact about that invocation, not about the deployment the document describes, so there's no flag here to write the other kind of skill. That section names the profile, the store, the executor kinds in play, and any -action that can't run here. It also carries a `kcmd action run` command line, -built by the same code that prints one under `kcmd action list` — so it arrives -with the flags this action's rules need and a typed placeholder per required -argument, and it's marked, in the skill itself, as the debugging path. `kcmd` is +action that can't run here. It also carries a `kcmd action-run` command line, +built by the same code that prints one under `kcmd action-list` — so it arrives +with a typed placeholder per required argument and nothing else, and it's +marked, in the skill itself, as the debugging path. `kcmd` is a command line for inspecting a model, not the runtime an agent should call in production; an agent that runs continuously should be handed these actions as tools by its own framework, which reaches the same runtime. @@ -309,7 +311,7 @@ tools by its own framework, which reaches the same runtime. ## See also * [Modeling write operations](actions.md) — declaring the actions a skill - describes, and `kcmd agent tools`, which prints the same derivation instead of + describes, and `kcmd agent-tools`, which prints the same derivation instead of writing it out * [Binding profiles](profiles.md) — the profile the deployment-specific section reads diff --git a/toolbox/mdcode/src/libts/gcp/gemini.ts b/toolbox/mdcode/src/libts/gcp/gemini.ts index 2fc9d639..73dab753 100644 --- a/toolbox/mdcode/src/libts/gcp/gemini.ts +++ b/toolbox/mdcode/src/libts/gcp/gemini.ts @@ -28,12 +28,12 @@ export const DEFAULT_JUDGE_MODEL = 'gemini-2.5-flash'; // Vertex serves models from a region, and not every region serves every model. -// A caller that knows better names one, through `--judge-location` or this -// option; everything else uses a region that serves Gemini. The region is also -// where the argument values are sent, so a project that has to keep them -// somewhere in particular names that region. What is deliberately NOT consulted -// is `gcloud config get-value compute/region`, which is whatever the user set -// for Compute Engine and is routinely somewhere Vertex is not -- `us`, say, +// A caller that knows better names one through this option; everything else +// uses a region that serves Gemini. The region is also where the argument +// values are sent, so a project that has to keep them somewhere in particular +// names that region. What is deliberately NOT consulted is `gcloud config +// get-value compute/region`, which is whatever the user set for Compute +// Engine and is routinely somewhere Vertex is not -- `us`, say, // which is not a Vertex endpoint at all. Reading it would make a judge // unreachable over an unrelated setting, and an unreachable judge refuses // writes that are fine. @@ -187,8 +187,9 @@ export class GeminiJudge extends ApiClient implements Judge { // this judge will ever be asked, and rebuilding it per call would put the // cost of describing the model on every guard. this._system = systemInstruction(options.store); - // Read off which model this is, so `--judge` and `--judge gemini-2.5-flash` - // send the same request. A budget of 0 is a per-model limit. The model this + // Read off which model this is, so a caller taking the default and a + // caller naming that same model send the same request. A budget of 0 is a + // per-model limit. The model this // file picked accepts it; gemini-2.5-pro rejects it outright with `The model // does not support setting thinking_budget to 0`, and an unreachable judge // refuses every guarded write. So every other model is sent no budget and diff --git a/toolbox/mdcode/src/libts/semantic/runtime/agent_tools.ts b/toolbox/mdcode/src/libts/semantic/runtime/agent_tools.ts index 976ad7ec..82997723 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/agent_tools.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/agent_tools.ts @@ -146,6 +146,16 @@ export interface ActionToolOptions { * refused mid-call. */ judge?: Judge; + /** + * Derive the tools as a caller that will not check the guards at all: the + * write happens and every rule the model states goes unenforced. For trying + * a model out where no judge is configured, which is otherwise a model whose + * every guarded action is unofferable. + * + * Passed to the call as well as to the derivation, for the reason `judge` is: + * a tool derived one way and called the other is advertised wrongly. + */ + skipGuards?: boolean; } @@ -177,8 +187,8 @@ function toolFor(action: Action, opts: ActionToolOptions): ActionTool { // model, then the runtime having no store, which is the same sentence on // every tool and says nothing about this one. const model = opts.runtime.model; - const blocked = - whyRefusedWithoutRunning(model, action, handler, opts.judge) ?? + const blocked = whyRefusedWithoutRunning( + model, action, handler, opts.judge, opts.skipGuards) ?? noStore(opts.runtime) ?? undefined; const tool: ActionTool = { name: snakeCase(action.name), @@ -193,6 +203,7 @@ function toolFor(action: Action, opts: ActionToolOptions): ActionTool { args, handler, judge: opts.judge, + skipGuards: opts.skipGuards, }); return describeOutcome(outcome); }, diff --git a/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts b/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts index 03f6a555..e49f9edd 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts @@ -1,8 +1,10 @@ // The database a judge may read while it settles a guard. // // `modelJudgeStore` builds the `JudgeStore` that judge.ts declares and -// gcp/gemini.ts consumes. A run creates one when it is given -// `--judge-reads-store`, and it holds two things: +// gcp/gemini.ts consumes. It is a library facility rather than a `kcmd` one: +// letting a model compose and send queries against a caller's data is a +// decision for whoever embeds the runtime, so an application asks for it and +// the curation CLI does not offer it. It holds two things: // // 1. SCHEMA. A block of text naming the tables and columns the judge may // use, which the caller puts in the model's instructions. It is composed @@ -182,7 +184,7 @@ export function modelJudgeStore( /** * An entity a judge can be told about: one table, and the columns behind it. */ -export interface ReadableEntity { +interface ReadableEntity { entity: Entity; table: string; fields: BoundField[]; @@ -195,11 +197,11 @@ export interface ReadableEntity { * * The same test the lookup tools apply, for the same reason: an abstract * entity has no table, a field bound to an expression is not a column, and a - * data source that is not a table reference cannot be read from. Exported so - * that `action list` can tell whether offering a reading judge would work - * before it prints a command line suggesting one. + * data source that is not a table reference cannot be read from. An entity + * this leaves out is one the judge is never told about, so a rule that turns + * on it is one the judge reports it cannot settle. */ -export function readableEntities( +function readableEntities( runtime: SemanticRuntime, dialect: SqlDialect): ReadableEntity[] { const readable: ReadableEntity[] = []; for (const entity of runtime.model.entities ?? []) { diff --git a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts index d07c8944..126d3842 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts @@ -50,9 +50,7 @@ import {Action, Constraint, SemanticModel,} from '../ir'; import {bindScalar, isParameterRequired, sentence, storeCodeFor,} from '../parameters'; import {leadingDmlVerb, referencedParameters} from '../sql_identifiers'; -import {dialectFor} from './dialect'; import {Judge, JudgeVerdict} from './judge'; -import {readableEntities} from './judge_store'; import {runtimeClient, SemanticRuntime} from './runtime'; export {bindScalar, isParameterRequired, sentence, storeCodeFor} from '../parameters'; @@ -125,6 +123,15 @@ export interface RunActionOptions { // "run those unjudged": an action with a guard to settle is refused, because // a judgment is the only body a constraint has and only a judge settles one. judge?: Judge; + // Runs the action without checking its guards at all. Not a weaker check -- + // no check: every refusal a guard would have produced is skipped and the + // write happens. It exists because the refusals above are total. An author + // trying a model out locally, against their own database, has no judge to + // supply and would find every guarded action unrunnable; the alternative is + // deleting the guards to test the write, which is worse. The run still + // reports each guard it did not check, so a caller reading the output is + // never told the write passed rules nothing consulted. + skipGuards?: boolean; } @@ -150,8 +157,8 @@ export async function runAction(opts: RunActionOptions): } // Decided BEFORE touching the store, so an action this runtime will not run // fails without having opened a transaction at all. - const refusal = - whyRefusedWithoutRunning(model, action, opts.handler, opts.judge); + const refusal = whyRefusedWithoutRunning( + model, action, opts.handler, opts.judge, opts.skipGuards); if (refusal) return {status: 'error', message: refusal}; // Checked before the judge as well. A judge is asked whether a rule holds @@ -186,12 +193,20 @@ export async function runAction(opts: RunActionOptions): warnings.push(...asked.warnings); } } - // Every guard still unsettled here is advisory, because anything stricter - // was refused above. An advisory rule nothing checked is a check the model + // An unsettled rule the caller did not ask to skip is a check the model // asked for and did not get, and a caller shown no line for it reads the - // write as having passed every rule the model states. - for (const {constraint, why} of unsettledGuards(model, action, opts.judge)) { - warnings.push(`${citation(constraint)} was not checked: ${why}`); + // write as having passed every rule the model states. Every one reaching + // here is advisory, because anything stricter was refused above. + // + // `skipGuards` is the one caller that gets no line, because it has already + // been told: it asked for the guards to go unchecked, and it says so where + // it asked. Repeating it here would quote every rule back at a caller who + // named them all a moment ago, and bury the outcome of the write under it. + if (!opts.skipGuards) { + for (const {constraint, why} of unsettledGuards( + model, action, opts.judge)) { + warnings.push(`${citation(constraint)} was not checked: ${why}`); + } } // Whether a transaction was ever opened. A session that could not be @@ -446,22 +461,18 @@ const DEFINITELY_NOT_COMMITTED = new Set([400, 401, 403, 404, 409, 412]); /** - * The `kcmd action run` line that would actually run this action here. + * The `kcmd action-run` line that would actually run this action here. * - * Exported for the same reason `whyRefusedWithoutRunning` is. Three things - * about a call are easy to re-derive and easy to get wrong: whether a judge is - * needed, whether that judge has to read the store, and which arguments are - * required. Each is a rule the runtime already owns, and a second copy drifts + * Exported for the same reason `whyRefusedWithoutRunning` is: which arguments + * a call requires is a rule the runtime already owns, and a second copy drifts * silently -- into a suggested command that is refused the moment it is run. * - * A guard counts only when it names a constraint the model declares, because - * an unresolved guard name is inert. `--judge-reads-store` rides along wherever - * the model has tables to read: a judgment comparing the call against what is - * recorded is refused without it, nothing in a constraint's wording marks which - * judgments those are, and a judge with nothing to look up looks nothing up. + * A guard changes none of this. `kcmd action-run` settles none of them however + * the action is written, so there is no flag about guards for the line to + * carry and no reader who needs one to make the call work. */ -export function runLine(a: Action, runtime: SemanticRuntime): string { - return [`kcmd action run ${a.name}`, ...runFlags(a, runtime)].join(' '); +export function runLine(a: Action): string { + return [`kcmd action-run ${a.name}`, ...runFlags(a)].join(' '); } /** @@ -474,19 +485,9 @@ export function runLine(a: Action, runtime: SemanticRuntime): string { * only thing in it to split on is ' --', which an action name is free to * contain. Nothing constrains what is in a name. */ -export function runFlags(a: Action, runtime: SemanticRuntime): string[] { - const model = runtime.model; - const guards = new Set(a.guards ?? []); - const judged = (model.constraints ?? []).some(c => guards.has(c.name)); - const canRead = judged && !!runtime.store && - readableEntities(runtime, dialectFor(runtime.store)).length > 0; - const flags: string[] = []; - if (judged) flags.push('--judge'); - if (canRead) flags.push('--judge-reads-store'); - for (const p of a.parameters.filter(isParameterRequired)) { - flags.push(`--arg ${p.name}=<${p.type ?? 'no type'}>`); - } - return flags; +export function runFlags(a: Action): string[] { + return a.parameters.filter(isParameterRequired) + .map(p => `--arg ${p.name}=<${p.type ?? 'no type'}>`); } /** @@ -501,7 +502,7 @@ export function runFlags(a: Action, runtime: SemanticRuntime): string[] { */ export function whyRefusedWithoutRunning( model: SemanticModel, action: Action, handler?: ActionHandler, - judge?: Judge): string|null { + judge?: Judge, skipGuards?: boolean): string|null { // No executor at all is a binding outcome, not a broken model: the executor // is a physical facet, so an action can be declared here and performable // only somewhere else. Say which it is, because the fix is in the profile @@ -525,8 +526,8 @@ export function whyRefusedWithoutRunning( // Nothing about a parameter can refuse an action here any more. Every // parameter is a scalar and binds as one, so a key with three parts is three // ordinary parameters and there is no shape of key this runtime cannot pass - // to a statement. - return unsafeToRunUnchecked(model, action, judge); + // to a statement. What is left is the guards. + return unsafeToRunUnchecked(model, action, judge, skipGuards); } @@ -544,7 +545,8 @@ export function whyRefusedWithoutRunning( // does is the author's, which is what `affects` describes and what the // evaluator will check against the statements once it exists. function unsafeToRunUnchecked( - model: SemanticModel, action: Action, judge?: Judge): string|null { + model: SemanticModel, action: Action, judge?: Judge, + skipGuards?: boolean): string|null { // A guard names a constraint the author says is checked before the call. // One whose `onViolation` is `warn` reports rather than refuses, so an // evaluator would let the write through, and refusing here would make a @@ -587,7 +589,14 @@ function unsafeToRunUnchecked( // Refusing it HERE is what keeps this function and `runAction` in agreement: // a tool advertised as runnable and then refused mid-call spends the // caller's turn and teaches it nothing. - if (guards.length && !judge) { + // + // `skipGuards` is the caller saying nobody will be asked, so this refusal + // stands down. Only this one: the two above are the model being wrong about + // its own rules -- a guard that quotes nothing, a guard that names nothing -- + // and not asking repairs neither. They are also what a push refuses, so + // standing them down here would make this runtime disagree with the + // validation that gates publishing the model. + if (!skipGuards && guards.length && !judge) { return `Action '${action.name}' is guarded by ${quoteList(guards)}, ` + `which ${guards.length === 1 ? 'is' : 'are'} settled by reading the ` + `call, and this runtime was given no judge to ask. Running it would ` + diff --git a/toolbox/mdcode/src/libts/semantic/runtime/runtime.ts b/toolbox/mdcode/src/libts/semantic/runtime/runtime.ts index 213407e3..f54a6cd6 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/runtime.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/runtime.ts @@ -8,7 +8,7 @@ // find a store for. // // Both halves were the CLI's private business until an agent needed them. An -// agent runs the same actions `kcmd action run` runs, against the same store, +// agent runs the same actions `kcmd action-run` runs, against the same store, // so it has to reach them the same way -- and a second loader that merges // profiles slightly differently is a demo that passes while the product fails. // The CLI calls this too, so there is one answer to "what does this scope say" diff --git a/toolbox/mdcode/src/libts/semantic/skills.ts b/toolbox/mdcode/src/libts/semantic/skills.ts index 658716cb..195e6a0d 100644 --- a/toolbox/mdcode/src/libts/semantic/skills.ts +++ b/toolbox/mdcode/src/libts/semantic/skills.ts @@ -529,13 +529,11 @@ function runningSection( const example = actions.find(t => t.runnable)!; const action = actionFor(example, runtime.model); // `runLine` is the runtime's own answer, and it is asked rather than - // reproduced. Whether a judge is needed, whether that judge has to read the - // store, and which arguments are required are three rules this module got - // wrong when it derived them itself: a guard was counted before it resolved - // to a declared constraint, `--judge-reads-store` was never offered, and - // every parameter was listed as if required. A line that is certain to be - // refused is worse than no line, so there is one copy of the rule. - const flags = action ? runFlags(action, runtime) : null; + // reproduced. Which arguments a call requires is a rule this module got + // wrong when it derived it itself, listing every parameter as if required. + // A line that is certain to be refused is worse than no line, so there is + // one copy of the rule. + const flags = action ? runFlags(action) : null; if (flags) { // The head is rebuilt so a name that needs shell quoting gets it -- // nothing constrains what is in an action name, and this is a block meant @@ -546,7 +544,7 @@ function runningSection( // with no flags at all would otherwise end on a continuation with nothing // after it. const parts = [ - `kcmd action run ${shellArg(example.actionName)}`, + `kcmd action-run ${shellArg(example.actionName)}`, `--profile ${shellArg(runtime.profile)}`, ...flags, ]; @@ -555,15 +553,16 @@ function runningSection( out.push('```'); out.push(''); } - if (flags?.includes('--judge')) { + // Said wherever the action states a rule, because the command writes either + // way. An agent that tried the line, saw it commit, and took that for the + // rules holding would have drawn the one conclusion this command cannot + // support. + if (action?.guards?.length) { out.push( - '`--judge` is what settles the rules stated in words. Without it a ' + - 'guarded action is refused rather than run unchecked.' + - (flags.includes('--judge-reads-store') ? - ' `--judge-reads-store` lets that judge read the model\'s own ' + - 'tables, which a rule about something on record rather than ' + - 'in the arguments cannot be settled without.' : - '')); + 'That command line settles no guard. It names the rules this action ' + + 'states and runs the write regardless, so it answers whether the ' + + 'call binds and the write lands, and nothing about whether the rules ' + + 'hold. The runtime your framework calls is what settles them.'); out.push(''); } return out; @@ -650,7 +649,7 @@ function referenceDocument( const out: string[] = []; out.push(`# ${action.name}`); out.push(''); - // Both names, once, here. `action.name` is what `kcmd action run` takes and + // Both names, once, here. `action.name` is what `kcmd action-run` takes and // what every command line in this package uses; `tool.name` is what the same // action is called when a framework hands it over as a tool. An agent meets // one or the other depending on how it was wired, and a page that showed diff --git a/toolbox/mdcode/src/libts/semantic/validate.ts b/toolbox/mdcode/src/libts/semantic/validate.ts index 636d0dd2..be1acd37 100644 --- a/toolbox/mdcode/src/libts/semantic/validate.ts +++ b/toolbox/mdcode/src/libts/semantic/validate.ts @@ -170,7 +170,7 @@ export function validatePushRequirements( } // The subset of the push checks that bear on RUNNING an action rather than on -// deploying a model. `kcmd action run` skips the deployment checks on purpose +// deploying a model. `kcmd action-run` skips the deployment checks on purpose // -- it deploys nothing -- but it must not skip these, because the runtime acts // on exactly what they verify. // diff --git a/toolbox/mdcode/src/tool/commands.ts b/toolbox/mdcode/src/tool/commands.ts index a7b68db9..48438cc5 100644 --- a/toolbox/mdcode/src/tool/commands.ts +++ b/toolbox/mdcode/src/tool/commands.ts @@ -9,7 +9,6 @@ import * as kcmd from '../libts'; import {BigQueryClient} from '../libts/gcp/bigquery'; import * as context from '../libts/gcp/context'; import * as dataplex from '../libts/gcp/dataplex'; -import {GeminiJudge} from '../libts/gcp/gemini'; import {SemanticModelLayout} from '../libts/layouts/semantic-model'; import {convertOwlToOsi} from '../libts/semantic/converters/owl/convert'; import * as deploy from '../libts/semantic/deploy_bigquery'; @@ -23,8 +22,6 @@ import {serializeModel} from '../libts/semantic/osi_converter'; import {pullKnowledgeCatalog} from '../libts/semantic/pull_kc'; import {AvailabilityReport, DEFAULT_PROFILE, mergeProfileOntoDoc, pruneUnavailable,} from '../libts/semantic/resolve_profiles'; import {ActionTool, EntityTool, modelTools} from '../libts/semantic/runtime/agent_tools'; -import {Judge, JudgeStore} from '../libts/semantic/runtime/judge'; -import {modelJudgeStore} from '../libts/semantic/runtime/judge_store'; import {isParameterRequired, runAction, runLine, whyRefusedWithoutRunning,} from '../libts/semantic/runtime/run_action'; import {createSemanticRuntimes, runtimeClient, SemanticRuntime} from '../libts/semantic/runtime/runtime'; import {dataClientFor, storeLine} from '../libts/semantic/runtime/store'; @@ -60,11 +57,11 @@ export interface PushOptions { // Unlike `force` above, this authorizes a destructive delete rather // than overriding a conflict. forceRemove?: boolean; - // Whether to push the Knowledge Catalog metadata leg. On by default; `--no-kc` - // sets it false to deploy only the graph. The catalog toggle is symmetric with - // the profile axis below: `--no-profile` gives a catalog-only push, `--no-kc` a - // graph-only push, and both together are an error (nothing to deploy). Ignored - // for non-semantic-model scopes. + // Whether to push the Knowledge Catalog metadata leg. On by default; + // `--no-kc` sets it false to deploy only the graph. The catalog toggle is + // symmetric with the profile axis below: `--no-profile` gives a catalog-only + // push, `--no-kc` a graph-only push, and both together are an error (nothing + // to deploy). Ignored for non-semantic-model scopes. kc?: boolean; // Print each pushed destination's generated artifact in that destination's // native format (BigQuery/Spanner Graph -> SQL DDL, Knowledge Catalog -> the @@ -141,10 +138,10 @@ export function checkPushSelection(sel: { // `deployment_target:` sugar or a GOOGLE custom_extension `deploymentTargets`. // Drives the push mode: a push whose models all declare no target governs the // logical model only -- it deploys no graph, so bindings and a target are not -// required and pruning is skipped (Knowledge Catalog publishes the whole model). -// On any ambiguity (unparseable YAML, malformed GOOGLE data) it returns true, so -// the strict load reports the problem rather than silently taking the logical -// path. +// required and pruning is skipped (Knowledge Catalog publishes the whole +// model). On any ambiguity (unparseable YAML, malformed GOOGLE data) it returns +// true, so the strict load reports the problem rather than silently taking the +// logical path. export function declaresGraphTarget(text: string): boolean { let doc: any; try { @@ -308,10 +305,11 @@ export async function push(options: PushOptions): Promise { const layout = snapshot.layout as SemanticModelLayout; const source = snapshot.manifest.source as SemanticModelSource; - // The push has two axes. The profile axis says how many binding profiles the - // graph deploys for: --no-profile (options.profile === false) deploys none -- - // a catalog-only push; --profile deploys that one; --all-profiles every - // one; and the default (undefined) the model's default binding + // The push has two axes. The profile axis says how many binding profiles + // the graph deploys for: --no-profile (options.profile === false) deploys + // none -- a catalog-only push; --profile deploys that one; + // --all-profiles every one; and the default (undefined) the model's default + // binding // (`default_profile` from catalog.yaml, else the inline bindings -- the // implicit 'default' profile). A profile's deployment target names the // backend; the command line never does. The catalog axis is --no-kc. @@ -337,9 +335,9 @@ export async function push(options: PushOptions): Promise { // Reserve the profile name 'default': it is the sentinel for the inline // bindings (never merged onto the document), so a // `.profiles/default.yaml` would be silently unreachable. Reject it - // rather than let it sit there doing nothing. Only when the graph axis is on: - // a graph push may resolve profiles, but a catalog-only --no-profile push - // reads no profile files, so it does not police their names. + // rather than let it sit there doing nothing. Only when the graph axis is + // on: a graph push may resolve profiles, but a catalog-only --no-profile + // push reads no profile files, so it does not police their names. if (graphEnabled) { for (const doc of layoutDocs) { const clash = layout.profileDocuments(doc.name).some( @@ -375,9 +373,8 @@ export async function push(options: PushOptions): Promise { console.error( `Error: unknown binding profile '${profileName}' for model '${ doc.name}'; ` + - (names.length ? - `defined profiles: ${names.join(', ')}.` : - `no profiles are defined for this model.`)); + (names.length ? `defined profiles: ${names.join(', ')}.` : + `no profiles are defined for this model.`)); return null; } const res = mergeProfileOntoDoc(doc.text, chosen.text, profileName); @@ -407,55 +404,55 @@ export async function push(options: PushOptions): Promise { }|null> => { const loaded = loadSemanticModels(docs, {defaultProject, bindingOptional: !prune}); - if (loaded.error) { - console.error('Error:', loaded.error); - return null; - } - for (const w of loaded.warnings) { - if (options.transpile && w.includes('needs transpilation')) continue; - console.warn(`Warning: ${w}`); - } - let models = loaded.models; - if (options.transpile) { - const transpiled = await transpileModels(models); - models = transpiled.models; - for (const w of transpiled.warnings) console.warn(`Warning: ${w}`); - } - if (prune) { - const availability: AvailabilityReport[] = []; - models = models.map(({document, model}) => { - const {model: pruned, report} = pruneUnavailable(model, profileName); - availability.push(report); - return {document, model: pruned}; - }); - for (const r of availability) { - const dropped = r.droppedEntities.length + r.droppedMetrics.length + - r.droppedRelationships.length + r.droppedActions.length; - if (r.unboundFields.length || dropped) { - console.warn( - `Note: profile '${r.profile}' leaves ${ - r.unboundFields.length} field(s) unbound` + - (dropped ? - `; ${r.droppedEntities.length} entity(ies), ${ - r.droppedMetrics.length} metric(s), ${ - r.droppedRelationships.length} relationship(s) ` + - `and ${r.droppedActions.length} action(s) ` + - `unavailable` : - '') + - '.'); - } - } - } - const validationErrors = validatePushRequirements( - models, {targetOptional: !prune, fieldsPruned: prune}); - if (validationErrors.length) { - for (const e of validationErrors) console.error(`Error: ${e}`); - return null; + if (loaded.error) { + console.error('Error:', loaded.error); + return null; + } + for (const w of loaded.warnings) { + if (options.transpile && w.includes('needs transpilation')) continue; + console.warn(`Warning: ${w}`); + } + let models = loaded.models; + if (options.transpile) { + const transpiled = await transpileModels(models); + models = transpiled.models; + for (const w of transpiled.warnings) console.warn(`Warning: ${w}`); + } + if (prune) { + const availability: AvailabilityReport[] = []; + models = models.map(({document, model}) => { + const {model: pruned, report} = pruneUnavailable(model, profileName); + availability.push(report); + return {document, model: pruned}; + }); + for (const r of availability) { + const dropped = r.droppedEntities.length + r.droppedMetrics.length + + r.droppedRelationships.length + r.droppedActions.length; + if (r.unboundFields.length || dropped) { + console.warn( + `Note: profile '${r.profile}' leaves ${ + r.unboundFields.length} field(s) unbound` + + (dropped ? + `; ${r.droppedEntities.length} entity(ies), ${ + r.droppedMetrics.length} metric(s), ${ + r.droppedRelationships.length} relationship(s) ` + + `and ${r.droppedActions.length} action(s) ` + + `unavailable` : + '') + + '.'); } - const bqModels = models.filter(m => hasTargetType(m, 'bigquery')); - const spannerModels = models.filter(m => hasTargetType(m, 'spanner')); - return {models, bqModels, spannerModels}; - }; + } + } + const validationErrors = validatePushRequirements( + models, {targetOptional: !prune, fieldsPruned: prune}); + if (validationErrors.length) { + for (const e of validationErrors) console.error(`Error: ${e}`); + return null; + } + const bqModels = models.filter(m => hasTargetType(m, 'bigquery')); + const spannerModels = models.filter(m => hasTargetType(m, 'spanner')); + return {models, bqModels, spannerModels}; + }; // Merge + prepare a profile once and reuse it. The graph leg and the // Knowledge Catalog leg both consume the default binding, so without this a @@ -465,8 +462,7 @@ export async function push(options: PushOptions): Promise { // --all-profiles may prepare a filtered subset of the documents) the set of // document names. type Prepared = { - models: LoadedModel[]; - bqModels: LoadedModel[]; + models: LoadedModel[]; bqModels: LoadedModel[]; spannerModels: LoadedModel[]; }; const mergeCache = @@ -514,20 +510,22 @@ export async function push(options: PushOptions): Promise { } } else { graphProfileNames.push( - namedProfile ?? snapshot.manifest.defaultProfile ?? DEFAULT_PROFILE); + namedProfile ?? snapshot.manifest.defaultProfile ?? + DEFAULT_PROFILE); } } // Deploy each selected profile's graph (BigQuery first within a profile, so // a fail-fast push stops before later legs). Only the documents that, after // the merge, declare a deployment target contribute a graph; the rest are - // left to the Knowledge Catalog leg (a profile that binds no target at all is - // skipped). The live BigQuery pre-flight runs before each BigQuery deploy so - // a push fails fast when a source table is unreachable; it also runs under - // --validate-only. A deployment target may be claimed by only one profile in - // a run: two profiles pointing at the same graph would have the second - // CREATE OR REPLACE silently overwrite the first, so that is an error rather - // than last-write-wins. + // left to the Knowledge Catalog leg (a profile that binds no target at all + // is skipped). The live BigQuery pre-flight runs before each BigQuery + // deploy so a push fails fast when a source table is unreachable; it also + // runs under + // --validate-only. A deployment target may be claimed by only one profile + // in a run: two profiles pointing at the same graph would have the second + // CREATE OR REPLACE silently overwrite the first, so that is an error + // rather than last-write-wins. const multiProfile = graphProfileNames.length > 1; const claimedTargets = new Map(); // target URI -> profile const deployedProfiles: string[] = []; @@ -538,8 +536,11 @@ export async function push(options: PushOptions): Promise { // Per model, what deploys through the Knowledge Catalog leg ALONE: // actions and constraints both have a catalog home and no graph one, so a // push that omits that leg has to account for either. - const catalogOnly = - new Map(); + const catalogOnly = new Map < string, { + actions: number; + constraints: number + } + >(); const noteCatalogOnly = (loaded: LoadedModel[]) => { for (const {model} of loaded) { const actions = model.actions?.length ?? 0; @@ -564,8 +565,8 @@ export async function push(options: PushOptions): Promise { if (!prepared) return 1; for (const d of docs) loadedDocs.add(d.name); noteCatalogOnly(prepared.models); - // Fail before any deploy if this profile's targets collide with a graph an - // earlier profile already claimed this run. + // Fail before any deploy if this profile's targets collide with a graph + // an earlier profile already claimed this run. for (const m of prepared.models) { for (const uri of deploy.deploymentTargetUris(m.model)) { const owner = claimedTargets.get(uri); @@ -601,8 +602,8 @@ export async function push(options: PushOptions): Promise { deployedProfiles.push(profileName); } // Under --all-profiles the per-leg "Deployed N" lines alone don't show the - // whole fan-out, so summarize which profiles deployed and which were skipped - // for declaring no deployment target. + // whole fan-out, so summarize which profiles deployed and which were + // skipped for declaring no deployment target. if (allProfiles && (deployedProfiles.length || skippedProfiles.length)) { console.log( `Deployed ${deployedGraphs} graph(s) across ${ @@ -626,8 +627,8 @@ export async function push(options: PushOptions): Promise { // does not define this profile is skipped rather than failing a warning. const kcProfileName = namedProfile ?? snapshot.manifest.defaultProfile ?? DEFAULT_PROFILE; - const rest = - (mergeOnce(kcProfileName, true) ?? []).filter(d => !loadedDocs.has(d.name)); + const rest = (mergeOnce(kcProfileName, true) ?? + []).filter(d => !loadedDocs.has(d.name)); if (rest.length) { const prepared = await prepareOnce(rest, kcProfileName, false); // prepareModels has already printed why it failed. Ignoring that here @@ -645,7 +646,8 @@ export async function push(options: PushOptions): Promise { // Knowledge Catalog records one canonical view of the logical model: the // single --profile selection, else the default binding. Alongside a graph // deploy the entries reflect that binding (pruned to what it answers); a - // catalog-only --no-profile push publishes the whole logical model unpruned. + // catalog-only --no-profile push publishes the whole logical model + // unpruned. if (kcEnabled) { const kcProfileName = namedProfile ?? snapshot.manifest.defaultProfile ?? DEFAULT_PROFILE; @@ -780,7 +782,8 @@ export async function profiles(): Promise { for (const d of report.droppedEntities) { withheld.push(`entity ${d.name} (${d.reason})`); } - for (const f of report.unboundFields) withheld.push(`field ${f} (unbound)`); + for (const f of report.unboundFields) + withheld.push(`field ${f} (unbound)`); for (const d of report.droppedRelationships) { withheld.push(`relationship ${d.name} (${d.reason})`); } @@ -1090,7 +1093,8 @@ export async function owl( // convertOwlToOsi throws only on malformed Turtle; main.ts's try/catch // reports it. - const result = convertOwlToOsi(turtle, modelName, {compactFlow: options.compact}); + const result = + convertOwlToOsi(turtle, modelName, {compactFlow: options.compact}); for (const w of result.warnings) { console.warn(`Warning: ${w}`); } @@ -1177,41 +1181,16 @@ export interface ActionOptions { // `string|boolean` for the same reason push's is: cac yields `true` for a // bare `--profile` and `false` for `--no-profile`. profile?: string|boolean; - // `--store`: print where a run would land and nothing else (`list` only). + // `--store`: print where a run would land and nothing else. store?: boolean; - // `--judge [model]`: settle the guards stated in words by asking Gemini. - // `true` for a bare `--judge`, which takes the default model. - judge?: string|boolean; - // `--judge-location `: the Vertex AI region to ask in. - judgeLocation?: string; - // `--judge-reads-store`: let the judge read the model's tables while it - // decides, so a rule stated in words can compare the call against what is - // recorded rather than only against what the caller said. - judgeReadsStore?: boolean; } -// Lists or runs a semantic model's actions. -// -// kcmd action list -// kcmd action run --arg = ... -// -// `list` answers "what can I run, and how": each action's parameters, executor, -// guards and blast radius, ending with the command line that runs it. `run` -// executes one against the store the model's deployment target names -- the -// command line never says where to write, the same rule push follows, so -// changing stores is changing profiles rather than remembering a flag. -// -// Returns a process exit code (0 on success). -export async function action( - command: string, name: string|undefined, - options: ActionOptions = {}): Promise { - if (command !== 'list' && command !== 'run') { - console.error(`Error: unknown action command '${ - command}'; expected 'list' or 'run'.`); - return 1; - } - +// Opens every semantic model in scope under the named binding profile, which +// is the one thing both action commands need before they can say anything. +// Returns the runtimes, or an exit code if the scope would not open. +async function openActionRuntimes(options: ActionOptions): + Promise { const ctx = context.ApiContext.default(); // cac hands back `true` for a bare `--profile` and mri `false` for // `--no-profile`; neither names a profile, and `??` would let both through @@ -1225,35 +1204,46 @@ export async function action( return 1; } - return command === 'list' ? listActions(opened, options) : - await runOneAction(opened, ctx, name, options); + return opened; } -// A `NOT RUNNABLE:` line wraps under the label column the other lines use, so -// a reason running to three lines still reads as one entry's answer. -const RUN_INDENT = ' '; +// Lists what a semantic model declares as runnable. +// +// kcmd action-list [name] +// +// Answers "what can I run, and how": each action's parameters, executor, +// guards and blast radius, ending with the command line that runs it. +// +// Returns a process exit code (0 on success). +export async function actionList( + _name: string|undefined, options: ActionOptions = {}): Promise { + const opened = await openActionRuntimes(options); + if (typeof opened === 'number') return opened; + return listActions(opened, options); +} -// Stands in for the judge the printed run line promises. +// Runs one of a semantic model's actions. +// +// kcmd action-run --arg = ... // -// `--judge` is a `run` flag; the listing does not take one and asks nothing. -// Handing the runtime nothing here would report every judged action as -// unrunnable and send the reader off to fix a model that is fine -- the -// command printed under it carries `--judge`, because `runLine` puts it there, -// and that command is what this listing is describing. So the question asked -// is asked with a judge in hand. +// Executes against the store the model's deployment target names -- the +// command line never says where to write, the same rule push follows, so +// changing stores is changing profiles rather than remembering a flag. // -// It throws rather than answering. Nothing in a listing may reach a judge, and -// a path that somehow got this far should say so loudly instead of settling a -// rule with a stub verdict. -const JUDGE_THE_RUN_LINE_SUPPLIES: Judge = { - name: 'the judge `--judge` supplies', - decide() { - throw new Error( - '`kcmd action list` lists what a run would do and never asks a judge.'); - }, -}; +// Returns a process exit code (0 on success). +export async function actionRun( + name: string|undefined, options: ActionOptions = {}): Promise { + const opened = await openActionRuntimes(options); + if (typeof opened === 'number') return opened; + return await runOneAction(opened, name, options); +} + + +// A `NOT RUNNABLE:` line wraps under the label column the other lines use, so +// a reason running to three lines still reads as one entry's answer. +const RUN_INDENT = ' '; // Prints what each model declares as runnable. The last line of every entry is @@ -1269,7 +1259,7 @@ function listActions( // action writes to, and the profile's deployment target is what decides // that; a second place to say it is a second place to say it differently. if (options.store) { - // One line, because the caller is `STORE=$(kcmd action list --store)` and + // One line, because the caller is `STORE=$(kcmd action-list --store)` and // a second line makes that variable address the wrong database. A scope // holding several models has no single answer, so it says so instead. if (runtimes.length > 1) { @@ -1335,12 +1325,18 @@ function listActions( // reachable is the same sentence on every action in the model and says // nothing about any of them, and the listing has already said it once, // at the top, where it belongs. - const blocked = whyRefusedWithoutRunning( - model, a, undefined, JUDGE_THE_RUN_LINE_SUPPLIES); + // + // Asked the way `kcmd action-run` runs: with nobody to settle a guard, + // and not held up for want of one. What survives that is a model this + // command could not run however it was invoked -- an executor it holds + // no handler for, a guard quoting nothing, a guard naming nothing -- + // which is a thing to fix rather than a flag to add. + const blocked = + whyRefusedWithoutRunning(model, a, undefined, undefined, true); if (blocked) { console.log(wrapTo(`NOT RUNNABLE: ${blocked}`, RUN_INDENT)); } else { - console.log(` run: ${runLine(a, runtime)}`); + console.log(` run: ${runLine(a)}`); } } } @@ -1352,15 +1348,12 @@ export interface AgentOptions { // `string|boolean` for the same reason the others are: cac yields `true` for // a bare `--profile` and `false` for `--no-profile`. profile?: string|boolean; - // `--judge [model]`: derive the listing as an agent holding a judge would - // see it. `true` for a bare `--judge`, which takes the default model. - judge?: string|boolean; } // Prints what an agent is handed when it is pointed at this model. // -// kcmd agent tools +// kcmd agent-tools // // Two halves and an instruction, all three derived: one lookup per entity, one // write per action, and what to tell the agent about using them. Nothing here @@ -1370,26 +1363,17 @@ export interface AgentOptions { // // A tool the runtime cannot run today is listed and marked rather than // dropped. The model declares it; what it is waiting on is the useful thing to -// print. `kcmd action run` calls the write half; the read half is a SELECT the +// print. `kcmd action-run` calls the write half; the read half is a SELECT the // tool would issue, and `gcloud spanner databases execute-sql` will run it. // -// What the listing can call depends on what the caller holds, so `--judge` -// takes the same argument `kcmd action run` does. Without it an action guarded -// by a rule stated in words is marked NOT RUNNABLE, which is accurate for a -// caller holding no judge and misleading about an agent that holds one -- the -// point of this command is to show what the agent will be handed, and the -// judge is part of what decides that. No model is called here either way: a -// judge is asked when an action runs, and the listing runs none. +// A guard is not what decides whether a tool is listed. Who settles one +// belongs to the application that embeds the runtime, and this command cannot +// know what that will be, so withholding a guarded tool here would describe a +// caller rather than the model. What it does say, in the tool's own +// description, is which rules the agent's calls will be held to. // // Returns a process exit code (0 on success). -export async function agent( - command: string, options: AgentOptions = {}): Promise { - if (command !== 'tools') { - console.error( - `Error: unknown agent command '${command}'; expected 'tools'.`); - return 1; - } - +export async function agentTools(options: AgentOptions = {}): Promise { const ctx = context.ApiContext.default(); const named = typeof options.profile === 'string' ? options.profile : undefined; @@ -1399,14 +1383,6 @@ export async function agent( return 1; } - // Built the way `kcmd action run --judge` builds it, from the context this - // command already holds, so the listing and the run agree about who answers. - const judge = options.judge ? - new GeminiJudge( - ctx, typeof options.judge === 'string' ? {model: options.judge} : {}) : - undefined; - if (judge) console.log(`Rules stated in words go to ${judge.name}.`); - let incomplete = false; for (const runtime of opened) { const {model, store, storeError, profile, entryGroup} = runtime; @@ -1422,8 +1398,9 @@ export async function agent( // uncallable, and a caller reading the exit code would take a listing that // offers nothing for a listing that offers everything. const unusable = store ? dataClientFor(store) : undefined; - const why = store ? (unusable && 'error' in unusable ? unusable.error : '') : - storeError ?? ''; + const why = store ? + (unusable && 'error' in unusable ? unusable.error : '') : + storeError ?? ''; if (!store || why) { console.log(' offers no tools under this profile.'); console.log(wrapTo(why, BODY_INDENT)); @@ -1434,7 +1411,8 @@ export async function agent( console.log(` store: ${storeLine(store)}`); console.log(); - const {lookups, actions, instruction} = modelTools({runtime, judge}); + const {lookups, actions, instruction} = + modelTools({runtime, skipGuards: true}); for (const tool of actions) printActionTool(tool); for (const tool of lookups) printLookupTool(tool); console.log(' instruction:'); @@ -1460,7 +1438,7 @@ export interface SkillsGenerateOptions { // Writing a model out as an Agent Skill. // -// `kcmd agent tools` prints what an agent is offered and forgets it. This +// `kcmd agent-tools` prints what an agent is offered and forgets it. This // writes the same derivation to disk in the form an agent loads by itself: a // directory per model, a `SKILL.md` a client reads, and the per-action detail // in files beside it. Same derivation, so a skill cannot describe a tool that @@ -1500,8 +1478,8 @@ export async function skillsGenerate(options: SkillsGenerateOptions = {}): // already been written -- the command refuses the scope and leaves half of // it on disk, under a name it has just said it cannot assign. `generateSkill` // is pure, so ordering the work this way costs only the order. - const planned: - Array<{model: string; dir: string; generated: SkillPackage;}> = []; + const planned: Array<{model: string; dir: string; generated: SkillPackage;}> = + []; // A skill name is a lossy form of a model name, so two models in one scope // can arrive at one directory. Writing both would leave the second on disk // and the first gone, with two "Wrote ..." lines and an exit code of 0 @@ -1701,11 +1679,11 @@ function describeParameter(p: ActionParameter): string { // Runs one action against the store its model's deployment target names. async function runOneAction( - runtimes: SemanticRuntime[], ctx: context.ApiContext, - name: string|undefined, options: ActionOptions): Promise { + runtimes: SemanticRuntime[], name: string|undefined, + options: ActionOptions): Promise { if (!name) { console.error( - 'Error: `kcmd action run` needs an action name; `kcmd action list` ' + + 'Error: `kcmd action-run` needs an action name; `kcmd action-list` ' + 'shows what this scope declares.'); return 1; } @@ -1765,49 +1743,25 @@ async function runOneAction( return 1; } - // Refused rather than ignored. A caller who asked for a reading judge and - // got an ordinary one is a caller whose rule about the order total quietly - // went unread, which is the failure this whole flag exists to avoid. - if (options.judgeReadsStore && !options.judge) { - console.error( - `Error: --judge-reads-store says what a judge may do; --judge is ` + - `what hires one. Pass both.`); - return 1; - } - // What the judge may read, if anything. Composed from this runtime, so the - // tables it can see are the ones the model declares under the profile this - // run is using, and its reads land on the database the write will land on. - let judgeStore: JudgeStore|undefined; - if (options.judgeReadsStore) { - const built = modelJudgeStore(runtime, { - // Printed as it is sent. A judge that read the store did something on - // the caller's behalf, and a transcript showing the verdict without the - // reads behind it is one nobody can check. - onRead: sql => - console.log(` the judge reads: ${sql.replace(/\s+/g, ' ').trim()}`), - }); - if ('error' in built) { - console.error(`Error: ${built.error}`); - return 1; - } - judgeStore = built; - } - - // Built from the context this command already holds, so judging costs no - // second trip to gcloud for a project and a token. - const judge = options.judge ? new GeminiJudge(ctx, { - ...(typeof options.judge === 'string' ? {model: options.judge} : {}), - ...(options.judgeLocation ? {location: options.judgeLocation} : {}), - ...(judgeStore ? {store: judgeStore} : {}), - }) : undefined; - console.log(`Running '${name}' on ${runtime.store.name}...`); - if (judge) console.log(` rules stated in words go to ${judge.name}`); - if (judgeStore) { - console.log(` it may read ${runtime.model.name}'s tables to settle them`); + // Said before the write rather than after it, and named rule by rule, so a + // reader watching the run knows what went unenforced while it is still + // happening. This command checks no guard at all: settling one takes a judge, + // and who that is belongs to whoever dispatches the call in earnest. Nothing + // is printed for an action that declares none, because nothing was skipped. + const guards = + (runtime.model.actions ?? []).find(a => a.name === name)?.guards ?? []; + if (guards.length) { + console.log( + ` NOT CHECKED: ${guards.join(', ')} -- this command settles ` + + `no guard, and the write still happens`); } - const outcome = - await runAction({runtime, actionName: name, args: parsed.args, judge}); + const outcome = await runAction({ + runtime, + actionName: name, + args: parsed.args, + skipGuards: true, + }); if (outcome.status === 'error') { console.error(`Error: ${outcome.message}`); return 1; diff --git a/toolbox/mdcode/src/tool/main.ts b/toolbox/mdcode/src/tool/main.ts index 20f19698..74101b00 100644 --- a/toolbox/mdcode/src/tool/main.ts +++ b/toolbox/mdcode/src/tool/main.ts @@ -133,30 +133,40 @@ cli.command( cli.command( - 'action [name]', - 'Semantic model actions (command: `list` what the model declares, or `run` one against its store)') - .option( - '--arg ', - 'Bind one action parameter; repeat the flag for each one (`run` only)') + 'action-list [name]', + 'List what a semantic model declares as runnable: parameters, executor, guards, blast radius, and the command that runs each one') .option( '--profile [name]', 'Read the model under this binding profile; its deployment target names the database the action runs against; defaults to default_profile, else the inline bindings') .option( '--store', - 'Print only where a run would land: project/instance/database for Spanner, and the backend named ahead of the path for any other store (`list` only)') - .option( - '--judge [model]', - 'Settle guards the model states in words by asking Gemini on Vertex AI, naming a model or taking the default; without it, an action guarded by such a rule is refused rather than run unchecked (`run` only)') + 'Print only where a run would land: project/instance/database for Spanner, and the backend named ahead of the path for any other store') + .action(async (name, options) => { + let exitCode = 1; + try { + exitCode = await commands.actionList(name, options); + } catch (err: any) { + console.error('Error:', err.message || err); + exitCode = 1; + } + + process.exit(exitCode); + }); + + +cli.command( + 'action-run ', + 'Run one of a semantic model\'s actions against the store its deployment target names; the guards it declares are NOT checked') .option( - '--judge-location ', - 'Ask the judge in this Vertex AI region, which is where the argument values are sent; defaults to us-central1 (`run` only)') + '--arg ', + 'Bind one action parameter; repeat the flag for each one') .option( - '--judge-reads-store', - 'Let the judge read the model\'s own tables while it decides, so a rule stated in words can compare the call against what is recorded; costs one model call more per guard and needs --judge (`run` only)') - .action(async (command, name, options) => { + '--profile [name]', + 'Read the model under this binding profile; its deployment target names the database the action runs against; defaults to default_profile, else the inline bindings') + .action(async (name, options) => { let exitCode = 1; try { - exitCode = await commands.action(command, name, options); + exitCode = await commands.actionRun(name, options); } catch (err: any) { console.error('Error:', err.message || err); exitCode = 1; @@ -167,18 +177,15 @@ cli.command( cli.command( - 'agent ', - 'Agent bindings for a semantic model (command: `tools`, what an agent is offered)') + 'agent-tools', + 'List what an agent holding this semantic model is offered') .option( '--profile [name]', 'Read the model under this binding profile; defaults to default_profile, else the inline bindings') - .option( - '--judge [model]', - 'List what an agent holding a judge is offered, naming a Gemini model or taking the default; without it, an action guarded by a rule stated in words is marked NOT RUNNABLE. No model is called either way') - .action(async (command, options) => { + .action(async (options) => { let exitCode = 1; try { - exitCode = await commands.agent(command, options); + exitCode = await commands.agentTools(options); } catch (err: any) { console.error('Error:', err.message || err); exitCode = 1; diff --git a/toolbox/mdcode/tests/libts/semantic/actions.test.ts b/toolbox/mdcode/tests/libts/semantic/actions.test.ts index 13c1e801..8c053727 100644 --- a/toolbox/mdcode/tests/libts/semantic/actions.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/actions.test.ts @@ -1118,7 +1118,7 @@ describe('parameter description, required, and default', () => { describe('a published statement and a run read the verb the same way', () => { // These are the forms `run_action.test.ts` already drives through a `sql` // executor. Before the readers were shared, every one of them ran in the - // library and was refused by `kcmd push` and `kcmd action run`, which both + // library and was refused by `kcmd push` and `kcmd action-run`, which both // call `sqlExecutorErrors` first -- so the tests below and those ones // disagreed about the same model. const target = diff --git a/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md b/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md index 7878c825..b60d93c9 100644 --- a/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md +++ b/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md @@ -53,15 +53,13 @@ Everything above is true of this model wherever it is deployed. This section is `kcmd` is a command line for inspecting and debugging a model, not the runtime an agent should call in production. Use it to try a call and to see what a refusal says. An agent that runs continuously should be handed these actions as tools by its own framework, which reaches the same runtime. ```bash -kcmd action run PlaceOrder \ +kcmd action-run PlaceOrder \ --profile default \ - --judge \ - --judge-reads-store \ --arg customer= \ --arg quantity= ``` -`--judge` is what settles the rules stated in words. Without it a guarded action is refused rather than run unchecked. `--judge-reads-store` lets that judge read the model's own tables, which a rule about something on record rather than in the arguments cannot be settled without. +That command line settles no guard. It names the rules this action states and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. ## What happens when you call one diff --git a/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts b/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts index fd22c6e2..5bef40ae 100644 --- a/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts @@ -375,7 +375,7 @@ describe('what counts as runnable is the runtime\'s answer, not a copy', () => { // `neverAsked` throws, so this passing is the assertion: a judge settles a // rule when an action runs, and listing what an agent is offered runs // none. A derivation that spent a model call per guarded action would make - // `kcmd agent tools` cost money to read. + // `kcmd agent-tools` cost money to read. const [tool] = actionTools({ runtime: rt(guardedBy(judged, 'CreditIsJustified')), judge: neverAsked diff --git a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts index 0cf29c41..08a3b841 100644 --- a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts @@ -1223,6 +1223,58 @@ describe('a guard settled by judgment', () => { expect(whyRefusedWithoutRunning(guarded, action, undefined, holds())) .toBeNull(); }); + + test('skipGuards clears the refusal for want of a judge', async () => { + // What the caller is saying is that nobody will be asked. A guarded action + // is otherwise unrunnable without a judge, and that refusal is total, so + // an author with no judge configured could not exercise their own write at + // all without deleting the guard -- which loses the guard and tests a + // different model. + const judged = guarding([justified]); + expect(whyRefusedWithoutRunning(judged, judged.actions![0])) + .toContain('no judge to ask'); + expect(whyRefusedWithoutRunning( + judged, judged.actions![0], undefined, undefined, true)) + .toBeNull(); + }); + + test('skipGuards reports no rule as unchecked', async () => { + // Without it, an advisory rule nobody could ask about is warned about -- + // the test above this one. With it, the caller has already been told, by + // itself: it named every one of these rules when it asked for them to go + // unchecked. Saying it again here, rule by rule with each judgment quoted + // back, buries what happened to the write under a list the caller wrote. + const fake = fakeStore(); + const outcome = await act({ + model: guarding([advisory]), + actionName: 'Credit', + args: {account: 'A1', amount: 100}, + client: fake.client, + skipGuards: true, + }); + if (outcome.status !== 'committed') throw new Error(outcome.message); + expect(fake.committed).toBe(true); + expect(outcome.warnings ?? []).toEqual([]); + }); + + test('skipGuards does not clear a guard that names nothing', async () => { + // Not checking the guards is not the same as not reading them. A guard + // naming a rule the model never declares is the model being wrong about + // itself -- a push refuses it on the same grounds -- and it refuses with a + // judge in hand, so standing the judge down was never what was wrong. The + // write it would apply is one the author believes is gated by something + // that does not exist, and the repair is a spelling, not a flag. + const undeclared = creditModel({ + actions: [{...credit, guards: ['NoSuchRule']}], + constraints: [], + }); + const action = undeclared.actions![0]; + expect(whyRefusedWithoutRunning(undeclared, action, undefined, holds())) + .toContain('not declared'); + expect(whyRefusedWithoutRunning( + undeclared, action, undefined, undefined, true)) + .toContain('not declared'); + }); }); @@ -1448,7 +1500,7 @@ describe('a constraint that only warns', () => { test( 'but a guard naming nothing the model declares still refuses', async () => { - // Validation makes that a hard error and `kcmd action run` now runs + // Validation makes that a hard error and `kcmd action-run` now runs // validation -- but a library caller reaching runAction directly gets // no such pass, and a guard this cannot account for is not something // to wave through on the grounds that it was not found. diff --git a/toolbox/mdcode/tests/libts/semantic/skills.test.ts b/toolbox/mdcode/tests/libts/semantic/skills.test.ts index 310a1f97..becb4e6e 100644 --- a/toolbox/mdcode/tests/libts/semantic/skills.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/skills.test.ts @@ -160,7 +160,7 @@ describe('SKILL.md frontmatter', () => { 'the description says what the model is and when to reach for it', () => { const description = fm['description'] as string; expect(description).toContain(model.description!); - // The name `kcmd action run` takes, which is the name every command + // The name `kcmd action-run` takes, which is the name every command // line in the package uses. expect(description).toContain('PlaceOrder'); expect(description.length).toBeLessThanOrEqual(1024); @@ -212,11 +212,11 @@ describe('SKILL.md is a router', () => { test('the router names the action the way the command line does', () => { // The index and the one executable instruction have to agree. Naming the - // row `place_order` while the command reads `kcmd action run PlaceOrder` + // row `place_order` while the command reads `kcmd action-run PlaceOrder` // sends an agent that routed off the table to an action the CLI rejects. const row = skill.split('\n').find(l => l.includes('references/'))!; const named = row.split('|')[1].trim().replace(/`/g, ''); - expect(skill).toContain(`kcmd action run ${named}`); + expect(skill).toContain(`kcmd action-run ${named}`); }); test('the per-argument detail is in the reference, not the router', () => { @@ -266,7 +266,7 @@ describe('SKILL.md is a router', () => { const noArgs = generate(rt(withAction(model, {...RUNNABLE, parameters: []}))); const lines = noArgs.files['SKILL.md'].split('\n'); - const start = lines.findIndex(l => l.startsWith('kcmd action run')); + const start = lines.findIndex(l => l.startsWith('kcmd action-run')); expect(start).toBeGreaterThan(-1); const end = lines.indexOf('```', start); expect(end).toBeGreaterThan(start); @@ -296,18 +296,19 @@ describe('an action reference', () => { expect(reference).toContain('`place_order`'); }); - test('a projected argument reads as the field, a declared one as itself', - () => { - // `customer` projects from `customer.c_custkey` and `quantity` states - // its own type, and the table does not say which is which -- by the - // time an agent reads this, both are one scalar to pass. What the - // projection buys is the wording: the field's datatype and the - // field's description, rather than an author restating them here and - // drifting from the column. - expect(reference).toContain( - '| `customer` | integer | yes | The customer\'s account number. |'); - expect(reference).toContain('| `quantity` | integer | yes |'); - }); + test( + 'a projected argument reads as the field, a declared one as itself', + () => { + // `customer` projects from `customer.c_custkey` and `quantity` states + // its own type, and the table does not say which is which -- by the + // time an agent reads this, both are one scalar to pass. What the + // projection buys is the wording: the field's datatype and the + // field's description, rather than an author restating them here and + // drifting from the column. + expect(reference).toContain( + '| `customer` | integer | yes | The customer\'s account number. |'); + expect(reference).toContain('| `quantity` | integer | yes |'); + }); test('carries the action\'s own guidance for a caller', () => { expect(reference).toContain( @@ -378,9 +379,12 @@ describe('when the runtime would refuse the call', () => { // The guard is settled in words, which is the runtime's job and not the // reading agent's: an agent that judged its own call would be the // constrained thing certifying itself. So the skill is written for a - // runtime that has a judge, and the command line it prints says `--judge`. + // runtime that has a judge, and carries no flag about one: the command + // line it prints is kcmd's, which settles no guard, and the skill says so + // rather than letting a commit read as a rule that held. const out = generate(rt(withAction(model, {executor: RUNNABLE.executor}))); - expect(out.files['SKILL.md']).toContain('--judge'); + expect(out.files['SKILL.md']).not.toContain('--judge'); + expect(out.files['SKILL.md']).toContain('settles no guard'); expect(out.warnings.join(' ')).not.toContain('runnable'); }); @@ -410,7 +414,7 @@ describe('when the runtime would refuse the call', () => { () => { const skill = generate(rt(model)).files['SKILL.md']; expect(skill).toContain('No action in this model can be run'); - expect(skill).not.toContain('kcmd action run'); + expect(skill).not.toContain('kcmd action-run'); }); test('a profile that binds no store says where the skill stands', () => { @@ -627,7 +631,7 @@ describe('text that would otherwise break the output', () => { const dashed = withAction(model, {...RUNNABLE, name: 'Place --Order'} as never); const skill = generate(rt(dashed)).files['SKILL.md']; - expect(skill).toContain(`kcmd action run 'Place --Order'`); + expect(skill).toContain(`kcmd action-run 'Place --Order'`); expect(skill).not.toContain('\n --Order'); }); @@ -637,7 +641,7 @@ describe('text that would otherwise break the output', () => { const spaced = withAction(model, {...RUNNABLE, name: 'Place Order'} as never); const skill = generate(rt(spaced)).files['SKILL.md']; - expect(skill).toContain(`kcmd action run 'Place Order'`); + expect(skill).toContain(`kcmd action-run 'Place Order'`); }); }); @@ -683,9 +687,10 @@ describe('a description that does not fit', () => { frontmatter(generate(rt(manyActions(60))).files['SKILL.md']) .description as string; expect(description.length).toBeLessThanOrEqual(1024); - expect(description).toContain( - 'Use when a request asks to change this data rather than only read ' + - 'it.'); + expect(description) + .toContain( + 'Use when a request asks to change this data rather than only read ' + + 'it.'); expect(description).toContain(' more.'); // The count is of what the model declares, not of what is listed, so a // partial list reads as one. @@ -711,22 +716,25 @@ describe('a description that does not fit', () => { {...model.actions![0], ...RUNNABLE, name: 'BBBBBBB'} as Action, ], }; - const description = frontmatter(generate(rt(twoNames)).files['SKILL.md']) - .description as string; + const description = + frontmatter(generate(rt(twoNames)).files['SKILL.md']).description as + string; expect(description.length).toBeLessThanOrEqual(1024); expect(description).not.toContain('and 1'); - expect(description).toContain( - 'Use when a request asks to change this data rather than only read ' + - 'it.'); + expect(description) + .toContain( + 'Use when a request asks to change this data rather than only read ' + + 'it.'); }); test('a list that fits is not abridged', () => { const description = frontmatter(generate(rt(manyActions(3))).files['SKILL.md']) .description as string; - expect(description).toContain( - 'Declares 3 actions: LongishActionName0, LongishActionName1, ' + - 'LongishActionName2.'); + expect(description) + .toContain( + 'Declares 3 actions: LongishActionName0, LongishActionName1, ' + + 'LongishActionName2.'); expect(description).not.toContain(' more.'); }); }); diff --git a/toolbox/mdcode/tests/tool/action.test.ts b/toolbox/mdcode/tests/tool/action.test.ts index cfb2372d..6add7419 100644 --- a/toolbox/mdcode/tests/tool/action.test.ts +++ b/toolbox/mdcode/tests/tool/action.test.ts @@ -1,8 +1,9 @@ -// Tests for `kcmd action` (src/tool/commands.ts, action()) -- the command in -// front of the semantic runtime. +// Tests for `kcmd action-list` and `kcmd action-run` (src/tool/commands.ts) +// -- the commands in front of the semantic runtime. // -// Almost nothing here reaches a store, and that is not a compromise: `list` -// never opens one, and every `run` covered but the last fails before the first +// Almost nothing here reaches a store, and that is not a compromise: +// `action-list` never opens one, and every `action-run` covered but the last +// fails before the first // request. The exception fakes the Spanner client's own surface, because what // it checks is the QUESTION the runtime asks the store. The // argument parse, the choice of database, and the runtime's own refusal to run @@ -20,7 +21,7 @@ import * as path from 'node:path'; import {ApiContext} from '../../src/libts/gcp/context'; import {SpannerDataClient} from '../../src/libts/gcp/spanner'; -import {action} from '../../src/tool/commands'; +import {actionList, actionRun} from '../../src/tool/commands'; const CTX = new ApiContext('test-project', 'us', 'test-token'); @@ -321,13 +322,13 @@ afterEach(() => { }); -describe('kcmd action list', () => { +describe('kcmd action-list', () => { test( 'prints each action with what it takes, what it touches, and the ' + 'command line that runs it', async () => { writeWorkspace(); - const code = await action('list', undefined); + const code = await actionList(undefined); expect(code).toBe(0); const out = logs.join('\n'); @@ -345,12 +346,11 @@ describe('kcmd action list', () => { expect(out).toContain('affects: Entry (create)'); // The point of the listing: the reader can copy this and run it. - // IssueCredit names a guard, and a guard is settled by asking, so the - // line carries the flags that supply a judge -- see the judge-flag - // test below for why both of them. + // Nothing but the arguments goes on the line: this command settles no + // guard, so there is no flag about guards to offer. expect(out).toContain( - 'run: kcmd action run IssueCredit --judge ' + - '--judge-reads-store --arg order= --arg amount='); + 'run: kcmd action-run IssueCredit ' + + '--arg order= --arg amount='); // An action with no description, guards or blast radius shows only // what it declares. It is executed by MCP, which this command holds no @@ -360,7 +360,7 @@ describe('kcmd action list', () => { expect(out).toContain('executor: mcp'); expect(out).toContain('NOT RUNNABLE:'); expect(out).toContain('is executed by MCP, which runs'); - expect(out).not.toContain('kcmd action run NotifyCustomer'); + expect(out).not.toContain('kcmd action-run NotifyCustomer'); }); test( @@ -377,7 +377,7 @@ describe('kcmd action list', () => { ' - {name: literalNull, type: String, default: "null"}\n' + ' - {name: memo, type: String, required: false}'); writeWorkspace(optionalModel); - const code = await action('list', undefined); + const code = await actionList(undefined); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain( @@ -386,8 +386,8 @@ describe('kcmd action list', () => { 'cleared (Boolean, default: null), literalNull (String, default: "null"), ' + 'memo (String, optional)'); expect(out).toContain( - 'run: kcmd action run IssueCredit --judge ' + - '--judge-reads-store --arg order= --arg amount='); + 'run: kcmd action-run IssueCredit ' + + '--arg order= --arg amount='); expect(out).not.toContain('--arg currency='); expect(out).not.toContain('--arg memo='); }); @@ -399,7 +399,7 @@ describe('kcmd action list', () => { // line for a write this binding cannot perform would send the reader // to a refusal, so it prints the fix instead. writeWorkspace(LOGICAL); - const code = await action('list', undefined, {profile: 'readonly'}); + const code = await actionList(undefined, {profile: 'readonly'}); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain('IssueCredit'); @@ -410,7 +410,7 @@ describe('kcmd action list', () => { 'NOT RUNNABLE: Action \'IssueCredit\' has no executor under this ' + 'binding'); expect(out).toContain('supplies one, and a profile that writes'); - expect(out).not.toContain('kcmd action run IssueCredit'); + expect(out).not.toContain('kcmd action-run IssueCredit'); }); test( @@ -425,51 +425,43 @@ describe('kcmd action list', () => { // out from the executor. writeWorkspace(MODEL.replace( 'guards: [CreditIsPositive]', 'guards: [NoSuchRule]')); - const code = await action('list', undefined); + const code = await actionList(undefined); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain('executor: sql'); expect(out).toContain('NOT RUNNABLE:'); expect(out).toContain('\'NoSuchRule\''); expect(out).toContain('declared by model \'commerce\''); - expect(out).not.toContain('kcmd action run IssueCredit'); + expect(out).not.toContain('kcmd action-run IssueCredit'); }); - test( - 'the run line names both judge flags when a guard is judged', - async () => { - // Copying the line is the whole point of printing it. A judged guard - // refuses without a judge, so a line omitting the flag would send the - // reader to a refusal it could have predicted. The same holds one step - // on: a judgment comparing the call against a stored row is refused - // without `--judge-reads-store`, and a constraint's wording does not - // say which judgments those are, so the line offers the read wherever - // the profile binds a table to read. - writeWorkspace(); - const code = await action('list', undefined); - expect(code).toBe(0); - const out = logs.join('\n'); - expect(out).toContain( - 'run: kcmd action run IssueCredit --judge ' + - '--judge-reads-store --arg order='); - // And nowhere else. The other action names no guard, so it must gain - // neither flag; it prints no run line at all here -- MCP is not - // runnable from this command -- so counting is what is left to check - // that the flags are attached to the guard rather than to the listing. - expect(out.match(/--judge(?!-)/g)?.length).toBe(1); - expect(out.match(/--judge-reads-store/g)?.length).toBe(1); - }); + test('the run line offers no flag for a judged guard', async () => { + // Copying the line is the whole point of printing it, so it must not + // suggest a flag this command line does not have. A guard is settled by + // asking somebody, and who that is belongs to whoever dispatches the call + // in earnest -- so no flag here offers it, and the listing says the guard + // is there without pretending it can be checked. + writeWorkspace(); + const code = await actionList(undefined); + expect(code).toBe(0); + const out = logs.join('\n'); + expect(out).toContain('guards: CreditIsPositive'); + expect(out).toContain( + 'run: kcmd action-run IssueCredit --arg order='); + expect(out).not.toContain('--judge'); + expect(out).not.toContain('--skip-guards'); + }); test('says so when a model declares no actions', async () => { writeWorkspace(NO_ACTIONS); - const code = await action('list', undefined); + const code = await actionList(undefined); expect(code).toBe(0); expect(logs.join('\n')).toContain('declares no actions.'); }); test('reads the model under a named profile', async () => { writeWorkspace(LOGICAL); - const code = await action('list', undefined, {profile: 'analytical'}); + const code = await actionList(undefined, {profile: 'analytical'}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'analytical\''); }); @@ -478,26 +470,19 @@ describe('kcmd action list', () => { 'names the profiles that exist when given one that does not', async () => { writeWorkspace(); - const code = await action('list', undefined, {profile: 'nope'}); + const code = await actionList(undefined, {profile: 'nope'}); expect(code).toBe(1); const out = logs.join('\n'); expect(out).toContain('unknown binding profile \'nope\''); expect(out).toContain('analytical'); }); - - test('rejects a subcommand that is neither list nor run', async () => { - writeWorkspace(); - const code = await action('explain', 'IssueCredit'); - expect(code).toBe(1); - expect(logs.join('\n')).toContain('expected \'list\' or \'run\''); - }); }); -describe('kcmd action run: what it will not send to a store', () => { +describe('kcmd action-run: what it will not send to a store', () => { test('needs an action name', async () => { writeWorkspace(); - const code = await action('run', undefined); + const code = await actionRun(undefined); expect(code).toBe(1); expect(logs.join('\n')).toContain('needs an action name'); }); @@ -506,7 +491,7 @@ describe('kcmd action run: what it will not send to a store', () => { 'names the declared actions when asked for one that is not there', async () => { writeWorkspace(); - const code = await action('run', 'IssueRefund'); + const code = await actionRun('IssueRefund'); expect(code).toBe(1); const out = logs.join('\n'); expect(out).toContain('declares an action \'IssueRefund\''); @@ -516,7 +501,7 @@ describe('kcmd action run: what it will not send to a store', () => { test('rejects an --arg that does not name a parameter', async () => { writeWorkspace(); const code = - await action('run', 'IssueCredit', {arg: ['order=12345', 'amount']}); + await actionRun('IssueCredit', {arg: ['order=12345', 'amount']}); expect(code).toBe(1); expect(logs.join('\n')) .toContain('--arg expects =, but got \'amount\''); @@ -525,7 +510,7 @@ describe('kcmd action run: what it will not send to a store', () => { test('rejects the same parameter given twice', async () => { writeWorkspace(); const code = - await action('run', 'IssueCredit', {arg: ['amount=30', 'amount=40']}); + await actionRun('IssueCredit', {arg: ['amount=30', 'amount=40']}); expect(code).toBe(1); expect(logs.join('\n')).toContain('--arg amount was given twice.'); }); @@ -535,45 +520,57 @@ describe('kcmd action run: what it will not send to a store', () => { async () => { writeWorkspace(); // Reaches the runtime rather than the argument parser: the refusal - // below is about the guard, which is proof the parse succeeded. - const code = await action('run', 'IssueCredit', {arg: 'amount=30'}); + // below names the parameter that was NOT given, which only something + // holding the parsed pair could report. + const code = await actionRun('IssueCredit', {arg: 'amount=30'}); expect(code).toBe(1); - expect(logs.join('\n')) - .toContain('this runtime was given no judge to ask'); + expect(logs.join('\n')).toContain('order'); }); test( 'refuses an action whose executor runs outside the transaction', async () => { writeWorkspace(); - const code = await action('run', 'NotifyCustomer', {arg: 'order=1'}); + const code = await actionRun('NotifyCustomer', {arg: 'order=1'}); expect(code).toBe(1); expect(logs.join('\n')) .toContain('which runs outside this transaction'); }); - test('refuses a guarded action when no judge was supplied', async () => { - writeWorkspace(); - const code = - await action('run', 'IssueCredit', {arg: ['order=12345', 'amount=30']}); - expect(code).toBe(1); - const out = logs.join('\n'); - expect(out).toContain('is guarded by \'CreditIsPositive\''); - // It got as far as choosing a database, so the refusal is the - // runtime's and not a wiring failure earlier on. - expect(out).toContain( + test('names the guards it is not going to check', async () => { + // The command checks no guard, and the one thing it must not do is let + // that pass unremarked: a reader watching a write land is owed the list of + // rules that did not stand between them and it, by name, before it lands. + writeWorkspace(); + await actionRun('IssueCredit', {arg: ['order=12345', 'amount=30']}); + const out = logs.join('\n'); + expect(out).toContain('NOT CHECKED: CreditIsPositive'); + expect(out).toContain('this command settles no guard'); + // Printed before the run banner's database line is reached, so it is read + // while the run is still a run. + expect(out.indexOf('NOT CHECKED')) + .toBeGreaterThan(out.indexOf('Running \'IssueCredit\'')); + expect(out).toContain( 'Running \'IssueCredit\' on projects/acme-ops/instances/prod/databases/commerce'); - }); + }); + + test('says nothing about guards for an action that declares none', async () => { + // An action with no guards skipped no check, so a line saying one went + // unchecked would be false -- and a caveat printed on every run is a + // caveat nobody reads on the run that needed it. + writeWorkspace(INHERITS); + await actionRun('Touch', {arg: 'who=Alice'}); + expect(logs.join('\n')).not.toContain('NOT CHECKED'); + }); }); -describe('kcmd action run: an action this binding cannot perform', () => { +describe('kcmd action-run: an action this binding cannot perform', () => { test('refuses an action whose executor the profile withdrew', async () => { // Nothing is wrong with the action. The binding is what says no, so the // message has to send the reader to the profile rather than to the model. writeWorkspace(LOGICAL); - const code = await action( - 'run', 'IssueCredit', + const code = await actionRun('IssueCredit', {profile: 'readonly', arg: ['order=1', 'amount=5']}); expect(code).toBe(1); const out = logs.join('\n'); @@ -583,11 +580,10 @@ describe('kcmd action run: an action this binding cannot perform', () => { }); -describe('kcmd action run: where the write would go', () => { +describe('kcmd action-run: where the write would go', () => { test('refuses a profile that deploys to BigQuery', async () => { writeWorkspace(LOGICAL); - const code = await action( - 'run', 'IssueCredit', + const code = await actionRun('IssueCredit', {profile: 'analytical', arg: ['order=12345', 'amount=30']}); expect(code).toBe(1); const out = logs.join('\n'); @@ -605,8 +601,7 @@ describe('kcmd action run: where the write would go', () => { 'its deployment target', async () => { writeWorkspace(LOGICAL); - const code = await action( - 'run', 'IssueCredit', + const code = await actionRun('IssueCredit', {profile: 'mismatched', arg: ['order=12345', 'amount=30']}); expect(code).toBe(1); const out = logs.join('\n'); @@ -624,8 +619,7 @@ describe('kcmd action run: where the write would go', () => { // Spanner table shares the name while the data the model describes // sat in BigQuery, untouched and unmentioned. writeWorkspace(LOGICAL); - const code = await action( - 'run', 'IssueCredit', + const code = await actionRun('IssueCredit', {profile: 'crossbound', arg: ['order=12345', 'amount=30']}); expect(code).toBe(1); const out = logs.join('\n'); @@ -640,7 +634,7 @@ describe('kcmd action run: where the write would go', () => { // A scope holds every document under the entry group, and `run` touches one of // them. An error in a document this call will not read is a real error to fix, // and refusing on it would report a model the reader did not name. -describe('kcmd action run: which model has to be valid', () => { +describe('kcmd action-run: which model has to be valid', () => { function withWarehouse(): void { writeWorkspace(); fs.writeFileSync( @@ -653,16 +647,14 @@ describe('kcmd action run: which model has to be valid', () => { 'a broken document elsewhere in the scope does not block the run', async () => { withWarehouse(); - const code = await action( - 'run', 'IssueCredit', {arg: ['order=12345', 'amount=30']}); - // Still refused -- IssueCredit is guarded and nothing evaluates a - // guard yet -- but refused on its OWN terms. + const code = await actionRun('IssueCredit', {arg: ['order=12345', 'amount=30']}); + // Refused, because nothing here stands up a Spanner client -- but + // refused on its OWN terms, not the other document's. expect(code).toBe(1); // The broken document is still WARNED about -- it is a real problem, // reported where it is. What must not happen is it becoming the // reason this call failed. const errors = logs.filter(l => l.startsWith('Error:')).join('\n'); - expect(errors).toContain('is guarded by \'CreditIsPositive\''); expect(errors).not.toContain('Pallet'); expect(errors).not.toContain('warehouse'); }); @@ -671,7 +663,7 @@ describe('kcmd action run: which model has to be valid', () => { 'the broken document is still refused when it is the one being run', async () => { withWarehouse(); - const code = await action('run', 'Restock', {arg: ['bin=B1']}); + const code = await actionRun('Restock', {arg: ['bin=B1']}); expect(code).toBe(1); const errors = logs.filter(l => l.startsWith('Error:')).join('\n'); expect(errors).toContain('Pallet'); @@ -682,7 +674,7 @@ describe('kcmd action run: which model has to be valid', () => { // cac and mri hand back values a flag's name does not suggest, and the shell // hands back names an object literal already has. Both look like a nuisance // and both change which model runs, or whether the run happens at all. -describe('kcmd action: what the command line can actually contain', () => { +describe('kcmd action-list/action-run: what the command line can hold', () => { test( 'a bare --profile falls back to the default rather than looking up ' + 'a profile called \'true\'', @@ -690,7 +682,7 @@ describe('kcmd action: what the command line can actually contain', () => { // cac yields `true` for `--profile` with no value. Reading it as a // name would fail the command with a profile the user never typed. writeWorkspace(); - const code = await action('list', undefined, {profile: true}); + const code = await actionList(undefined, {profile: true}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'default\''); }); @@ -698,14 +690,14 @@ describe('kcmd action: what the command line can actually contain', () => { test('--no-profile does not become a profile name either', async () => { // mri yields `false`, which `??` would pass straight through. writeWorkspace(); - const code = await action('list', undefined, {profile: false}); + const code = await actionList(undefined, {profile: false}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'default\''); }); test('a named profile still selects that profile', async () => { writeWorkspace(LOGICAL); - const code = await action('list', undefined, {profile: 'analytical'}); + const code = await actionList(undefined, {profile: 'analytical'}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'analytical\''); }); @@ -716,18 +708,17 @@ describe('kcmd action: what the command line can actually contain', () => { // On a plain object `'toString' in args` is true before anything is // parsed, so this would report a duplicate the caller never gave. writeWorkspace(); - await action( - 'run', 'IssueCredit', {arg: ['toString=x', 'order=1', 'amount=5']}); + await actionRun( + 'IssueCredit', {arg: ['toString=x', 'order=1', 'amount=5']}); const out = logs.join('\n'); expect(out).not.toContain('given twice'); - // It gets as far as the refusal, which is where this model stops. - expect(out).toContain('CreditIsPositive'); + // It gets as far as the run, which is proof the parse let it through. + expect(out).toContain('Running \'IssueCredit\''); }); test('a genuinely repeated argument is still reported', async () => { writeWorkspace(); - const code = await action( - 'run', 'IssueCredit', {arg: ['order=1', 'order=2', 'amount=5']}); + const code = await actionRun('IssueCredit', {arg: ['order=1', 'order=2', 'amount=5']}); expect(code).toBe(1); expect(logs.join('\n')).toContain('--arg order was given twice'); }); @@ -736,20 +727,20 @@ describe('kcmd action: what the command line can actually contain', () => { // `run` skips the deployment checks on purpose -- it deploys nothing -- but // not the ones the runtime's refusal gate depends on. -describe('kcmd action run: --arg has to be a pair', () => { +describe('kcmd action-run: --arg has to be a pair', () => { test('a bare value is reported rather than crashing the parse', async () => { // cac does not hand back a string for every `--arg`: it coerces a bare // numeric value, so `--arg amount 30` arrives here as the NUMBER 30. Left // as it came, `pair.indexOf` threw a TypeError past the parser and the // message written for exactly this typo was unreachable. writeWorkspace(); - expect(await action('run', 'IssueCredit', {arg: 30 as any})).toBe(1); + expect(await actionRun('IssueCredit', {arg: 30 as any})).toBe(1); expect(logs.join('\n')).toContain('--arg expects ='); }); }); -describe('kcmd action run: a subtype inherits its fields', () => { +describe('kcmd action-run: a subtype inherits its fields', () => { test('projects a parameter from an inherited field', async () => { // Both push legs resolve inheritance and this path did not, so a subtype // arrived at the runtime with only the fields it declares itself. @@ -778,7 +769,7 @@ describe('kcmd action run: a subtype inherits its fields', () => { return ok({rows: []}); }); - expect(await action('run', 'Touch', {arg: 'who=Alice'})).toBe(0); + expect(await actionRun('Touch', {arg: 'who=Alice'})).toBe(0); expect(asked).toHaveLength(1); expect(asked[0].sql).toContain('CustomerId = @who'); expect(asked[0].params.who).toBe('Alice'); @@ -787,7 +778,64 @@ describe('kcmd action run: a subtype inherits its fields', () => { }); -describe('kcmd action run: the model has to be valid to run', () => { +describe('kcmd action-run: the guards go unchecked', () => { + test('runs a guarded action rather than refusing it', async () => { + // An author trying their own model against their own database has no judge + // to stand up, and a guard refusal is total -- so a command that insisted + // on one would leave them deleting the guard to test the write, which + // loses the guard and tests a different model. Checking guards belongs to + // whoever dispatches the call in earnest; this command is for seeing + // whether the statements do what the author meant. + writeWorkspace(); + const asked: string[] = []; + const ok = (result: unknown) => + Promise.resolve({status: 200, result} as any); + spyOn(SpannerDataClient.prototype, 'createSession') + .mockImplementation(() => ok({name: 'sessions/1'})); + spyOn(SpannerDataClient.prototype, 'deleteSession') + .mockImplementation(() => ok({})); + spyOn(SpannerDataClient.prototype, 'beginReadWrite') + .mockImplementation(() => ok({id: 'txn-1'})); + spyOn(SpannerDataClient.prototype, 'rollback') + .mockImplementation(() => ok({})); + spyOn(SpannerDataClient.prototype, 'commit') + .mockImplementation( + () => ok({commitTimestamp: '2026-09-20T00:00:00Z'})); + spyOn(SpannerDataClient.prototype, 'executeSql') + .mockImplementation((_s: any, _t: any, stmt: any) => { + asked.push(stmt.sql); + return ok({rows: []}); + }); + + const code = + await actionRun('IssueCredit', {arg: ['order=12345', 'amount=30']}); + const out = logs.join('\n'); + // Not stopped by the guard, and it reached the store. + expect(code).toBe(0); + expect(out).not.toContain('is guarded by \'CreditIsPositive\''); + expect(asked.length).toBeGreaterThan(0); + expect(out).toContain('NOT CHECKED: CreditIsPositive'); + }); + + test( + 'still refuses a guard the model never declares, which no judge would ' + + 'have fixed', + async () => { + // Not checking the guards is not the same as not reading them. A guard + // naming nothing is the model being wrong about its own rules -- a + // push refuses it too -- and running it anyway would apply a write the + // author believes is gated by something that does not exist. + writeWorkspace(MODEL.replace( + 'guards: [CreditIsPositive]', 'guards: [NoSuchRule]')); + const code = + await actionRun('IssueCredit', {arg: ['order=1', 'amount=5']}); + expect(code).toBe(1); + expect(logs.join('\n')).toContain('\'NoSuchRule\''); + }); +}); + + +describe('kcmd action-run: the model has to be valid to run', () => { const TYPO = MODEL.replace( '- {concept: Entry, operation: create}', '- {concept: Etnry, operation: create}'); @@ -801,13 +849,13 @@ describe('kcmd action run: the model has to be valid to run', () => { // over an entry that resolves to nothing. writeWorkspace(TYPO); const code = - await action('run', 'IssueCredit', {arg: ['order=A1', 'amount=5']}); + await actionRun('IssueCredit', {arg: ['order=A1', 'amount=5']}); expect(code).toBe(1); expect(logs.join('\n')).toContain('\'Etnry\''); }); test('but listing it still works, because listing runs nothing', async () => { writeWorkspace(TYPO); - expect(await action('list', undefined)).toBe(0); + expect(await actionList(undefined)).toBe(0); }); }); diff --git a/toolbox/mdcode/tests/tool/main_cli.test.ts b/toolbox/mdcode/tests/tool/main_cli.test.ts index 799ceb5b..0722ce58 100644 --- a/toolbox/mdcode/tests/tool/main_cli.test.ts +++ b/toolbox/mdcode/tests/tool/main_cli.test.ts @@ -19,7 +19,8 @@ const MAIN = path.join(process.cwd(), 'src', 'tool', 'main.ts'); let cwd: string; beforeAll(() => { - if (!fs.existsSync(MAIN)) throw new Error(`cannot find CLI entrypoint ${MAIN}`); + if (!fs.existsSync(MAIN)) + throw new Error(`cannot find CLI entrypoint ${MAIN}`); cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'kcmd-cli-')); }); @@ -106,12 +107,12 @@ describe('kcmd: --help and --version', () => { }); test('`--help` past a command that takes arguments still succeeds', () => { - // `action [name]` has its own name taken out of `cli.args` -- - // which arrives holding `list`, a word that names no command -- so - // `process.argv` is the only place the verb survives to be checked. - const {code, out} = run('action', 'list', '--help'); + // `action-list [name]` has its own name taken out of `cli.args`, which + // therefore arrives empty here, so `process.argv` is the only place the + // verb survives to be checked. + const {code, out} = run('action-list', '--help'); expect(code).toBe(0); - expect(out).toContain('kcmd action'); + expect(out).toContain('kcmd action-list'); }); test('`--help` before an unknown verb is still an error', () => { @@ -124,7 +125,7 @@ describe('kcmd: --help and --version', () => { }); test('`--help` before a known verb still succeeds', () => { - const {code, out} = run('--help', 'action'); + const {code, out} = run('--help', 'action-list'); expect(code).toBe(0); expect(out).not.toContain('Unknown command'); }); From ea74c23640e55f495a4ffd3eb82bfd8b7fe40a17 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Sun, 20 Sep 2026 22:25:27 +0000 Subject: [PATCH 02/11] feat(mdcode)!: move the store read-back to `profiles --print-store` `action-list --store` printed where a run would land and exited, suppressing the listing entirely: a mode flag on a listing command, wearing the name of a target flag. Nothing on any kcmd command line selects a store -- `--profile` picks a binding and the binding's deployment target decides where writes land -- so a flag called `--store` that takes no value and only prints is one a reader has to try before they can tell which direction it goes. It moves to the command that owns bindings, under a name that says so: `kcmd profiles --print-store`. The output is byte-identical, so the demo's setup script changes only the command it calls. `profiles` also gains `--profile [name]`. It narrows the report to one profile and is what picks the store `--print-store` prints; without it the move would drop what `action-list --store --profile alloydb` could already do. Naming a profile the model does not declare exits 1, because an empty report reads as "this profile withholds nothing", which is the opposite of what a typo means. `action-list` is left with one flag. --- .../mdcode/docs/semantic-model/reference.md | 29 ++++- .../src/libts/semantic/runtime/store.ts | 6 +- toolbox/mdcode/src/tool/commands.ts | 110 +++++++++++++----- toolbox/mdcode/src/tool/main.ts | 13 ++- toolbox/mdcode/tests/tool/profiles.test.ts | 54 +++++++++ 5 files changed, 171 insertions(+), 41 deletions(-) diff --git a/toolbox/mdcode/docs/semantic-model/reference.md b/toolbox/mdcode/docs/semantic-model/reference.md index 629bbcc9..4b90bc54 100644 --- a/toolbox/mdcode/docs/semantic-model/reference.md +++ b/toolbox/mdcode/docs/semantic-model/reference.md @@ -63,6 +63,31 @@ scope you authored under. See [Pull](README.md#pull) for behavior. | `--dry-run` | Reconstruct from the catalog and report what would be written, but write no files. | | `--force-remove` | Replace a differently-named local model with the catalog's (see [Pull](README.md#pull)); without it, a pull that would leave the entry group holding two models fails. | +### profiles + +```bash +kcmd profiles +``` + +Reports each binding profile the model declares: its deployment target, the +source each entity binds to, and what the profile cannot answer or cannot run. +Read-only — it merges and prunes each profile the way `push` does, but deploys +nothing and runs no live probe, so you can compare coverage before choosing one. + +| Flag | Effect | +|------|--------| +| `--profile [name]` | Report only this profile. Naming one the model does not declare is an error, not an empty report. Defaults to every profile. | +| `--print-store` | Print only the store the profile deploys to, on one line and nothing else, for a script to read rather than parse out of the report: `project/instance/database` for a Spanner store, and the backend named ahead of the path for any other (`alloydb:project/region/cluster/instance/database`, `bigquery:project/dataset`). Errors when the scope holds more than one model, since those may name different databases. | + +This is a read of the binding, never a choice of one. Nothing on any `kcmd` +command line names a store directly; `--profile` selects a binding and the +binding's deployment target decides where writes land. A script that creates, +seeds or drops that database asks for the name rather than repeating it: + +```bash +IFS=/ read -r PROJECT INSTANCE DATABASE <<<"$(kcmd profiles --print-store)" +``` + ### action-list ```bash @@ -76,7 +101,9 @@ model. See [Run it](actions.md#7-run-it). | Flag | Effect | |------|--------| | `--profile [name]` | Read the model under this binding profile. Its deployment target names the database the action runs against, so this is how you change stores. Defaults to `default_profile`, else the model's inline bindings. | -| `--store` | Print only where a run would land, on one line and nothing else, for a script to read rather than parse back out of the listing: `project/instance/database` for a Spanner store, `bigquery:project/dataset` for a BigQuery one. Errors when the scope holds more than one model, since those may name different databases. | + +To read back the store a profile deploys to, ask the binding rather than the +listing: [`profiles --print-store`](#profiles). ### action-run diff --git a/toolbox/mdcode/src/libts/semantic/runtime/store.ts b/toolbox/mdcode/src/libts/semantic/runtime/store.ts index 67a7bb1a..7488329f 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/store.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/store.ts @@ -342,9 +342,9 @@ export async function closeStore(store: Store): Promise { * * Here rather than in whichever caller needed it first, because more than one * now answers "where would this land" -- the listing a person reads, the - * `--store` line a script reads, and the section a generated skill writes into - * a file that outlives the run. Two of those disagreeing is a reader sent to - * the wrong database. + * `--print-store` line a script reads, and the section a generated skill writes + * into a file that outlives the run. Two of those disagreeing is a reader sent + * to the wrong database. */ export function storeLine(store: Store): string { switch (store.kind) { diff --git a/toolbox/mdcode/src/tool/commands.ts b/toolbox/mdcode/src/tool/commands.ts index 48438cc5..6713d9d4 100644 --- a/toolbox/mdcode/src/tool/commands.ts +++ b/toolbox/mdcode/src/tool/commands.ts @@ -711,12 +711,73 @@ export async function push(options: PushOptions): Promise { } +export interface ProfilesOptions { + // `string|boolean` for the same reason push's is: cac yields `true` for a + // bare `--profile` and `false` for `--no-profile`. + profile?: string|boolean; + // `--print-store`: print the store this profile deploys to and nothing else. + printStore?: boolean; +} + + +// Prints the store one profile deploys to, on a line with nothing else on it. +// +// kcmd profiles --print-store [--profile ] +// +// A script that creates, seeds or drops the database an action writes to has +// to address the database the action writes to, and the profile's deployment +// target is what decides that; naming it a second time in the script is a +// second place to name it differently. So the script asks. The answer is a +// read of the binding, never a choice of one -- `--profile` is the only thing +// that picks, here as everywhere else. +// +// Returns a process exit code (0 on success). +async function printStore(options: ProfilesOptions): Promise { + const ctx = context.ApiContext.default(); + const named = + typeof options.profile === 'string' ? options.profile : undefined; + + const opened = await createSemanticRuntimes({profile: named, ctx}); + if ('error' in opened) { + console.error(`Error: ${opened.error}`); + return 1; + } + + // One line, because the caller is `STORE=$(kcmd profiles --print-store)` and + // a second line makes that variable address the wrong database. A scope + // holding several models has no single answer, so it says so instead. + if (opened.length > 1) { + console.error( + `Error: this scope holds ${opened.length} models, which may name ` + + `different databases, so --print-store has no single answer. Narrow ` + + `the scope to one model.`); + return 1; + } + for (const {store, storeError} of opened) { + if (!store) { + console.error(`Error: ${storeError}`); + return 1; + } + console.log(storeLine(store)); + } + return 0; +} + + // Lists a semantic model's binding profiles and, per profile, its resolved // deployment target and sources plus what it cannot answer (the availability // report). Read-only: it merges and prunes each profile the way push does, but // deploys nothing and runs no live probe, so a user can see coverage before // choosing a profile. Returns a process exit code (0 on success). -export async function profiles(): Promise { +export async function profiles(options: ProfilesOptions = {}): Promise { + if (options.printStore) return await printStore(options); + + // `--profile` narrows the report to one profile; anything that is not a name + // (a bare `--profile`, `--no-profile`) narrows nothing, the same read every + // other command does. + const only = + typeof options.profile === 'string' ? options.profile : undefined; + const ctx = context.ApiContext.default(); const snapshot = await kcmd.CatalogSnapshot.fromPath('.', ctx); if (snapshot.manifest.source.type !== Sources.SEMANTIC_MODEL) { @@ -734,15 +795,28 @@ export async function profiles(): Promise { return 0; } + // Set when `--profile` names something no model declares, so a typo exits + // non-zero rather than reporting an empty scope as a clean one. + let missing = false; + for (const doc of docs) { console.log(`Model '${doc.name}' (${source.entryGroup}):`); - const available = layout.profileDocuments(doc.name); - if (!available.length) { + const declared = layout.profileDocuments(doc.name); + if (!declared.length) { console.log( ` no binding profiles; the model document is its own inline ` + `'default' binding.`); continue; } + const available = only ? declared.filter(p => p.name === only) : declared; + // Naming a profile the model does not declare is a typo, not an empty + // report: saying nothing would read as "this profile withholds nothing". + if (only && !available.length) { + console.error(` no profile '${only}'; this model declares ${ + declared.map(p => `'${p.name}'`).join(', ')}.`); + missing = true; + continue; + } for (const {name, text} of available) { const res = mergeProfileOntoDoc(doc.text, text, name); if ('error' in res) { @@ -806,7 +880,7 @@ export async function profiles(): Promise { } } } - return 0; + return missing ? 1 : 0; } @@ -1181,8 +1255,6 @@ export interface ActionOptions { // `string|boolean` for the same reason push's is: cac yields `true` for a // bare `--profile` and `false` for `--no-profile`. profile?: string|boolean; - // `--store`: print where a run would land and nothing else. - store?: boolean; } @@ -1253,32 +1325,6 @@ const RUN_INDENT = ' '; // is waiting on instead. function listActions( runtimes: SemanticRuntime[], options: ActionOptions): number { - // `--store` answers one question -- where would a run land -- on one line - // with nothing else on it, so a script can read it. Creating, seeding and - // dropping the database an action writes to has to address the database the - // action writes to, and the profile's deployment target is what decides - // that; a second place to say it is a second place to say it differently. - if (options.store) { - // One line, because the caller is `STORE=$(kcmd action-list --store)` and - // a second line makes that variable address the wrong database. A scope - // holding several models has no single answer, so it says so instead. - if (runtimes.length > 1) { - console.error( - `Error: this scope holds ${runtimes.length} models, which may ` + - `name different databases, so --store has no single answer. Narrow ` + - `the scope to one model.`); - return 1; - } - for (const {store, storeError} of runtimes) { - if (!store) { - console.error(`Error: ${storeError}`); - return 1; - } - console.log(storeLine(store)); - } - return 0; - } - for (const runtime of runtimes) { const {model, store, storeError, profile, entryGroup} = runtime; console.log(`Model '${model.name}' (${entryGroup}), profile '${profile}':`); diff --git a/toolbox/mdcode/src/tool/main.ts b/toolbox/mdcode/src/tool/main.ts index 74101b00..bb62995b 100644 --- a/toolbox/mdcode/src/tool/main.ts +++ b/toolbox/mdcode/src/tool/main.ts @@ -97,10 +97,16 @@ cli.command('push', 'Push catalog entries') cli.command( 'profiles', 'List a semantic model\'s binding profiles and what each can answer') - .action(async () => { + .option( + '--profile [name]', + 'Report only this binding profile; defaults to every profile the model declares') + .option( + '--print-store', + 'Print only the store the profile deploys to, on one line and nothing else, for a script to read: project/instance/database for Spanner, and the backend named ahead of the path for any other store') + .action(async (options) => { let exitCode = 1; try { - exitCode = await commands.profiles(); + exitCode = await commands.profiles(options); } catch (err: any) { console.error('Error:', err.message || err); exitCode = 1; @@ -138,9 +144,6 @@ cli.command( .option( '--profile [name]', 'Read the model under this binding profile; its deployment target names the database the action runs against; defaults to default_profile, else the inline bindings') - .option( - '--store', - 'Print only where a run would land: project/instance/database for Spanner, and the backend named ahead of the path for any other store') .action(async (name, options) => { let exitCode = 1; try { diff --git a/toolbox/mdcode/tests/tool/profiles.test.ts b/toolbox/mdcode/tests/tool/profiles.test.ts index 0f353c01..259c4b9a 100644 --- a/toolbox/mdcode/tests/tool/profiles.test.ts +++ b/toolbox/mdcode/tests/tool/profiles.test.ts @@ -143,6 +143,60 @@ describe('kcmd profiles', () => { expect(code).toBe(0); expect(logs.join('\n')).not.toContain('(default)'); }); + + test('`--profile` reports only the profile it names', async () => { + writeWorkspace('analytical'); + const code = await profiles({profile: 'operational'}); + expect(code).toBe(0); + const out = logs.join('\n'); + expect(out).toContain("profile 'operational'"); + expect(out).not.toContain("profile 'analytical'"); + }); + + // An empty report would read as "this profile withholds nothing", which is + // the opposite of what a misspelled name means. + test('`--profile` naming an undeclared profile is an error', async () => { + writeWorkspace('analytical'); + const code = await profiles({profile: 'operatonal'}); + expect(code).toBe(1); + const out = logs.join('\n'); + expect(out).toContain("no profile 'operatonal'"); + expect(out).toContain("'analytical', 'operational'"); + }); + + // A bare `--profile` reaches cac as `true` and `--no-profile` as `false`. + // Neither names a profile, so neither may narrow the report -- filtering on + // one would report zero profiles for a flag the caller left blank. + for (const profile of [true, false]) { + test(`\`--profile ${profile}\` narrows nothing`, async () => { + writeWorkspace('analytical'); + const code = await profiles({profile}); + expect(code).toBe(0); + const out = logs.join('\n'); + expect(out).toContain("profile 'analytical'"); + expect(out).toContain("profile 'operational'"); + }); + } +}); + + +// `--print-store` is a read of the binding, never a choice of one: it prints +// where the selected profile deploys to and nothing else, so a setup script can +// address the same database the actions write to instead of naming it twice. +describe('kcmd profiles --print-store', () => { + test('prints the selected profile\'s store on one line', async () => { + writeWorkspace('operational'); + const code = await profiles({printStore: true}); + expect(code).toBe(0); + expect(logs).toEqual(['acme-ops/prod/commerce']); + }); + + test('`--profile` is what picks which store is printed', async () => { + writeWorkspace('operational'); + const code = await profiles({printStore: true, profile: 'analytical'}); + expect(code).toBe(0); + expect(logs).toEqual(['bigquery:acme-analytics/sales']); + }); }); From 2c65c35d9cb58e9180d8bd7c5d40e90365709d68 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 01:17:59 +0000 Subject: [PATCH 03/11] fix(mdcode): re-export readableEntities for the skills generator The trim unexported it, on the grounds that letting a model compose queries against a caller's data is a decision for whoever embeds the runtime rather than something the curation CLI offers. That reasoning still holds for the judge, but #451's skills generator calls the same function for a different purpose: naming the tables a generated skill tells an agent it may read. The two PRs were in flight together, so the textual merge was clean and only the type checker caught it. --- toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts b/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts index e49f9edd..125cf738 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/judge_store.ts @@ -201,7 +201,7 @@ interface ReadableEntity { * this leaves out is one the judge is never told about, so a rule that turns * on it is one the judge reports it cannot settle. */ -function readableEntities( +export function readableEntities( runtime: SemanticRuntime, dialect: SqlDialect): ReadableEntity[] { const readable: ReadableEntity[] = []; for (const entity of runtime.model.entities ?? []) { From 713fd44627cddfdd21ebcef13a289743e16f58de Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 01:27:39 +0000 Subject: [PATCH 04/11] docs(mdcode): the skill demo stops documenting the judge command line #453 shipped this demo hours before the trim removed the flags it runs on. Twelve command lines and nineteen judge flags in one README, and two sections whose entire subject was a flag pair that no longer exists. Section 4 is rewritten around what the command line actually does now, with two live runs: a credit that commits under the NOT CHECKED banner, and the same command asked for twenty dollars against an eighteen dollar order -- the case CreditWithinOrderTotal exists to stop -- committing and leaving the order at a total of negative five dollars. That is the cost of the trim, shown rather than asserted, and it is a better argument for where guards belong than the prose it replaces. The guarded transcripts are kept and marked. They are the only record of one action carrying four rules through all three on_violation outcomes, and they are pasted as they were printed, two-word verb and all, because a transcript nobody ever saw is not evidence. The agent runs in sections 7 through 9 get the same treatment: those calls were refused because the generated command line carried a judge at the time, and the skill as it generates today would commit them. Three claims the demo made are now false and say so instead of quietly standing: that the rules are enforced underneath whatever reads the skill, that an agent ignoring the description cannot get past them, and that the split is impossible because of the runtime rather than because the harness happened to behave. Limits leads with the honest version -- no caller in this repository settles a guard today, because the CLI declares them unchecked and the one application that hired a judge was the ADK agent #453 removed. The library is untouched and still tested; what is missing is something wired to it. --- .../demo/semantic-model/skill/README.md | 194 ++++++++++++++---- 1 file changed, 158 insertions(+), 36 deletions(-) diff --git a/toolbox/mdcode/demo/semantic-model/skill/README.md b/toolbox/mdcode/demo/semantic-model/skill/README.md index 21ac62bf..ede753f1 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/README.md +++ b/toolbox/mdcode/demo/semantic-model/skill/README.md @@ -16,8 +16,16 @@ the harness. There is no agent in this directory. Nothing here is written against a particular framework, and nothing here has to be rewritten to move to another -one. The skill is a folder of Markdown, and the rules are enforced underneath it -whatever reads it. +one. The skill is a folder of Markdown, and the rules are stated in the model +underneath it whatever reads it. + +One thing to know before you start, because the rest of this page depends on +it: **the executor this demo hands the agent does not currently settle those +rules.** `kcmd action-run` is a curation command line. It names the rules an +action states and performs the write regardless. The runtime beneath it settles +them when the caller gives it a judge, and no caller in this repository does +today. Where that matters, this page says so, and the transcripts recorded when +a command line did settle them are marked as such. ## What this demo is arguing @@ -34,9 +42,12 @@ from them. So: - **The skill is generated, not written.** Change the model, regenerate. There is no second copy of the business to keep in step. - **The rules are not in the skill.** The skill *describes* them so the reading - agent knows what it is up against, but they are settled by the runtime that - performs the write. An agent that ignores the description still cannot get - past them. + agent knows what it is up against; settling them is the job of the runtime + that performs the write, not of the agent that asks. That separation is the + argument, and it is the part this revision does not currently demonstrate: + the runtime settles a rule only when its caller supplies a judge, and the + `kcmd` executor the skill names supplies none. An agent that ignores the + description gets past them today. - **Any harness will do.** The output is the published Agent Skill layout, so Claude Code, Gemini CLI, Cursor and the other clients that read that layout all take it as-is. @@ -276,7 +287,7 @@ Before generating anything, see what `kcmd` reads out of the scope. Run this from this directory, the one holding `catalog.yaml`: ```console -$ kcmd action list +$ kcmd action-list Model 'commerce' (commerce_demo), profile 'spanner': store: my-project/my-instance/semantic_skill_demo IssueCredit: Credit a customer against one order -- a late delivery, a coupon, a shipping charge applied in error. The credit is added as a negative line and the order total is recomputed from the lines. @@ -284,20 +295,92 @@ Model 'commerce' (commerce_demo), profile 'spanner': executor: sql guards: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview affects: LineItem (create), Order (modify) - run: kcmd action run IssueCredit --judge --judge-reads-store --arg order= --arg amount= --arg memo= + run: kcmd action-run IssueCredit --arg order= --arg amount= --arg memo= ``` That is the model, the binding, and the command line to try a call, all derived. -`kcmd` is a debugging command line rather than the runtime an agent should call -in production, but a call it makes goes through the same runtime, so it is the -fastest way to see what a rule does. +The four names on the `guards` line are the rules this action states, and they +are worth reading before the next command, because the next command does not +check any of them. + +### What the command line settles: nothing + +`kcmd action-run` binds the arguments, opens one transaction and applies the +statements. It settles none of the rules. It says so before it writes, naming +every guard it is passing over, so that nobody reads a committed write as a +checked one: + +```console +$ kcmd action-run IssueCredit \ + --arg order=12346 --arg amount=3.00 --arg memo="Coupon applied late" +Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... + NOT CHECKED: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this command settles no guard, and the write still happens +Committed at 2026-09-21T01:19:39.284744Z. +``` + +That is the useful half for curating a model, and it is genuinely useful: +whether an action binds its arguments, writes the line it says it writes and +leaves the store consistent is a question about SQL, and one command against +your own database answers it without standing up a judge first. -### Why the judge has to be able to read +It is also the whole of what this command tells you. Order 12346 was seeded at +$18.00 and the credit above took it to $15.00. Here is the same command asked +for $20.00 against it — a credit larger than the order it credits, which is +precisely what `CreditWithinOrderTotal` exists to stop: -`--judge` hires the judge that settles rules stated in words. -`--judge-reads-store` lets it query the model's own tables while deciding. The -difference is not a nicety. Without the read, `CreditWithinOrderTotal` is a rule -about a number the judge cannot see: +```console +$ kcmd action-run IssueCredit \ + --arg order=12346 --arg amount=20.00 --arg memo="Shipping charge applied in error" +Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... + NOT CHECKED: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this command settles no guard, and the write still happens +Committed at 2026-09-21T01:19:55.792326Z. +``` + +```console +$ gcloud spanner databases execute-sql "$DATABASE" \ + --instance="$INSTANCE" --project="$PROJECT" \ + --sql='SELECT order_id, total FROM Orders WHERE order_id = 12346' +order_id total +12346 -5 +``` + +An order with a total of **negative five dollars**, committed, with the rule +that forbids it sitting right there in the model. The banner is not a formality. + +### Where the rules are settled instead + +Settling a rule stated in words takes a language model, and hiring one is a +decision for whoever dispatches the call — it costs a model call per guard and +credentials to reach one, and an author checking a `WHERE` clause should not +have to stand either up. So the runtime takes a judge from the application that +embeds it, at construction: + +```ts +const judge = new GeminiJudge(ctx, {model: 'gemini-2.5-flash'}); +``` + +Given one, the runtime puts each guard to it before the transaction opens and +routes the verdict by `on_violation`. Given none, it refuses the call rather +than running a write the model says must be checked. `kcmd action-run` is the +one caller that opts out of both: it asks for no judge and refuses nothing, +which is why it prints the banner instead. + +**No command line in this repository settles a guard.** Until one does, the +transcripts below are the record of what settling them looked like. + +### What the rules caught, when something was settling them + +> These four were recorded when the command took a judge and could give that +> judge the store to read. Neither is a flag any more, and the verb was two +> words then, so each transcript below opens `kcmd action run` rather than +> `kcmd action-run`. They are pasted as they were rather than corrected, +> because a transcript that was never printed is not evidence of anything. +> They are kept because what they show — one action, four rules, and every +> one of the three `on_violation` outcomes — is not shown anywhere else, and +> because the contrast with the two runs above is the point of this section. + +Without the store, `CreditWithinOrderTotal` is a rule about a number the judge +cannot see: ```console $ kcmd action run IssueCredit --judge \ @@ -313,7 +396,7 @@ number. What it does instead is not predictable: this run said it could not tell, and an earlier run of the same call answered that $3.00 exceeded a total of $2.50 — a figure nothing in the database supports. A judge that cannot read either blocks a call it should pass or invents the fact it was missing, and you -do not get to choose which. With the read, it looks: +do not get to choose which. Given the store, it looks: ```console $ kcmd action run IssueCredit --judge --judge-reads-store \ @@ -325,13 +408,14 @@ Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/sem Error: Action 'IssueCredit' is guarded by 'CreditWithinOrderTotal' (...), and gemini-2.5-flash (us-central1) judged that it does not hold for this call: The credit amount of 20.00 exceeds the order total of 18.00. Please request a credit amount that does not exceed the order total. The model marks this rule 'escalate', so an approver may allow it; nothing here can. A credit cannot exceed the total of the order it credits. Lower the credit amount, or split it across the orders it actually covers. No transaction was opened, so nothing was written. ``` -It wrote its own `SELECT`, from the model, and the refusal is now about the real -total. Note what the message carries: which rule, the rule's own words, the -judge's reason, the consequence the model attaches, the advice the model wrote -for this case, and the fact that nothing was written. That whole message is what -a reading agent gets back. +That is the same call that now commits and leaves the order at -$5.00. The judge +wrote its own `SELECT`, from the model, and refused against the real total. Note +what the message carries: which rule, the rule's own words, the judge's reason, +the consequence the model attaches, the advice the model wrote for this case, +and the fact that nothing was written. That whole message is what a reading +agent gets back. -### With no judge at all +Handed no judge at all, the runtime fails closed rather than writing unchecked: ```console $ kcmd action run IssueCredit \ @@ -339,10 +423,11 @@ $ kcmd action run IssueCredit \ Error: Action 'IssueCredit' is guarded by 'CreditWithinOrderTotal', 'CreditUnderReviewThreshold' and 'CreditIsNotSplitToAvoidReview', which are settled by reading the call, and this runtime was given no judge to ask. Running it would apply a write the model says must be checked first, so it is refused rather than run unchecked. ``` -Fail closed. A guarded action with nothing able to settle its guards does not -run. +That refusal is still the runtime's behaviour for any caller that supplies no +judge. What changed is that the command line no longer asks: it declares the +guards unchecked and proceeds, which is the banner in the two runs above. -### The reject consequence +And the `reject` consequence, the one an approver cannot wave through: ```console $ kcmd action run IssueCredit --judge --judge-reads-store \ @@ -467,8 +552,17 @@ Everything above is true of this model wherever it is deployed. This section is - Executor: `sql` ``` -with the command line, and a note that `kcmd` is for trying calls rather than for -production. +with the command line, and — because that command line is `kcmd` — the +generator writes the same warning the CLI prints at run time into the skill +itself, so the reading agent has it before it calls: + +```markdown +That command line settles no guard. It names the rules this action states and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. +``` + +An agent handed this skill is therefore told, in the skill, that the command it +has been given is not the one that enforces the rules it just read. What it does +with that is section 7. Then **What happens when you call one** — the three states a call comes back in, and the fourth case that is not a state: @@ -514,7 +608,7 @@ this directory will offer the skill. ### Where the harness has to be running -The command line in the skill is `kcmd action run IssueCredit --profile spanner +The command line in the skill is `kcmd action-run IssueCredit --profile spanner ...`, with no path to the model — and `kcmd` takes the scope from the current directory, the one holding `catalog.yaml`. There is no flag to point it elsewhere. So the harness has to be working in the scope directory, and `kcmd` @@ -536,6 +630,16 @@ no SQL. The four runs are consecutive against the seed from [section 3](#3-create-the-store), so the state each one starts from is the state the previous one left. +> **These four were recorded when the generated command line was +> `kcmd action run ... --judge --judge-reads-store`, and they are kept because +> nothing else shows an agent meeting a rule it cannot talk its way past.** Read them for what the +> agent did with a refusal, not as what this skill does today. Regenerate the +> skill now and the command it writes settles no guard, so the first run below +> would commit the $30 credit the desk is not allowed to approve, and the third +> would land without the advisory ever being raised. The agent's own judgment +> would be the only thing between the request and the write — which is exactly +> the distinction the fourth run is about, applied to all four. + ### A request that gets refused > Morgan Ellis emailed about the order she placed on Labor Day. She was charged @@ -692,7 +796,8 @@ something that was not JSON, and the runtime refused the call — *"judge […] returned an answer that is not JSON. No transaction was opened, so nothing was written."* A guard that cannot be settled is a refusal, not a pass. That is the same fail-closed rule as [running with no judge at -all](#with-no-judge-at-all), reached by accident rather than by configuration. +all](#what-the-rules-caught-when-something-was-settling-them), reached by +accident rather than by configuration. ### An instruction to break a rule @@ -722,11 +827,13 @@ approval control is behaviour a capable harness brings with it. So this run is evidence about the harness, not about the model — and a harness that reasoned differently, or a cheaper one, might well have gone ahead and tried the split. -What makes the split impossible is the runtime, and the case that shows it is -the one forced at the command line in [section 4](#the-reject-consequence): the -same rule, the same wording, refused before a transaction opened with nothing -written. That one is a guarantee. This one is a harness behaving well on top of -it. +What makes the split impossible is the runtime, not the harness, and the case +that shows it is the `reject` transcript in [section +4](#what-the-rules-caught-when-something-was-settling-them): the same rule, the +same wording, refused before a transaction opened with nothing written. That one +is a guarantee — but read the note above it. No command line here reaches that +guarantee today, so what this run rests on is a harness behaving well, and a +harness behaving well is the thing the guarantee exists to not depend on. ### Where the store ended up @@ -791,8 +898,11 @@ Committed at 2026-09-20T20:01:35.712448Z. Order 12347 recomputed from its lines, $175.00 → $167.00. Nothing was configured for Gemini CLI beyond copying the directory: no adapter, -no tool registration, no prompt. The rules held because they are enforced in the -runtime under the action, not by whichever agent happens to be reading. +no tool registration, no prompt. That part is the portability claim and it still +holds. The rules held in these two runs because the command line the skill +carried at the time put them to a judge — see the note in +[section 7](#7-run-it); the same two requests against the skill as it generates +today would both commit. ## 9. The same skill against a different database @@ -830,6 +940,18 @@ what may be done and under what rules is the same bytes. ## 10. Limits +**Nothing in this repository settles a guard right now, so this demo no longer +runs end to end as written.** `kcmd action-run` declares the guards unchecked +and writes; the runtime settles them only for a caller that hands it a judge, +and the one caller that did — the ADK agent this directory used to hold — was +removed when the demo became a skill. The rules, the judgments, the +`on_violation` routing and the judge itself are all still in the library and +still tested; what is missing is a caller wired to them. Until one is back, the +guarded transcripts on this page are a record rather than something you can +reproduce, and the two unguarded runs in [section +4](#what-the-command-line-settles-nothing) are what you get if you follow the +commands. + **The AlloyDB skill has no way to find a record.** The `gcloud` read snippet and the schema block are emitted for Spanner stores only. Under `alloydb` the skill still describes the action and its rules correctly and the action still runs, but From 1d5591472b6705a1ee242b188917761dc5173806 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:23:31 +0000 Subject: [PATCH 05/11] fix(mdcode): a name no model declares is an error, not an empty listing Three ways the trimmed command lines answered a question nobody asked. `action-list ` took the positional and ignored it, printing every action in scope. The ignoring predates this branch, but renaming the command promoted the argument into an explicit promise, so it now checks the name across the whole scope before anything prints and names what is declared when it finds nothing. `profiles --profile ` exited 0 on a model with no profile documents: the "no binding profiles" line returned before the name was ever checked, so a script keying off the exit code read a misspelling as success. The check moves ahead of that line. `profiles --profile default` then broke, because 'default' is the sentinel for the inline bindings rather than a profile document -- a push rejects a file that claims the name -- so it is the one profile name always valid against every model. It reports the inline bindings instead of being called undeclared. Also restores the formatting of `catalogOnly`, which an editor formatter had mangled into `new Map < string, {...} > ()`, and names its value type so there is no longer an object literal inside a generic argument for a formatter to break. --- toolbox/mdcode/src/tool/commands.ts | 75 ++++++++++--- toolbox/mdcode/tests/tool/action.test.ts | 28 ++++- toolbox/mdcode/tests/tool/profiles.test.ts | 117 +++++++++++++++------ 3 files changed, 170 insertions(+), 50 deletions(-) diff --git a/toolbox/mdcode/src/tool/commands.ts b/toolbox/mdcode/src/tool/commands.ts index 6713d9d4..6bd680da 100644 --- a/toolbox/mdcode/src/tool/commands.ts +++ b/toolbox/mdcode/src/tool/commands.ts @@ -41,6 +41,15 @@ export interface InitOptions { } +// What a model would deploy through the Knowledge Catalog leg alone. Named +// rather than written inline at its one use site: an object literal inside a +// generic argument is what editor formatters mangle first, and this one has +// been mangled into `new Map < string, {...} > ()` more than once. +interface CatalogOnlyCounts { + actions: number; + constraints: number; +} + export interface PushOptions { // Generic push flag for non-semantic-model (CatalogSync) scopes; // forwarded to CatalogSync.push. The semantic-model legs ignore it. @@ -536,11 +545,7 @@ export async function push(options: PushOptions): Promise { // Per model, what deploys through the Knowledge Catalog leg ALONE: // actions and constraints both have a catalog home and no graph one, so a // push that omits that leg has to account for either. - const catalogOnly = new Map < string, { - actions: number; - constraints: number - } - >(); + const catalogOnly = new Map(); const noteCatalogOnly = (loaded: LoadedModel[]) => { for (const {model} of loaded) { const actions = model.actions?.length ?? 0; @@ -802,21 +807,38 @@ export async function profiles(options: ProfilesOptions = {}): Promise { for (const doc of docs) { console.log(`Model '${doc.name}' (${source.entryGroup}):`); const declared = layout.profileDocuments(doc.name); - if (!declared.length) { + // 'default' is not a profile document and never can be -- a push rejects a + // file by that name. It names the inline bindings, which every model has, + // so it is the one profile name that is always right and has nothing to + // merge or prune. Reported here rather than falling into the typo check + // below, which would call a universally valid name undeclared. + if (only === DEFAULT_PROFILE) { console.log( - ` no binding profiles; the model document is its own inline ` + - `'default' binding.`); + ` profile '${DEFAULT_PROFILE}': the model document's own inline ` + + `bindings, as authored -- nothing merged, nothing withheld.`); continue; } const available = only ? declared.filter(p => p.name === only) : declared; // Naming a profile the model does not declare is a typo, not an empty // report: saying nothing would read as "this profile withholds nothing". + // Checked before the inline-binding line below, because a model that + // declares no profile documents still has to call a typo a typo -- saying + // "no binding profiles" and exiting 0 answers a question nobody asked. if (only && !available.length) { - console.error(` no profile '${only}'; this model declares ${ - declared.map(p => `'${p.name}'`).join(', ')}.`); + console.error( + ` no profile '${only}'; this model declares ` + + (declared.length ? + `${declared.map(p => `'${p.name}'`).join(', ')}.` : + `none, and its inline bindings are its 'default'.`)); missing = true; continue; } + if (!declared.length) { + console.log( + ` no binding profiles; the model document is its own inline ` + + `'default' binding.`); + continue; + } for (const {name, text} of available) { const res = mergeProfileOntoDoc(doc.text, text, name); if ('error' in res) { @@ -1285,14 +1307,16 @@ async function openActionRuntimes(options: ActionOptions): // kcmd action-list [name] // // Answers "what can I run, and how": each action's parameters, executor, -// guards and blast radius, ending with the command line that runs it. +// guards and blast radius, ending with the command line that runs it. Naming +// one narrows the listing to it; a name no model in scope declares is an +// error, because an empty listing reads as "this model declares nothing". // // Returns a process exit code (0 on success). export async function actionList( - _name: string|undefined, options: ActionOptions = {}): Promise { + name: string|undefined, options: ActionOptions = {}): Promise { const opened = await openActionRuntimes(options); if (typeof opened === 'number') return opened; - return listActions(opened, options); + return listActions(opened, name, options); } @@ -1324,9 +1348,29 @@ const RUN_INDENT = ' '; // when the runtime would refuse the call before opening a transaction, what it // is waiting on instead. function listActions( - runtimes: SemanticRuntime[], options: ActionOptions): number { + runtimes: SemanticRuntime[], only: string|undefined, + options: ActionOptions): number { + // A name nothing declares is a typo, and printing every action under it + // would answer a question the caller did not ask while looking like the + // answer to the one they did. Checked across the whole scope before + // anything prints, so the error is not buried under a model's heading. + if (only) { + const known = + runtimes.flatMap(r => (r.model.actions ?? []).map(a => a.name)); + if (!known.includes(only)) { + console.error( + `Error: no model in this scope declares an action '${only}'` + + (known.length ? `; declared: ${known.sort().join(', ')}.` : '.')); + return 1; + } + } + for (const runtime of runtimes) { const {model, store, storeError, profile, entryGroup} = runtime; + // A scope can hold several models and only one of them declare the action + // that was named. The others have nothing to say about it, and a heading + // over an empty listing reads as an answer. + if (only && !(model.actions ?? []).some(a => a.name === only)) continue; console.log(`Model '${model.name}' (${entryGroup}), profile '${profile}':`); // Where a run lands, said once at the top rather than left to be inferred // from a profile file the reader would have to go open. @@ -1336,7 +1380,8 @@ function listActions( } else { console.log(` store: ${storeLine(store)}`); } - const actions = model.actions ?? []; + const declared = model.actions ?? []; + const actions = only ? declared.filter(a => a.name === only) : declared; if (!actions.length) { console.log(' declares no actions.'); continue; diff --git a/toolbox/mdcode/tests/tool/action.test.ts b/toolbox/mdcode/tests/tool/action.test.ts index 6add7419..f2c79db2 100644 --- a/toolbox/mdcode/tests/tool/action.test.ts +++ b/toolbox/mdcode/tests/tool/action.test.ts @@ -323,6 +323,30 @@ afterEach(() => { describe('kcmd action-list', () => { + // The command is declared `action-list [name]`, so the positional has to + // do something. It used to be accepted and dropped: naming one action on a + // model that declares several printed them all and exited 0, which reads + // as the answer to the question that was asked. + test('a named action narrows the listing to it', async () => { + writeWorkspace(); + const code = await actionList('NotifyCustomer'); + expect(code).toBe(0); + const out = logs.join('\n'); + expect(out).toContain('NotifyCustomer'); + expect(out).not.toContain('IssueCredit: Credit an order'); + }); + + // An empty listing under a misspelled name reads as "this model declares + // nothing", so the name is checked across the scope before anything prints. + test('a name no model declares is an error', async () => { + writeWorkspace(); + const code = await actionList('IssueCredits'); + expect(code).toBe(1); + const out = logs.join('\n'); + expect(out).toContain("no model in this scope declares an action 'IssueCredits'"); + expect(out).toContain('IssueCredit, NotifyCustomer'); + }); + test( 'prints each action with what it takes, what it touches, and the ' + 'command line that runs it', @@ -546,8 +570,8 @@ describe('kcmd action-run: what it will not send to a store', () => { const out = logs.join('\n'); expect(out).toContain('NOT CHECKED: CreditIsPositive'); expect(out).toContain('this command settles no guard'); - // Printed before the run banner's database line is reached, so it is read - // while the run is still a run. + // After the run banner, so the reader has been told which database is + // about to be written to before being told what will go unchecked on it. expect(out.indexOf('NOT CHECKED')) .toBeGreaterThan(out.indexOf('Running \'IssueCredit\'')); expect(out).toContain( diff --git a/toolbox/mdcode/tests/tool/profiles.test.ts b/toolbox/mdcode/tests/tool/profiles.test.ts index 259c4b9a..a770414c 100644 --- a/toolbox/mdcode/tests/tool/profiles.test.ts +++ b/toolbox/mdcode/tests/tool/profiles.test.ts @@ -84,8 +84,19 @@ function writeWorkspace(defaultProfile?: string): void { const eg = path.join(dir, 'catalog', 'EntryGroups', 'commerce_eg'); fs.mkdirSync(path.join(eg, 'commerce.profiles'), {recursive: true}); fs.writeFileSync(path.join(eg, 'commerce.yaml'), LOGICAL); - fs.writeFileSync(path.join(eg, 'commerce.profiles', 'analytical.yaml'), ANALYTICAL); - fs.writeFileSync(path.join(eg, 'commerce.profiles', 'operational.yaml'), OPERATIONAL); + fs.writeFileSync( + path.join(eg, 'commerce.profiles', 'analytical.yaml'), ANALYTICAL); + fs.writeFileSync( + path.join(eg, 'commerce.profiles', 'operational.yaml'), OPERATIONAL); +} + +// A model whose bindings are inline: it declares no profile documents, so +// `profileDocuments` is empty and 'default' is the only binding there is. +function writeInlineOnlyWorkspace(): void { + fs.writeFileSync(path.join(dir, 'catalog.yaml'), catalogYaml(undefined)); + const eg = path.join(dir, 'catalog', 'EntryGroups', 'commerce_eg'); + fs.mkdirSync(eg, {recursive: true}); + fs.writeFileSync(path.join(eg, 'commerce.yaml'), LOGICAL); } beforeEach(() => { @@ -111,31 +122,32 @@ afterEach(() => { describe('kcmd profiles', () => { - test('lists each profile with its target, sources, and withheld coverage', - async () => { - writeWorkspace('analytical'); - const code = await profiles(); - expect(code).toBe(0); - const out = logs.join('\n'); - - // Both profiles listed; the default one is marked. - expect(out).toContain("profile 'analytical' (default)"); - expect(out).toContain("profile 'operational'"); - expect(out).not.toContain("profile 'operational' (default)"); - - // Resolved targets and normalized sources. - expect(out).toContain( - 'target: //bigquery.googleapis.com/projects/acme-analytics/datasets/sales/propertyGraphs/commerce'); - expect(out).toContain('Customer -> acme-analytics.sales.customer'); - expect(out).toContain( - 'Customer -> //spanner.googleapis.com/projects/acme-ops/instances/prod/databases/commerce/tables/Customer'); - - // Withheld coverage: availableCredit only under analytical, - // lifetimeValue (and the metric on it) only under operational. - expect(out).toContain('field Customer.availableCredit (unbound)'); - expect(out).toContain('field Customer.lifetimeValue (unbound)'); - expect(out).toContain('metric avg_lifetime_value'); - }); + test( + 'lists each profile with its target, sources, and withheld coverage', + async () => { + writeWorkspace('analytical'); + const code = await profiles(); + expect(code).toBe(0); + const out = logs.join('\n'); + + // Both profiles listed; the default one is marked. + expect(out).toContain('profile \'analytical\' (default)'); + expect(out).toContain('profile \'operational\''); + expect(out).not.toContain('profile \'operational\' (default)'); + + // Resolved targets and normalized sources. + expect(out).toContain( + 'target: //bigquery.googleapis.com/projects/acme-analytics/datasets/sales/propertyGraphs/commerce'); + expect(out).toContain('Customer -> acme-analytics.sales.customer'); + expect(out).toContain( + 'Customer -> //spanner.googleapis.com/projects/acme-ops/instances/prod/databases/commerce/tables/Customer'); + + // Withheld coverage: availableCredit only under analytical, + // lifetimeValue (and the metric on it) only under operational. + expect(out).toContain('field Customer.availableCredit (unbound)'); + expect(out).toContain('field Customer.lifetimeValue (unbound)'); + expect(out).toContain('metric avg_lifetime_value'); + }); test('marks no profile default when none is configured', async () => { writeWorkspace(undefined); @@ -149,8 +161,8 @@ describe('kcmd profiles', () => { const code = await profiles({profile: 'operational'}); expect(code).toBe(0); const out = logs.join('\n'); - expect(out).toContain("profile 'operational'"); - expect(out).not.toContain("profile 'analytical'"); + expect(out).toContain('profile \'operational\''); + expect(out).not.toContain('profile \'analytical\''); }); // An empty report would read as "this profile withholds nothing", which is @@ -160,10 +172,49 @@ describe('kcmd profiles', () => { const code = await profiles({profile: 'operatonal'}); expect(code).toBe(1); const out = logs.join('\n'); - expect(out).toContain("no profile 'operatonal'"); - expect(out).toContain("'analytical', 'operational'"); + expect(out).toContain('no profile \'operatonal\''); + expect(out).toContain('\'analytical\', \'operational\''); }); + // The typo has to be caught whether or not the model declares any profile + // documents. This used to report the inline binding and exit 0, because the + // "no binding profiles" line returned before the name was ever checked -- + // so a script keying off the exit code read a misspelling as success. + test( + '`--profile` naming an undeclared profile is an error with inline bindings', + async () => { + writeInlineOnlyWorkspace(); + const code = await profiles({profile: 'operatonal'}); + expect(code).toBe(1); + const out = logs.join('\n'); + expect(out).toContain('no profile \'operatonal\''); + // Says what this model does have, which is not a list of names. + expect(out).toContain('inline bindings'); + expect(out).not.toContain('no binding profiles;'); + }); + + // 'default' is the sentinel for the inline bindings, so it is valid against + // every model and a push rejects a profile file that claims the name. The + // typo check above must not treat the one always-correct name as a + // misspelling just because no profile document declares it -- which it did, + // for both a model with profiles and one without. + for (const [label, write] of [ + ['profiles declared', writeWorkspace], + ['inline bindings only', writeInlineOnlyWorkspace], + ] as const) { + test( + `\`--profile default\` reports the inline bindings (${label})`, + async () => { + write(); + const code = await profiles({profile: 'default'}); + expect(code).toBe(0); + const out = logs.join('\n'); + expect(out).toContain('profile \'default\''); + expect(out).toContain('inline'); + expect(out).not.toContain('no profile \'default\''); + }); + } + // A bare `--profile` reaches cac as `true` and `--no-profile` as `false`. // Neither names a profile, so neither may narrow the report -- filtering on // one would report zero profiles for a flag the caller left blank. @@ -173,8 +224,8 @@ describe('kcmd profiles', () => { const code = await profiles({profile}); expect(code).toBe(0); const out = logs.join('\n'); - expect(out).toContain("profile 'analytical'"); - expect(out).toContain("profile 'operational'"); + expect(out).toContain('profile \'analytical\''); + expect(out).toContain('profile \'operational\''); }); } }); From 4c7d58a06ce916ffc5aedd81319225908dbf90ff Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:23:58 +0000 Subject: [PATCH 06/11] fix(mdcode): the generated skill's guard caveat is keyed to the model The caveat that `kcmd action-run` settles no guard was emitted only when the action the example command line happened to pick was itself guarded. Pick order is declaration order, so a model whose first runnable action is unguarded shipped a SKILL.md with the caveat missing -- while its reference pages still told the agent "Each is settled before anything is written." The one place the skill corrects that claim was the place that dropped out. It now keys on whether any action in the model declares a guard, and says so: the command line settles no guard for this action or any other. Also drops a comment in the demo model document that still described `kcmd` as offering a judge flag, and re-syncs the copy of the caveat pasted into the demo README. --- .../demo/semantic-model/skill/README.md | 2 +- .../EntryGroups/commerce_demo/commerce.yaml | 5 ++-- toolbox/mdcode/src/libts/semantic/skills.ts | 22 ++++++++++------ ...ions_place_order.sql_bound.skill.golden.md | 2 +- .../tests/libts/semantic/skills.test.ts | 25 +++++++++++++++++++ 5 files changed, 44 insertions(+), 12 deletions(-) diff --git a/toolbox/mdcode/demo/semantic-model/skill/README.md b/toolbox/mdcode/demo/semantic-model/skill/README.md index ede753f1..6ece5cc1 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/README.md +++ b/toolbox/mdcode/demo/semantic-model/skill/README.md @@ -557,7 +557,7 @@ generator writes the same warning the CLI prints at run time into the skill itself, so the reading agent has it before it calls: ```markdown -That command line settles no guard. It names the rules this action states and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. +That command line settles no guard, for this action or any other in this model. It names whatever rules the action it runs states, and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. ``` An agent handed this skill is therefore told, in the skill, that the command it diff --git a/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml b/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml index 6bb047fb..cf6e2bdf 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml +++ b/toolbox/mdcode/demo/semantic-model/skill/catalog/EntryGroups/commerce_demo/commerce.yaml @@ -23,8 +23,9 @@ # a judge can settle only if it can read the database. Running this demo # therefore takes a judge that has been given one -- a `store` on the judge the # agent hires. Giving a judge your tables belongs to whoever embeds the -# runtime, so no kcmd flag offers it, and `kcmd action-run --judge` stops on -# that rule rather than guessing at the number it names. +# runtime, so no kcmd flag offers it. `kcmd action-run` settles no guard at +# all -- it names them and writes anyway -- so running this demo from the +# command line reaches none of the rules below. # # What this costs: every gate is a model call, and a model can answer two # identical calls differently. Don't copy the `$25` ceiling below -- a threshold diff --git a/toolbox/mdcode/src/libts/semantic/skills.ts b/toolbox/mdcode/src/libts/semantic/skills.ts index 195e6a0d..471fe7f2 100644 --- a/toolbox/mdcode/src/libts/semantic/skills.ts +++ b/toolbox/mdcode/src/libts/semantic/skills.ts @@ -553,15 +553,21 @@ function runningSection( out.push('```'); out.push(''); } - // Said wherever the action states a rule, because the command writes either - // way. An agent that tried the line, saw it commit, and took that for the - // rules holding would have drawn the one conclusion this command cannot - // support. - if (action?.guards?.length) { + // Said wherever ANY action in the model states a rule, not just the one the + // example line happens to name. The line is a template an agent adapts to + // whichever action it means to call, so keying the caveat to the example + // would drop it from a model whose first runnable action is unguarded and + // whose second is not -- leaving the reference page's "settled before + // anything is written" as the only thing said about running a guarded call. + // An agent that tried the line, saw it commit, and took that for the rules + // holding would have drawn the one conclusion this command cannot support. + const anyGuarded = (runtime.model.actions ?? []).some(a => a.guards?.length); + if (anyGuarded) { out.push( - 'That command line settles no guard. It names the rules this action ' + - 'states and runs the write regardless, so it answers whether the ' + - 'call binds and the write lands, and nothing about whether the rules ' + + 'That command line settles no guard, for this action or any other ' + + 'in this model. It names whatever rules the action it runs states, ' + + 'and runs the write regardless, so it answers whether the call ' + + 'binds and the write lands, and nothing about whether the rules ' + 'hold. The runtime your framework calls is what settles them.'); out.push(''); } diff --git a/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md b/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md index b60d93c9..806b0b3d 100644 --- a/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md +++ b/toolbox/mdcode/tests/libts/semantic/fixtures/actions_place_order.sql_bound.skill.golden.md @@ -59,7 +59,7 @@ kcmd action-run PlaceOrder \ --arg quantity= ``` -That command line settles no guard. It names the rules this action states and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. +That command line settles no guard, for this action or any other in this model. It names whatever rules the action it runs states, and runs the write regardless, so it answers whether the call binds and the write lands, and nothing about whether the rules hold. The runtime your framework calls is what settles them. ## What happens when you call one diff --git a/toolbox/mdcode/tests/libts/semantic/skills.test.ts b/toolbox/mdcode/tests/libts/semantic/skills.test.ts index becb4e6e..1651c33d 100644 --- a/toolbox/mdcode/tests/libts/semantic/skills.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/skills.test.ts @@ -388,6 +388,31 @@ describe('when the runtime would refuse the call', () => { expect(out.warnings.join(' ')).not.toContain('runnable'); }); + // The example command line is built from the FIRST runnable action, but an + // agent adapts it to whichever action it means to call. Keying the caveat to + // that example dropped it from a model whose first runnable action happens + // to be unguarded -- leaving the guarded action's reference page, which says + // its rules are "settled before anything is written", as the only thing the + // skill said about running one. + test('the caveat survives an unguarded first action', () => { + const [guarded] = model.actions!; + const unguarded: Action = { + ...guarded, + name: 'CloseOrder', + guards: [], + executor: RUNNABLE.executor, + }; + const both: SemanticModel = { + ...model, + actions: [unguarded, {...guarded, executor: RUNNABLE.executor}], + }; + const out = generate(rt(both)); + // The example line is the unguarded action's, and the caveat is still + // there, because the model declares a guarded one. + expect(out.files['SKILL.md']).toContain('kcmd action-run CloseOrder'); + expect(out.files['SKILL.md']).toContain('settles no guard'); + }); + test('an executor the runtime cannot roll back is reported as such', () => { // The fixture's own MCP executor: the write would commit in a system this // runtime does not control. From 19000899bd7e5e90d4f3cf821b60b53b8afe026c Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:24:06 +0000 Subject: [PATCH 07/11] fix(mdcode): skipGuards asks no judge, even when one is handed over A caller passing both a judge and skipGuards reached the judge anyway and then had its verdicts suppressed into warnings. That spends a model call to produce a refusal nobody acts on, and it means the two inputs disagreed about whether guards were being checked with no way to tell from the result which one won. skipGuards is the answer: it is the statement that this run settles nothing. --- .../src/libts/semantic/runtime/run_action.ts | 8 +++++- .../libts/semantic/runtime/run_action.test.ts | 26 +++++++++++++++++++ 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts index 126d3842..e5a7c7d3 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts @@ -182,7 +182,13 @@ export async function runAction(opts: RunActionOptions): // never the state the write produced, which means a rule about the RESULT of // a write is out of reach here and belongs in the schema. const warnings: string[] = []; - if (opts.judge) { + // `skipGuards` means nobody is asked -- the whole point of it -- so it + // stands the asking down too, not just the refusal for want of a judge + // and the unsettled-guard warnings. A caller that passed both used to + // reach the judge with those warnings suppressed, so a guard whose + // judgment states nothing, or one that threw while being asked, committed + // with no line about it anywhere. + if (opts.judge && !opts.skipGuards) { const judged = judgedGuards(model, action); if (judged.length) { // Returned rather than thrown. A throw from here reaches the catch at diff --git a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts index 08a3b841..e3062a6d 100644 --- a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts @@ -1257,6 +1257,32 @@ describe('a guard settled by judgment', () => { expect(outcome.warnings ?? []).toEqual([]); }); + test('skipGuards asks no judge, even when one is handed over', async () => { + // `skipGuards` means nobody is asked. A caller that passed both used to + // reach the judge anyway while the unsettled-guard warnings stayed + // suppressed -- the worst of both, since a judge that threw or returned + // nothing then committed with no line about it anywhere. + const fake = fakeStore(); + let asked = 0; + const outcome = await act({ + model: guarding([justified]), + actionName: 'Credit', + args: {account: 'A1', amount: 100}, + client: fake.client, + skipGuards: true, + judge: { + name: 'a judge nobody should reach', + decide: async () => { + asked++; + return {holds: false, reason: 'refused'}; + }, + }, + }); + expect(asked).toBe(0); + if (outcome.status !== 'committed') throw new Error(outcome.message); + expect(fake.committed).toBe(true); + }); + test('skipGuards does not clear a guard that names nothing', async () => { // Not checking the guards is not the same as not reading them. A guard // naming a rule the model never declares is the model being wrong about From 9ed88548b555154370369aa5506c7f4d14ceb277 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:51:19 +0000 Subject: [PATCH 08/11] fix(mdcode): action-list takes no name The verb was declared `action-list [name]` and the positional was accepted and thrown away: naming one action on a model that declares several printed them all and exited 0, which reads as the answer to the question that was asked. Two ways to settle that. Make the name filter, or drop it. Dropped: this is a listing command in a change that is trimming the surface, and a listing that always lists everything is one fewer thing to know. Nothing is lost -- a scope's actions are few enough to read, and the name is still how `action-run` is addressed. Removes the scope-wide unknown-name check that landed with the filter, since there is no longer a name to be wrong about. --- .../mdcode/docs/semantic-model/reference.md | 2 +- toolbox/mdcode/src/tool/commands.ts | 36 +++---------- toolbox/mdcode/src/tool/main.ts | 12 ++--- toolbox/mdcode/tests/tool/action.test.ts | 50 +++++-------------- toolbox/mdcode/tests/tool/main_cli.test.ts | 14 ++++-- 5 files changed, 36 insertions(+), 78 deletions(-) diff --git a/toolbox/mdcode/docs/semantic-model/reference.md b/toolbox/mdcode/docs/semantic-model/reference.md index 4b90bc54..8a583a86 100644 --- a/toolbox/mdcode/docs/semantic-model/reference.md +++ b/toolbox/mdcode/docs/semantic-model/reference.md @@ -91,7 +91,7 @@ IFS=/ read -r PROJECT INSTANCE DATABASE <<<"$(kcmd profiles --print-store)" ### action-list ```bash -kcmd action-list [name] +kcmd action-list ``` Prints every action the models in the scope declare, with the store a run would diff --git a/toolbox/mdcode/src/tool/commands.ts b/toolbox/mdcode/src/tool/commands.ts index 6bd680da..edea636d 100644 --- a/toolbox/mdcode/src/tool/commands.ts +++ b/toolbox/mdcode/src/tool/commands.ts @@ -1304,7 +1304,7 @@ async function openActionRuntimes(options: ActionOptions): // Lists what a semantic model declares as runnable. // -// kcmd action-list [name] +// kcmd action-list // // Answers "what can I run, and how": each action's parameters, executor, // guards and blast radius, ending with the command line that runs it. Naming @@ -1312,11 +1312,11 @@ async function openActionRuntimes(options: ActionOptions): // error, because an empty listing reads as "this model declares nothing". // // Returns a process exit code (0 on success). -export async function actionList( - name: string|undefined, options: ActionOptions = {}): Promise { +export async function actionList(options: ActionOptions = {}): + Promise { const opened = await openActionRuntimes(options); if (typeof opened === 'number') return opened; - return listActions(opened, name, options); + return listActions(opened); } @@ -1347,30 +1347,9 @@ const RUN_INDENT = ' '; // the listing is enough to make the call without going back to the YAML -- or, // when the runtime would refuse the call before opening a transaction, what it // is waiting on instead. -function listActions( - runtimes: SemanticRuntime[], only: string|undefined, - options: ActionOptions): number { - // A name nothing declares is a typo, and printing every action under it - // would answer a question the caller did not ask while looking like the - // answer to the one they did. Checked across the whole scope before - // anything prints, so the error is not buried under a model's heading. - if (only) { - const known = - runtimes.flatMap(r => (r.model.actions ?? []).map(a => a.name)); - if (!known.includes(only)) { - console.error( - `Error: no model in this scope declares an action '${only}'` + - (known.length ? `; declared: ${known.sort().join(', ')}.` : '.')); - return 1; - } - } - +function listActions(runtimes: SemanticRuntime[]): number { for (const runtime of runtimes) { const {model, store, storeError, profile, entryGroup} = runtime; - // A scope can hold several models and only one of them declare the action - // that was named. The others have nothing to say about it, and a heading - // over an empty listing reads as an answer. - if (only && !(model.actions ?? []).some(a => a.name === only)) continue; console.log(`Model '${model.name}' (${entryGroup}), profile '${profile}':`); // Where a run lands, said once at the top rather than left to be inferred // from a profile file the reader would have to go open. @@ -1380,8 +1359,7 @@ function listActions( } else { console.log(` store: ${storeLine(store)}`); } - const declared = model.actions ?? []; - const actions = only ? declared.filter(a => a.name === only) : declared; + const actions = model.actions ?? []; if (!actions.length) { console.log(' declares no actions.'); continue; @@ -1405,7 +1383,7 @@ function listActions( .join(', ')}`); } // Asked of the runtime rather than worked out here, for the reason - // `agent tools` asks: two copies of "can this run" drift, and neither + // `agent-tools` asks: two copies of "can this run" drift, and neither // direction of the drift is visible to the reader. This used to notice // only a missing executor, so an action executed by HTTP -- which this // command has no handler for and could not roll back -- printed a run diff --git a/toolbox/mdcode/src/tool/main.ts b/toolbox/mdcode/src/tool/main.ts index bb62995b..59066c7b 100644 --- a/toolbox/mdcode/src/tool/main.ts +++ b/toolbox/mdcode/src/tool/main.ts @@ -139,15 +139,15 @@ cli.command( cli.command( - 'action-list [name]', + 'action-list', 'List what a semantic model declares as runnable: parameters, executor, guards, blast radius, and the command that runs each one') .option( '--profile [name]', 'Read the model under this binding profile; its deployment target names the database the action runs against; defaults to default_profile, else the inline bindings') - .action(async (name, options) => { + .action(async (options) => { let exitCode = 1; try { - exitCode = await commands.actionList(name, options); + exitCode = await commands.actionList(options); } catch (err: any) { console.error('Error:', err.message || err); exitCode = 1; @@ -264,9 +264,9 @@ try { // actually typed, which cac does not keep once it has cleared the match, so // read it off `process.argv` rather than off `cli.args`. `cli.args` cannot // answer this: cac strips a matched command's own name from it and clears the -// match in the same breath, so `action list --help` arrives holding `list` and -// `bogusverb --help` holding `bogusverb`, and neither of those words names a -// command. +// match in the same breath, so `action-run IssueCredit --help` arrives holding +// `IssueCredit` and `bogusverb --help` holding `bogusverb`, and neither of those +// words names a command. // // The verb is the first token that is not a flag, not the first token: the // flag may come first, and `kcmd --help bogusverb` still misspells a diff --git a/toolbox/mdcode/tests/tool/action.test.ts b/toolbox/mdcode/tests/tool/action.test.ts index f2c79db2..7b3edba4 100644 --- a/toolbox/mdcode/tests/tool/action.test.ts +++ b/toolbox/mdcode/tests/tool/action.test.ts @@ -323,36 +323,12 @@ afterEach(() => { describe('kcmd action-list', () => { - // The command is declared `action-list [name]`, so the positional has to - // do something. It used to be accepted and dropped: naming one action on a - // model that declares several printed them all and exited 0, which reads - // as the answer to the question that was asked. - test('a named action narrows the listing to it', async () => { - writeWorkspace(); - const code = await actionList('NotifyCustomer'); - expect(code).toBe(0); - const out = logs.join('\n'); - expect(out).toContain('NotifyCustomer'); - expect(out).not.toContain('IssueCredit: Credit an order'); - }); - - // An empty listing under a misspelled name reads as "this model declares - // nothing", so the name is checked across the scope before anything prints. - test('a name no model declares is an error', async () => { - writeWorkspace(); - const code = await actionList('IssueCredits'); - expect(code).toBe(1); - const out = logs.join('\n'); - expect(out).toContain("no model in this scope declares an action 'IssueCredits'"); - expect(out).toContain('IssueCredit, NotifyCustomer'); - }); - test( 'prints each action with what it takes, what it touches, and the ' + 'command line that runs it', async () => { writeWorkspace(); - const code = await actionList(undefined); + const code = await actionList(); expect(code).toBe(0); const out = logs.join('\n'); @@ -401,7 +377,7 @@ describe('kcmd action-list', () => { ' - {name: literalNull, type: String, default: "null"}\n' + ' - {name: memo, type: String, required: false}'); writeWorkspace(optionalModel); - const code = await actionList(undefined); + const code = await actionList(); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain( @@ -423,7 +399,7 @@ describe('kcmd action-list', () => { // line for a write this binding cannot perform would send the reader // to a refusal, so it prints the fix instead. writeWorkspace(LOGICAL); - const code = await actionList(undefined, {profile: 'readonly'}); + const code = await actionList({profile: 'readonly'}); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain('IssueCredit'); @@ -449,7 +425,7 @@ describe('kcmd action-list', () => { // out from the executor. writeWorkspace(MODEL.replace( 'guards: [CreditIsPositive]', 'guards: [NoSuchRule]')); - const code = await actionList(undefined); + const code = await actionList(); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain('executor: sql'); @@ -466,7 +442,7 @@ describe('kcmd action-list', () => { // in earnest -- so no flag here offers it, and the listing says the guard // is there without pretending it can be checked. writeWorkspace(); - const code = await actionList(undefined); + const code = await actionList(); expect(code).toBe(0); const out = logs.join('\n'); expect(out).toContain('guards: CreditIsPositive'); @@ -478,14 +454,14 @@ describe('kcmd action-list', () => { test('says so when a model declares no actions', async () => { writeWorkspace(NO_ACTIONS); - const code = await actionList(undefined); + const code = await actionList(); expect(code).toBe(0); expect(logs.join('\n')).toContain('declares no actions.'); }); test('reads the model under a named profile', async () => { writeWorkspace(LOGICAL); - const code = await actionList(undefined, {profile: 'analytical'}); + const code = await actionList({profile: 'analytical'}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'analytical\''); }); @@ -494,7 +470,7 @@ describe('kcmd action-list', () => { 'names the profiles that exist when given one that does not', async () => { writeWorkspace(); - const code = await actionList(undefined, {profile: 'nope'}); + const code = await actionList({profile: 'nope'}); expect(code).toBe(1); const out = logs.join('\n'); expect(out).toContain('unknown binding profile \'nope\''); @@ -706,7 +682,7 @@ describe('kcmd action-list/action-run: what the command line can hold', () => { // cac yields `true` for `--profile` with no value. Reading it as a // name would fail the command with a profile the user never typed. writeWorkspace(); - const code = await actionList(undefined, {profile: true}); + const code = await actionList({profile: true}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'default\''); }); @@ -714,14 +690,14 @@ describe('kcmd action-list/action-run: what the command line can hold', () => { test('--no-profile does not become a profile name either', async () => { // mri yields `false`, which `??` would pass straight through. writeWorkspace(); - const code = await actionList(undefined, {profile: false}); + const code = await actionList({profile: false}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'default\''); }); test('a named profile still selects that profile', async () => { writeWorkspace(LOGICAL); - const code = await actionList(undefined, {profile: 'analytical'}); + const code = await actionList({profile: 'analytical'}); expect(code).toBe(0); expect(logs.join('\n')).toContain('profile \'analytical\''); }); @@ -869,7 +845,7 @@ describe('kcmd action-run: the model has to be valid to run', () => { 'refused rather than run', async () => { // A push rejects this outright, and running the model is running the - // same typo, so `action run` reports it by name rather than passing + // same typo, so `action-run` reports it by name rather than passing // over an entry that resolves to nothing. writeWorkspace(TYPO); const code = @@ -880,6 +856,6 @@ describe('kcmd action-run: the model has to be valid to run', () => { test('but listing it still works, because listing runs nothing', async () => { writeWorkspace(TYPO); - expect(await actionList(undefined)).toBe(0); + expect(await actionList()).toBe(0); }); }); diff --git a/toolbox/mdcode/tests/tool/main_cli.test.ts b/toolbox/mdcode/tests/tool/main_cli.test.ts index 0722ce58..8eef0305 100644 --- a/toolbox/mdcode/tests/tool/main_cli.test.ts +++ b/toolbox/mdcode/tests/tool/main_cli.test.ts @@ -107,12 +107,16 @@ describe('kcmd: --help and --version', () => { }); test('`--help` past a command that takes arguments still succeeds', () => { - // `action-list [name]` has its own name taken out of `cli.args`, which - // therefore arrives empty here, so `process.argv` is the only place the - // verb survives to be checked. - const {code, out} = run('action-list', '--help'); + // The command has to be one that takes a positional, or the case this + // covers does not arise: cac strips a matched command's own name from + // `cli.args` and clears the match together, so what is left here is + // `['IssueCredit']` -- a word that names no command, which is exactly what + // reading `cli.args` instead of `process.argv` would misread as a + // misspelled verb. `action-list` takes no positional and leaves `cli.args` + // empty, so it cannot stand in for this. + const {code, out} = run('action-run', 'IssueCredit', '--help'); expect(code).toBe(0); - expect(out).toContain('kcmd action-list'); + expect(out).toContain('kcmd action-run'); }); test('`--help` before an unknown verb is still an error', () => { From fe72c37eeea5b19514b79509b9cd160b6be0d453 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:51:52 +0000 Subject: [PATCH 09/11] fix(mdcode): the guards a run passed over ride on its outcome `skipGuards` said, in its own documentation, that the caller knows it asked and so needs no telling. That is true of the caller and of nobody else. `describeOutcome` hands an outcome to an agent as the answer to its tool call, and an agent told `applied: true` and nothing further has been told the write met every rule the model states, which is the one thing it did not. A skipped guard was reaching the command line, which printed its own line for it, and stopping there. So it travels with the outcome. `runAction` names every guard a `skipGuards` run passed over in `warnings`, in the order the action states them, and everything downstream gets it without having to remember: the command line, `describeOutcome`, and whatever calls the runtime next. One warning for the whole skip rather than one per rule. The rules went unchecked for a single reason, and repeating it four times buries the outcome of the write under a list that says the same thing each time. Drops the pre-run banner that said this. It was printed before `runAction` had decided there would be a run at all, so a call with a missing argument announced that the write happens and then errored without opening a transaction. It is a fact about a write that was made, so it is not said until one has been. Re-records both `action-run` transcripts in the skill demo against a live Spanner database, and adds the reset the second half of that page needs to start from the state it describes. --- .../demo/semantic-model/skill/README.md | 34 ++- .../src/libts/semantic/runtime/run_action.ts | 47 +++- toolbox/mdcode/src/tool/commands.ts | 19 +- .../semantic/runtime/agent_tools.test.ts | 48 +++- .../libts/semantic/runtime/run_action.test.ts | 224 +++++++++--------- toolbox/mdcode/tests/tool/action.test.ts | 37 +-- 6 files changed, 241 insertions(+), 168 deletions(-) diff --git a/toolbox/mdcode/demo/semantic-model/skill/README.md b/toolbox/mdcode/demo/semantic-model/skill/README.md index 6ece5cc1..592cde3b 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/README.md +++ b/toolbox/mdcode/demo/semantic-model/skill/README.md @@ -306,18 +306,22 @@ check any of them. ### What the command line settles: nothing `kcmd action-run` binds the arguments, opens one transaction and applies the -statements. It settles none of the rules. It says so before it writes, naming -every guard it is passing over, so that nobody reads a committed write as a -checked one: +statements. It settles none of the rules, and the run says so itself: every +guard it passed over is named in the outcome, alongside the commit, so that +nobody reads a committed write as a checked one: ```console $ kcmd action-run IssueCredit \ --arg order=12346 --arg amount=3.00 --arg memo="Coupon applied late" Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... - NOT CHECKED: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this command settles no guard, and the write still happens -Committed at 2026-09-21T01:19:39.284744Z. +Warning: guards were not checked: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this run was told to skip them, and the write was made anyway +Committed at 2026-09-21T02:33:20.747065Z. ``` +That warning is a property of the run, not of this command line. It rides on the +outcome, so an agent handed these actions as tools by its own framework reads +the same sentence in the tool's result rather than a bare `applied: true`. + That is the useful half for curating a model, and it is genuinely useful: whether an action binds its arguments, writes the line it says it writes and leaves the store consistent is a question about SQL, and one command against @@ -332,8 +336,8 @@ precisely what `CreditWithinOrderTotal` exists to stop: $ kcmd action-run IssueCredit \ --arg order=12346 --arg amount=20.00 --arg memo="Shipping charge applied in error" Running 'IssueCredit' on projects/my-project/instances/my-instance/databases/semantic_skill_demo... - NOT CHECKED: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this command settles no guard, and the write still happens -Committed at 2026-09-21T01:19:55.792326Z. +Warning: guards were not checked: CreditWithinOrderTotal, CreditUnderReviewThreshold, CreditMemoNamesAServiceFailure, CreditIsNotSplitToAvoidReview -- this run was told to skip them, and the write was made anyway +Committed at 2026-09-21T02:33:30.039525Z. ``` ```console @@ -345,7 +349,21 @@ order_id total ``` An order with a total of **negative five dollars**, committed, with the rule -that forbids it sitting right there in the model. The banner is not a formality. +that forbids it sitting right there in the model. The warning is not a +formality. + +Those two writes really happened, so put 12346 back before going on — section 7 +runs against the seed from section 3, and expects this order at $18.00: + +```bash +gcloud spanner databases execute-sql "$DATABASE" \ + --instance="$INSTANCE" --project="$PROJECT" \ + --sql="DELETE FROM LineItem WHERE order_id = 12346 AND type = 'credit'" + +gcloud spanner databases execute-sql "$DATABASE" \ + --instance="$INSTANCE" --project="$PROJECT" \ + --sql="UPDATE Orders SET total = NUMERIC '18.00' WHERE order_id = 12346" +``` ### Where the rules are settled instead diff --git a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts index e5a7c7d3..30522ab4 100644 --- a/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts +++ b/toolbox/mdcode/src/libts/semantic/runtime/run_action.ts @@ -128,9 +128,10 @@ export interface RunActionOptions { // write happens. It exists because the refusals above are total. An author // trying a model out locally, against their own database, has no judge to // supply and would find every guarded action unrunnable; the alternative is - // deleting the guards to test the write, which is worse. The run still - // reports each guard it did not check, so a caller reading the output is - // never told the write passed rules nothing consulted. + // deleting the guards to test the write, which is worse. The outcome names + // every guard the run passed over, in `warnings`, so nothing that reads the + // outcome -- a command line, or an agent handed the result of a tool call -- + // is ever told the write passed rules nothing consulted. skipGuards?: boolean; } @@ -199,16 +200,27 @@ export async function runAction(opts: RunActionOptions): warnings.push(...asked.warnings); } } - // An unsettled rule the caller did not ask to skip is a check the model - // asked for and did not get, and a caller shown no line for it reads the - // write as having passed every rule the model states. Every one reaching - // here is advisory, because anything stricter was refused above. + // An unsettled rule is a check the model asked for and did not get, and a + // caller shown no line for it reads the write as having passed every rule the + // model states. Every one reaching here is advisory, because anything + // stricter was refused above. // - // `skipGuards` is the one caller that gets no line, because it has already - // been told: it asked for the guards to go unchecked, and it says so where - // it asked. Repeating it here would quote every rule back at a caller who - // named them all a moment ago, and bury the outcome of the write under it. - if (!opts.skipGuards) { + // This travels with the outcome rather than being left to whoever called, + // including under `skipGuards`. A caller that asked for the skip does know it + // asked, but it is not the only one reading the result: `describeOutcome` + // hands these warnings to an agent as the tool's own answer, and an agent + // told only `applied: true` has been told the write met every rule the model + // states, which is the one thing it did not. + // + // One line for the whole skip rather than one per rule. The rules were not + // checked for one reason, and repeating it four times buries the outcome of + // the write under a list that says the same thing each time. + const skipped = skippedGuards(action, opts.skipGuards); + if (skipped.length) { + warnings.push( + `guards were not checked: ${skipped.join(', ')} -- this run was ` + + `told to skip them, and the write was made anyway`); + } else { for (const {constraint, why} of unsettledGuards( model, action, opts.judge)) { warnings.push(`${citation(constraint)} was not checked: ${why}`); @@ -732,6 +744,17 @@ function citation(constraint: Constraint): string { // `unsafeToRunUnchecked` has refused everything stricter, so what turns up // here is advisory: it did not stop the write, and it still has to be // reported rather than left to read as a rule that passed. +// The guards a `skipGuards` run passed over, in the order the action names +// them. Reads the action rather than the model's constraints, because a guard +// naming a constraint that does not exist was refused before this point and a +// run that reaches here names only real ones. +function skippedGuards( + action: Action, skipGuards?: boolean): readonly string[] { + if (!skipGuards) return []; + return action.guards ?? []; +} + + function unsettledGuards(model: SemanticModel, action: Action, judge?: Judge): ReadonlyArray<{constraint: Constraint; why: string}> { const named = new Set(action.guards ?? []); diff --git a/toolbox/mdcode/src/tool/commands.ts b/toolbox/mdcode/src/tool/commands.ts index edea636d..03c0d856 100644 --- a/toolbox/mdcode/src/tool/commands.ts +++ b/toolbox/mdcode/src/tool/commands.ts @@ -1813,18 +1813,13 @@ async function runOneAction( } console.log(`Running '${name}' on ${runtime.store.name}...`); - // Said before the write rather than after it, and named rule by rule, so a - // reader watching the run knows what went unenforced while it is still - // happening. This command checks no guard at all: settling one takes a judge, - // and who that is belongs to whoever dispatches the call in earnest. Nothing - // is printed for an action that declares none, because nothing was skipped. - const guards = - (runtime.model.actions ?? []).find(a => a.name === name)?.guards ?? []; - if (guards.length) { - console.log( - ` NOT CHECKED: ${guards.join(', ')} -- this command settles ` + - `no guard, and the write still happens`); - } + // Nothing is said about the guards here. This command settles none of them -- + // that takes a judge, and who that is belongs to whoever dispatches the call + // in earnest -- but saying so before the run meant saying it before + // `runAction` had decided there would be a run at all, so a call with a + // missing argument announced that the write happens and then errored without + // opening a transaction. The run reports what it passed over in its own + // warnings, below, where it is a fact about a write that was made. const outcome = await runAction({ runtime, actionName: name, diff --git a/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts b/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts index 5bef40ae..32a3171b 100644 --- a/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/runtime/agent_tools.test.ts @@ -273,8 +273,7 @@ describe('a tool this runtime would refuse', () => { test('a handler makes a remote executor runnable again', () => { const handler = async () => ({statements: []}); const ungated = withExecutor(model, {guards: []}); - const [tool] = - actionTools({runtime: rt(ungated), handler}); + const [tool] = actionTools({runtime: rt(ungated), handler}); expect(tool.runnable).toBe(true); }); @@ -391,16 +390,16 @@ describe('what counts as runnable is the runtime\'s answer, not a copy', () => { // to fetch a judge, who fetched one and was refused again, has been // sent the wrong way. const bodyless: Constraint = { - name: 'QuantityIsPositive', - onViolation: 'reject', - }; + name: 'QuantityIsPositive', + onViolation: 'reject', + }; const [tool] = actionTools({ runtime: rt(guardedBy(bodyless, 'QuantityIsPositive')), judge: neverAsked, }); - expect(tool.runnable).toBe(false); + expect(tool.runnable).toBe(false); expect(tool.unavailable).toContain('states no rule to put to a judge'); - }); + }); }); @@ -602,6 +601,35 @@ describe('a handler does not displace an action\'s own statements', () => { }); +// A tool derived with `skipGuards` is the only way a guarded action is offered +// as callable at all, so what it says when it commits is the whole of what the +// agent learns about the rules. +describe('a skipped guard reaches the agent, not just the caller', () => { + const model = loadFixtureModel('actions_place_order.yaml'); + + test('the tool result names the guard the run passed over', async () => { + // The run used to come back `applied: true` and nothing else. The + // suppression was justified by the caller already knowing it asked for the + // skip -- true of the caller, and irrelevant to the agent reading the + // tool's result, which never saw the call that built the tool. An agent + // told only that the write applied has been told it met every rule the + // model states. + const store = new FakeStore(); + const [tool] = actionTools({ + runtime: + rt(withExecutor(model, {executor: RUNNABLE.executor}), store.client), + skipGuards: true, + }); + expect(tool.runnable).toBe(true); + const result = await tool.invoke({customer: 1, quantity: 2}); + expect(result.applied).toBe(true); + expect(result.warnings ?? []).toHaveLength(1); + expect((result.warnings ?? [])[0]) + .toContain('guards were not checked: OrderWithinCustomerCredit'); + }); +}); + + describe('entity tools', () => { const model = loadFixtureModel('actions_place_order.yaml'); const tools = entityTools({runtime: rt(model)}); @@ -849,10 +877,10 @@ describe('sorting the tools an adapter can actually offer', () => { const guarded = { ...withExecutor(model, {...RUNNABLE, guards: ['UnderReview']}), constraints: [{ - name: 'UnderReview', + name: 'UnderReview', judgment: 'The quantity must be under 25.', - onViolation: 'escalate', - }] as Constraint[], + onViolation: 'escalate', + }] as Constraint[], }; const {callable, withheld} = callableTools(modelTools({runtime: rt(guarded)})); diff --git a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts index e3062a6d..27dc0838 100644 --- a/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts +++ b/toolbox/mdcode/tests/libts/semantic/runtime/run_action.test.ts @@ -867,42 +867,42 @@ describe('a guarded action is refused, not run unchecked', () => { test( 'an action writing data a constraint reads runs, if it names no guard', - async () => { - // Credit affects Account and NonNegativeBalance reads Account.balance. - // That overlap is not what gives the rule effect over this call, and - // refusing on it would mean publishing a rule silently stopped calls - // that worked the day before -- the thing a constraint's reference - // rule exists to prevent. - const outcome = await runWith({constraints: [balance]}); - if (outcome.status !== 'committed') throw new Error(outcome.message); - }); + async () => { + // Credit affects Account and NonNegativeBalance reads Account.balance. + // That overlap is not what gives the rule effect over this call, and + // refusing on it would mean publishing a rule silently stopped calls + // that worked the day before -- the thing a constraint's reference + // rule exists to prevent. + const outcome = await runWith({constraints: [balance]}); + if (outcome.status !== 'committed') throw new Error(outcome.message); + }); test( 'an action that declares no affects runs, constraints or not', - async () => { - // `affects` describes the blast radius; it is not a switch that turns - // checking on, and its absence is not a reason to refuse. The same - // DML runs either way, down to the row it inserts. - const outcome = await runWith({ - actions: [{...credit, affects: undefined}], - constraints: [balance], - }); - if (outcome.status !== 'committed') throw new Error(outcome.message); - }); + async () => { + // `affects` describes the blast radius; it is not a switch that turns + // checking on, and its absence is not a reason to refuse. The same + // DML runs either way, down to the row it inserts. + const outcome = await runWith({ + actions: [{...credit, affects: undefined}], + constraints: [balance], + }); + if (outcome.status !== 'committed') throw new Error(outcome.message); + }); test( 'a guard is refused even when the model states no such constraint', - async () => { - // An unresolved guard fails the push, so this model should not exist. - // If one reaches the runtime anyway, the action still claims to be - // checked, and running it would still be running it unchecked. - const outcome = await runWith({ - actions: [{...credit, guards: ['NoSuchRule']}], - constraints: [], - }); - if (outcome.status !== 'error') throw new Error('expected an error'); + async () => { + // An unresolved guard fails the push, so this model should not exist. + // If one reaches the runtime anyway, the action still claims to be + // checked, and running it would still be running it unchecked. + const outcome = await runWith({ + actions: [{...credit, guards: ['NoSuchRule']}], + constraints: [], + }); + if (outcome.status !== 'error') throw new Error('expected an error'); expect(outcome.message).toContain('guarded by \'NoSuchRule\''); - }); + }); test('several guards are all named', async () => { const outcome = await runWith({ @@ -995,14 +995,14 @@ describe('a guard settled by judgment', () => { test( 'the judge is given the arguments as the caller stated them', - async () => { - // Before resolution, which is the point of asking here: 'A1' is what - // the caller said, and the key it resolves to would tell a judge - // nothing. - const judge = holds(); - await runWith([justified], judge); - expect(judge.asked[0].arguments).toEqual({account: 'A1', amount: 100}); - }); + async () => { + // Before resolution, which is the point of asking here: 'A1' is what + // the caller said, and the key it resolves to would tell a judge + // nothing. + const judge = holds(); + await runWith([justified], judge); + expect(judge.asked[0].arguments).toEqual({account: 'A1', amount: 100}); + }); test( 'a verdict that does not hold refuses before anything opens', @@ -1021,18 +1021,18 @@ describe('a guard settled by judgment', () => { test( 'the refusal carries the author words and the judge reason', async () => { - const outcome = await runWith([justified], doesNot()); - if (outcome.status !== 'error') throw new Error('expected an error'); + const outcome = await runWith([justified], doesNot()); + if (outcome.status !== 'error') throw new Error('expected an error'); expect(outcome.message).toContain('\'CreditIsJustified\''); - expect(outcome.message) - .toContain('The memo must name a specific service failure.'); - expect(outcome.message).toContain('A credit needs a stated reason.'); + expect(outcome.message) + .toContain('The memo must name a specific service failure.'); + expect(outcome.message).toContain('A credit needs a stated reason.'); expect(outcome.message).toContain('The memo names no service failure.'); - // A caller told a transaction rolled back goes looking for a write - // that never reached the store. - expect(outcome.message).toContain('No transaction was opened'); - expect(outcome.message).not.toContain('rolled back'); - }); + // A caller told a transaction rolled back goes looking for a write + // that never reached the store. + expect(outcome.message).toContain('No transaction was opened'); + expect(outcome.message).not.toContain('rolled back'); + }); test('an escalation says an approver may allow it', async () => { // `escalate` states that an approver exists. Nothing here is one, and a @@ -1134,33 +1134,33 @@ describe('a guard settled by judgment', () => { test( 'a judgment with no words refuses rather than asking about nothing', - async () => { - // An empty rule put to a judge comes back "not enough to tell", so - // every call would be refused and the citation could not quote what - // was broken. - const judge = holds(); - const blank: Constraint = {...justified, judgment: ' '}; - const outcome = await runWith([blank], judge); - if (outcome.status !== 'error') throw new Error('expected an error'); - expect(judge.asked).toHaveLength(0); - expect(outcome.message).toContain('CreditIsJustified'); - expect(outcome.message) - .toContain( + async () => { + // An empty rule put to a judge comes back "not enough to tell", so + // every call would be refused and the citation could not quote what + // was broken. + const judge = holds(); + const blank: Constraint = {...justified, judgment: ' '}; + const outcome = await runWith([blank], judge); + if (outcome.status !== 'error') throw new Error('expected an error'); + expect(judge.asked).toHaveLength(0); + expect(outcome.message).toContain('CreditIsJustified'); + expect(outcome.message) + .toContain( `'CreditIsJustified', which states no rule to put to a judge.`); - }); + }); test( 'an advisory judgment with no words is reported, never asked', - async () => { - // An advisory guard is never refused, so this is the one path on - // which an empty rule could still have reached a judge. - const judge = holds(); - const blank: Constraint = {...advisory, judgment: ''}; - const outcome = await runWith([blank], judge); - if (outcome.status !== 'committed') throw new Error(outcome.message); - expect(judge.asked).toHaveLength(0); - expect(outcome.warnings?.[0]).toContain('states no words'); - }); + async () => { + // An advisory guard is never refused, so this is the one path on + // which an empty rule could still have reached a judge. + const judge = holds(); + const blank: Constraint = {...advisory, judgment: ''}; + const outcome = await runWith([blank], judge); + if (outcome.status !== 'committed') throw new Error(outcome.message); + expect(judge.asked).toHaveLength(0); + expect(outcome.warnings?.[0]).toContain('states no words'); + }); test('a verdict missing its answer is reported, not thrown', async () => { // `Judge` is a seam a caller implements, so a verdict can arrive without @@ -1176,13 +1176,13 @@ describe('a guard settled by judgment', () => { test( 'a verdict that does not hold and states no reason still refuses', - async () => { - const terse = new ScriptedJudge( - {holds: false, reason: undefined as unknown as string}); - const outcome = await runWith([justified], terse); - if (outcome.status !== 'error') throw new Error('expected an error'); - expect(outcome.message).toContain('does not hold for this call'); - }); + async () => { + const terse = new ScriptedJudge( + {holds: false, reason: undefined as unknown as string}); + const outcome = await runWith([justified], terse); + if (outcome.status !== 'error') throw new Error('expected an error'); + expect(outcome.message).toContain('does not hold for this call'); + }); test( 'an advisory guard nobody could ask about is reported, not dropped', @@ -1201,16 +1201,16 @@ describe('a guard settled by judgment', () => { test( 'a guard stating no rule is refused whatever judge is given', - async () => { + async () => { // Supplying a judge does not give a bodyless constraint something to // put to it, and a message about a missing judge would send a caller // who already supplied one the wrong way. - const outcome = await runWith( + const outcome = await runWith( [{name: 'UnderCeiling', onViolation: 'reject'}], holds()); - if (outcome.status !== 'error') throw new Error('expected an error'); + if (outcome.status !== 'error') throw new Error('expected an error'); expect(outcome.message).toContain('states no rule to put to a judge'); expect(outcome.message).not.toContain('was given no judge to ask'); - }); + }); test('what a run does and what a tool advertises agree', async () => { // agent_tools.ts asks this before offering the action. A tool advertised @@ -1238,24 +1238,32 @@ describe('a guard settled by judgment', () => { .toBeNull(); }); - test('skipGuards reports no rule as unchecked', async () => { - // Without it, an advisory rule nobody could ask about is warned about -- - // the test above this one. With it, the caller has already been told, by - // itself: it named every one of these rules when it asked for them to go - // unchecked. Saying it again here, rule by rule with each judgment quoted - // back, buries what happened to the write under a list the caller wrote. - const fake = fakeStore(); - const outcome = await act({ - model: guarding([advisory]), - actionName: 'Credit', - args: {account: 'A1', amount: 100}, - client: fake.client, - skipGuards: true, - }); - if (outcome.status !== 'committed') throw new Error(outcome.message); - expect(fake.committed).toBe(true); - expect(outcome.warnings ?? []).toEqual([]); - }); + test( + 'skipGuards names every guard it passed over, on the outcome', + async () => { + // One line for the whole skip rather than one per rule, since they went + // unchecked for one reason -- but a line, not silence. This used to + // return no warnings at all, on the reasoning that the caller asking + // for the skip already knew. The caller is not the only reader: + // `describeOutcome` hands these warnings to an agent as a tool's own + // answer, and an agent told only `applied: true` has been told the + // write met every rule the model states. + const fake = fakeStore(); + const outcome = await act({ + model: guarding([advisory]), + actionName: 'Credit', + args: {account: 'A1', amount: 100}, + client: fake.client, + skipGuards: true, + }); + if (outcome.status !== 'committed') throw new Error(outcome.message); + expect(fake.committed).toBe(true); + expect(outcome.warnings ?? []).toHaveLength(1); + expect((outcome.warnings ?? [])[0]) + .toContain('guards were not checked: CreditIsJustified'); + expect((outcome.warnings ?? [])[0]) + .toContain('the write was made anyway'); + }); test('skipGuards asks no judge, even when one is handed over', async () => { // `skipGuards` means nobody is asked. A caller that passed both used to @@ -1632,16 +1640,16 @@ describe('a date or a timestamp argument', () => { test( 'a date in some other order is refused, naming the parameter', - async () => { - // '03/04/2026' is the fourth of March to one reader and the third of - // April to another, so it is not a date this can accept. - const {fake, outcome} = scheduling({day: '03/04/2026'}); - const result = await outcome; - if (result.status !== 'error') throw new Error('expected an error'); + async () => { + // '03/04/2026' is the fourth of March to one reader and the third of + // April to another, so it is not a date this can accept. + const {fake, outcome} = scheduling({day: '03/04/2026'}); + const result = await outcome; + if (result.status !== 'error') throw new Error('expected an error'); expect(result.message).toContain('\'day\' is a Date'); - expect(result.message).toContain('YYYY-MM-DD'); - expect(fake.committed).toBe(false); - }); + expect(result.message).toContain('YYYY-MM-DD'); + expect(fake.committed).toBe(false); + }); test('a date with the right shape and no such day is refused', async () => { const {outcome} = scheduling({day: '2026-02-30'}); diff --git a/toolbox/mdcode/tests/tool/action.test.ts b/toolbox/mdcode/tests/tool/action.test.ts index 7b3edba4..1438964f 100644 --- a/toolbox/mdcode/tests/tool/action.test.ts +++ b/toolbox/mdcode/tests/tool/action.test.ts @@ -537,22 +537,18 @@ describe('kcmd action-run: what it will not send to a store', () => { .toContain('which runs outside this transaction'); }); - test('names the guards it is not going to check', async () => { - // The command checks no guard, and the one thing it must not do is let - // that pass unremarked: a reader watching a write land is owed the list of - // rules that did not stand between them and it, by name, before it lands. - writeWorkspace(); - await actionRun('IssueCredit', {arg: ['order=12345', 'amount=30']}); - const out = logs.join('\n'); - expect(out).toContain('NOT CHECKED: CreditIsPositive'); - expect(out).toContain('this command settles no guard'); - // After the run banner, so the reader has been told which database is - // about to be written to before being told what will go unchecked on it. - expect(out.indexOf('NOT CHECKED')) - .toBeGreaterThan(out.indexOf('Running \'IssueCredit\'')); - expect(out).toContain( - 'Running \'IssueCredit\' on projects/acme-ops/instances/prod/databases/commerce'); - }); + test('says nothing about guards until there is a write to say it about', + async () => { + // This used to be announced before the run, which meant announcing + // that "the write still happens" to a call that then failed to bind + // and wrote nothing at all. It is a fact about a write that was made, + // so it is not said until one has been. + writeWorkspace(); + await actionRun('IssueCredit', {arg: 'order=12345'}); + const out = logs.join('\n'); + expect(out).toContain('was not given a value'); + expect(out).not.toContain('guards were not checked'); + }); test('says nothing about guards for an action that declares none', async () => { // An action with no guards skipped no check, so a line saying one went @@ -560,7 +556,7 @@ describe('kcmd action-run: what it will not send to a store', () => { // caveat nobody reads on the run that needed it. writeWorkspace(INHERITS); await actionRun('Touch', {arg: 'who=Alice'}); - expect(logs.join('\n')).not.toContain('NOT CHECKED'); + expect(logs.join('\n')).not.toContain('not checked'); }); }); @@ -814,7 +810,12 @@ describe('kcmd action-run: the guards go unchecked', () => { expect(code).toBe(0); expect(out).not.toContain('is guarded by \'CreditIsPositive\''); expect(asked.length).toBeGreaterThan(0); - expect(out).toContain('NOT CHECKED: CreditIsPositive'); + // Named rule by rule, because a reader watching a write land is owed the + // list of rules that did not stand between them and it. + expect(out).toContain('guards were not checked: CreditIsPositive'); + expect(out).toContain('the write was made anyway'); + expect(out).toContain( + 'Running \'IssueCredit\' on projects/acme-ops/instances/prod/databases/commerce'); }); test( From 98eb1629974af044627e2e16760acacfa30a3df7 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:52:34 +0000 Subject: [PATCH 10/11] docs(mdcode): the guides stop saying this repository settles a guard Three places said the commerce demo is where the guards are "actually settled", "live", against the same model. That was true when the generated command line was `kcmd action run --judge --judge-reads-store`. Those flags are gone, the demo's executor is `kcmd action-run`, and it settles nothing. The recorded transcripts are still there and still worth reading -- they are the only place a guard is shown holding an agent to something -- but they are a record of what hiring a judge takes, not of what this repository does. Also names the two verbs where the text still said `kcmd action`. --- toolbox/mdcode/docs/semantic-model/actions.md | 14 ++++++++------ toolbox/mdcode/docs/semantic-model/model_spec.md | 6 +++--- toolbox/mdcode/docs/semantic-model/reference.md | 10 +++++----- 3 files changed, 16 insertions(+), 14 deletions(-) diff --git a/toolbox/mdcode/docs/semantic-model/actions.md b/toolbox/mdcode/docs/semantic-model/actions.md index 6f2aa05d..80bc396e 100644 --- a/toolbox/mdcode/docs/semantic-model/actions.md +++ b/toolbox/mdcode/docs/semantic-model/actions.md @@ -1124,8 +1124,10 @@ names the guards it did not check; it is for finding out whether your statements do what you meant, not for finding out whether your rules hold. The two demands pull apart: a judge costs a model call per guard and credentials to reach one, and an author checking a `WHERE` clause should not have to stand either up. The -[commerce demo](../../demo/semantic-model/skill/README.md) is where the guards -are actually settled, against the same model, live. +[commerce demo](../../demo/semantic-model/skill/README.md) is where what it +takes to settle them is shown, against the same model — in runs recorded while a +command line still hired a judge, because nothing in this repository hires one +today. The rules below run against the commerce model under `demo/semantic-model/skill` — the [credit policy worked through earlier](#a-credit-policy-worked-through), @@ -1141,7 +1143,7 @@ declares `warn` on the memo rule; the three outputs below come from setting that one field to each of its values in turn, so a single rule shows all three branches. -> These three were recorded through `kcmd action-run`, back when it took a judge +> These three were recorded through `kcmd action run`, back when it took a judge > and could give that judge the store to read — which is why each of them shows > the judge reading `Orders`. Neither is a command-line flag any more, for the > reason above. They are @@ -1592,9 +1594,9 @@ withheld that would have worked is never tried. `kcmd agent-tools` prints these tools; `modelTools` returns them. Both take a **semantic runtime**: one model paired with the store your profile binds it to. -`createSemanticRuntimes` assembles them the way `kcmd action` does, so your -agent reads the model the CLI reads, under the same profile, with the same merge -and the same warnings: +`createSemanticRuntimes` assembles them the way `kcmd action-list` and +`kcmd action-run` do, so your agent reads the model the CLI reads, under the +same profile, with the same merge and the same warnings: ```ts import {createSemanticRuntimes} from './src/libts/semantic/runtime/runtime'; diff --git a/toolbox/mdcode/docs/semantic-model/model_spec.md b/toolbox/mdcode/docs/semantic-model/model_spec.md index a9619e60..8d8d5e57 100644 --- a/toolbox/mdcode/docs/semantic-model/model_spec.md +++ b/toolbox/mdcode/docs/semantic-model/model_spec.md @@ -540,9 +540,9 @@ reads the document ([§6](#6-the-extension-mechanism)). refused rather than run past its rules. No command line here does that: `kcmd action-run` checks no guard, because who settles one belongs to whoever dispatches the call in earnest. The [commerce - demo](../../demo/semantic-model/skill/README.md) is where it is shown. Rules - in - [Reference → Validation](reference.md#validation). + demo](../../demo/semantic-model/skill/README.md) shows what hiring a judge + takes, in recorded runs — nothing in this repository embeds one today. + Rules in [Reference → Validation](reference.md#validation). A constraint says two things about a violation, under two separate keys. **`on_violation`** is what a violation does to the write that tripped it: diff --git a/toolbox/mdcode/docs/semantic-model/reference.md b/toolbox/mdcode/docs/semantic-model/reference.md index 8a583a86..ade8146a 100644 --- a/toolbox/mdcode/docs/semantic-model/reference.md +++ b/toolbox/mdcode/docs/semantic-model/reference.md @@ -123,11 +123,11 @@ watching one land sees what did not stand between them and it. Running an action is not what `kcmd` is for — the command exists so that an author can exercise a model they are curating, and find out whether the -statements do what they meant, without first standing up an agent. To see the -guards actually settled, run the model through something that embeds the -runtime: the [commerce demo](../../demo/semantic-model/skill/README.md) hires -a judge, gives it the model's tables to read, and refuses the call when a rule -does not hold. +statements do what they meant, without first standing up an agent. Settling the +guards takes something that embeds the runtime and hires a judge for it. Nothing +in this repository does today — the [commerce +demo](../../demo/semantic-model/skill/README.md) shows what it takes and keeps +recorded runs of it, but its own executor is this command, which settles none. What this command still refuses is a model that is wrong about its own rules: a guard naming a constraint the model does not declare, or one whose `judgment` From ff19a5511126cfac85840380e73ea8179e245f71 Mon Sep 17 00:00:00 2001 From: Bei Li Date: Mon, 21 Sep 2026 02:57:31 +0000 Subject: [PATCH 11/11] docs(mdcode): the skill demo stops calling the warning a banner Two sentences still pointed at the line that was printed before the run. There is no such line; the fact is on the outcome now, and the two runs above them show it as a warning beside the commit. --- toolbox/mdcode/demo/semantic-model/skill/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/toolbox/mdcode/demo/semantic-model/skill/README.md b/toolbox/mdcode/demo/semantic-model/skill/README.md index 592cde3b..4f47c1f5 100644 --- a/toolbox/mdcode/demo/semantic-model/skill/README.md +++ b/toolbox/mdcode/demo/semantic-model/skill/README.md @@ -381,7 +381,7 @@ Given one, the runtime puts each guard to it before the transaction opens and routes the verdict by `on_violation`. Given none, it refuses the call rather than running a write the model says must be checked. `kcmd action-run` is the one caller that opts out of both: it asks for no judge and refuses nothing, -which is why it prints the banner instead. +which is why its runs come back carrying that warning instead. **No command line in this repository settles a guard.** Until one does, the transcripts below are the record of what settling them looked like. @@ -442,8 +442,9 @@ Error: Action 'IssueCredit' is guarded by 'CreditWithinOrderTotal', 'CreditUnder ``` That refusal is still the runtime's behaviour for any caller that supplies no -judge. What changed is that the command line no longer asks: it declares the -guards unchecked and proceeds, which is the banner in the two runs above. +judge. What changed is that the command line no longer asks: it proceeds, and +the run reports the guards it passed over, which is the warning in the two runs +above. And the `reject` consequence, the one an approver cannot wave through: