demo(mdcode): the commerce demo installs a generated skill, not an agent - #453
Merged
Merged
Conversation
`kcmd skills-generate` derives an Agent Skill from a semantic model, so the demo's hand-written ADK agent was the one part of it that had to be maintained by hand and was true of exactly one framework. Delete it and make the demo about the thing that replaces it: generate the skill, install it into a harness, hand the harness a support request in English. demo/semantic-model/agent/ becomes demo/semantic-model/skill/. agent.ts and its package.json go, and with them the @google/adk dependency the demo installed on the side. The model, the two binding profiles and the two schema files are unchanged except for the placeholder database name, which follows the rename. The README is rewritten around the new spine, and every listing in it comes from a run today against the seed it publishes: the four command-line cases that show what a judge with and without store access settles, what happens with no judge, and the reject consequence; then four Claude Code runs against the installed skill reaching escalate, applied, applied-with-warning, and a refusal to break a rule it was told to break. A fifth outcome turned up by accident -- the judge returned non-JSON and the runtime refused rather than run unchecked -- and is transcribed as it happened. It also states three limits it did not before: the AlloyDB profile generates no read path, so an agent under it cannot find a key; kcmd takes its scope from the working directory, so the skill's command line only works from the scope directory; and a judgment that returns well-formed nonsense is not caught. docs/semantic-model/actions.md loses its claim that the directory holds "an agent built this way" with a 72-line agent.ts, and the agent_tools.ts header loses its pointer to that file.
… read "Finding a record" gave a Spanner-backed skill the gcloud line to read the store and stopped there, so a harness given a customer's name and an action wanting a key had a command and no schema. Driving Claude Code against the commerce demo, it spent two turns querying INFORMATION_SCHEMA before it could read a row. Emit the tables under that snippet, reusing readableEntities() -- the same listing the reading judge is given, so the two cannot drift. Rendering it as `customer_id is Customer.customerId` was read backwards on the next run: the agent wrote `o.customerId`, got INVALID_ARGUMENT: Name customerId not found inside o, and fell back to INFORMATION_SCHEMA anyway. The rest of the skill is written in the model's names and a statement has to contain the store's, so a pairing that does not say which side goes in the SQL is worse than ambiguous. The physical name is labelled `column` and the sentence above the block says the quoted name is the one to write. With that, the next run read two rows and issued the action with no schema discovery at all. Field descriptions come along, because a coded column's description carries what its values are -- `item, tax, fee, or credit` -- and an agent that guesses filters on a value the column never holds and gets an empty answer back, which reads like the record not existing. The section is Spanner-only, as the snippet above it already was. A model bound to AlloyDB still gets no read path; docs/semantic-model/skills.md now says so under what it doesn't generate yet, and its cross-profile diff is refreshed from a live run.
…output
Parameter projection changed both the demo model and what the generator
emits, so the README's quoted strings and its transcripts no longer matched
a run. Re-ran every command in the guide and replaced them with what came
back.
What moved:
* The model excerpt now shows `{name: order, concept: Order, field:
orderId}` and the `orderId` description it projects, and the prose says
the parameter is projected rather than typed as an entity.
* `kcmd action list` prints `order (Integer from Order.orderId)` and
`--arg order=<Integer>`.
* SKILL.md's "Finding a record" opens on what a wrong key costs rather
than on an argument that accepted text, and the schema block carries
`order_id`'s description. Both quoted here and in docs/skills.md.
* All four harness transcripts are new runs. Three land where they did
before; the fourth no longer attempts the honest credit after refusing
to split, and the section now says plainly that refusing to structure
around a control is the harness's own behaviour and only the
command-line case in section 4 is a guarantee.
* The judge-without-a-store case answered that it could not tell the
order total rather than inventing one, as it had before. Both outcomes
are recorded, because which one you get is not predictable.
* The non-JSON judge answer did not recur; it is kept as a note about an
earlier run rather than presented as part of this transcript.
Goldens pick up main's fixture change to `c_custkey`. `kcmd
skills-generate` is in main now, so the limit saying it is on a branch is
gone.
The guide claimed a skill installs into any harness on the strength of the file format alone. Copy the same generated directory into the place Gemini CLI looks and run two of section 7's requests through it: one refused at the $25 ceiling, one committed a $8 credit against order 12347. No adapter, no tool registration, no prompt beyond the request. Record what was actually run rather than what the layout implies, and say in Limits that two harnesses were exercised and the rest of the install table is from each tool's documentation.
libei
marked this pull request as ready for review
September 20, 2026 22:12
libei
added a commit
to libei/knowledge-catalog
that referenced
this pull request
Sep 21, 2026
GoogleCloudPlatform#453 shipped this demo hours before the trim removed the flags it runs on. Twelve command lines and nineteen judge flags in one README, and two sections whose entire subject was a flag pair that no longer exists. Section 4 is rewritten around what the command line actually does now, with two live runs: a credit that commits under the NOT CHECKED banner, and the same command asked for twenty dollars against an eighteen dollar order -- the case CreditWithinOrderTotal exists to stop -- committing and leaving the order at a total of negative five dollars. That is the cost of the trim, shown rather than asserted, and it is a better argument for where guards belong than the prose it replaces. The guarded transcripts are kept and marked. They are the only record of one action carrying four rules through all three on_violation outcomes, and they are pasted as they were printed, two-word verb and all, because a transcript nobody ever saw is not evidence. The agent runs in sections 7 through 9 get the same treatment: those calls were refused because the generated command line carried a judge at the time, and the skill as it generates today would commit them. Three claims the demo made are now false and say so instead of quietly standing: that the rules are enforced underneath whatever reads the skill, that an agent ignoring the description cannot get past them, and that the split is impossible because of the runtime rather than because the harness happened to behave. Limits leads with the honest version -- no caller in this repository settles a guard today, because the CLI declares them unchecked and the one application that hired a judge was the ADK agent GoogleCloudPlatform#453 removed. The library is untouched and still tested; what is missing is something wired to it.
This was referenced Sep 21, 2026
libei
added a commit
that referenced
this pull request
Sep 21, 2026
…454) `kcmd` is for curating a semantic model, not for dispatching its actions. The action command lines had drifted into the second job. Every flag that made `kcmd` behave like a runtime is gone -- `--judge`, `--judge-location`, `--judge-reads-store` -- while `Judge`, `GeminiJudge`, `modelJudgeStore` and `runAction`'s `judge` option are untouched and still tested. Only the command lines stopped pretending to be the application that hires them. What is left is two flags over four command-flag pairs, with no combinations: `action-list` (`--profile`), `action-run` (`--arg`, `--profile`), `agent-tools` (`--profile`). `action <command>` and `agent <command>` become `action-list`, `action-run` and `agent-tools`. cac cannot match a nested subcommand, so the space form was a positional dispatcher that accepted either verb's flags and printed their union under `--help`. `action-list` takes no positional at all; it lists everything, and says so. `action-list --store` becomes `kcmd profiles --print-store`. It was a mode flag wearing a target flag's name -- it suppressed the listing entirely and only printed, while nothing on any `kcmd` command line ever selects a store. `profiles` gains `--profile [name]` so what `action-list --store --profile alloydb` did survives, and gains a `reference.md` section it never had. `action-run` binds the arguments, opens one transaction and applies the statements. It settles no guard, and the outcome says so: `runAction` names every guard a `skipGuards` run passed over in `ActionOutcome.warnings`, so the command line, `describeOutcome` and whatever calls the runtime next all get it without having to remember. An agent handed the action as a tool reads that sentence in the result rather than a bare `applied: true`. `agent-tools` derives with `skipGuards`, so a guarded action is offered rather than marked `[NOT RUNNABLE]` -- who settles a rule belongs to whoever dispatches the call, which a listing cannot know. A tool is still withheld for what no judge would repair: no executor under the binding, an executor no handler runs, or a guard naming a rule the model never declares. The skills generator emits the flat verbs and no judge flag, and keys its "this command line settles no guard" caveat on the model rather than on the action it picked for the example. Docs: `actions.md` §7, `reference.md` and `model_spec.md` stop saying this repository settles a guard -- true only while the demo's command line carried `--judge --judge-reads-store`. The skill demo's section 4 is rewritten against a live store, where a $20.00 credit against an $18.00 order commits and leaves the order at -$5.00 with `CreditWithinOrderTotal` sitting in the model. The transcripts recorded while a judge was hired are kept and marked historical. Between this and #453 no caller in the repository settles a guard. The judge, the `on_violation` routing and the constraint library are all still there and still tested; nothing is wired to them. That is the demo's top limit. Replaces #448, which could not be updated in place: a rebased head cannot be published over an existing PR without a force-push.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
demo/semantic-model/agent/demo was built around a 72-line ADK agent. PR#451 made the agent the wrong payload:
kcmd skills-generateturns a semanticmodel into an Agent Skill, and a skill installs into
any harness that reads the format. So the demo's spine is now generate →
install → hand the harness a support request in English, and there is no agent
in the directory at all.
What changed
demo/semantic-model/agent/→demo/semantic-model/skill/.agent.ts,package.jsonand the@google/adkdependency are gone, along with everyreference that cited them as proof of a running agent. The model, the two
binding profiles and the two schemas are unchanged apart from the placeholder
database name.
The README is rewritten end to end. It walks the model and its four
word-stated rules, creates the store, shows what
kcmdreads out of thescope, generates the skill, installs it, and then runs four support requests
through a headless coding agent with nothing but the installed skill — no
order numbers, no table names, no SQL. One is refused at the self-service
ceiling, one lands, one lands carrying an advisory warning, and one is an
instruction to split a credit to dodge review. Section 9 diffs the skill
generated under
spanneragainst the one generated underalloydb: thereference page is byte-identical, and
SKILL.mddiffers only in thedeployment section and the Spanner-only read path.
Every command line, every transcript and every diff in the guide is a live
run against a real Spanner database, not a plausible reconstruction.
Two harnesses, not one claim about a format
"A skill installs into any harness" was asserted from the layout. Section 8
now checks it: the same generated directory, copied unchanged into the path
Gemini CLI reads, with no adapter, no tool registration and no prompt beyond
the request. Gemini CLI discovered the skill on its own —
activate_skillfirst, then the reference page — wrote its own
SELECTs from the schemablock, and walked a customer name and "Labor Day" down to one order. The
first request was refused at the $25 ceiling with nothing written; the second
committed an $8 credit and took order 12347 from $175.00 to $167.00.
The rules held in both harnesses because they are enforced in the runtime
under the action, not by whichever agent happens to be reading.
One generator fix
A generated skill offers writes, and a request names a person and a day where
an action wants a key — so SKILL.md tells an agent to read the store. It gave
it a
gcloudline and nothing else. An agent that follows that has to inventthe table and column names, and one did: it wrote
o.customerId, gotName customerId not found inside o, and fell back toINFORMATION_SCHEMA.So the read section now names the tables, bound through the same profile the
rest of the skill is generated from. The physical name is labelled
columnand the model's name follows it for cross-reference, because the first
rendering paired them without saying which was which and was read backwards.
Field descriptions come along, since a coded column's description is what
carries
item, tax, fee, or credit— and an agent that guesses those filterson a value the column never holds and gets an empty answer back, which reads
like the record not existing.
This is Spanner-only, which the guide's Limits section states: under
alloydbthe skill describes the action and its rules correctly and theaction still runs, but an agent handed a customer name has nothing telling it
how to reach a key.
What the guide says it does not show
The harness refused the split before running a single command, which is the
harness's own judgment and not the model's. The guide says so, and points at
the command-line case where the same rule refuses the same call in the
runtime with nothing written. Only that one is a guarantee.
Two judgments also misbehaved while this was being written: one returned an
answer that was not JSON, which fails closed; one invented an order total
when it had no read. Both are recorded.
Two harnesses were run, not every harness. The remaining rows in the install
table come from each tool's own documentation.