What
find-and-kick as a file passes its own gate. The same task as a --goal does not.
| form |
verdict, 6 runs |
outcome |
quackd run find-and-kick |
feasible 6 |
success 6 |
quackd run --goal "Find the ball and kick it." |
uncertain 5, feasible 1 |
aborted 5 |
Same robot, seed, model and words. --goal is the first thing in run --help.
Which difference does it
duck_from_goal differs from that duck file four ways at once, so I rebuilt the --goal
contract as a .duck file and changed one thing at a time. The reconstruction is exact: its
system prompt diffs to zero lines against a real --goal run, and it reproduces the same
5/1 split.
| variant |
verdicts, 6 runs |
0, reconstruction of the --goal contract |
uncertain 5, feasible 1 |
| 1, only the persona changed to "Determined and cheerful" |
uncertain 5, feasible 1 |
| 2, only the success criterion made concrete |
uncertain 5, feasible 1 |
| 3, only the allowlist narrowed to six, duck's verb names |
feasible 5, uncertain 1 |
| 6, only the allowlist narrowed to six, goal's own verb names |
feasible 6 |
| 4, only the five step strategy added |
feasible 3, uncertain 3 |
5, the shipped find-and-kick file |
feasible 6 |
The allowlist decides it. Persona and success criterion do nothing. Variant 6 exists
because variant 3 renamed two verbs as a side effect, and it rules naming out: the same six
verbs under the goal's own names give 6 of 6.
Why, from the pilot's own reasons
With fifteen verbs the pilot answers about the world:
"Since the camera currently detects nothing, I cannot determine feasibility yet."
"I cannot assess whether this body can do the task without first observing the position
and distance of the ball."
With six it answers about the body and the list:
"The robot can rotate to scan for the ball using the search_scan verb, move toward it
using go_to, and kick it using the kick verb. No payload, reach, or height requirements
exceed the robot's limits."
A short allowlist hands the pilot a route it can check against a datasheet. A long one
leaves the route open, so it looks at the world first and files "I have not looked yet" as
uncertain, which then asks a human and, with nobody there, aborts the run.
observe is in BEFORE_VERDICT precisely so a pilot can look before judging. On the long
list it says it cannot judge without observing instead of calling observe.
Suggestion, not a patch
I have not sent code for this because the fix is a judgement call, not a line. Candidates:
say in the gate's own prompt paragraph that an unexamined world is a reason to call
observe rather than to answer uncertain; or narrow what --goal allows by default. The
measurement says the second would work and the first is untested.
What this does not say
One model (Qwen3-32B-AWQ on vLLM), microduck:sim2d, one seed, six runs a cell, thinking
off. 48 repeated runs elsewhere in this measurement put the noise floor near plus or minus 2
per six run cell, and the swings here are 5.
Measurement and data from the same session as #24.
What
find-and-kickas a file passes its own gate. The same task as a--goaldoes not.quackd run find-and-kickquackd run --goal "Find the ball and kick it."Same robot, seed, model and words.
--goalis the first thing inrun --help.Which difference does it
duck_from_goaldiffers from that duck file four ways at once, so I rebuilt the--goalcontract as a
.duckfile and changed one thing at a time. The reconstruction is exact: itssystem prompt diffs to zero lines against a real
--goalrun, and it reproduces the same5/1 split.
--goalcontractfind-and-kickfileThe allowlist decides it. Persona and success criterion do nothing. Variant 6 exists
because variant 3 renamed two verbs as a side effect, and it rules naming out: the same six
verbs under the goal's own names give 6 of 6.
Why, from the pilot's own reasons
With fifteen verbs the pilot answers about the world:
With six it answers about the body and the list:
A short allowlist hands the pilot a route it can check against a datasheet. A long one
leaves the route open, so it looks at the world first and files "I have not looked yet" as
uncertain, which then asks a human and, with nobody there, aborts the run.observeis inBEFORE_VERDICTprecisely so a pilot can look before judging. On the longlist it says it cannot judge without observing instead of calling
observe.Suggestion, not a patch
I have not sent code for this because the fix is a judgement call, not a line. Candidates:
say in the gate's own prompt paragraph that an unexamined world is a reason to call
observerather than to answeruncertain; or narrow what--goalallows by default. Themeasurement says the second would work and the first is untested.
What this does not say
One model (Qwen3-32B-AWQ on vLLM),
microduck:sim2d, one seed, six runs a cell, thinkingoff. 48 repeated runs elsewhere in this measurement put the noise floor near plus or minus 2
per six run cell, and the swings here are 5.
Measurement and data from the same session as #24.