Repository navigation
fix: describe tool schema fields and show one request payload - #7
Merged
DevMortimer merged 2 commits intoSep 22, 2026
Merged
Conversation
Models read `parameters` and `promptGuidelines` before their first call, and neither said what the payload must look like: the emitted schema carried no `description` on `state`, `questions`, `type`, `instructions`, or `criteria`, and the canonical sample only reached `/typesafe test` and the playground. The syntax guide lived in the validation error, so a first-call failure was the only way to learn the shape — `questions` authored as an array failed every time. Descriptions on the fields the agent authors, plus the existing sample inlined as a single-line guideline. Accepted input is unchanged. Refs DevMortimer#6
The guideline test only checked that some line contained `"state":`; it now parses the payload out of the guideline and runs it through parseEvaluationRequest, so the example the model copies is one the schema accepts. The sample's score rubric said in sentences what three words say, and every session pays for that line on every tool listing, so it is short now and the test caps its length. The schema test walks the serialized schema instead of pinning TypeBox's patternProperties/anyOf layout. `entry` takes the description directly, and the questions description drops the em-dash the rest of the file avoids. Version 0.6.2 with a changelog line. Refs DevMortimer#6, DevMortimer#7.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Fixes #6.
Models inspecting
typesafe_evaluateconsistently fail their first call because neither the JSON Schema nor the prompt guidelines showed what a valid payload looks like:evaluationSchemainsrc/schema.tshad 0 property-leveldescriptionfields onstate,questions,type,instructions, andcriteria. LLMs had to infer structural constraints from rawanyOfandpatternPropertiesunions./typesafe testand the playground, while prompt guidelines contained usage policies but no syntax example.questions.This PR implements the suggested scope discussed in #6:
descriptionannotations to agent-authored schema fields:state,questions,typeliterals,instructions,criteria(per question type), andmodel.promptGuidelinesreusing the existing module-levelsample.Verification
npm ciandnpm run checkpass: build (tsc -p tsconfig.build.json) and typecheck (tsc --noEmit) succeed cleanly.tests/schema.test.tsverifying all authored schema fields and question variants carry descriptions.tests/extension.test.tsverifyingpromptGuidelinescontains the single-line payload example without newlines.0o600vs NTFS0o666occur identically before and after this change; CI runs on Linux and macOS where they pass).Live testing: Not applicable. Transport and response validation are untouched.
Compatibility and release notes