Skip to content

Add validation support for llm_generation steps - #12

Merged
wyrdbound merged 8 commits into
mainfrom
addValidationToLLMGenerationStep
Aug 13, 2025
Merged

Add validation support for llm_generation steps#12
wyrdbound merged 8 commits into
mainfrom
addValidationToLLMGenerationStep

Conversation

@wyrdbound

Copy link
Copy Markdown
Owner

When LLMs are invoked, their outputs are often error prone and ill-formatted. With validation blocks, we will now be able to ask the engine to validate JSON output is parseable and/or abides by a given schema:

Example:

  - id: determine_saving_throw_type
    name: "Determine Saving Throw Type"
    type: llm_generation
    prompt_id: determine_saving_throw_type
    prompt_data:
      context_summary: "{{ inputs.context_summary }}"
      saving_throw_ability: "{{ variables.saving_throw_ability }}"
    validation:
      type: "json_schema"
      schema:
        type: "object"
        properties:
          type:
            type: "string"
            enum: ["basic", "advantage", "disadvantage"]
          reason:
            type: "string"
            minLength: 10
        required: ["type", "reason"]
    actions:
      - set_value:
          path: "variables.saving_throw_type"
          value: "{{ llm_result.type }}"
      - log_event:
          type: "llm_result"
          data: "Saving Throw Type Determined: {{ variables.saving_throw_type }}, Reason: {{ llm_result.reason }}"

@wyrdbound
wyrdbound merged commit 9f81dcc into main Aug 13, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant