Skip to content

Belief update parser rejects valid JSON responses lacking markdown fences #71

@crest-agent

Description

@crest-agent

Problem

The checkBeliefUpdate function in src/beliefs.ts only accepts LLM responses that are either exactly "NO_UPDATE" or a JSON object wrapped in triple backticks with the language identifier 'json' (e.g., json\n{...}\n). If the model returns a JSON object without the fences, or with a different fence format (e.g., \n{...}\n), the response is ignored, preventing belief updates even when the model has provided a valid update.

What to build

Improve the response parsing to be more forgiving while maintaining safety. Specifically:

  1. First, check for the exact string "NO_UPDATE".
  2. Then, attempt to extract a JSON object from the response by:
    • Looking for a substring that starts with '{' and ends with '}' (with balanced braces) OR
    • Looking for a markdown code block (with or without language identifier) and parsing the content as JSON.
  3. If JSON is found, validate it has the required fields (newBeliefs and changelogEntry as strings).
  4. If no valid JSON is found, log a warning and skip the update.

Done when

The belief update function correctly processes LLM responses that contain valid JSON in common formats (with or without markdown fences) and updates beliefs when appropriate, while still rejecting clearly malformed responses.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions