Skip to content

Improve AGENTS.md and agent workflow #408

Description

@lin-yang-ly

Reason

During PR #402 (Add validation case val-2j), several issues were observed with the AI agent workflow in TMAP8 that could be improved for future development:

  1. Agents consistently forgot to copy session .jsonl files to doc/content/agent/<issue-number>/ before committing, despite the requirement in AGENTS.md. The current instruction is buried in a paragraph and easy to skip.

  2. Agents did not recognize that a separate light test case is unnecessary when the heavy case already runs in less than 2 seconds.

  3. Context size may be a limiting factor for complex validation cases — reading C++ code, input files, and documentation consumes a large part of the available context.

  4. Agents sometimes failed to define acronyms on first use (e.g., ODEs).

  5. The symlink step needed to generate documentation figures is not documented in AGENTS.md.

Design

Proposed improvements to AGENTS.md and agent workflow:

  1. Pre-commit checklist: Move the JSONL copy and commit trailer requirements from prose into a bold, standalone checklist that agents cannot skim past. Optionally, automate via a Claude Code pre-commit hook in .claude/settings.json or a /commit skill. A possible solution to the large size of .jsonl files is to include a simplified and summarized .jsonl file for each PR.

  2. Light/heavy test guideline: Add a rule that a separate light test case is unnecessary if the heavy case completes in under 2 seconds.

  3. Context management strategies: For complex validation cases that may exceed agent context:

    • Define agent teams in AGENTS.md to separately read codespace, documentation, and inputs, so sub-agents can summarize and transfer information to the main agent without impacting its context.
    • Introduce a "skills" structure within TMAP8 so tasks can be divided into sub-agents with subtasks without requiring large shared context.
    • Future models with larger context windows may also address this.
  4. Acronym convention: Add a rule to AGENTS.md requiring full definitions on first use of acronyms.

  5. Documentation figure symlinks: Add instructions to AGENTS.md for creating symlinks needed to generate documentation figures (e.g., ln -s ~/projects/TMAP8/test/tests/val-2j/comparison_val-2j.py ~/projects/TMAP8/doc/content/verification_and_validation/comparison_val-2j.py).

Impact

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    T: TaskAn enhancement to the software.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions