Skip to content

Bug_199_EVALUATE: PROMPT-FUNC-001 — Current date in system prompt goes stale for long-running sessions #442

@steadhac

Description

@steadhac

Component: finbot/agents/chat.py → VendorChatAssistant._get_system_prompt (line 524) and CoPilotAssistant._get_system_prompt (line 789)

Root cause:

# chat.py line 524 (Vendor) and line 789 (CoPilot)
Current date: {datetime.now(UTC).strftime("%Y-%m-%d")}

datetime.now(UTC) is evaluated once when _get_system_prompt() is called at session
initialization. The result is a static string frozen into the prompt. If the session
remains active past midnight, the date in the prompt is yesterday's date for the rest
of the session.

Steps to reproduce:

  1. Create a VendorChatAssistant or CoPilotAssistant.
  2. Call _get_system_prompt() at 23:58.
  3. Wait until after midnight.
  4. Call _get_system_prompt() again on the same instance.

Expected: both calls return today's date for the time of the call.
Actual: both calls return the date from session initialization — the second call is stale.

How to execute:

pytest tests/unit/agents/test_chat_assistant.py::TestVendorSystemPrompt::test_chat_prompt_005_vendor_prompt_contains_current_date -v
pytest tests/unit/agents/test_chat_assistant.py::TestCoPilotPromptExtended::test_chat_prompt_040_copilot_prompt_date_format_is_iso -v

Note: these tests pass today because they call _get_system_prompt() immediately after
construction. The staleness only surfaces in a session that spans midnight.

Proposed fix:

_get_system_prompt() is already called fresh on each conversation turn (it is not cached).
The bug is that datetime.now(UTC) is evaluated at call time but the string it produces
reflects the moment the f-string was rendered — which is correct behavior for a single call.
The real fix is to ensure a new session is created per conversation turn, or document
explicitly that sessions must not be reused across calendar days.

If the session object is long-lived by design, move the date to be injected per message
rather than per session:

# In stream_response(), inject date into the first user message context
# rather than hardcoding it in the system prompt at construction time.

Impact: The LLM uses the date for scheduling, deadline calculations, and date-aware
reporting (CoPilot). A session initialized on March 31 that continues into April 1 will
generate reports and responses dated March 31 for the entire April 1 session. In a
financial context, off-by-one-day errors in reports or payment references have audit
implications.

Affects: Both VendorChatAssistant and CoPilotAssistant.

Acceptance criteria:

  • _get_system_prompt() returns today's actual date regardless of when the session was initialized
  • Both prompt date tests continue to pass
  • No regression in existing prompt content tests

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions