Skip to content

Bug_198_EVALUATE: CHAT-BOUNDARY-015 — String vendor_id forwarded to DB layer without type coercion; behavior is DB-dependent #419

@steadhac

Description

@steadhac

Component: finbot/agents/chat.py → VendorChatAssistant._call_get_vendor_details (line 661)

Root cause:

The LLM can produce a tool call where vendor_id is a string (e.g. "42" or "abc")
instead of an integer. The agent layer performs no type coercion. The string is passed
directly to get_vendor_details, which passes it to the ORM layer.

Steps to reproduce:

  1. Patch get_vendor_details to record its arguments.
  2. Call agent._call_get_vendor_details(vendor_id="abc").
  3. Assert captured["vendor_id"] == "abc".

Expected: int("abc") raises ValueError with a clean error JSON, OR the field is
coerced to int before the DB call.
Actual: "abc" forwarded to the DB layer as-is. SQLite may silently coerce the string;
PostgreSQL raises DataError. Behavior is environment-dependent.

How to execute:

pytest tests/unit/agents/test_chat_assistant.py::TestBoundaryAndTypeValues::test_chat_boundary_015_get_vendor_details_vendor_id_string_propagates -v

Proposed fix:

async def _call_get_vendor_details(self, vendor_id: int) -> str:
    try:
        vendor_id = int(vendor_id)
    except (TypeError, ValueError):
        return json.dumps({"error": f"vendor_id must be an integer, got {vendor_id!r}"})
    result = await get_vendor_details(vendor_id, self.session_context)
    ...

Impact: In development (SQLite), string "42" may silently match vendor 42 — tests
pass, the bug is invisible. In production (PostgreSQL), the same call raises DataError
and crashes the agent method. This is a classic SQLite/PostgreSQL divergence: the test
environment masks a production bug. The same pattern applies to all tool methods that
accept an integer ID.

Acceptance criteria:

  • test_chat_boundary_015_get_vendor_details_vendor_id_string_propagates updated to assert type error response
  • _call_get_vendor_details coerces vendor_id to int or returns a structured error
  • Audit all vendor_id: int and invoice_id: int parameters across tool methods for the same gap

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions