[Bug] Needle HTTP server (--serve) KV cache state persistence ignores /reset and returns sticky/stale responses

[willdom-kahari]

Summary

When running Needle in HTTP server mode (--serve), the engine caches static prefix KV states and conversational history across requests. Issuing POST /reset returns a success response ({}) but fails to clear or invalidate the in-memory KV cache/prefix state, causing subsequent independent requests to return sticky/stale responses tied to the first or previous turn.

Details & Reproduction

1. Start the Needle HTTP server:

/macos-arm64/needle --model needle3.cact --tools tools.json --serve --port 8086

2. Send an initial request.

curl --location 'http://127.0.0.1:8086/complete' \
--header 'Content-Type: application/json' \
--data '{"input": "dim the living room to 30"}'

3. The response i get is far from what i asked

{
    "type": "call",
    "success": true,
    "error": null,
    "error_code": null,
    "reason": null,
    "function_calls": [
        {
            "name": "setThermostatTool",
            "arguments": {
                "temperature": 18,
                "mode": ""
            }
        }
    ],
    "suppressed_calls": [],
    "reasoning": "'set thermostat to 18' -> set_thermostat_tool with temperature 18, no mode or other params.",
    "confidence": 0.4796,
    "prefill_tps": 648.4,
    "decode_tps": 790,
    "peak_ram_mb": 83.3,
    "validation": {
        "ungrounded": [
            "set_thermostat_tool.temperature"
        ],
        "negation": false
    }
}

4. Send POST /reset.

curl --location --request POST 'http://127.0.0.1:8086/reset'

5. The response indicated a success

{}

6. Send a completely different query
7. Observed Behavior: The server ignores the new query intent and returns the cached previous turn's tool call response with stale arguments. Response is the same as in step 3
8. Expected Behavior: POST /reset should fully flush the session state and KV cache so subsequent prompts are evaluated cleanly and independently (matching stateless CLI execution).

Here is my tools.json