Investigation request: clarify and harden the NPC/dialog interaction contract

[Wellwisher533]

I have encountered recurring NPC, dialogue, blessing, reward, and gadget interaction failures while running missions, dungeons, and vanquishes in-game. These have occurred across unrelated widgets, including Skills Unlocker, EOTN Storyline, Mission Runner, dungeon bots, and faction vanquishes.

I initially built an opt-in shared reliability layer after repeatedly addressing the same behavior. Before proposing that as a mergeable solution, I am requesting clarification about which behavior belongs in the existing helpers and whether any interaction-related migration or binding surface is incomplete.

Environment

The tested Clean Test checkout is based on:

9bdb0d31992f108287f8b57f93e48f73b9095105
fix YMLAD roaming Koren objective
August 2, 2026

The working tree contains unrelated local widget changes, so I am not claiming a pristine-checkout reproduction. The native interaction files inspected during the original investigation were unmodified relative to that commit:

• Py4GWCoreLib/botting_src/subclases_src/INTERACT_src.py
• Py4GWCoreLib/botting_src/subclases_src/DIALOGS_src.py
• Py4GWCoreLib/routines_src/behaviourtrees_src/composite.py
• Py4GWCoreLib/routines_src/behaviourtrees_src/player.py
• Py4GWCoreLib/UIManager.py

Existing helper contracts

Coroutine/FSM interaction

_INTERACT._coro_with_agent() currently:

1. Calls Routines.Yield.Agents.InteractWithAgentXY().
2. Optionally calls Player.SendDialog(dialog_id).
3. Waits a fixed 500 ms.
4. Sets dialog_at_succeeded=True.

InteractWithAgentXY() verifies that a target was selected and that the player reached interaction range, but it does not verify that an NPC dialogue became visible.

The subsequent dialogue send does not verify that the intended context opened, that it belongs to the current target, or that a quest, reward, map, effect, bundle, or gadget state changed. Consequently, dialog_at_succeeded=True appears to mean that the calls were dispatched rather than that the requested game interaction succeeded.

BottingTree interaction

BTPlayer.InteractAgent() returns success immediately after Player.Interact() and uses a fixed 350 ms aftercast. BTPlayer.SendDialog() returns success immediately after Player.SendDialog() and uses a fixed 300 ms aftercast.

The interaction composites sequence those dispatches but do not poll dialogue visibility or verify a concrete game-state postcondition.

There is already better behavior for automatic choices: BTPlayer.SendAutomaticDialog() polls active dialogue state for up to three seconds. There is also a separate visible-frame API:

• UIManager.IsNPCDialogVisible()
• UIManager.GetDialogButtonCount()
• UIManager.ClickDialogButton(choice)

UIManager.ClickDialogButton() is explicitly one-based, while SendAutomaticDialog() uses its own indexing and dialogue mechanism. Raw context IDs, visible button indexes, reward frames, and automatic triggers are therefore not interchangeable.

Reproduced failure classes

1. Raw context sent through the wrong dialogue mechanism or stale context

In Finding Gadd, Gadd was model 6772 with intended raw context 0x833304. The Inscription Stone was model 15 with intended raw context 0x833307.

The bot called AutoDialog(0x833304) even though that helper expected a button index rather than a raw context ID. After correcting that call-site error, Gadd's dialogue could remain open while the bot switched to the Inscription Stone. The stone's context was then sent while the stale Gadd context remained active. Closing the old dialogue before changing targets corrected that specific failure.

An EOTN Storyline audit found 51 coordinate-based MoveAndDialog calls using dispatch/fixed-delay behavior, plus several AutoDialog calls containing values that were actually raw context IDs.

Some of this is bot misuse, but the current API boundaries make it easy to confuse visible button indexes, raw dialogue context IDs, reward frames, and automatic triggers.

2. Visible blessing choice is not equivalent to raw 0x84

Darkrime Delves blessing NPCs display the visible choice:

Yes, I will take the blessing.

Sending raw 0x84/decimal 132 did not reliably select that visible choice.

The successful flow was:

1. Resolve the intended live NPC or beacon.
2. Approach using a safe player coordinate.
3. Reacquire the runtime agent after movement.
4. Interact once.
5. Wait asynchronously for the NPC dialogue to become visible.
6. Wait for its buttons to populate.
7. Click visible button 1 using UIManager.ClickDialogButton(1).
8. Verify the blessing effect.
9. Close the delayed follow-up popup when present.

The outside-Longeye/Bjora blessing worked consistently through this flow. This suggests visible choices and raw context sends should remain explicitly separate mechanics.

3. Dispatch treated as success without a game-state postcondition

Across existing helpers, success was sometimes defined as an interaction call, dialogue ID, or button click being dispatched. Those events do not prove that the intended result occurred.

Reliable results required checking an observable postcondition such as quest state, blessing effect, reward receipt, bundle state, hostility, door/gadget state, or map transition.

4. Route-to-interaction boundary failure

This is separate from dialogue handling but affects the same interactions.

Mission: Genius Operated Living Enchanted Manifestation
Step: 03. Reach Mamp
Ordinary route endpoint: (16082.28, 15253.92)

Observed nearby agents:

• NPC model 51 at (16043.93, 15260.60), approximately 39 units from the route endpoint.
• Mamp model 6766 at approximately (16051, 15183).

Diagnostics repeatedly showed that the current step remained 03. Reach Mamp, movement entered avoiding_obstacle, the player oscillated near the endpoint, step 04. Start G.O.L.E.M. at Mamp never received control, and no reliable-interaction events were emitted.

The Mamp interaction itself was already configured to resolve model 6766, use visible choices (1, 1), and verify a map transition. None of that code ran.

This indicates a missing route-to-interaction handoff contract. Ordinary navigation should stop at a safe standoff and hand final approach to the interaction routine before entering an NPC or object's collision area.

5. Current-target and binding-surface inconsistencies

Silent Surf capture NPC-20260810-181737-210 was taken while the Luxon Priest dialogue was visibly open. The record contained:

• PyAgent.get_target_id() == 0
• Player.GetTargetID() == 0
• visible NPC dialogue: true
• visible buttons: two
• initialized received-dialog callback journal entries: zero
• live nearby priest: model 3692 at (11339, 24455)

The callback journal initialization completed without error, but PyDialog.PyDialog.get_dialog_callback_journal_received() returned an empty list for this interaction.

The documented UI-message-log path was also unavailable. UIManager.GetUIMessageLogs() failed because the installed native binding did not expose the method used by the wrapper:

AttributeError: type object 'PyUIManager.UIManager' has no attribute 'get_ui_message_logs'

A separate Cyrus merchant interaction showed the same zero values from both current-target accessors.

To distinguish broken capture data from invalid agents, I added an explicit user-selected runtime-agent lock. That path successfully captured Cyrus (model 3672, runtime agent 27) and the Luxon Priest (model 3692, runtime agent 33), including exact target coordinates, separate player approach coordinates, and the priest's visible dialogue/button frames.

This confirms that the live agents and model/location data are valid. The explicit lock is a workaround for target ownership during capture, not an explanation for why the target, journal, and documented UI-message-log surfaces did not identify the interaction.

What is established versus unknown

Established from code inspection and runtime evidence:

• Several interaction actions report success after dispatch rather than UI or game-state success.
• Raw contexts and visible choices are different APIs.
• Dialogue contexts can remain stale while targets change.
• Fixed delays do not reliably cover asynchronous dialogue population.
• A preceding route can prevent an interaction helper from running.
• Both current-target accessors can return zero during the tested merchant and priest interactions.
• The initialized received-dialog callback journal remained empty for the tested priest dialogue.
• The installed native PyUIManager.UIManager does not expose the documented get_ui_message_logs method used by the wrapper.
• Explicit runtime-agent selection, reacquisition, asynchronous polling, and concrete postconditions improve reliability.

Not established:

• I have not identified a single regression commit.
• I am not claiming that every failure has the same cause.
• I do not know whether the dispatch-only semantics are intentional low-level contracts.
• The target being cleared after interaction may be intentional, but the intended replacement source for identifying the active dialogue owner is unclear.
• I do not know whether the empty callback journal is expected for these dialogue paths.
• I do not know whether the missing UI-message-log binding is a build mismatch, an incomplete migration, or stale wrapper/stub documentation.

Request for maintainer guidance

Could a maintainer familiar with the interaction and binding migrations clarify:

1. Are InteractAgent, SendDialog, _coro_with_agent, and the interaction composites intentionally dispatch-only primitives?
2. If so, which existing layer is intended to own runtime-agent reacquisition, asynchronous dialogue readiness, visible-choice versus raw-context selection, retries, stale-dialog cleanup, and concrete postcondition verification?
3. What is the intended authoritative way to identify the NPC or object owning a visible dialogue after the current target becomes zero?
4. Should the received-dialog callback journal contain entries for merchant and blessing interactions?
5. Should PyUIManager.UIManager.get_ui_message_logs exist in the current native binding, given that it is used by the wrapper and documented in the stubs?
6. Was a higher-level interaction orchestration layer planned or removed during migration?
7. Should existing helpers be hardened, or should an opt-in high-level composite be added above the dispatch primitives?
8. Where should route-to-interaction standoff and handoff behavior belong?

Supporting work available

I have a locally tested opt-in reference implementation and focused tests covering exact live-agent resolution, separate target and safe-player coordinates, asynchronous visible-dialogue/button polling, separate visible-choice and raw-context profiles, postcondition-based success, bounded retries, stale-dialog cleanup, HeroAI/movement restoration, route-boundary validation, capture-source diagnostics, explicit user-selected runtime-agent capture, and blocking invalid captures rather than saving target: null.

The current suites include:

• 33 passing shared reliable-interaction tests.
• 16 passing NPC/location capture tests.

The explicit runtime-agent capture path has also been validated in-game with both Cyrus and the Silent Surf Luxon Priest.

I am not requesting that the reference implementation be merged unchanged. It is available as evidence, a behavioral reference, and a possible source of focused tests after maintainers confirm where these responsibilities belong.

I can provide the relevant log excerpts, captures, focused tests, and reference implementation. Maintainer guidance on the intended native ownership would help determine whether the appropriate next step is strengthening an existing helper or proposing a small opt-in orchestration layer.