Gwau3

[caustic-kronos]

Hello,

I’d like to open a discussion here regarding gwau3, with the goal of sharing observations and gathering feedback.

To be clear from the start: I currently have no intention of switching the hub to gwau3. That said, I’m very open to suggestions, ideas, and counter-arguments. If you think I’m missing something or disagree with my conclusions, please don’t hesitate to share your perspective here — this is meant to be a discussion.

While applying the fixes that Greg and Kleutschi made in gwau3 for Reforged to the hub, I noticed several patterns and ideas that could potentially be brought over, as well as others that I’d prefer not to adopt. I’ll try to outline both below.

My broader goal would be to move closer to gwau3 where it makes sense, especially for compatibility, easier fixes, and reusing good work, while still preserving the design choices of the hub.

Areas where I’m hesitant to follow gwau3’s approach

1. AutoIt naming style (Hungarian notation)

gwau3 uses a Hungarian-style naming convention, for example:

$a_i_MaxRegions

The hub, on the other hand, follows a more Java-style, descriptive (semantic) naming convention, which I personally find much easier to read, understand, and reason about.

In my experience, Hungarian notation adds visual noise without providing much additional value, especially given modern screen resolutions and tooling. In most cases, a name like maximumRegions (or maxRegions) already conveys enough information, including the expected type.

This is ultimately a stylistic preference, but it’s one where I strongly favor the hub’s current approach.

2. Direct memory access per field vs. structured reads

gwau3 frequently accesses structure fields directly by offset, for example:

Func Agent_GetAgentInfo($a_i_AgentID = -2, $a_s_Info = "")
    Local $l_p_AgentPtr = Agent_GetAgentPtr($a_i_AgentID)
    If $l_p_AgentPtr = 0 Or $a_s_Info = "" Then Return 0
    Switch $a_s_Info
        Case "vtable"
            Return Memory_Read($l_p_AgentPtr, "ptr")
        Case "h0004"
            Return Memory_Read($l_p_AgentPtr + 0x4, "dword")
        ...

Compared to the hub’s approach:

Func GetAgentInfo($agentID, $field = "")
    Local $agentStruct = DllStructCreate($agentStructTemplate)
    Local $agentPtr = GetAgentPtr($agentID)
    DllCall($kernelHandle, 'int', 'ReadProcessMemory', 'int', GetProcessHandle(), 'int', $agentPtr, 'ptr', DllStructGetPtr($agentStruct), 'int', DllStructGetSize($agentStruct), 'int', 0)
    Return DllStructGetData($agentStruct, $field)
EndFunc

ReadProcessMemory is relatively expensive. If only a single field is needed, the per-field approach makes sense. However, as soon as multiple fields are accessed, reading the full structure once tends to perform significantly better.

From a maintenance standpoint, defining structures also centralizes offsets and makes updates easier when a structure layout changes. For these reasons, I personally disagree with the overall direction taken by gwau3 on this topic, even though there are valid use cases for both approaches.

3. Interface split across many files and folders

I generally agree with the idea of splitting code into specialized files to keep things clear and focused. However, in my opinion, gwau3 takes this quite far, resulting in a structure that feels overly fragmented and harder to navigate.

This is again subjective, but I prefer a slightly more consolidated layout.

Things I would like to bring over

1. Analyzing the existing direct field accesses and using them to define proper structures

2. Porting certain functions that are present in gwau3 but missing from gwa2 / the hub

I’m very interested in hearing other opinions on this — especially from people who have worked extensively with this the hub or gwau3. My goal here isn’t to dismiss that work, but to figure out where alignment makes sense and where different design choices are preferable.

[Gahais]

I have big respect for Greg and Kleutschi, they are programming geniuses :)

There might be some good ideas and more hardware solutions implemented in GwAu3 that could optimize GWA2 and make it working faster or more stable against crashing

I admit that I'm a bit uncomfortable with that Hungarian notation, it makes variable names unnecessarily broader with not much information added, and still a bit cryptic, imho
I like the current Java like style, descriptive (semantic) naming convention that is currently used in BotsHub in quite standardized way. I think we could stick to it, changing it may require a lot of effort

hmm, regarding these single fields of struct access this feature could be added in some new sections of GWA2 to be used as substitute for calls like DllStructGetData($agent, 'ID')
But extracting whole struct first and then accessing its single fields might be sufficiently good solution probably

Regarding splitting GWA into many files and folders.
I like current way the GWA2 is written, It holds all the stuff in one file which makes easy for searching for API functions and definitions/declarations, easier to maintain it
In GwAu3 there might be a lot of files and folders. It is quite professionally modularized, I respect that
I think, in BotsHub GWA2 could be optionally split into several more files, like 5-15
Most distinctive #regions in current GWA2 could be moved outside to form new files for these #regions
For example

• maps, movement, traveling into a separate file, like GWA2_Travel.au3
• low-level hardware, assembly stuff into a separate file, like GWA2_Assembly.au3
• skills usage, targeting, commands into a separate file, like GWA2_Commands.au3
• agents, npcs, player, heroes, teams management into a separate file, like GWA2_Agents.au3
• others and miscellaneous into a separate file, like GWA2_Misc.au3
• Ids from Guild Wars game as it is already in GWA2_ID.au3
• Guild Wars headers as it is already in GWA2_Headers.au3-
• most GWA-like stuff from Utils.au3 could be moved into a separate file like GWA2_Utils.au3
• Utils-Items_Modstructs.au3 could be renamed into GWA2_Items_Modstructs.au3
• Utils-Debugger.au3 could be renamed into GWA2_Debugger.au3
• Utils-Storage-Bot.au3 could be renamed into GWA2_Storage.au3 and can move some storage functions into it from Utils.au3, that remain there, like DefaultShouldPickItem()
• Utils-Omnifarmer.au3 could be moved to src directory alongside other farms

So it might be more clear to read and a bit more modularized

Quite interesting topic is to import all constant IDs from GwAu3 with little effort into GWA2
I see that in GwAu3 there are probably all in-game constant IDs written in Api/Constants directory
All constant IDs for skills, maps, items, quests, titles could be imported from there to GWA2_ID.au3 file and just replace name prefixes. So there would not be a need to check every ID from GWToolbox
Can store all these IDs in GWA2_ID.au3 file or split this file into several more like GWA2_ID_Maps.au3, GWA2_ID_Skills.au3, GWA2_ID_Quests.au3, GWA2_ID_Items.au3, GWA2_ID_Misc.au3

Yeah, I think it would be a good idea to read through GwAu3 project's code and try to port best ideas and functionalities from GwAu3 into GWA2 to implement them in BotsHub. I may try doing some of this if I find some free time :)

[logan-77]

Hey Kronos,

1. I agree about about the notation, it is a little bit overwhelming and carries redundend/useless information and makes the variable names very long and hard to write. Thats why I don't use it in my project, but I also struggle to find a consitent naming convention.
   I guess it makes the project more "official" and the API itself is easier to maintain with the more formal naming conventions.

2. This is the most important difference between both projects.

Generally, you will notice that Gwau3 bots use way less CPU load then GWA2 bots.
From my experience ReadProcessMemory is not that expensive, nevertheless I optimize my bots, to use it as little as possible.
However what is more expensive is creating huge Structs and move them around in CPU cache.
One GetAgentArray() call creates an array of N (number of agents in compass range) times 500 bytes (approx size of AgentStruct): N*500 bytes.
That same array created with GetAgentPtrArray() is N*4 bytes (it only contains the pointer to the Agent), which is a very big difference. Granted you have to follow up with a few MemoryReads afterwards. But the diffence in CPU load between both functions is really big in favor of the latter. (In my subjective estimate CPU load is 90% less with the latter)

Kleutschi created the golden way, between both worlds. Since, most of the time, you only use about ~10 fields of the AgentStruct, you really only want to read these fields and not the rest of the noise. He creates a partial AgentStruct and reads all (and only) the relevant fields with one DllCall. Example:

; Agent - MoveXY
Global $g_d_MoveXYStruct = DllStructCreate( _
    "byte[" & $GC_I_OFFSET_AGENT_MOVE_X & "];float moveX; float moveY" _
)
Global $g_i_MoveXYStructSize = DllStructGetSize($g_d_MoveXYStruct)

; Agent - PosXY
Global $g_d_PosXYStruct = DllStructCreate( _
    "byte[" & $GC_I_OFFSET_AGENT_POS_X & "];float posX; float posY" _
)
Global $g_i_PosXYStructSize = DllStructGetSize($g_d_PosXYStruct)

Of course everbody can create their own partial structs as needed, i.e. PosXY and MoveXY could be combined into one struct.

2.1.

From a maintenance standpoint, defining structures also centralizes offsets and makes updates easier when a structure layout changes.

We plan to define all fields of the AgentStruct as a global constant and use these constants in GetAgentInfo() and everywhere else, which will centralize the offsets.

3. Imho, for GWA2 the best way is to keep everything in one file (Ctrl+F is your friend)
   As there is no "official" team, which maintains GWA2 and at different times different people step up to update GWA2 to the newest release of Guild Wars.
   For Gwau3 it makes more sense to split everything up, because the scope of the project is quite a bit larger and it does include more then the API for Guild Wars funtions, but also a lot of algorithms, like Smartcast and Pathfinding.
   Also all functions are prefixed, i.e. Agent_, Skill_, which makes debugging easier. I'd say for GWA2 this is not a must have, but makes sense for Gwau3.

I hope I could provide some useful information to the topic at hand and am happy to engagne in further discussion about pros/cons between GWA2 and GwAu3.

[caustic-kronos]

Back from winter holidays 😄
Thank you so much for participating, Gahais and Logan 🎉

@Gahais
Concerning separation of concerns on the Hub, I agree that we will need to split GWA2 (and also separate the GUI from the BotsHub file). Not entirely sure I agree with the separation - many files will somewhat overlap with each other (agents are used everywhere for instance). But I definitely agree that all low level stuff (what changes at every update) should have its own file, that will also simplify fixes.

I kept GWA2_ and Utils_ as two different prefixes to differentiate functional and non functional requirements. But some renaming for coherence does make sense. And yes, there are some overlaps between Utils and Utils-Storage-Bot which will need to be fixed at some point. Omnifarmer is indeed not a farm and more of a utility class here.

IDs will indeed need to be split in several files as it becomes way too crowded by now. I don't think it's worth making a file for every type though, but definitely separate items from skills/professions from maps for instance.

All good points, thanks for bringing that up.

@logan-77
I'm not overly familiar with AutoIt way of dealing with memory, so maybe I became too spoiled by java garbage collector. So excuse me if I make mistakes here.

Even if we have 200 agents in range and each of these agents takes 500 bytes (that's about the current size of an agent I believe), those structures only live when we need them, so memory will be immediately freed once those structures are not referenced anymore. Which means that the memory used by those 200 agents will be used only in the GetAgent function, once it returns, only one structure will be left in memory. So, memory usage is extremely limited in time.

Secondly, 500 bytes times 200 is only 100 Kb - you'd need to have 10 bots fetching all those agents at the exact same time to even reach a single Mb. I'm not sure the memory improvement here is worth the additional CPU.

But I agree that the in-between solution from Kleutschi is splendid. Fetching only the fields you care about uses the best of both worlds: a single ReadMemory with a smaller structure footprint. That could be an improvement to port to GWA2, though I'm not sure it's worth the trouble - I'd need to see some memory benchmarks to confirm/contradict my previous napkin calculations.

Concerning centralization of offsets, you might already be going with this option, but I'd suggest automatically calculating offsets. It's a cheap thing to do once at bot start or first usage and be free of it forever, and no more having to manually update offsets after updates.

And yes, I think I agree with you that keeping things mostly centered (it's already split with headers and GWA2 so we could probably split it off a little bit more) is probably correct for GWA2, that's how it survived until today.

You did provide useful informations, thank you 😎

Something I forgot to mention that I absolutely want to bring up from gwau3 is the regroupement of the memory pattern with their offset. It makes sense and will make future fixes much easier.

[logan-77]

Hey,

Gonna be really honest here, I don't know the first thing about, how AutoIt runs the code on the hardware side.
I'm just trying to understand the code we run as good as possible and make the best educated guess I can.
Your explanation makes a lot of sense and I tried to use ChatGPT to get more information about this. (I think I found some interesting info)

What I know for a fact is, that my CPU load went down like 90%, when I switched from the regular GetAgentArray() to GetAgentPtrArray+necessary Dllcalls.
My scripts basically never go above 0.1% and are 0% a lot of the time.

I just looked at GetAgentArray and fed it into ChatGPT and it pointed out this loop:

Local $deadlock = TimerInit()
Do
    Sleep(1)
    $count = MemoryRead($agentCopyCount, 'long')
Until $count >= 0 Or TimerDiff($deadlock) > 5000

To me it seems, this might be a reason for the high CPU load, I was talking about above, because it might potentially do a lot of Dllcalls?

Another thing ChatGPT noticed is, that the whole AgentArray is physically copied twice (ReadProcessMemory → buffer, buffer → per-agent struct).
The second copy can be avoided with the following change:

> DllStructSetData($struct, 1, DllStructGetData($buffer, $i + 1)) ; copies each element

> ; (not sure if the syntax is correct, but I hope you get what its trying to do)
> $returnArray[$i] = DllStructCreate($agentStructTemplate, DllStructGetPtr($buffer) + ($i * 448)) ; only the pointer/reference is written to the new array

(ChatGPT:) Modern CPUs hate: Big linear memory copies, Cache pollution, Repeated Variant creation
This might be confirming my point from the last post.

I also asked ChatGPT, why my GetAgentPtrArray function causes way less CPU load, then GetAgentArray:

Your AutoIt thread:
Is not executing AutoIt bytecode
Is often waiting on kernel operations
Consumes little or no user-mode CPU time

So paradoxically:
More DLL calls → less AutoIt CPU usage

The conclusion was, that a hybrid is the best of both worlds. Batch ReadProcessMemory calls, but only copy the fields we actually use from process memory to the buffer.

[caustic-kronos]

That loop is very interesting. Looking at it I thought it would only ever run once. So I had the opposite idea that it would basically never do a lot of DllCalls. I was surprised to find it does run more than once sometimes.
Updated code like so:

	Local $numberOfMemoryRead = 0
	Do
		Sleep(1)
		$count = MemoryRead($agentCopyCount, 'long')
		$numberOfMemoryRead += 1
	Until $count >= 0 Or TimerDiff($deadlock) > 5000
	If $count < 0 Then $count = 0
	Out($numberOfMemoryRead & ' memory reads to get agent array of type ' & $type)

And here is what I saw, running raptors:

22:46:44 - 2 memory reads to get agent array of type 219
22:46:52 - Moving to Cave
22:47:05 - Moving Hero away
22:47:05 - Gathering Raptors
22:47:05 - 4 memory reads to get agent array of type 219
22:47:22 - 1 memory reads to get agent array of type 219
22:47:22 - 1 memory reads to get agent array of type 219
22:47:23 - 1 memory reads to get agent array of type 219
22:47:23 - 1 memory reads to get agent array of type 219
22:47:25 - Clearing Raptors
22:47:26 - 2 memory reads to get agent array of type 219
22:47:26 - 1 memory reads to get agent array of type 219
22:47:28 - 1 memory reads to get agent array of type 219
22:47:28 - 1 memory reads to get agent array of type 219
22:47:29 - 1 memory reads to get agent array of type 219
22:47:29 - Spiking 30 raptors
22:47:29 - 1 memory reads to get agent array of type 219
22:47:29 - 1 memory reads to get agent array of type 219
22:47:29 - 1 memory reads to get agent array of type 219
22:47:31 - Looting
22:47:31 - 1 memory reads to get agent array of type 1024
22:47:33 - 1 memory reads to get agent array of type 1024
22:47:34 - 1 memory reads to get agent array of type 219
22:47:34 - Returning to Rata Sum (outpost)

It's still mostly 1 memory read, though there are sometimes little spikes at 3-4-5 memory read (saw 9-10 a few times too). I supposed worst case scenario is some disconnection. Even failure doesn't affect those read.
Overall, this loop shouldn't be a problem. It's not a busy read either, so it's fine.

As for the correction ChatGPT offered, it's indeed a performance improvement, but it causes a much bigger issues down the line.
Here is what we do:

			$returnArray[$i] = SafeDllStructCreate($agentStructTemplate)
			$struct = SafeDllStructCreate('byte[448]', DllStructGetPtr($returnArray[$i]))
			DllStructSetData($struct, 1, DllStructGetData($buffer, $i + 1))

i.e: read from memory into buffer, create structure inside array, copy from buffer into structure. We end up with a duplicate data, the cost is we copied it.
If we go for CGPT correction:

DllStructSetData($struct, 1, DllStructGetData($buffer, $i + 1)) ; copies each element
$returnArray[$i] = DllStructCreate($agentStructTemplate, DllStructGetPtr($buffer) + ($i * 448))

We read memory into buffer, and we point our structure in returnArray directly on the memory in buffer (not sure what struct does here, probably chatGPT gibberish). Anyway, the problem is that we return returnArray from function, so it will be referenced after function lifetime, but not buffer. So now, we have returnArray, made of structures pointing into the void, and they can read all kind of gibberish if that memory is allocated somewhere else.
So yep, this is not a good solution, and if it works, it's just by luck because nothing uses memory there.

[Gahais]

Logan-77 seems to be right that the whole AgentArray is physically copied twice
hmm, how about this alternative approach?
instead of skipping second copying - buffer → per-agent struct, skip first copying - ReadProcessMemory → buffer
something like that - do many small ReadProcessMemory calls for each agent struct, so only 1 copying should be done

		For $i = 0 To $count - 1
			$returnArray[$i] = SafeDllStructCreate($agentStructTemplate)
			SafeDllCall13($kernelHandle, 'int', 'ReadProcessMemory', 'int', GetProcessHandle(), 'int', $agentCopyBase, 'ptr', DllStructGetPtr($returnArray[$i]), 'int', DllStructGetSize($returnArray[$i]), 'int', 0)
		Next

and remove these lines

	Local $struct
	SafeDllCall13($kernelHandle, 'int', 'ReadProcessMemory', 'int', GetProcessHandle(), 'int', $agentCopyBase, 'ptr', DllStructGetPtr($buffer), 'int', DllStructGetSize($buffer), 'int', 0)
	$struct = SafeDllStructCreate('byte[448]', DllStructGetPtr($returnArray[$i]))
	DllStructSetData($struct, 1, DllStructGetData($buffer, $i + 1))

[caustic-kronos]

@Gahais no, this is not a good solution.
ReadProcessMemory is not a cheap operation on CPU. You can refer to this discussion on stackOverflow for more details:
https://stackoverflow.com/questions/23515468/c-readprocessmemory-slowing-my-prog
What this means is that we prefer to read 500 bytes in one call than 100 calls reading 5 bytes.
(Edit) here is some more documentation on the topic: https://www.ownedcore.com/forums/world-of-warcraft/world-of-warcraft-bots-programs/wow-memory-editing/453135-reading-game-memory-slow-looking-advice.html

There is no better way than to read the entire block in one call, and then copy from that block into what we want to return.
The only improvements would be to copy less data from the buffer to the structures (but then we need to know what we need in advance).

[Gahais]

okay, got it, maybe let's leave it as it is

[logan-77]

Kronos you know your stuff and you're right. The ChatGPT explanation just sounded so good and kinda logical, but it was a total hallucination...

Now I'm more confused then ever, why my GWA2 bots use much much much more CPU% then my GwAu3 bots.
Even though my GwAu3 bots probably use more DllCalls.

[caustic-kronos]

@logan-77 If you find the cause for the increased CPU usage in GWA2 that you see, don't hesitate to share ! I'm super interested since I could port that improvement back to the hub and improve performances 😄

[KleuTSchi]

@caustic-kronos Automatically calculating the offsets sounds like an even better idea than just centralizing them, agreed. Its definitely something I want to do but I am currently procrastinating as I dread having to look through all functions and replacing all offsets.
I initally planned to also replace the Switches in the GetXYZInfo functions with lookup tables but that comes with its own fair share of issues as some values are not just a series of memory reads but also involve some basic arithmetic, like calculating the exact current HP, some bitmasks, etc. This unfortunately makes my initial idea of the lookup tables less viable than I had hoped for.

Regarding the notation, when Greg and I were working on the refactoring of GwAu3 we needed to settle on something and then stick to it, since collaborating on a project and every second function using a different naming convention is a nightmare in the making.
In our case that was the current Hungarian notation. While I agree that this in some cases leads to redundant information or potentially reduce readability (i.e. g_p_Agent vs. g_p_AgentPtr) we still decided to settle on it as this makes it abundantly clear for readers what datatype is expected, what scope we are currently in and in my opinion helps getting a quicker grasp on all the data types you are currently dealing with as some variables might be filled by other function calls you otherwise would have to look up.
At the end of the day its a stylistic choice and there is no real correct answer. But I agree that we have some very bad offenders here and there when it comes to variable names 😄

As for the difference in CPU usage @logan-77 experienced I cannot really give you guys much of a clue I believe. For most of the heavy lifting (I am also not a huge fan of the individual memory field reads we currently have when it comes to performance), I use custom structs to avoid ReadProcessMemory and other heavy DllCalls as much as possible. I also cache a lot of information whenever possible, like party pointers as they are valid until an agent leaves compass range, IDs of party members are also constant until the map changes, etc. This also avoids some unnecessary memory reads. However, all of this doesn't really apply to what someone would experience if they migrate their scripts.

If there are any other questions I can help answering I am happy to do so, either here or via Discord.

[caustic-kronos]

Hey @KleuTSchi, thanks for chiming in, much appreciated 😄

Yeah... replacing hardcoded offsets definitely sounds like the kind of can I would kick further down the road for a few months before finally taking care of it.

On the lookup-table vs switch topic: I don’t think values that require a bit of computation necessarily disqualify lookup-based approaches, as long as the contract is clear. If something returns HP%, that’s fine - just make it explicit that it’s a percentage and not raw HP. Same idea with pointers: returning a pointerToX instead of immediately following it inside the switch makes the intent very clear and keeps responsibilities separated.
At least that’s the mental model I tend to gravitate toward.

Regarding notation: I completely get why you and Greg settled on one convention and stuck to it.
I still very much dislike Hungarian-style naming though; for me it tends to leak implementation details into places where the variable's meaning should already be obvious. It's also making code harder to approach for people less familiar with development, folks struggle more with gwau3 because of that.
I’ve been trying to keep my own rules pretty simple: scope via casing (Global Const, Global, Local) and names that describe what the thing is rather than how it's stored. $agent vs $agentID is usually enough for my brain.
That said... yeah, we all have a few naming crimes buried somewhere in our repos 😄

As for CPU usage, I never used gwau3 extensively, so I can’t compare directly - but based on what you describe, the performance improvements make some sense. Caching stable pointers, avoiding repeated RPM calls, and relying on logic/timers instead of brute-force reads are all solid design choices. I try to follow the same philosophy where I can, but I’m not surprised if Logan noticed a difference.

Anyway, thanks again for taking the time to talk about all this, always interesting to see how other people approached similar problems. I might take you up on that Discord offer at some point 🙂

Cheers !