Gwau3 #98
Replies: 13 comments
|
I have big respect for Greg and Kleutschi, they are programming geniuses :) There might be some good ideas and more hardware solutions implemented in GwAu3 that could optimize GWA2 and make it working faster or more stable against crashing I admit that I'm a bit uncomfortable with that Hungarian notation, it makes variable names unnecessarily broader with not much information added, and still a bit cryptic, imho hmm, regarding these single fields of struct access this feature could be added in some new sections of GWA2 to be used as substitute for calls like Regarding splitting GWA into many files and folders.
So it might be more clear to read and a bit more modularized Quite interesting topic is to import all constant IDs from GwAu3 with little effort into GWA2 Yeah, I think it would be a good idea to read through GwAu3 project's code and try to port best ideas and functionalities from GwAu3 into GWA2 to implement them in BotsHub. I may try doing some of this if I find some free time :) |
|
Hey Kronos,
Generally, you will notice that Gwau3 bots use way less CPU load then GWA2 bots. Kleutschi created the golden way, between both worlds. Since, most of the time, you only use about ~10 fields of the AgentStruct, you really only want to read these fields and not the rest of the noise. He creates a partial AgentStruct and reads all (and only) the relevant fields with one DllCall. Example: ; Agent - MoveXY
Global $g_d_MoveXYStruct = DllStructCreate( _
"byte[" & $GC_I_OFFSET_AGENT_MOVE_X & "];float moveX; float moveY" _
)
Global $g_i_MoveXYStructSize = DllStructGetSize($g_d_MoveXYStruct)
; Agent - PosXY
Global $g_d_PosXYStruct = DllStructCreate( _
"byte[" & $GC_I_OFFSET_AGENT_POS_X & "];float posX; float posY" _
)
Global $g_i_PosXYStructSize = DllStructGetSize($g_d_PosXYStruct)Of course everbody can create their own partial structs as needed, i.e. PosXY and MoveXY could be combined into one struct. 2.1.
We plan to define all fields of the AgentStruct as a global constant and use these constants in GetAgentInfo() and everywhere else, which will centralize the offsets.
I hope I could provide some useful information to the topic at hand and am happy to engagne in further discussion about pros/cons between GWA2 and GwAu3. |
|
Back from winter holidays 😄 @Gahais I kept GWA2_ and Utils_ as two different prefixes to differentiate functional and non functional requirements. But some renaming for coherence does make sense. And yes, there are some overlaps between Utils and Utils-Storage-Bot which will need to be fixed at some point. Omnifarmer is indeed not a farm and more of a utility class here. IDs will indeed need to be split in several files as it becomes way too crowded by now. I don't think it's worth making a file for every type though, but definitely separate items from skills/professions from maps for instance. All good points, thanks for bringing that up. @logan-77 Even if we have 200 agents in range and each of these agents takes 500 bytes (that's about the current size of an agent I believe), those structures only live when we need them, so memory will be immediately freed once those structures are not referenced anymore. Which means that the memory used by those 200 agents will be used only in the GetAgent function, once it returns, only one structure will be left in memory. So, memory usage is extremely limited in time. Secondly, 500 bytes times 200 is only 100 Kb - you'd need to have 10 bots fetching all those agents at the exact same time to even reach a single Mb. I'm not sure the memory improvement here is worth the additional CPU. But I agree that the in-between solution from Kleutschi is splendid. Fetching only the fields you care about uses the best of both worlds: a single ReadMemory with a smaller structure footprint. That could be an improvement to port to GWA2, though I'm not sure it's worth the trouble - I'd need to see some memory benchmarks to confirm/contradict my previous napkin calculations. Concerning centralization of offsets, you might already be going with this option, but I'd suggest automatically calculating offsets. It's a cheap thing to do once at bot start or first usage and be free of it forever, and no more having to manually update offsets after updates. And yes, I think I agree with you that keeping things mostly centered (it's already split with headers and GWA2 so we could probably split it off a little bit more) is probably correct for GWA2, that's how it survived until today. You did provide useful informations, thank you 😎 Something I forgot to mention that I absolutely want to bring up from gwau3 is the regroupement of the memory pattern with their offset. It makes sense and will make future fixes much easier. |
|
Hey, Gonna be really honest here, I don't know the first thing about, how AutoIt runs the code on the hardware side. What I know for a fact is, that my CPU load went down like 90%, when I switched from the regular GetAgentArray() to GetAgentPtrArray+necessary Dllcalls. I just looked at GetAgentArray and fed it into ChatGPT and it pointed out this loop:
To me it seems, this might be a reason for the high CPU load, I was talking about above, because it might potentially do a lot of Dllcalls? Another thing ChatGPT noticed is, that the whole AgentArray is physically copied twice (ReadProcessMemory → buffer, buffer → per-agent struct). > DllStructSetData($struct, 1, DllStructGetData($buffer, $i + 1)) ; copies each element
> ; (not sure if the syntax is correct, but I hope you get what its trying to do)
> $returnArray[$i] = DllStructCreate($agentStructTemplate, DllStructGetPtr($buffer) + ($i * 448)) ; only the pointer/reference is written to the new array(ChatGPT:) Modern CPUs hate: Big linear memory copies, Cache pollution, Repeated Variant creation I also asked ChatGPT, why my GetAgentPtrArray function causes way less CPU load, then GetAgentArray:
The conclusion was, that a hybrid is the best of both worlds. Batch ReadProcessMemory calls, but only copy the fields we actually use from process memory to the buffer. |
|
That loop is very interesting. Looking at it I thought it would only ever run once. So I had the opposite idea that it would basically never do a lot of DllCalls. I was surprised to find it does run more than once sometimes. And here is what I saw, running raptors: It's still mostly 1 memory read, though there are sometimes little spikes at 3-4-5 memory read (saw 9-10 a few times too). I supposed worst case scenario is some disconnection. Even failure doesn't affect those read. As for the correction ChatGPT offered, it's indeed a performance improvement, but it causes a much bigger issues down the line. i.e: read from memory into buffer, create structure inside array, copy from buffer into structure. We end up with a duplicate data, the cost is we copied it. We read memory into buffer, and we point our structure in returnArray directly on the memory in buffer (not sure what struct does here, probably chatGPT gibberish). Anyway, the problem is that we return returnArray from function, so it will be referenced after function lifetime, but not buffer. So now, we have returnArray, made of structures pointing into the void, and they can read all kind of gibberish if that memory is allocated somewhere else. |
|
Logan-77 seems to be right that the whole AgentArray is physically copied twice and remove these lines |
|
@Gahais no, this is not a good solution. There is no better way than to read the entire block in one call, and then copy from that block into what we want to return. |
|
okay, got it, maybe let's leave it as it is |
|
Kronos you know your stuff and you're right. The ChatGPT explanation just sounded so good and kinda logical, but it was a total hallucination... Now I'm more confused then ever, why my GWA2 bots use much much much more CPU% then my GwAu3 bots. |
|
@logan-77 If you find the cause for the increased CPU usage in GWA2 that you see, don't hesitate to share ! I'm super interested since I could port that improvement back to the hub and improve performances 😄 |
|
@caustic-kronos Automatically calculating the offsets sounds like an even better idea than just centralizing them, agreed. Its definitely something I want to do but I am currently procrastinating as I dread having to look through all functions and replacing all offsets. Regarding the notation, when Greg and I were working on the refactoring of GwAu3 we needed to settle on something and then stick to it, since collaborating on a project and every second function using a different naming convention is a nightmare in the making. As for the difference in CPU usage @logan-77 experienced I cannot really give you guys much of a clue I believe. For most of the heavy lifting (I am also not a huge fan of the individual memory field reads we currently have when it comes to performance), I use custom structs to avoid ReadProcessMemory and other heavy DllCalls as much as possible. I also cache a lot of information whenever possible, like party pointers as they are valid until an agent leaves compass range, IDs of party members are also constant until the map changes, etc. This also avoids some unnecessary memory reads. However, all of this doesn't really apply to what someone would experience if they migrate their scripts. If there are any other questions I can help answering I am happy to do so, either here or via Discord. |
|
Hey @KleuTSchi, thanks for chiming in, much appreciated 😄 Yeah... replacing hardcoded offsets definitely sounds like the kind of can I would kick further down the road for a few months before finally taking care of it. On the lookup-table vs switch topic: I don’t think values that require a bit of computation necessarily disqualify lookup-based approaches, as long as the contract is clear. If something returns HP%, that’s fine - just make it explicit that it’s a percentage and not raw HP. Same idea with pointers: returning a pointerToX instead of immediately following it inside the switch makes the intent very clear and keeps responsibilities separated. Regarding notation: I completely get why you and Greg settled on one convention and stuck to it. As for CPU usage, I never used gwau3 extensively, so I can’t compare directly - but based on what you describe, the performance improvements make some sense. Caching stable pointers, avoiding repeated RPM calls, and relying on logic/timers instead of brute-force reads are all solid design choices. I try to follow the same philosophy where I can, but I’m not surprised if Logan noticed a difference. Anyway, thanks again for taking the time to talk about all this, always interesting to see how other people approached similar problems. I might take you up on that Discord offer at some point 🙂 Cheers ! |
|
I’ve updated the bot to use the gwau3 backend 🍾 What this means:
What this does not mean:
Why this is useful:
Changes and improvements made during the port - while porting gwau3, I also refactored and improved several parts.
Also found a small issue during the port: QueuePtr is referenced but never defined anywhere. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hello,
I’d like to open a discussion here regarding gwau3, with the goal of sharing observations and gathering feedback.
To be clear from the start: I currently have no intention of switching the hub to gwau3. That said, I’m very open to suggestions, ideas, and counter-arguments. If you think I’m missing something or disagree with my conclusions, please don’t hesitate to share your perspective here — this is meant to be a discussion.
While applying the fixes that Greg and Kleutschi made in gwau3 for Reforged to the hub, I noticed several patterns and ideas that could potentially be brought over, as well as others that I’d prefer not to adopt. I’ll try to outline both below.
My broader goal would be to move closer to gwau3 where it makes sense, especially for compatibility, easier fixes, and reusing good work, while still preserving the design choices of the hub.
Areas where I’m hesitant to follow gwau3’s approach
gwau3 uses a Hungarian-style naming convention, for example:
$a_i_MaxRegionsThe hub, on the other hand, follows a more Java-style, descriptive (semantic) naming convention, which I personally find much easier to read, understand, and reason about.
In my experience, Hungarian notation adds visual noise without providing much additional value, especially given modern screen resolutions and tooling. In most cases, a name like maximumRegions (or maxRegions) already conveys enough information, including the expected type.
This is ultimately a stylistic preference, but it’s one where I strongly favor the hub’s current approach.
gwau3 frequently accesses structure fields directly by offset, for example:
Compared to the hub’s approach:
ReadProcessMemory is relatively expensive. If only a single field is needed, the per-field approach makes sense. However, as soon as multiple fields are accessed, reading the full structure once tends to perform significantly better.
From a maintenance standpoint, defining structures also centralizes offsets and makes updates easier when a structure layout changes. For these reasons, I personally disagree with the overall direction taken by gwau3 on this topic, even though there are valid use cases for both approaches.
I generally agree with the idea of splitting code into specialized files to keep things clear and focused. However, in my opinion, gwau3 takes this quite far, resulting in a structure that feels overly fragmented and harder to navigate.
This is again subjective, but I prefer a slightly more consolidated layout.
Things I would like to bring over
Analyzing the existing direct field accesses and using them to define proper structures
Porting certain functions that are present in gwau3 but missing from gwa2 / the hub
I’m very interested in hearing other opinions on this — especially from people who have worked extensively with this the hub or gwau3. My goal here isn’t to dismiss that work, but to figure out where alignment makes sense and where different design choices are preferable.
All reactions