Jokolay and window managers

[coderedart]

Window Manager is what deals with the layout/position of the windows , decorations, window frames/borders, focus, stacking order etc..
Compositor deals with the drawing part. it draws the decorations on a offscreen buffer, copies the window's framebuffer into the decorations, deals with the translucency and other color stuff before finally rendering it onto the monitor.
First, we must understand the 3 states that Guild Wars 2 can be in.

1. Window: in this mode, it acts just like a normal window with decorations. especially useful if you want to put this on left side of monitor and another app like firefox/video player on the right side to watch a movie. or when you want to keep an eye on it, but not let it occupy the whole screen. very similar in windows or linux, but there is a chance of crashing if using arcdps on linux.
2. FullScreen: you can select various resolutions, and gw2 will render in that resolution by taking over the desktop. the game draws directly on to the monitor's framebuffer instead of the copying that compositor usually makes in window mode. when you alt-tab, game will be minimized and your desktop resolution restored to normal to draw the windows as the control is back to compositor now which does the framebuffer copy into a offscreen texture thingy for the visible windows. back in the day, this was the best way to get full performance as all resources are given to game, and compositor doesn't do much at all.
3. Windowed FullScreen: it is a hybrid mode where the game's decorations are stripped off, game resized to the monitor, but it is still a window. the compositor is still copying the game's window into a seperate texture where it builds the who used to have "slightly" reduced performance in a game in past, but it is almost as performant as fullscreen these days due to compositors d letting the game draw directly on to the framebuffer of monitor unless there's another window visible, at which point, it will take draw that window on top of the game.

On Windows, we use the TOP_MOST attribute. refer window style hints and setting it dynamically . this will set the window to be above windowed fullscreen gw2.

The main issue we will be dealing with is that linux does not differentiate between windowed fullscreen and fullscreen. most window managers will try to "emulate" the behavior of windows. when using windowed fullscreen, they will allow you to alt-tab without minimizing game window and the game's resolution must still match that of the desktop/compositor. so, if gw2 is running on a desktop with FHD resolution, then the windowed fullscreen mode is running at FHD. otoh, if we use fullscreen with different resolutions in gw2, then the desktop also changes resolution just like windows and will even minimize gw2 when alt-tabbing.

But it is just an emulation. the spec does not care about it at all. what the spec recommends is the stack order https://specifications.freedesktop.org/wm-spec/wm-spec-1.4.html#STACKINGORDER which says that a "always on top" window must be BELOW a "focused fullscreen" window. it means that the overlay will be pushed into background when gw2 window is focused (by clicking on it for example) either in fullscreen or windowed fullscreen mode. the key word is "focused", so overlay can be technically on top of windowed fullscreen gw2 as long as it is focused. but its still a recommendation and wm like i3 do not implement that order.

On Linux, we have to use window mode for now. even if overlay stayed above windowed fullscreen mode gw2 in any particular DE, it is accidental and cannot be relied upon as standard behavior. we have a temporary workaround to get "almost" the desired behavior. It must be remembered that there's a lot of window managers and we must target the spec rather than a particular WM to make sure the most people can run it.
Workaround:
We can resize gw2 window to be borderless (decoration less) and resize it to the monitor. this will feel like a fullscreen window except for the taskbar panel being visible. we can set taskbar panel to autohide mode. now, overlay can stay on top of the gw2 window while also feeling like its in fullscreen. one caveat is that resizing without decorations is not fun at all.

A more permanent solution would be to use the "Transient_For" hint and set gw2 as the parent window of our overlay. i have not looked much into this because it would mean adding the dependency for X11 and sending manual commands.
future reference: https://stackoverflow.com/questions/31225743/x11-non-clipped-child-windows/31436071#31436071
The stacking order in netwm spec recommends that the child windows that have "transient_for" set to a parent window must appear on top of it. this also means that we now need to find the gw2 window on the linux side for the particular mumble link we are dealing with.

there's also the option to completely skip the window manager by using override_redirect hint. then, we can avoid place jokolay wherever we want and on top of all the other applications. we would need to experiment and find out whether window manager can place their focused windows on "top" of our override_redirected overlay window.

[coderedart]

I made a bug report that windowed fullscreen must be below "always on top" windows in kde forums. https://bugs.kde.org/show_bug.cgi?id=441074
although idk what answer i am expecting. they can add additional hint only in kde or add it to spec. not expecting much though.

[coderedart]

i3 developer has specified that the stacking order is more of a recommendation and not necessary. In issue i3/i3#4478 , he has mentioned that i3 will not be supporting stacking order between floating windows, but floating windows will definitely stay on top of tiled windows, so gw2 has to be in tiled mode and overlay has to be in floating mode for this to work.

[coderedart]

kde https://bugs.kde.org/show_bug.cgi?id=441074 . They have recommended to just use a kde specific hint like critical notification https://api.kde.org/frameworks/kwindowsystem/html/classNET.html#a4b3115c0f40e7bc8e38119cc44dd60e0 . we can try to see if that will keep it above windowed fullscreen gw2.

[coderedart]

to use the transient_for method or to get the gw2 window normally, we will need to find the xid of the gw2 process. especially considering multiboxing. wine seems to provide a way to get xid of an app via simple method call. https://github.com/robbert-vdh/yabridge/blob/6eac5ce336c8e756fd7df52a5d4bf1c20d07a1c6/src/wine-host/editor.cpp#L1340-L1343
The above reference link is provided by a helpful robbert person in the wine libera chat irc channel. <3
another reference: https://github.com/wine-mirror/wine/blob/e909986e6ea5ecd49b2b847f321ad89b2ae4f6f1/dlls/winex11.drv/window.c#L2071

We can just write xid at the end of mumble data.

1. we need to check if we can get the xid of a different app.
2. we need to check how it works with rust. otherwise, we will need to rewrite Jokolink in cpp.

[coderedart]

alright, this works. we can get the xid from jokolink reliably https://github.com/coderedart/jokolink/issues/1. and by using that, we can start experimenting with TRANSIENT_FOR and other ways of using x11 against the xid of gw2 window including getting the window size (which we relied on jokolink to provide, but that has been deprecated as we can just make that function in gw2.

We will try continuing with x11rb as it seems to be a mature, but a little clunky in terms of api. we will only be setting the parent once, and mostly using x11 or other linux utilities to check if gw2 window is closed (so we can close too) or get its size. both of these are OS dependent, so we might use a trait so that we can have all os specific functions defined in the same place.

EDIT: setting gw2 as x11 parent of jokolay is bad because child windows clip their parent windows, and all the area covered by jokolay will be a black clipped rectangle. https://stackoverflow.com/a/31436071/7484780
so, transient_for should be enough for most usecases.

[coderedart]

window manager specific hint for keeping an app above even fullscreen windows for kde is at https://github.com/KDE/kwin/blob/master/src/libkwineffects/kwineffects.h#L2403

_KDE_NET_WM_WINDOW_TYPE_CRITICAL_NOTIFICATION

[coderedart]

was able to successfully keep Jokolay above Guild wars 2 in windowed fullscreen AND fullscreen modes. with either transient_for or the critical notification window type hint. we will ignore the critical notification type as it is not useful outside of kde.
But once you focus the gw2 window, the pointer is "grabbed" and it is not possible to interact with the visible jokolay.
ways to work out focusing properly:
we can check when egui needs input and do window.focus(). this has two issues. first is that we can't "unfocus" our overlay when we don't need it. second is that it will also focus it when we are rightlicking in game window to control camera, and the hidden pointer "moves" to the egui area and suddenly overlay gets focused. but if we keep track of the pressed mouse buttons of camera, we should be able to not focus under this corner case. for the first problem we can simply set focus to gw2 window using x11 when we don't need the focus.

the slighly more complicated way is to send a "_NET_ACTIVE_WINDOW" event. this might be a better way, but we need more x11 knowledge. xdotool has this stuff https://github.com/jordansissel/xdotool/blob/6c1db901e22f21eb8ffa1ea03da562a59c115067/xdo.c#L480 . we can also research more on focus vs active window. especially in terms of grabbing, and how it works with window mode normally.

the most complicated method is to grab the pointer with the event mask of Button_Press, that way, when the mouse hovers over egui, we can grab and when it clicks we will get the focus (i think). and when it leaves the egui area, we can just ungrab the pointer. we will need to keep track of pointer grabs/focus and other stuff, but this will be reasonably clean and intuitive.

IDEALLY, we will want to simply check the source code of window managers and find out how they deal with focus stealing. because when gw2 is in window mode, we don't have to think about all this stuff. if mouse goes over to egui, and we click, we can be sure that egui will receive that click and gw2 won't. this must be a way of pointer grabbing. while the mouse is hovering over egui, the keyboard events like wasd are still going to gw2. replicating this behavior would be the ideal path forward.

for now, we will go with the easiest way of setting input focus manually or even ignore the whole "windowed fullscreen" stuff. we already spent too much time on polishing stuff, and it is good for the morale to move fast to a reasonable alpha release so we can get some testers who can give us feedback on all the above methods.

we are tracking what the intended behavior for focus stealing is with windowed fullscreen in kde at https://bugs.kde.org/show_bug.cgi?id=442720 .

we have already verified in gnome/kde/i3/xfce that windowed fullscreen won't let jokolay be focusable. so, unless we get some kind of new idea/method from that bug report, this seems to be a dead end for now.

[coderedart]

using Xembed is not ideal because both parent/child windows need to communicate about their focus and other properties. and gw2 won't communicate as either parent or child, unless we mess around with wine code and make a large effort like yabridge repo does.

[coderedart]

to know about the virtual desktops, the source of wmctrl is a good reference https://github.com/dancor/wmctrl/blob/master/main.c#L1063

and to know about the events happening on gw2 window, we can use xev as reference https://gitlab.freedesktop.org/xorg/app/xev/-/blob/master/xev.c#L1407

we can also simply use xev from terminal with
xev -id 0x06800005 -event keyboard -event expose -event visibility -event structure -event substructure -event focus -event owner_grab_button -event randr -event button
replace the window id by getting the proper window id of gw2 via either xprop or something like wmctrl -l -G

[coderedart]

wayland has support for passthrough as input_regions https://wayland-book.com/surfaces-in-depth/surface-regions.html

and ofcourse, glfw also supports that.

Need to check if set_transient_for is required in wayland or not.