Correct out-of-the-box HiDPI support in SDL, GLFW and Android apps

[mosra]

Got myself a Spectre x360 with 4K screen and the current HiDPI "support" in Magnum is making me sad so I did some research. Saving it for the moment, postponing to after #233 is done as this is apparently a bit more work than just flipping a switch in SDL.

What needs to be done:

• Static way to query display scaling (SDL_GetDisplayDPI(), calculating from milimeters in GLFW). Win/Linux and macOS have different default DPI. However, getting physical monitor DPI on Linux is probably not what one wants (e.g. I have 282 DPI but set the scaling only to 240% (230 DPI). SDL_GetDisplayDPI() returns 282, xrdb -query | grep Xft.dpi returns (correct) 230). Further info about how to implement and how to get this in GLFW. Make the latter the default on Linux (and possible to fall back to physical? or does xrdb return the physical value if not overriden?) On the other hand, does the builtin DPI query in SDL report the correct virtual (not physical) DPI on Win / macOS? GLFW will have this in 3.3.
  • support in SDL2 on Linux and Emscripten -- ae31c3c
  • GLFW Linux support -- dba35ba
  • Windows support in SDL2 and GLFW -- 2253987
  • Use glfwGetContentScale() on GLFW 3.3+ instead of the direct queries? Unfortunately the code has a fallback to physical DPI if retrieval of given value fails (https://github.com/glfw/glfw/blob/1bf892f603b2479666b0035eb1bc214b05364ad7/src/x11_init.c#L794-L828), which was subsequently removed as KDE seems to unset Xft.dpi if content scaling is 100% (glfw/glfw@399c082) so it's not really any better than mine as the fallback is implicit without giving the user any option to pick one or the other.
  • Use some builtin API on SDL2 instead of the direct queries? Seems like SDL_GetDisplayDPI() does both as of 2.0.22 (In x11, GetDisplayDPI can give incorrect or unusable DPI information.… libsdl-org/SDL#4220), but in my case (on 2.0.28) it doesn't seem to work and returns some imprecise value instead. So the same implicit fallback out of user control as with GLFW but without a workaround KDE unsetting Xft.dpi, so again, the direct queries I have seem to be superior.
    • SDL3 has everything completely changed, behaving like a Mac on all platforms: https://github.com/libsdl-org/SDL/blob/main/docs/README-highdpi.md
  • Add KDE detection (KDE_SESSION_UID env var) and fallback for the Xft.dpi code
  • Add a Wayland variant of the X11 Xft.dpi code (branching based on XDG_SESSION_TYPE being x11 or wayland? getting some Wayland "context" handle from GLFW and SDL?)
    • Essential code to handle the fractional scaling protocol (new since Nov 2022) in SDL is in wayland: Support wp-fractional-scale-v1 protocol libsdl-org/SDL#5906
    • Still need to figure out how to actually query it, adding an async listener or some such feels extremely overcomplicated? Hello world app: https://github.com/emersion/hello-wayland/blob/master/main.c, some tutorial: https://wayland-book.com/libwayland/interfaces.html
    • What about the original non-fractional scaling? Or can one assume that all WMs will eventually use the fractional scaling protocol?
• Enabled-by-default option to treat the requested size as "a physical size of the window that one would get on non-HiDPI systems" and scaling it to the current (virtual) DPI using the above query (meaning the window and framebuffer pixel size will get bigger). Make it possible to control that programatically through Application::Configuration (e.g. override to 1.73, completely disable when things go really bad (misconfigured X11, e.g.), switching between physical and virtual dpi?)
  • support in SDL2 on Linux and Emscripten -- ae31c3c, 56a933b,
  • GLFW Linux support -- dba35ba
  • Windows support in SDL2 and GLFW -- 2253987
• Ability to get back an unscaled window size (for e.g. saving and restoring window size to avoid it growing every time) -- by dividing window size by dpi scaling -- done for SDL2 on Linux and Emscripten in ae31c3c, for GLFW in dba35ba
• Ability to control such scaling from a --magnum-dpi-scaling command-line option and environment -- done for SDL2 on Linux and Emscripten in ae31c3c, for GLFW in dba35ba
• Resizing a HiDPI window is no longer about just scaling the framebuffer and event handling the same way, one will need to ask for SDL_GL_GetDrawableSize() (and SDL_Vulkan_GetDrawableSize() explicitly next to the new reported window size from the event. Do it via a new ViewportEvent type, deprecate the old event function. -- done for all apps in c0125fa, 25d0bb8, ed0a719
• I should use glfwGetMonitorContentScale() to get virtual DPI scaling on Windows -- 444b925
  • And some WINAPI to get physical DPI scaling in SDL2 (because the current is only virtual) -- c3878c9
• What should specifying custom DPI scaling actually do? On EmscriptenApp, in case of ImGui, it makes the app change size as well (I would assume it only changes pixel density), on desktop it changes window size. This also further complicates reacting to DPI change events (DPI change events in Sdl2App and GlfwApp #423).
• Document best practices, hint framebuffer blit to save rendering power on HiDPI systems

With this, an 800x600 window should ideally have the same physical size regardless of monitor DPI. Question is about relation of framebuffer size and virtual screen size (coordinate system for events):

• Should it be always the same as the requested size? That might not be possible under X11, because there it seems to be always 1:1 relation between pixels and points. SDL has SDL_RenderSetLogicalSize(), but that's not for GL/Vulkan it seems. GLFW 3.3 will have GLFW_HIDPI_RESIZE.
• Or, does it matter at all if its consistent or if its consistent across platforms? In the end there's some projection matrix that does the final scaling of scene coordinates to window coordinates, the only problem is event handling, but for that we can tell users to always query windowSize() and scale their event coordinates (currently reported as integers in both SDL and GLFW App implementation) according to that value. Moreover, the UI library already is designed with this in mind.
• There are three separate concepts now: window size (for events), framebuffer size (for rendering) and DPI scaling (to know how big the elements should be on the screen)

The following needs to be verified:

• On macOS, if an app doesn't have NSHighResolutionCapable and doesn't allow HiDPI via SDL, it's treated as "old" and gets some scaling from the system to avoid appearing too small. Does the framebuffer size stay as requested? -- Yes.
• On Windows, under similar conditions, the app should get also a scaled window. Does it? Is the framebuffer size also preserved? -- Yes.
• When I enable NSHighResolutionCapable or the Windows equivalent and request a scaled size, does it do the expected thing (on both SDL and GLFW)? Or is the window scaled by the system again? -- On macOS the window and framebuffer size is no longer the same (so one has to request the same size always), on Windows one has to request larger window size.
• Can I detect presence of the flag programatically and disable the application-side scaling to avoid the window being scaled too much? -- Possible (and implemented) on both iOS and macOS

Further work:

• There needs to be some way to check the framebuffer and window ratio on macOS and iOS before opening a window -- for example to decide if/how much MSAA is needed. The dpiScaling() query is relative to window size, which is 1 on macOS, so not helpful. Or make dpiScaling() relative to framebuffer size? Would that solve it? Or break something else?
• Creating a SDL window on iOS with default size will make SDL "pick a resolution", is this resolution the full Retina? Or do I have to force that explicitly? Can some "just give me all pixels" behavior be implemented? How does Android work here?
• Responding to events when the app is moved across monitors with different DPI. Extremely complex topic on its own, moved to DPI change events in Sdl2App and GlfwApp #423.
• Events in Emscripten apps in the browser on Android probably suffer from some DPI scaling problem, investigate (can't click on anything in the UI gallery) -- not a problem in the new EmscriptenApplication anymore, was something in the emscripten SDL emulation layer (Emscripten application #300)
• Implement support in the AndroidApplication as well -- getting separate window, framebuffer (there is ANativeWindow_Buffer, could that be used?) and DPI scaling values (NDK can give me only an enum for DPI scaling, do I need to use JNI? https://stackoverflow.com/a/18858569 What about the framebuffer size?) -- "just works" as long as the app is targeting new enough SDK, FB size is then same as window size
• Since the go-to way to enable HiDPI on Windows is via a manifest file, provide a nice way to embed it via CMake. While it's probably builtin for MSVC (just adding a manifest file to sources?), it's harder with MinGW. 1, 2 -- not so hard after all, done with 2253987 (docs)
• Provide a way to supply Windows manifests via INTERFACE_SOURCES of Magnum::*Application CMake targets? Could make it nicely "just work" for all examples, without need for manual boilerplate. What about macOS / iOS? Also needs a way to disable such behavior.

Related info:

• https://twitter.com/czmosra/status/996804570016821248
• In order to position a window on another monitor, SDL2 puts all displays into one big "virtual screen". However, there is a bug that miscalculates screen sizes if Windows has DPI scaling enabled. The last comment (from 2016!!) suggests adding an API to query display scaling. No reply since.

[noisiak]

I just came across Magnum for the first time and it shocks me that all the WebGL showcase examples in the website look blurred and non-retina.

Good to know this is being addressed, hope the web demos in the showcase could be updated soon, this way new people interested on using Magnum wouldn't get confused about retina support.

[mosra]

@noisiak oh, thanks for the reminder, almost forgot it's a problem on WebGL as well. Didn't investigate yet how is it with Emscripten (and WebGL in general) and HiDPI. Any hints where I should look?

[elmindreda]

Responding to events when the app is moved across monitors with different DPI. Windows has WM_DPICHANGED, but that doesn't seem to be exposed to SDL. Submit a patch? What about GLFW?

(The still unreleased) GLFW 3.3 has a window content scale callback corresponding to WM_DPICHANGED or viewDidChangeBackingProperties.

[mosra]

@elmindreda yay, awesome, thanks for letting me know :) GLFW is great.

[mosra]

Initial work in SDL2 apps for Linux, macOS and Emscripten is done in ae31c3c, 769bc0d, c0125fa, 25d0bb8, ed0a719 and 56a933b. For the next release I still want to make this working on Windows and have feature parity with GLFW. Android support will come probably later.

[mosra]

Showcase on http://magnum.graphics/showcase/ is updated with proper HiDPI support since mosra/magnum-website@f0ffb93.

[mosra]

Initial support in GLFW apps (matching the SDL support 1:1) is done in dba35ba.

[elmindreda]

Enabled-by-default option to treat the requested size as "a physical size of the window that one would get on non-HiDPI systems" and scaling it to the current (virtual) DPI using the above query (meaning the window and framebuffer pixel size will get bigger).

(The still unreleased) GLFW 3.3 now has the GLFW_SCALE_TO_MONITOR window hint that does this on Windows and X11, though it's disabled by default for compatibility reasons.

[mosra]

@elmindreda amazing, thank you again 👍 Looking forward to all the code I can delete once 3.3 is out :)