Skip to content

main: search for a CUDA VA floor - #68

Open
luizhtss wants to merge 1 commit into
Jetsku:mainfrom
luizhtss:cuda-floor-auto
Open

luizhtss wants to merge 1 commit into
Jetsku:mainfrom
luizhtss:cuda-floor-auto

Conversation

@luizhtss

Copy link
Copy Markdown
Contributor

With the floor at 1024 GiB, cuInit fails every time with CUDA_ERROR_OUT_OF_MEMORY (2), so KYTY_CUDA_EARLY_INIT=1 does nothing and Smooth Motion stays off. CUDA wants its range below 1 TiB. I checked this by holding every free range below a floor and calling cuInit in a plain process (RTX 5070 Ti, driver 617.42): 512 and 1008 GiB work, 1024 GiB fails. Inside the emulator 1008 GiB failed too, because CUDA asked for more room there, and 944 GiB worked.

So the floor is now searched. The first try is the end of the guest range, which leaves guest VA alone when CUDA fits above it. A failed cuInit can be retried, so on failure the floor drops 64 GiB at a time. If CUDA ends up inside the guest range, the startup line says how many MiB it took. KYTY_CUDA_VA_FLOOR_GB still forces a single floor, now without the 1024 GiB minimum.

The end of the guest range moves to kernel/memory.h as GUEST_ADDRESS_SPACE_END, and HOST_USER_MAX is derived from it, so the floor follows if the guest range ever changes.

…024 GiB

With the floor at 1024 GiB, cuInit fails every time with
CUDA_ERROR_OUT_OF_MEMORY (2), so KYTY_CUDA_EARLY_INIT=1 does nothing and
Smooth Motion stays off. CUDA wants its range below 1 TiB. I checked this by
holding every free range below a floor and calling cuInit in a plain process
(RTX 5070 Ti, driver 617.42): 512 and 1008 GiB work, 1024 GiB fails. Inside
the emulator 1008 GiB failed too, because CUDA asked for more room there, and
944 GiB worked.

So the floor is now searched. The first try is the end of the guest range,
which leaves guest VA alone when CUDA fits above it. A failed cuInit can be
retried, so on failure the floor drops 64 GiB at a time. If CUDA ends up
inside the guest range, the startup line says how many MiB it took.
KYTY_CUDA_VA_FLOOR_GB still forces a single floor, now without the 1024 GiB
minimum.

The end of the guest range moves to kernel/memory.h as
GUEST_ADDRESS_SPACE_END, and HOST_USER_MAX is derived from it, so the floor
follows if the guest range ever changes.

Tested with Astro Bot. Kyty prints "CUDA: initialized with the floor at
944 GiB, inside the guest address space (ends at 1008 GiB): CUDA holds
57431 MiB of guest VA", and the NvPresent log shows "Feature Smooth Motion
requested. System support: 1. Device support: 1" followed by its swapchain.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant