Which is the use for the new modules support in WSL 2.5?

[crramirez]

Pardon my ignorance about this topic, I'll explain it.

With kernel 5 in WSL2 if I wanted to have a module / driver that is not available, I go to the Microsoft kernel sources, enable the module, for example, ZSWAP=Y and then compile the kernel. Then I put the kernel path in the .wslconfig. Each time Microsoft releases a new kernel, I need to do the same.

With the new kernel 6, if I want, for example, ZSWAP, I go to the Microsoft kernel source, enable the module like ZSWAP=M and compile. Then after running some scripts, I get the kernel binary and the kernel_modules.vhdx. I figure that I only need to put the kernel modules path in .wslconfig and not the kernel binary. The worst case I need to put both. As I remember from running Linux on bare metal, each time the kernel changed, I needed to recompile the NVIDIA driver module or download the precompiled ones. It means, as I understand, each time Microsoft releases a new kernel, I'll need to recompile and generate again the kernel modules vhdx.

As I described, it looks like the new approach is worse.

Can anyone correct my logic or highlight why it is better?

@benhillis @OneBlue @craigloewen-msft and any other that can help.

Best

[lars18th]

Hi @crramirez ,

Some of your assumptions are wrong:

• The first is this: when recompiling a new WSL kernel you will always obtain the kernel and the modules. The difference between the 5.x and 6.x kernels in WSL is that Microsoft is moving the modules to a “packaged” version. That's a new VHDX file system with all the modules. However, you can manually copy them to your root file system (on every WSL boot). But doing it this way isn't easy to use. And for that reason support for new vhdx modules volume support has been added.
• The second is the following: You can compile out of tree any module for the kernel. You don't need to recompile a new kernel. You can compile binaries for a running kernel. However, this is not easy: you need the same sources, the same configuration and the same compiler. But you can get any .ko file you need. The problem here is how to use these files. And for this reason we have requested support for multiple volumes of “modules vhdx”.

Please keep commenting in the original thread: #12586 😉

[crramirez]

hello @lars18th ,

Thanks for your answer. I didn't explain me well enough. What I am trying to illustrate is that in WSL prior 2.5, if I need a new "driver" like for example zswap in all the distros, I needed to compile the kernel and bake the "driver" into it. Now with WSL 2.5 I can define my driver as a module ZSWAP=M and after the compilation I'll get my kernel that I don't need to use, and my module is now in a vhdx file.

Fine, my question is. What is the advantage of the new method? In both if I want a driver that it is not included in the kernel or as a module, I will need to recompile every time a new kernel arrives.

About the thread that you mentioned. This is an issue asking for documentation on how to tell wsl about the modules.vhdx. This question was answered and the documentation was generated. In my opinion, this issue should be closed. But a subthread was created inside that issue asking for multiple vhdx, that in my opinion, should be a new discussion. I simply don't want to add more mess to it, and I prefer creating a new one, because it is not related to the documentation or multiple volumes.

[lars18th]

What is the advantage of the new method?

Compiling a new driver only takes less than a minute. Compiling an entire kernel takes a some minutes (or more). In addition, for drivers outside the tree it's necessary to compile them as modules. And to support “modules” outside the distribution file system it's necessary to mount an additional file system with the modules linked to the running kernel.

[crramirez]

I need to get involved more in the process of only compile modules. My only experience was NVIDIA drivers on bare metal Linux and this needed at least the kernel headers.

[craigloewen-msft]

Is the main issue here that you have to compile both the kernel and the modules for each new kernel change? I understand this adds an additional step when using a customer kernel. But this would be expected behaviour for us when we moved to kernel modules, which gives benefits to the regular user.

[lars18th]

Is the main issue here that you have to compile both the kernel and the modules for each new kernel change?

No. We know how to compile kernel modules for the stock kernel. New kernel is not necessary.

I understand this adds an additional step when using a customer kernel. But this would be expected behaviour for us when we moved to kernel modules, which gives benefits to the regular user.

The problem is how to use multiple volumes of stock/custom kernel modules. With only one volume you need to replace and merge to obtain only one VHDX file.

[crramirez]

Hello @craigloewen-msft ,

I talked about the compiling of the kernel to illustrate an example.

But my real question is in fact "which are the benefits to the regular user"

@lars18th the thing about multiple volumes belongs to the thread that you mentioned. My question is unrelated to that. And this is not what I am asking.

[lars18th]

Hi @crramirez ,

As to the question of “what are the benefits to the regular user” I have this answer:

• The original WSL kernel includes only the necessary drivers. And now some of the drivers with the 6.x kernel are compiled as modules to reduce memory overhead. But in order to use them the new VHDX volume has been added with the modules. This is necessary to disaggregate the modules (linked to the kernel) from the linux distribution file system (not linked to the kernel).
• If you want to add new drivers to the WSL kernel you can compile them into the kernel (driver=Y) or as a module (driver=M). And this is no different with 5.x and 6.x kernels. Only the stock support for the modules.vhdx volume is new.
• If you decide to compile the new drivers as modules, then you don't need to recompile a new kernel and can use the stock one. With the modules.vhdx this opens up a new way to use them.

I hope these comments have clarified your question.

And other different questions are: And why would a user need to compile a new driver (i.e. USB devices)? Or, why would several modules.vhdx be needed (i.e. third-party drivers)?
😉

[crramirez]

This means that it is possible that WSL devs include optional modules, or include more modules than the bare minimum like today?

For example a user is asking for the modules (ashmem, binder, and memfd) in #12601. I may expect that they will be included and the user only needs to activate them on the kernel parameters in .wslconfig? If this is the case I see a lot of benefits for the end user. But if they will include exactly the same drivers but now as modules and if I need a new one I need to compile all of them + mine, I don't see the advantage.

[lars18th]

PING: We need support for third-party drivers !

[Andarwinux]

This is essentially pointless and only increases complexity. A machine capable enough to run WSL2 is clearly not low-memory embed shit, so just build all features as builtin (=Y). Is compile time really that important? You can build a ThinLTO kernel in just a few minutes using PGOed clang.

The primary use case for modules in traditional kernel builds is to isolate hardware-specific drivers, but WSL2 has no need for this approach because its drivers are either generic or merely platform-specific.

[lars18th]

Hi,

Your assumption is incorrect on these two points:

1. In some environments, WSL2 is used for cross-platform development, and USB devices are employed. In this environment, installing third-party drivers is necessary.
2. Futermone, when you need to use WSL2-based tools (such as Docker Desktop), it's preferable to keep the default MS-Linux kernel. The reason for supporting third-party drivers is to provide a clean way of using them.

Please remember that not all environments are the same and that yours isn't everyone's.

[Andarwinux]

1. In some environments, WSL2 is used for cross-platform development, and USB devices are employed. In this environment, installing third-party drivers is necessary.

WSL2 isn't even natively compatible with USB passthrough. I can't imagine any masochist trying to debug USB hardware using USB over IP. And I also can't imagine a embed hardware developer lacking the resources or ability to install Linux on baremetal.

2. Futermone, when you need to use WSL2-based tools (such as Docker Desktop), it's preferable to keep the default MS-Linux kernel. The reason for supporting third-party drivers is to provide a clean way of using them.

This is purely a skill issue. If you know which kernel features Docker Desktop relies on, you don’t need to worry about this.

[lars18th]

Hi,

Two things: firstly, I'm using and developing USB drivers via WSL2 within Docker Desktop. So please don't jump to conclusions. Secondly, the use of USB devices in WSL2 isn't affected in any way by the usbipd layer. The problem is with the current method of using third-party drivers.

In my opinion, the WSL2 developers would do well to provide a simple way to load any kernel driver from a VHDX file. 😃

[Andarwinux]

Two things: firstly, I'm using and developing USB drivers via WSL2 within Docker Desktop. So please don't jump to conclusions. Secondly, the use of USB devices in WSL2 isn't affected in any way by the usbipd layer. The problem is with the current method of using third-party drivers.

What’s going on here? As a Linux driver developer, hasn’t your employer provided you with a Linux workstation? Or are you voluntarily making things difficult for yourself? If that’s the case, then I’d have to quote you back: not all environments are the same and that yours isn't everyone's.

And even without vhdx-based module support, you can still load ko manually. What I object to is the current approach: modules for the sake of modules, which modularizes even non hardware-specific drivers. No one objects to you manually loading your own hardware driver kernel modules.

[lars18th]

I don’t want to start an endless debate. I just want to point out that, in some environments, it’s essential to be able to load custom kernel modules distributed within a VHDX file that can be easily added to the standard path /lib/modules. That’s all. I’d appreciate your comments.