How to add a custom module when using disnixos + nixops?

[ibizaman]

Hi!

I want to make some changes to the postgresql-database.in module. I figured the easiest would be to copy it, modify it, and ship it with another name so dysnomia could see the new module. But I fail to do that in at least two ways:

1. All the @var@ variables do not get substituted correctly (especially the #!/bin/bash at the top).
2. I don't think dysnomia sees the module I copied over using nixops

I can share my full config if needed but here are the relevant parts:

In network.nix:

{
  myserver = { pkgs, ... }:
    let
      dysnomiaExtraModulePath = "dysnomia-modules";
    in
    rec {
      dysnomia = {
        enable = true;
        enableLegacyModules = false;
        extraModulePaths = ["/etc/${dysnomiaExtraModulePath}"];
      };

      environment.etc = {
        "${dysnomiaExtraModulePath}/postgresql-database-secret" = {
          source = ./dysnomia-modules/postgresql-database-secret.in;
          # mode = "0777";
        };
      };
    };
}

For the purpose of this issue, there's no need to show the specific changes to the dysnomia module I made, the issue is relevant with a verbatim copy of any module.

Issue about replacing shebang and variables

Afterwards I deploy using nixops:

$ nixops deploy -d vboxtest

And can see the file but none of the variables were replaced (which makes sense):

$ nixops ssh myserver
$ head /etc/dysnomia-modules/postgresql-database-secret -n 1
#!/bin/bash

Looking at an official module, the shebang gets replaced correctly:

$ nixops ssh myserver
$ head /nix/store/9d7jhh7xsyyx933blk0c504qff609qkl-dysnomia-0.10.1/libexec/dysnomia/postgresql-database -n 1
#!/nix/store/bm7jr70d9ghn5cczb3q0w90apsm05p54-bash-5.1-p8/bin/bash

I'm guessing I'm missing some call to one of the substitute* functions but I tried to grep in your repos and I don't see where that happens.

Issue about not finding module

The second issue comes from not being able to locate the module. At least that's what it looks like but I know the bash error messages can be cryptic and can hide a permission issue as something else for example.

Anyway, deploying returns this error:

$ disnixos-env -s services.nix -n network.nix -d distribution.nix --use-nixops
[...]
[coordinator]: Executing activation of services:
[target: myserver]: Activating service with key: 6c7779e925f58420b6761b4b04b8ada3412f8fcc8071ea2bc823dd943d5f6544 and package: /nix/store/47l1mkclinkqwfwgwd9r1i95vk1msrfy-ttrss with module: postgresql-database-secret in container: postgresql-database-secret, arguments: 
env: ‘/etc/dysnomia-modules/postgresql-database-secret’: No such file or directory
[target: arsenic]: Activation failed of service: 6c7779e925f58420b6761b4b04b8ada3412f8fcc8071ea2bc823dd943d5f6544
[...]

Although it looks like it should be able to find the module correctly:

$ nixops ssh myserver
$ echo $DYSNOMIA_MODULES_PATH 
/etc/dysnomia-modules:/etc/dysnomia/modules

I grepped github for any clue on how to create a new module but couldn't find anything useful. Can you give me pointers on how to achieve this?

[svanderburg]

I probably need to investigate this, to fully uncover the problem. Can you try out a few things?

An easier way to configure the modules path is by using the dysnomia.extraModulesPath NixOS configuration property. It simply needs to refer to a directory in which a script that corresponds to your module name. So if you create a Nix derivation that creates a directory with your test module, this should suffice.

Another potential problem is how SSH works -- when you intend to remotely execute instructions over SSH and the login shell of the remote user is bash, then it will take the settings from .bashrc, rather than .bash_profile. It may happen that the modules path is missing because it is loaded from the profile. Then the solution is to modify the remote user's .bashrc file and a DYSNOMIA_MODULES_PATH there.

I have not fully documented yet how to create Dysnomia modules, but when I want to make modifications and test them, I typically use Dysnomia's test suite.

If you clone the Git repo, then you can run the PostgreSQL tests as follows:

$ nix-build release.nix -A tests.modules.postgresql-database

if you want to make changes, then simply modify the script in dysnomia-modules/ and then run the above instruction

[svanderburg]

One of my future plans is to make the development and deployment of Dysnomia modules easier. Right now, they are stored in a mono repo, but since we have Nix at our disposal, why not making it easier to deploy each plugin by using Nix in our advantage?

[ibizaman]

Hi @svanderburg, I'm getting back to this lately. I tried a few variations of extraModulePaths but can't get it right.

Let's say I created a new file called postgresql-database2.in somewhere and that I want to make it accessible to dysnomia. I updated my network.nix to have:

{
  mymachine = { system, pkgs, lib, ... }: {
    environment.etc."dysnomia/modules/postgresql-database2".source = pkgs.substituteAll {
      src = ./modules/dysnomia/modules/postgresql-database2.in;
    };

    dysnomia = {
      enable = true;
      extraModulePaths = ["/etc/dysnomia/modules/"];
    }
  };
} 

Then, if I run nixops deploy, I do get a file in /etc/dysnomia/modules/postgresql-database2 but nothing got substituted in it.
And if I run disnixos-env ..., it tells me it couldn't find the postgresql-database2 module.

I think I'm going in circles here because it's still the same problems as before.

I'll go the route of making a clone of the git repo for the time being but I'd love to know how to do it this way, and to finally understand what I'm missing.

Thank you 🙏

[ibizaman]

Actually... I made a local clone of dysnomia and wanted to use that one. So, in my configuration.nix, I did:

dysnomia = {
  enable = true;
  enableLegacyModules = false;
  package = pkgs.lib.mkForce (pkgs.dysnomia.overrideAttrs (finalAttrs: previousAttrs: {
    version = "0.10.2";
    src = /home/me/projects/dysnomia;
  }));
}

But that fails with:

building Nix...
building the system configuration...
these 12 derivations will be built:
  /nix/store/a8frw9m68p627cy4vn0b1vz8x0wng99n-dysnomia-0.10.2.drv
  /nix/store/1kl3xm6n4gzmpi56dinih6v5dmai0wbk-system-path.drv
  /nix/store/69xyq3zd31pbn2l6lk2yzm4ni42axzsw-unit-polkit.service.drv
  /nix/store/f1mn8f23rv341snc3wq5mapd8r0zybbw-dbus-1.drv
  /nix/store/6khwzkgbjsym4grkyfs6qzvmzi3ic0nn-unit-dbus.service.drv
  /nix/store/7ybxpbzmyhd1c23kg1gihd8pa644vwq3-user-units.drv
  /nix/store/da2msf20hlkzg5d6lcbfdy2srrfp3dpp-unit-systemd-fsck-.service.drv
  /nix/store/g1q1b9xi0b4syk1z6v164hb9394bjaca-unit-dbus.service.drv
  /nix/store/q51nyl4p0mmx9lbwv3xhjxc5cf33lsx3-unit-accounts-daemon.service.drv
  /nix/store/ph2klm1hmdpq0lcyk74hqnjj5sb5h8f7-system-units.drv
  /nix/store/sgbv7hgxzrybq3bj5916vmqxq89qa3vb-etc.drv
  /nix/store/s127vq9g08ycnh37rywmjkhzxsywj9s4-nixos-system-tinux-22.05.2889.67e45078141.drv
building '/nix/store/a8frw9m68p627cy4vn0b1vz8x0wng99n-dysnomia-0.10.2.drv'...
unpacking sources
unpacking source archive /nix/store/hpqy3fifrkjyjchy116vafaiz4p4f5s5-dysnomia
source root is dysnomia
patching sources
configuring
no configure script, doing nothing
building
no Makefile, doing nothing
installing
install flags: SHELL=/nix/store/xbdqbi2mscmhl5wcpbgpjdwxbsrvpkil-bash-5.1-p16/bin/bash install
make: *** No rule to make target 'install'.  Stop.
error: builder for '/nix/store/a8frw9m68p627cy4vn0b1vz8x0wng99n-dysnomia-0.10.2.drv' failed with exit code 2
error: 1 dependencies of derivation '/nix/store/1kl3xm6n4gzmpi56dinih6v5dmai0wbk-system-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/s127vq9g08ycnh37rywmjkhzxsywj9s4-nixos-system-tinux-22.05.2889.67e45078141.drv' failed to build

To understand what's missing, I checked the tar.gz content of a dysnomia release. And there are differences with the repo itself!
So I have 2 additional questions: how do you make a release and how would I use my local repo?

Edit:
If I replace src above with /home/me/downloads/dysnomia-0.10.1 which is the extracted tar.gz release, then everything works.