Feature: Ability to define partial configurations

[Doc-Steve]

Hi there,

When using all the existing dot-file managers, I’m curious if Nix could offer a unique feature that would make it significantly superior to traditional dot-management: defining partial configurations. Currently, the only option is to either set all configurations for a program in the Nix code or leave them completely unmanaged. There‘s no middle ground.

Let me elaborate on this:

I often find myself wanting to define only a subset of the possible configuration settings for a program within my NixOS infrastructure code. I want to leave other settings untouched or allow users to modify them. It would be incredibly helpful if I could define settings for a JSON, YAML, or INI configuration file, and NixOS would automatically insert or replace these settings in the existing dot files.

I’m wondering if such a feature is even feasible and if this is something that could be done with hjem?

[eclairevoyant]

I think this is similar to #113, is my understanding correct?

[Doc-Steve]

It’s something different.

#113 is meant for complete files, I think similar to a “mutable” option like this.

My feature request extends beyond this. Imagine a definition like this:

        xdg.config.files.”config.json” = {
          generator = lib.generators.toJSON { };
          value = {
            templates = “/templatesdir”;
          };
          # add or reset only defined values, leave everything else untouched
          partial = true;
        };

This would simply add or modify the value templates of an existing config.json file.

{
  “background”: “green”,
  “warnings”: true
}

->

{
  “background”: “green”,
  “warnings”: true,
  “templates”: "/templatesdir"
}

The user can still use the app GUI to change other settings, but every NixOS switch will only revert the value of templates to its defined state.

I hope I described it in an understandable way.

[HigherOrderLogic]

Maybe ignore_modification in smfh will solve this.

[eclairevoyant]

No, it will not; it would only solve #113.

[eclairevoyant]

This is essentially asking hjem to parse the existing (modified) config file before activation, merge it with the declarative configuration, and then deploy the merged file again. That seems out of scope for hjem. At most what might be in scope for hjem would be to provide the machinery to allow passing a function to parse existing config files, or a command that transforms to a language readable by nix (e.g. JSON).

[Doc-Steve]

Sounds interesting. How would the „flow“ applying these self defined merger functions (if I understood right) then be like?

[eclairevoyant]

Currently we have e.g.

        xdg.config.files."test/bar.json" = {
          generator = lib.generators.toJSON { };
          value = {
            foo = 1;
            bar = "Hello world!";
            baz = false;
          };
        };

I envision a new option alongside generator - let's call it parser - that will read the file. We already have fromJSON and fromTOML as builtins, but this would support other cases.

Then I think we would also have to support some kind of new type like "merge".

The main concern I see with something like this is the level of impurity needed to achieve it...

The other possibility would be to define an option that lets you run arbitrary commands to read the file and handle the merging - but if that command fails, then activation fails. I'm not so confident about that approach either atm. @NotAShelf if you have any thoughts?

[Doc-Steve]

So the imaginary new format would be something like this?

        xdg.config.files."test/bar.json" = {
          generator = lib.generators.toJSON { };
          parser = lib.parsers.fromJSON { };
          value = {
            foo = 1;
            bar = "Hello world!";
            baz = false;
          };
        };

That means if "parser != null" then the "bar.json" would be read/parsed first and then merged with "value" before "generator" would be executed.

You are right about the impurity of "bar.json", only the "value" parts would be "pure". It reminds me of the homemanger "mkOutofStoreSymlink" workarounds for impurity.

I totally agree that this is some kind of a philosophical question, what means "purity" in this context - and making a decision on project level will lead to discussions.

How about letting the user decide if/how he deals with the purity question himself. A "solution" achieving this could be something like this:

        xdg.config.files."test/bar.json" = {
          generator = lib.generators.toJSON { };
          writeHook = "script merging files $GENERATED & $DESTINATION and write the new file to $DESTINATION";
          value = {
            foo = 1;
            bar = "Hello world!";
            baz = false;
          };
        };

The hook could be similar to backupCommand in homeManager.

The "writeHook" function will be responsible for creating the "$DESTINATION" file. The user will decide how this is done, considering all the potential implications.

This script is called after the generator creates the file from "values" but instead of replacing the previous file in the destination directory.

I appreciate the open discussion. I believe I'm not the only one who would find many use cases for such a "partial" dot-file feature.

[eclairevoyant]

This is not a space to advertise projects with (undisclosed, even) AI usage, thanks. (We also won't entertain moderation discussions on this point, so let's keep replies on-topic regarding any possible fixes here.)

[pupto-bismol]

Could you say, provide the config.files.<>.parser option and then have it so that a non-null parser would trigger this logic:

1. parse any existing file at .target into a basic attribute set
2. apply the nixos's pre-included mkMerge to the parsed attribute set and the .value attribute set, merging values as if they were just module options
3. insert the merged attribute set into the user's configuration as if they were to set it as the .value key
4. optionally, during step 1, you can pass additional .source files to the parser and each one will be merged in the same manner as the other options (as if it were a nixos module declaration)

I like the idea of custom hooks for people to do whatever they want, but (unless I'm missing something here) it would be a fairly straightforward default to just treat all the options as if they were modules using the same logic nixos users would be used to. It would also be pretty expected and should not be too confusing for users since it tracks with how hjem already goes for an approach that limits non-nix-code fiddling/hacking to be straightforward and easily understood.

[NotAShelf]

I've been meaning to respond to this issue but couldn't quite formulate what I wanted to say, and got distracted. The design I've been going for was similar to parser (which I've just named to parser now) and simplfied a little bit:

files.".config/starship.toml" = {
  source    = ./starship.toml;
  parser    = path: builtins.fromTOML (builtins.readFile path);
  generator = (pkgs.formats.toml {}).generate "starship.toml";
  value = {
    add_newline = false;
  };
};

The parser lets you declare a base file (your existing dotfile, checked into your repo) alongside Nix-managed overrides in value. Hjem merges them at build time and produces the final file. The app sees one complete config file; you only write Nix for the parts you care about. Worth nothing that source must be a path Nix can read at evaluation time, i.e., a file in your repo, a builtins.toFile, etc. It cannot transparently read the live file from ~/.config at rebuild time without --impure (or implicitly allowing impure behaviour) so "merge with whatever the app last wrote" isn't supported.

We've established early in Hjem's existence that onChange hooks à la HM are not something we plan to support due to their fragile and implicit nature. This means we will not be executing things on file change, file link, etc. unless we agree that this should be done in SMFH, which I don't think will happen anytime soon. That said, I think the above setup is a good interface for managing pure configurations that are version controlled in your setup. I'll have my changes ready to be pushed tomorrow, but I'm happy to hear your feedback until then.

Edit: I want to note that this is not a complete design and will fail on some .source setups due to my existing design choices but it's fixable. I've went with config.parser config.source after validating the two, which works but isn't perfect.

[eclairevoyant]

Hjem merges them at build time and produces the final file.

Can't do that, impure. At best we can have a service to handle that. They mentioned having users have the ability to override the config which would imply working with stateful files. At that point we should have parser be a binary path instead that handles files in the correct format. But that does open up a lot of issues in terms of changing generations...