Gripes with nilla

[nrabulinski]

I started migrating my config over to nilla, while using flakes for dependency management and because that's currently the only way to get pure evaluation.
I'll be updating this issue with my thoughts and gripes I encounter
If you want to follow my journey, I'll be pushing my changes here https://git.rab.lol/nrabulinski/settei/src/branch/experiments/nilla

COMPLAINT - solved/acknowledged and not applicable

Thoughts so far:

• It'd be nice to have a default set of systems a'la flake-parts. It's good you started out with having each package and shell say explicitly which systems it's compatible with, I think that's a better approach than a single perSystem attr like how flake-parts does it, but I also shouldn't have to repeat the same set of systems for every attribute.
• 1000 nixpkgs instances is back - it seems like you're importing and evaling a new nixpkgs instance for every shell, package, etc. This seems very unnecessary unless a package has special requirements. It's also easily solvable with a custom builder, but it shouldn't be the default IMO
• But speaking of perSystem being a bad design choice: I just migrated my wrapper-manager packages to use nilla (https://git.rab.lol/nrabulinski/settei/commit/265fba390834b131743b1c673d7d75257db3cb11) and I had to use a bad hack to re-export all of my wrappers under packages. Turns out having some way to do a single evaluation per system could be nice
• I'd like nilla modules to be shorthanded by default so I don't have to prefix everything with config.
• Not about nilla specifically but about the module system you've got - even when you do define a shorthand submodule, if there's config attribute it's by default treated as non-shorthand which in practice means you can't really define a submodule where one of the options is called config. (Well, you can, but then there's no point in making it a shorthand since you need to use config.config anyway
• I'm not sure if your module system has something akin to _module.args but if it does, reexporting _module.args.inputs = builtins.mapAttrs (_: input: input.result) config.inputs would be very nice
• First-class formatter package - pretty self-explanatory. While that, again, is currently possible, having a special formatter output (or at least having the cli with fmt subcommand which would look for packages.formatter) would be nice

[jakehamilton]

This feedback is very much appreciated! Some of these are things that I have also run into and have been thinking about how to build things correctly without letting ux suffer. Nilla is absolutely not perfect, nowhere remotely close. Having feedback like this and people actually testing its limits help to move things in the right direction. I'll keep my ears and eyes open as you continue trying to use the project!

[jakehamilton]

Making some progress on these :)

It'd be nice to have a default set of systems [...]

I've gone ahead and overhauled the nixpkgs loader to now take a list of systems as opposed to a single system value. The loaded result is now an attribute set of system names to package sets (plus a lib attribute for easy access). You can reference these default systems via config.inputs.nixpkgs.settings.systems if you want to reuse them for your own package definitions.

1000 nixpkgs instances is back [...]

Yeah, the initial Nixpkgs handling was very much "just get things working". We should now at least only be creating one instance of nixpkgs per-system per-project. I am considering the utility of a proxy loader which would make it easy to directly patch in loaded inputs for Nilla dependencies in your project.

I had to use a bad hack to re-export all of my wrappers [...]

I don't totally follow on this one, can you clarify which parts you're bumping up against?

I'd like nilla modules to be shorthanded by default [...]

I'm not sure this will change. Aux Lib's module system intentionally made the decision to disallow root level shorthand to avoid confusion with users not realizing they were working with modules (as happened a lot in NixOS).

[...] even when you do define a shorthand submodule, if there's config attribute it's by default treated as non-shorthand [...]

Yeah, this is less-than-ideal. Though, we need to keep compatibility with using root-level module attributes in order to ensure users can provide more dynamic modules even when a submodule has opted-in to shorthand syntax. The answer here, at least for now, is to not name an option config since that confuses things anyway. The only time I've come up against this was with the Nixpkgs loader due to Nixpkgs wanting an argument named config, but we can manage that in the loader submodule using a different name.

[...] something akin to _module.args [...]

Yes, you can set dynamic module arguments using config.__module__.args.dynamic.<name>. So your example may be translatable as:

{ config }:
{
  config.__module__.args.dynamic.inputs = builtins.mapAttrs (name: input: input.result) config.inputs;
}

[nrabulinski]

That's awesome to hear, thanks!

I don't totally follow on this one, can you clarify which parts you're bumping up against?

Absolutely. The issue is, with flake-parts I could do this

{ inputs, ... }:
{
  perSystem =
    { pkgs, inputs', ... }:
    let
      wrapped = inputs.wrapper-manager-hm-compat.lib {
        inherit pkgs;
        modules = [
          ./starship
          ./helix
          # TODO: Enable again
          # ./rash
          ./fish
          ./wezterm
        ];
        specialArgs = {
          inherit inputs inputs';
        };
      };
      all-packages = wrapped.config.build.packages;
    in
    {
      packages = all-packages;
    };
}

The exact implementation doesn't matter, what does matter is that all-packages ends up being evaluated once per system, and then flake-parts takes care of transposing <system>.packages = { <all my wrappers> } into packages.<system> = { <all my wrappers> }. In contrast with how this file looks after migrating to nilla:

{ lib, config }:
let
  systems = [
    "x86_64-linux"
    "aarch64-linux"
    "aarch64-darwin"
  ];
  wrappedPerSystem = lib.attrs.generate systems (
    system:
    config.inputs.wrapper-manager-hm-compat.result.lib {
      pkgs = config.inputs.nixpkgs.result.legacyPackages.${system};
      modules = [
        ./starship
        ./helix
        # TODO: Enable again
        # ./rash
        ./fish
        ./wezterm
      ];
      specialArgs.inputs = builtins.mapAttrs (_: input: input.result) config.inputs;
    }
  );
  wrappedPerSystem' = builtins.mapAttrs (_: wrapped: wrapped.config.build.packages) wrappedPerSystem;
  wrapperNames = builtins.attrNames wrappedPerSystem'."x86_64-linux";
in
{
  config.packages = lib.attrs.generate wrapperNames (wrapper: {
    inherit systems;
    builder = "custom-load";
    package = { system }: wrappedPerSystem'.${system}.${wrapper};
  });
}

I still managed to have wrappers evaluate once per system at least, but because of how the package module is structured, I can't just "re-export" my wrappers, I need to get the list of wrappers (in this case with builtins.attrNames wrappedPerSystem'."x86_64-linux") and then use lib.attrs.generate to generate an attrset of package modules per wrapper.

What I'm trying to say I guess is it could be nice to have the option of moving the package building logic "level higher" I.e. have the package builder return a package set instead. This would simplify situations like this one where you're operating on sets of packages instead of singular packages.

[jakehamilton]

Ah, that makes a lot of sense, thank you!

I think that the way you've done it is the appropriate way to handle this for config.packages.*. Though, that structure seems at odds with how wrappers are build, customized, and extended. It may be better to avoid config.packages entirely and create your own config.wrappers which can handle all of this as well as extension/customization afterwards.

Lots of things won't fit cleanly into the base structures that Nilla comes with and that's okay. Part of the trouble with flakes were that you had no options if something couldn't go in packages. Now, however, we have the ability to extend the schema in any way we see fit. Another module for managing different kinds of things can be added. That can then patch over to things like config.packages if you want to use nilla build, but I recommend adding your own binary prefixed with nilla- (eg. nilla-wrappers) which is automatically detected by the Nilla CLI if it is in your path environment variable. This way you can both extend the pure-nix side in your Nilla project and interact with these pieces via the CLI however you see fit! For example you could interact with it via something like nilla wrappers build helix.

A simple variant of this command line tool which only handles building packages could actually also proxy to nilla build since that accepts a full attribute path:

#!/usr/bin/env bash
# This file is `nilla-wrappers`, executable, and in $PATH

command=$1
name=$2

if [[ $command == "build" ]]; then
  system=$(nix eval --impure --raw --expr 'builtins.currentSystem')

  nilla build wrappers.${system}.${name}
fi

---

I hope this comment doesn't come off as dismissive of the problem. Rather, what I'm hoping to communicate is that Nilla itself should facilitate plugging solutions for these things in rather than trying to solve every single problem itself out of the box.

[nrabulinski]

I hope this comment doesn't come off as dismissive of the problem. Rather, what I'm hoping to communicate is that Nilla itself should facilitate plugging solutions for these things in rather than trying to solve every single problem itself out of the box.

No, that makes perfect sense. I still think it could be useful to be able to operate on package sets instead of packages individually so at some point maybe nilla will get a module for that so people don't have to bring their own, but also, I do understand this would be a rather big change and that you may not want to take nilla in this direction (for now?) 😅

You can reference these default systems via config.inputs.nixpkgs.settings.systems if you want to reuse them for your own package definitions

That's better but I personally still think it could be useful to have a default system list, don't you think so? Then the nixpkgs loader could also by default reference those systems, or you could overwrite that list per nixpkgs instance and per package/shell. Having to specify systems = config.inputs.nixpkgs.settings.systems, while at least means the default system list is consistent, is a bit unwieldy.

Re: module system - I respect that, thanks for answering.

[nrabulinski]

I hope I'm not too pushy in suggesting my own views and preferences about the structure etc, but I am excited about nilla and aux and I think there is value to providing batteries for the majority use case. As in, if I started a project today and wanted to package it, ideally I shouldn't have to define any modules by myself.

I know I will have to define modules for my personal config because it's huge and complicated and intertwined in weird ways, but I'm talking a more or less standard web app or something.

Also also, aux lib docs are down heh

[jakehamilton]

[...] this would be a rather big change and that you may not want to take nilla in this direction [...]

I would highly encourage people to experiment with disabling the packages module entirely and trying to write a better one! I decided to go with a flat structure of (simplified) packages.<name>.<system>, but there may be a better way! Aux Tidepool enforces namespaces for all packages for example so there it would be packages.<namespace>.<name>.versions.<version>. I think experimentation is a good idea right now for Nilla.

That's better but I personally still think it could be useful to have a default system list [...]

I think that's probably right. I am trying to figure out whether that means I should pull in Aux Tidepool's lib.systems or not. For now I'm keeping it small while working on some other features, but these quality of life improvements will be necessary at some point!

I hope I'm not too pushy in suggesting my own views and preferences about the structure [...]

Not at all! I absolutely welcome the feedback, it is very helpful for me while I am working on the project. I run into some issues when I use Nilla, but there are loads more projects in the world than my own. Being able to hear about things that Nilla could do better is beneficial :)

[...] ideally I shouldn't have to define any modules by myself [...]

I think a good amount of things will be handled by built-in modules like packages and shells, but other pieces should be handled by separate nixos / macos / home / anything projects and plugged in. Those may be in the Nilla org on GitHub or they may be written by third-parties. I think the flexibility is valuable here. If you find yourself wanting to do something that Nilla doesn't currently support, please feel free to write your own module to support that use case! The community sharing modules will be the way that Nilla solves problems.

Also also, aux lib docs are down heh

Uh oh, I'll go ping people..

[jakehamilton]

Re: aux lib docs

Oh, I misread that originally. Did you mean the Aux Wiki? That seems to be up for me. We don't currently have docs for Aux Lib so unfortunately you'll have to look at the doc comments in the source for reference and read the tests (which there aren't enough of) for examples.

[nrabulinski]

Re: aux lib docs

Oh, I misread that originally. Did you mean the Aux Wiki?

I meant docs.auxolotl.org, sorry for the confusion. I'm not sure if it's a problem with git.gay or with the docs repo in particular but I can't navigate that site, it just 404s for me

[jakehamilton]

Oh, gotcha. I don't think we had anything super important on there. Will see what folks know.

[jakehamilton]

Looks like it was down for some maintenance on the underlying provider. Back up now :)

[nrabulinski]

I also started a project with nilla+npins so I hope to give some more feedback from another perspective too :)

[psentee]

I've started converting one of my personal projects to Nilla (trying to see the world outside of flakes & how aux lib/modules work), and I love it so far! I'm writing down a similar list of nitpicks / missing pieces / nice-to-haves: mostly it's about some UX quality-of-life and edge cases, maybe some missing extension points. Nothing breaking so far, (almost) all issues have reasonable workarounds by dropping down to raw nix eval.

Q: when my wishlist is more complete, should I post it in this issue, or open a new one?

[jakehamilton]

@psentee please feel free to open a new issue! Would love to have your feedback tracked as well :)