ZFS woes after NIXOS_LUSTRATE from Arch

Help
1

Thanks to the community for all the help lately. After having fun with NixOS on Pis and VMs, when Samba stopped working on my Arch plex + backup server for the zillionth time this week I decided to take the plunge and try to convert it to NixOS via NIXOS_LUSTRATE.

The conversion process went okay with a few minor hiccups (GRUB didn’t like my EFI mountpoint at /efi but putting it at /boot/efi and rerunning switch-to-configuration boot seems to have fixed things). EDIT: Now that I think about it, there were also some issues about the zpool previously being on a different system; I think some combination of zpool import -f tank and zpool export tank and zpool import tank got rid of this.

Unfortunately I’m running into some issues with ZFS:

  • my non-root ZFS pool doesn’t automatically import at boot
  • my ZFS-based Samba share fails

Currently I have commented out all of the automatically generated zfs-relevant mountpoints in my hardware-configuration.nix. I have mountpoint=foo and canmount=on set for all of the datasets. A manual zpool import tank mounts the datasets as expected, albeit with an SMB error: cannot share 'tank/my/share: system error': SMB share creation failed.

Upon reboot the zfs mounts are not mounted. systemctl status zfs-mount seems to have succeeded (no errors). systemctl --failed shows no errors.

Trying to solve the issue, I’m finding some conflicting information regarding legacy mountpoints:

https://nixos.wiki/wiki/ZFS:

By default, all ZFS pools available to the system will be forcibly imported during boot.

zfs-mount service is enabled by default on NixOS 22.05.
To automatically mount a dataset at boot, you only need to set canmount=on and mountpoint=/mount/point on the respective datasets.

This makes it seem like I don’t need legacy mountpoints or references in my hardware-configuration.nix as of 22.05 (I’m running 22.11).

The docs for boot.zfs.extraPools suggest the opposite approach:

Instead, you should set the mountpoint property of ZFS filesystems to legacy and add the ZFS filesystems to NixOS’s fileSystems option

I’m wondering if the SMB error from above is causing an issue? Also not sure how to debug the SMB share problem. I have services.samba enabled and configured (importing my previously working config), and it actually seems to be working (I can import the share from another computer). Why the ZFS error upon zpool import?

Should I just set everything to legacy and go the old-school route? Maybe I have some kind of race condition that’s preventing zfs-mount from working automatically? TIA for any ideas.

2

I’ve only ever Lustrated cloud machines, with ext4…

You may have better results just blasting your arch install, and installing nixos fresh.

But i’m gonna take guess that you got a Brazilian terrabytes of data, and can’t move it, or back it up?

Good luck…

3

Thanks for your input.

It’s actually on BTRFS root with ZFS just for a large (40 TB) storage dataset. Most of my NixOS installs have been on BTRFS and have worked great.

The BTRFS stuff and root seems to be working fine, so I’m not convinced that the issue is related to LUSTRATE – but maybe?

1 Like
4

This is why the pool is not being imported. Basically, NixOS adds steps in stage 1 to ensure that all dependencies for a set of filesystem paths that are marked neededForBoot are prepared, this can include opening LUKS or LVM partitions, and importing pools. Without that, they’ll be deferred until later in stage 2.

I’m not sure about the SMB issue, I suspect it’s a little more than just service dependency-order timing, because the error still happens on a manual import when presumably any services that were racing to start already have.

1 Like
5

Update: added boot.zfs.extraPools = [ "tank" ]; and I’m now getting everything mounted at boot time as desired (no other changes, still not listed in my filesystem).

Still getting the SMB error with zfs-share.service, but the SMB share is actually working. ¯\_(ツ)_/¯

I guess next step will be figuring out how to get some debug info from that command, looks like there is no -v flag and the manpage is rather sparse.

$ sudo zfs share -a
cannot share 'tank/my/share: system error': SMB share creation failed
1 Like
6

Apologies, I misread and didn’t see that the zfs pool was data only, no boot dependency.

Yes, for this you’ll need to add it to extraPools (because the mechanism above won’t find any dependencies on this pool to cause it to be added automatically, in either stage 1 or 2)

3 Likes
7

@uep, the force is strong with you.

3 Likes
8

Solutions:

  • my non-root ZFS pool doesn’t automatically import at boot → https://discourse.nixosstag.fcio.net/t/zfs-woes-after-nixos-lustrate-from-arch/24217/6?u=n8henrie
  • my ZFS-based Samba share fails → I had zfs configured as sharesmb=on (retained setting from my Arch system) but had also configured a samba share in my configuration.nix, it looks like the error was from this essentially being double configured. A simple zfs set sharesmb=off tank/my/share resolved the issue. Thanks to all for your input and help!
1 Like