How to change `RUSTFLAGS` in `build-rust-package` -- list / string merging problem?

I’m trying to get debug symbols to work for rust packages on darwin (when built as debug).

Issue: rust debug symbols don't work on aarch64-darwin · Issue #262131 · NixOS/nixpkgs · GitHub
PR (draft): rustPlatform.buildRustPackage: provide debug symbols on darwin by n8henrie · Pull Request #327136 · NixOS/nixpkgs · GitHub

I can get this to work locally by adding RUSTFLAGS = "-C split-debuginfo=packed"; (as well as copying then into $out).

I’ve been hoping to incorporate this into the build-rust-package tooling, but I’m having a hard time!

Using an approach like this:

# rustflags = args.RUSTFLAGS or "" + lib.optionalString isDarwinDebug "-C split-debuginfo=packed ";
rustflags = args.RUSTFLAGS or [] ++ lib.optionals isDarwinDebug ["-Csplit-debuginfo=packed"];

sysroot = callPackage ./sysroot { } {
    inherit target;
    RUSTFLAGS = rustflags;
    shortTarget = stdenv.hostPlatform.rust.cargoShortTarget;
    originalCargoToml = src + /Cargo.toml; # profile info is later extracted
  };

...
stdenv.mkDerivation ((removeAttrs...) // {
RUSTFLAGS = rustflags ++ lib.optionals useSysroot ["--sysroot ${sysroot}"];
...
}

When I define RUSTFLAGS as a string, I get build failures with unrelated packages that define it as a list (e.g. asusctl). (error: cannot coerce a list to a string)

When I redefine it as a list, I get build failures for packages like caligula that define it as a string.

I thought that perhaps I was going about this all wrong and tried defining NIX_RUSTFLAGS instead, but then I get failures about sysroot being used multiple times (when building for a custom target).

Is there an example I could pull from regarding how to add some default RUSTFLAGS specifically when stdenv.isDarwin && buildType == "debug";?

Thanks for any suggestions!

Convert it to a string first, then use only strings? But that’d require some testing to make sure it doesn’t break any existing packages.

Unfortunately that’s what I tried (and tried to describe above) – either way breaks existing packages (some having defined RUSTFLAGS as a string and others as a list of strings).

I don’t see any mention of toString above?

True – I tried both string coercion, toString, as well as making both rustflags and RUSTFLAGS into strings as seen in the PR: rustPlatform.buildRustPackage: provide debug symbols on darwin by n8henrie · Pull Request #327136 · NixOS/nixpkgs · GitHub

As I mentioned, asusctl uses RUSTFLAGS = [a_list]; and even with toString I get this build failure (note the toString rustflags).

    … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'asusctl-6.0.9'
         whose name attribute is located at /nix/store/6zmjqqiawihw5wasbz0krvh5dvvrnsk3-source/pkgs/stdenv/generic/make-derivation.nix:334:7

       … while evaluating attribute 'RUSTFLAGS' of derivation 'asusctl-6.0.9'

         at /nix/store/6zmjqqiawihw5wasbz0krvh5dvvrnsk3-source/pkgs/build-support/rust/build-rust-package/default.nix:108:3:

          107| # } // {
          108|   RUSTFLAGS = (toString rustflags) + lib.optionalString useSysroot "--sysroot ${sysroot} " + lib.optionalString isDarwinDebug "-C split-debuginfo=packed ";
             |   ^
          109|   # RUSTFLAGS = rustflags ++ lib.optionals useSysroot ["--sysroot ${sysroot}"];

       error: cannot coerce a list to a string

EDIT: I also tried with lib.concatStringsSep " " rustflags

EDIT2: To clarify, I get error: value is a string while a list was expected in this case.

Might not be the best solution, but this works:

  rustflags =
    lib.optionalString isDarwinDebug "-C split-debuginfo=packed "
    + (if lib.isList (args.RUSTFLAGS or "") then (lib.concatStringsSep " " args.RUSTFLAGS) else "");

I’ll take a look at the PR later, but toString should normally just work

nix-repl> toString [ "a" "b" "c" ] + lib.optionalString true " test"
"a b c test"

nix-repl> toString "a b c" + lib.optionalString true " test"
"a b c test"

So something else is happening in the builder

@eljamm your code would only work for lists, not strings

You’re right. Using toString, this works as well:

rustflags = toString (args.RUSTFLAGS or "") + lib.optionalString isDarwinDebug "-C split-debuginfo=packed ";

It looks like this works – I think my problem was that I was trying to toString at the RUSTFLAGS = ... part lower down, which is where the error message indicated was the problem. However, having toString args.RUSTFLAGS higher up seems to work – the code for the debug symbols seems to build in my test case, and both asusctl and caligula now evaluate successfully.

Thanks!

Darn, well that gets me to my next error at least (which I’ve also seen in my tinkering on this project):

 error: The ‘env’ attribute set cannot contain any attributes passed to derivation. The following attributes are overlapping: RUSTFLAGS

Now failing with gitbutler, presumably due to defining env.RUSTFLAGS: nixpkgs/pkgs/by-name/gi/gitbutler/package.nix at 04101568c0249894781fe857046efad568aa0c42 · NixOS/nixpkgs · GitHub

We can just define RUSTFLAGS directly and use that as necessary. How about this?

  RUSTFLAGS =
    toString (args.env.RUSTFLAGS or args.RUSTFLAGS or "")
    + lib.optionalString isDarwinDebug "-C split-debuginfo=packed "
    + lib.optionalString useSysroot "--sysroot ${sysroot} ";

  sysroot = callPackage ./sysroot { } {
    inherit target RUSTFLAGS;
    shortTarget = stdenv.hostPlatform.rust.cargoShortTarget;
    originalCargoToml = src + /Cargo.toml; # profile info is later extracted
  };

stdenv.mkDerivation (
  (removeAttrs args [
    "depsExtraArgs"
    "cargoUpdateHook"
    "cargoLock"
  ])
  // lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; }
  // lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; }
  // {
  ...

We can just define RUSTFLAGS directly and use that as necessary.

I’ve tried that before, it gives me infinite recursion errors. (Just confirmed with your suggested changes.)

EDIT: specifically

          105|   lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
          106|   lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {
             |                                                 ^
          107|

       error: infinite recursion encountered

       at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/sysroot/default.nix:10:10:

            9| in rustPlatform.buildRustPackage {
           10|   inherit target RUSTFLAGS;

Interesting. Do you get this when trying to build one of the above packages or is it something else?

Hmmm, well now I’m very confused. The above packages work, but I get it with both github:n8henrie/nix-rust-debug-info#sysroot-debug and github:n8henrie/nix-rust-debug-info#sysroot-release, which are outputs I made with a custom sysroot to make sure I didn’t break this functionality (it pulls in my nixpkgs branch with these changes). The resulting binary just exits with exit status 42.

Somehow I get infinite recursion locally, but not when I build from the repo. I’m sure I’m missing something obvious.

Locally fails to build with infinite recursion, no relevant local modifications that are unpushed:

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        notes.md

nothing added to commit but untracked files present (use "git add" to track)
$ git remote -v
origin  git@github.com:n8henrie/nix-rust-debug-info.git (fetch)
origin  git@github.com:n8henrie/nix-rust-debug-info.git (push)
$
$
$ nix build --no-eval-cache .#sysroot-debug --out-link ./result-sysroot-debug --print-build-logs --show-trace
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-rust-debug-info-x86_64-unknown-linux-gnu'
         whose name attribute is located at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/stdenv/generic/make-
derivation.nix:334:7

       … while evaluating attribute 'RUSTFLAGS' of derivation 'nix-rust-debug-info-x86_64-unknown-linux-gnu'

         at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/default.nix:10
6:49:

          105|   lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
          106|   lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {
             |                                                 ^
          107|

       … from call site

         at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/default.nix:90
:7:

           89|     + lib.optionalString isDarwinDebug "-C split-debuginfo=packed "
           90|     + lib.optionalString useSysroot "--sysroot ${sysroot} ";
             |       ^
           91|

       … while calling 'optionalString'

         at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/lib/strings.nix:268:5:

          267|     # String to return if condition is true
          268|     string: if cond then string else "";
             |     ^
          269|

       … while evaluating derivation 'custom-sysroot-x86_64-unknown-linux-gnu'
         whose name attribute is located at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/stdenv/generic/make-derivation.nix:334:7

       … while evaluating attribute 'RUSTFLAGS' of derivation 'custom-sysroot-x86_64-unknown-linux-gnu'

         at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/default.nix:106:49:

          105|   lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
          106|   lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {
             |                                                 ^
          107|

       error: infinite recursion encountered

       at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/sysroot/default.nix:10:10:

            9| in rustPlatform.buildRustPackage {
           10|   inherit target RUSTFLAGS;
             |          ^
           11|

However building from github runs and works:

$ nix build --no-eval-cache github:n8henrie/nix-rust-debug-info#sysroot-debug --out-link ./result-sysroot-debug --print-build-logs --show-trace
$ ./result/bin/nix-rust-debug-info
$ echo $?
42

Hmm, I’m sure I’m doing something silly here but it’s not jumping out at me.

$ nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.8.12, NixOS, 24.11 (Vicuna), 24.11.20240804.f97c714`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - channels(root): `""`
 - nixpkgs: `/nix/store/df089l85f9a9kfrlpgwgpix3xrgdfqvy-source`

Hmm, very odd.

$ cd $(mktemp -d)
$ git clone git@github.com:n8henrie/nix-rust-debug-info.git .
Cloning into '.'...
remote: Enumerating objects: 89, done.
remote: Counting objects: 100% (89/89), done.
remote: Compressing objects: 100% (63/63), done.
remote: Total 89 (delta 44), reused 68 (delta 23), pack-reused 0
Receiving objects: 100% (89/89), 12.94 KiB | 339.00 KiB/s, done.
Resolving deltas: 100% (44/44), done.
$ nix build .#sysroot-debug
error:
       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       … while evaluating derivation 'nix-rust-debug-info-x86_64-unknown-linux-gnu'
         whose name attribute is located at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/stdenv/generic/make-derivation.nix:334:7

       … while evaluating attribute 'RUSTFLAGS' of derivation 'nix-rust-debug-info-x86_64-unknown-linux-gnu'

         at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/default.nix:106:49:

          105|   lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
          106|   lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {
             |                                                 ^
          107|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: infinite recursion encountered

       at /nix/store/hrvag6fyx93x37pg1p2p8r1jkgcihblk-source/pkgs/build-support/rust/build-rust-package/sysroot/default.nix:10:10:

            9| in rustPlatform.buildRustPackage {
           10|   inherit target RUSTFLAGS;
             |          ^
           11|
$
$
$
$ nix build github:n8henrie/nix-rust-debug-info#sysroot-debug
$ nix build --no-eval-cache --rebuild github:n8henrie/nix-rust-debug-info#sysroot-debug
$ ./result/bin/nix-rust-debug-info
$ echo $?
42

Guess adding RUSTFLAGS to the derivation is useless since sysroot inherits them already, which causes an infinite recursion if we add them again.

stdenv.mkDerivation ((removeAttrs args [ "depsExtraArgs" "cargoUpdateHook" "cargoLock" ]) //
-  lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
-  lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {

With this, all the examples here, including the nix-rust-debug-info repo, seem to work fine.

Curious why it worked the first time when you built from GitHub, though.

Yes, very odd.

On the same machine, with no changes, I am now getting infinite recursion from both local and github URLs. I assume some kind of cache messing with nix’s purity.

With that diff it should fail to evaluate (due to removal of the trailing {), right? Won’t adding it back in (adding the diff → removing those lines) fail to provide the debug symbols for darwin?

Yes, unfortunately with those lines commented out, it fails to fulfill the purpose of the PR; there are no .dSYM files in result/bin (EDIT: produced – I have a separate patch that puts them in $out/bin). Confirmed that it works again if I uncomment those lines (revert that diff):

$ nix-info -m
 - system: `"aarch64-darwin"`
 - host os: `Darwin 23.5.0, macOS 14.5`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.5`
 - channels(n8henrie): `""`
 - channels(root): `""`
 - nixpkgs: `/nix/store/h5m0sr1l5cwqga83zfdm7qbbrnm1iill-source`
$ git -C ~/git/nixpkgs diff

pkgs/build-support/rust/build-rust-package/default.nix
────────────────────────────────────────────────────────────────────────────────────────────────────────────

────────┐
102: in │
────────┘
assert useSysroot -> !(args.doCheck or true);

stdenv.mkDerivation ((removeAttrs args [ "depsExtraArgs" "cargoUpdateHook" "cargoLock" ]) //
  lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
  lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } // {
  # lib.optionalAttrs (args ? env.RUSTFLAGS) { env.RUSTFLAGS = RUSTFLAGS; } //
  # lib.optionalAttrs (args ? RUSTFLAGS) { inherit RUSTFLAGS; } //
  {

  inherit buildAndTestSubdir cargoDeps;
$
$
$ nix build .#debug --override-input nixpkgs ~/git/nixpkgs
$ find ./result/bin -name '*.dSYM'
$

Also, thank you for your time and help. I’ve tried so many permutations without any luck, it’s nice to have someone helping me brainstorm this!

EDIT2: Sorry, delta has removed the plus and minus markers in my git diff above, making it confusing. It is correct. Apparently I need to set keep-plus-minus-markers = true.

1 Like

Do you know how to cross-compile this from an x86_64-linux machine?