Cannot run `nix-build -A phoronix-test-suite.tests`

Help
1

Hey! I try to follow Nixpkgs 22.11 manual to run passthru.tests, but without success. In that manual, you can find:

You can run these tests with:

$ cd path/to/nixpkgs
$ nix-build -A phoronix-test-suite.tests

but when I try that command for the examples in Linking NixOS module tests to a package I got this

$ nix-build -A phoronix-test-suite.tests
error: cannot evaluate a function that has an argument without a value ('callPackage')

       Nix attempted to evaluate a function as a top level expression; in
       this case it must have its arguments supplied either by default
       values, or passed explicitly with '--arg' or '--argstr'. See
       https://nixos.org/manual/nix/stable/expressions/language-constructs.html#functions.

       at /home/hadi/dev/nixpkgs/pkgs/development/libraries/nanopb/default.nix:1:3:

            1| { callPackage
             |   ^
            2| , cmake

Also tried this

$ nix-build -A phoronix-test-suite.tests -E '(import <nixpkgs> {}).callPackage ./default.nix {}'     
error: attribute 'phoronix-test-suite' in selection path 'phoronix-test-suite.tests' not found

How can I run those tests?

2

I just copied exactly nix-build -A phoronix-test-suite.tests in my local version of nixpkgs 39bc9634f64232 and it works fine:

$ nix-build -A phoronix-test-suite.tests   
/nix/store/iw6k3w3zs9vgyczqns7j2pjaykzf9nkx-phoronix-test-suite-tests

so are you actually running this exact command, or are you editing some files in nixpkgs or anything else? Maybe try to get the latest nixpkgs version.

3

Oh, I think I understand what you did: the cd path/to/nixpkgs goes to the root of the nixpkg’s repository, not to the folder containing the package you want to test. Then if you want to test various packages, just give the name of the package to test followed by .tests, and it will compile/run all its tests:

$ nix-build -A nanopb.tests
4

Thanks @tobiasBora I’m on master, fab9a5b367008dea29dc5c9d829a07f2b20773ea. I have to mention that I created a default.nix in a directory with this content:

{ pkgs ? import <nixpkgs> {} }:

pkgs.stdenv.mkDerivation {
  name = "example-test";
  meta.timeout = 60;

  passthru.tests = {
    full-sdk-shell = pkgs.runCommand "shell-test" {} ''
      exit 1
    '';
  };
}

then only by mentioning '<nixpkgs>' I got

$ nix-build '<nixpkgs>' -A phoronix-test-suite.tests            
/nix/store/nf32war8jsfj96sibv4nsj6m46hq081p-phoronix-test-suite-tests

which is not failed! I expect it to fail.

5

So you have to understand what this does:

$ nix-build '<nixpkgs>' -A phoronix-test-suite.tests

will go to the folder pointed by <nixpkgs> (chech echo $NIX_PATH for details), then it will evaluate the file default.nix in this folder. In nixpkgs this file outputs all top-level packages, including notably the programs loaded in pkgs/top-level/all-packages.nix. In particular, if you see this file, you can see that it contains the phoronix-test-suite program:

$ cat pkgs/top-level/all-packages.nix | grep phoronix
  phoronix-test-suite = callPackage ../tools/misc/phoronix-test-suite { };

If you look at this file you will see that it contains a passthru:

$ cat pkgs/tools/misc/phoronix-test-suite/default.nix
…
  passthru.tests = {
    simple-execution = callPackage ./tests.nix { };
  };
…

So after evaluating default.nix (itself calling pkgs/tools/misc/phoronix-test-suite/default.nix), you end up with a huge attribute set containing:

{
   …
   phoronix-test-suite = {
       tests = {
         simple-execution = DERIVATION FOR THE TEST WRITTEN IN pkgs/tools/misc/phoronix-test-suite/tests.nix;
      }
   }
   …
}

The role of -A phoronix-test-suite.tests is exactly to compile all the derivations in phoronix-test-suite.tests, so it will compile and pass the test with success!

Now if you want to compile your own test, say that you created in /tmp/a.nix this file:

{ pkgs ? import <nixpkgs> {} }:
pkgs.stdenv.mkDerivation {
  name = "example-test";
  meta.timeout = 60;
  src = ./.;
  passthru.tests = {
    full-sdk-shell = pkgs.runCommand "shell-test" {} ''
      echo "I'm running the test"
      exit 1
    '';
  };
}

Then you can run your test by doing:

$ nix-build /tmp/a.nix -A tests
this derivation will be built:
  /nix/store/nwcva6f5dpra77byb9fyhz46g6k7kkyx-shell-test.drv
building '/nix/store/nwcva6f5dpra77byb9fyhz46g6k7kkyx-shell-test.drv'...
I'm running the test
error: builder for '/nix/store/nwcva6f5dpra77byb9fyhz46g6k7kkyx-shell-test.drv' failed with exit code 1;
       last 1 log lines:
       > I'm running the test
       For full logs, run 'nix log /nix/store/nwcva6f5dpra77byb9fyhz46g6k7kkyx-shell-test.drv'.

that fails as expected as it will evaluate a.nix, the passthru will provide an attribute called tests, which is used to compile all the derivations in this attribute.

Note that if you rename /tmp/a.nix into /tmp/default.nix, then once you cd /tmp you don’t need to specify the .nix file to use as it will automatically pick default.nix.

I hope it’s clearer.

6

Thank you so much @tobiasBora It’s a great help :slight_smile: