Build multi-layered project using nix

I have a project that follows the following build structure:

  • some rust parts are compiled down using wasm-pack to wasm
  • the package is then included in a frontend, the frontend is built using npm run buld
  • the resulting dist folder (index.html) is then included in my backend (again written in rust)

My question know is how can I build such a project using nix, my current build strucuture is the one specified below:

FROM rust:alpine AS development
RUN apk update && apk upgrade && apk add npm git alpine-sdk openssh curl
RUN curl -fsS https://dotenvx.sh | sh
RUN cargo install wasm-pack
RUN cargo install --locked bacon
RUN rustup component add rustfmt

FROM development AS build
ARG BUILD_VARIANT=release
WORKDIR /workspace
COPY . .
RUN VARIANT=${BUILD_VARIANT} make all
RUN VARIANT=${BUILD_VARIANT} make test
WORKDIR /artifacts
RUN cp /workspace/target/${BUILD_VARIANT}/backend .
RUN strip backend

FROM scratch AS run
WORKDIR /app
COPY --from=build /artifacts/backend .
CMD ["/app/backend"]

And the makefile is here:

cargo fetch
dotenvx run -- wasm-pack build --${VARIANT} --out-dir ${CURRENT_ROOT}/wasm-pkgs/backend_interfaces sources/rust/backend_interfaces
npm install --workspaces --include=optional
npm run build --workspaces --${VARIANT}
dotenvx run -- cargo build ${CARGO_RELEASE_FLAG}

In the last step (cargo build), the backend embeds the generated index.html in the resulting binary using rust_embed.

The generated wasm package is included in my package.json like this:

  "optionalDependencies": {
    "backend_interfaces": "file:../../../wasm-pkgs/backend_interfaces"
  }

My question here is how can I download the dependencies from npm and cargo before actually running the build scripts? Do you have any pointers on this?
So my main two problems are:

  1. My frontend needs some rust parts (that are included via wasm) to be built, those are referenced via the file: URL in package.json
  2. My backend needs the frontend to be already built, because the resulting html files get included in the binary using rust_embed

I have come up with:

{
  description = "Nix environment";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/8b27c1239e5c421a2bbc2c65d52e4a6fbf2ff296";
    flake-utils.url = "github:numtide/flake-utils/04c1b180862888302ddfb2e3ad9eaa63afc60cf8";
  };

  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachSystem [
      flake-utils.lib.system.x86_64-linux
      flake-utils.lib.system.aarch64-linux

      flake-utils.lib.system.x86_64-darwin
      flake-utils.lib.system.aarch64-darwin
    ] ( system:
      let
        pkgs = import nixpkgs {
          inherit system;
          config.allowUnfree = false;
        };

        stdEnv = pkgs.swiftPackages.stdenv;

        globalPackages = with pkgs; [
            # tools
            dotenvx
            zsh
            git
            gnumake
            cmake

            # rust toolchain
            rustc
            cargo
            wasm-pack

            # typescript toolchain
            nodejs_23
        ];

       package = stdEnv.mkDerivation {
            name = "package";
            src = ./.;

            buildInputs = with pkgs; globalPackages;

            phases = [ "unpackPhase" "buildPhase" "installPhase" ];

            buildPhase = ''
              ??
            '';

            installPhase = ''
              mkdir -p $out/bin
              cp my-executable $out/bin/
            '';
          };
      in {
        devShells = {
          default = pkgs.mkShell.override
          {
            stdenv = stdEnv;
          }
          {
            packages = with pkgs; globalPackages ++ [
              (vscode-with-extensions.override {
                  vscode = vscodium;
                  vscodeExtensions = with vscode-extensions; [
                      bbenoist.nix
                      streetsidesoftware.code-spell-checker
                      vscode-extensions.rust-lang.rust-analyzer
                      vscode-extensions.dbaeumer.vscode-eslint
                      vscode-extensions.esbenp.prettier-vscode
                  ] ++ vscode-utils.extensionsFromVscodeMarketplace [ ];
              })
            ];

            DONT_PROMPT_WSL_INSTALL = true; #  to supress warnings issued by vscode on wsl
          };
        };

        packages = {
          default = package;
        };
      }
    );
}

Had a similar problem, i had to run make to build the static files:

As i see, fetchFromGitHub fetches the source code of a release and builds from there, so this builds manually, you can override buildPhase and installPhase vars to build yout way, by using your scripts before building. Or if you have a pre-built binary from GH releases, you can use fetchurl to fetch this binary, which wont be built, but you need to use pkgs.stdenv.mkDerivation instead.

What is your general nix level of experience? Have you looked at the nixpkgs manual for js projects? Have you looked at how buildRustProject functions? Both language specific methods use a series of hooks that inject FODs that vendor dependencies upfront s.t. that the build has them available when the network is sandboxed. You might be able to use these hooks off the shelf, or you might need to customise them, but there is no magic behind the curtain.

I have now implemented it something like that: from-scratch/flake.nix at 107cf4f2115b71bf7e85c1b5be927a61574046b8 · b521f771d8991e6f1d8e65ae05a8d783/from-scratch · GitHub

This allowed me to use the Makefile - the same I use for regular non-nix builds - in a nix flake. I did not want to do layered builds, so I just made a “download-layer” and copied node_modules over from the download layer.