Skip to content

Auto-stage external_libraries in mkLogosModule's dev shell #97

Description

@danisharora099

Auto-stage external_libraries in mkLogosModule's dev shell

The problem

Every universal C++ module that wraps an external library (a Rust cdylib, a vendored C lib, etc.) ends up writing the same ~40-line devShells override so that nix develop actually works for non-Nix tooling — clangd, IDEs, plain cmake from the shell.

The override always does the same four things:

  1. Build the external library via Nix.
  2. Symlink its .dylib / .so + headers into the module's ./lib dir so the module's existing CMakeLists.txt (find_library(... PATHS lib NO_DEFAULT_PATH)) resolves it.
  3. Export DYLD_LIBRARY_PATH / LD_LIBRARY_PATH + CMAKE_LIBRARY_PATH + CMAKE_INCLUDE_PATH.
  4. Export CMAKE_EXPORT_COMPILE_COMMANDS=ON for clangd.

Inside the Nix sandbox this all works because logos-plugin-qt's buildPlugin stages the libraries itself. Outside the sandbox (i.e. nix develop), mkLogosModule leaves the consumer to wire it up.

Most recent example: eth-lez-atomic-swaps PR #26 hand-rolled this in swap-module/flake.nix. The mapping it expresses — swap_ffi cdylib → swap-module/lib/libswap_ffi.dylib — is already declared in the module's metadata.json ("nix.external_libraries": [{"name": "swap_ffi"}]) and in its externalLibInputs. Everything mkLogosModule needs to generate the dev shell itself is already there.

The proposal

When externalLibInputs is non-empty, have mkLogosModule's dev shell:

  • Resolve the external libs (reusing the same resolveExtInput + mkExternalLib.buildExternalLibs path the package outputs already use).
  • Symlink each one's lib/ + include/*.h into ./lib.
  • Export the four library-path env vars + CMAKE_EXPORT_COMPILE_COMMANDS=ON.

After this, downstream nix develop Just Works and the ~40-line override block in PR #26 disappears.

Backward compatibility

  • Modules with no external_libraries: zero change (the new behaviour is gated on the list being non-empty).
  • Modules that already override devShells.<system>.default: keep working; nothing about the input surface changes.
  • Build derivations (nix build): untouched. Only nix develop is affected.

Proof-of-concept

Branch feat/devshell-stage-external-libs on a fork. +62 / −2 in lib/mkLogosModule.nix. nix flake check passes on aarch64-darwin (all 4 checks, including devShells.aarch64-darwin.default).

PR: #98 — happy to iterate on it or to file a fresh one once the design questions below are settled.

Open design questions

  1. Symlink or copy? PR feat: bump standalone app #26 uses symlinks (rebuilds picked up automatically next time the shell is entered). The in-sandbox copyExternalLibsToLib uses cp (sandbox-private paths). The PoC uses symlinks for dev shells — matches PR feat: bump standalone app #26 — but the asymmetry is worth a sentence.
  2. Header glob. Currently *.h only. Should it cover *.hpp and extension-less cbindgen outputs?
  3. Opt-out toggle. Right now: "auto whenever externalLibInputs is non-empty". Worth adding an autoStageExternalLibs = false escape hatch, or is "if you declared external libs, you want them staged" obvious enough?
  4. Explicit metadata.json field? e.g. nix.external_libraries[].source = "swap-ffi-source". The PoC keeps the current implicit convention (consumer passes externalLibInputs). Leaning: keep implicit; auto-discovery is a separate proposal.
  5. Hoist resolveExtInput. It's defined inside the packages let and closes over system. The PoC adds a second local copy in devShells for minimal blast radius; a follow-up could lift it to the outer scope and parameterise on system.

Out of scope

  • external_libraries shapes other than the "Nix-builds-a-derivation-with-$out/lib-+-$out/include" pattern (Go libs, prebuilt C libs, vendor submodules). The implementation generalises trivially but the motivating use case is Rust cdylibs.
  • Scaffold-side ergonomics (lgs basecamp develop <module>) — separate downstream ask.
  • Removing the downstream override in eth-lez-atomic-swaps — follow-up PR once this lands.

Pin context

eth-lez-atomic-swaps pins logos-module-builder @ 434b98a. The devShells block this proposal changes is byte-identical between 434b98a and master @ b3f1d65, so the change applies unchanged to both.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions