Record environment intent in exported lockfiles by jezdez · Pull Request #132 · conda/conda-lockfiles

jezdez · 2026-04-24T11:11:40Z

Closes #8.

Summary

Populates user-requested specs into both supported lockfile formats so exports/round-trips carry "what the user asked for", not just the full explicit install set.

Intent is sourced from conda-meta/history via a new conda_lockfiles.history.requested_specs_from_prefix helper. We re-derive from history rather than trust env.requested_packages because conda populates that field with every installed package by default (Environment.from_prefix(from_history=False), tracked upstream at conda/conda#15961).

Storage

conda-lock-v1 — metadata.custom_metadata.requested_specs, JSON-encoded list of MatchSpec strings:

metadata:
  custom_metadata:
    created_by: conda-lockfiles 0.1.x
    requested_specs: '["python_abi"]'

Widens our custom_metadata model to free-form dict[str, str] to match upstream conda-lock and CEP 37.
Key name requested_specs follows CEP 32's terminology for MatchSpec-string lists; no namespacing since conda-lock has no convention and no CEP mandates it.
Low interop risk: conda-lock's LockMeta is StrictModel (extra fields rejected), but custom_metadata itself is explicitly a user-extensible dict[str, str].

rattler-lock-v6 — environments.<name>.requested-packages as a per-platform map:

environments:
  default:
    channels:
      - url: conda-forge
    packages:
      linux-64:
        - conda: https://.../python_abi-3.13-7_cp313.conda
    requested-packages:
      linux-64:
        - python_abi

Kebab-case key matches rattler's serialization convention for multi-word fields (SolveOptions uses #[serde(rename_all = "kebab-case")]).
rattler's deserializer does NOT use deny_unknown_fields, so pixi install --frozen keeps working and ignores the extension.
Caveat: any rattler-driven re-lock (pixi add, pixi update) will drop the annotation. This is advisory, not durable. Documented inline.

Loader round-trip

Both CondaLockV1Loader and RattlerLockV6Loader decode the fields and populate Environment.requested_packages as list[MatchSpec]. Specs whose package name isn't in the platform's explicit_packages are dropped because Environment.__post_init__ enforces the subset relationship and would otherwise raise.

Test coverage

New tests/test_requested_specs.py with 7 tests covering the history helper, fixture contents, loader population for both formats, and malformed-payload tolerance. The existing single_package fixtures (conda-lock.yml, pixi.lock) are updated to carry the new field, so the existing test_export_to_conda_lock_v1[single-package] and test_export_to_rattler_lock_v6[single-package] round-trip comparisons now also validate serialization.

Full suite: 55 passed locally on test-py313.

References

Upstream issue about the from_history=False default: Reconsider Environment.from_prefix(from_history=False) as the default conda#15961
conda.history.History: https://github.com/conda/conda/blob/main/conda/history.py
Environment.from_prefix: https://github.com/conda/conda/blob/main/conda/models/environment.py
CEP 32 (requested_specs terminology): https://github.com/conda/ceps/blob/main/cep-0032.md
CEP 37 (conda-lock v1 custom_metadata): https://github.com/conda/ceps/blob/main/cep-0037.md
conda-lock LockMeta: https://github.com/conda/conda-lock/blob/main/conda_lock/lockfile/v1/models.py
rattler lock deserializer (no deny_unknown_fields): https://github.com/conda/rattler/blob/main/crates/rattler_lock/src/parse/deserialize.rs

Populate user-requested specs into both lockfile formats so round-trips actually carry "what the user asked for" rather than just the full explicit install set. Source of intent is `conda-meta/history` via a new `conda_lockfiles.history.requested_specs_from_prefix` helper. We re-derive from history rather than trusting `env.requested_packages` because conda populates that with every installed package by default (`conda.models.environment.Environment.from_prefix(from_history=False)`, tracked upstream at conda/conda#15961). Storage per format: - **conda-lock-v1**: under `metadata.custom_metadata.requested_specs` as a JSON-encoded list of MatchSpec strings. The model field is widened to free-form `dict[str, str]` to match upstream conda-lock and CEP 37. Name choice follows CEP 32. - **rattler-lock-v6**: under `environments.<name>.requested-packages` as a per-platform map of MatchSpec strings. rattler silently ignores unknown fields on parse, so pixi `install --frozen` is unaffected. Any rattler-driven re-lock (`pixi add`, `pixi update`) will drop the annotation; this is advisory, not durable. Loaders populate `Environment.requested_packages` from these fields, filtering out specs whose package name is not in `explicit_packages` since `Environment.__post_init__` enforces that subset relationship.

Reverting part of the previous commit: we shouldn't drop the typed custom_metadata wrapper just because the wire format is dict[str, str]. Named Python-side fields for created_by and requested_specs are more useful for readers/writers than a raw dict, and extra="allow" on the BaseModel preserves any unknown keys that upstream consumers may have put there. The JSON-stringification of the requested_specs value is still required for interop with conda-lock (StrictModel on LockMeta.custom_metadata with dict[str, str]), but that constraint is worth revisiting at the CEP level: see conda/ceps#163.

jezdez · 2026-04-27T14:44:08Z

@soapy1 Not sure if you have thoughts on this, but I remember talking to you about it..

soapy1 · 2026-04-28T16:50:09Z

+    from conda.common.path import PathType
+
+
+def requested_specs_from_prefix(prefix: PathType | None) -> list[str]:


What is the motivation for getting information from the history in the plugin? I would expect that the plugin should be as simple as possible, just converting Environments to their target file format and back. Then gathering information from history is something that core conda should do when required.

For example, consider a new lockfile plugin (eg. maybe contributed from the community). It would be nice for this plugin to also get the correct user requested specs without needing to copy this code + plug it into the right places.

soapy1 · 2026-04-28T16:54:05Z

+            try:
+                ms = MatchSpec(spec_str)
+            except Exception:
+                continue


Does this error represent a spec_str that is an invalid matchspec? That seems like a pretty significant error. Probably worth warning the user in some way. Maybe like a log or warning. +1 to not throwing an error and exiting the program.

soapy1 · 2026-04-28T16:56:04Z

+        explicit_names = {pkg.name for pkg in resolved_explicit}
+        for spec_str in platform_specs:
+            if not isinstance(spec_str, str):
+                continue


In which cases are these spec_str's not strings? Should the user be notified that their environment file has something wrong with it?

soapy1 · 2026-04-28T17:05:49Z

+    # kebab-case convention for multi-word keys. rattler silently ignores
+    # unknown fields on read, but WILL drop them on any re-serialize
+    # (``pixi add``, ``pixi update``, ...), so treat this as advisory.
+    requested_packages: Annotated[


Is it safe to be extending external lockfile specifications like this? I see in this case it is since rattler will silently ignore unknown fields. But will other specifications also do this?

soapy1 · 2026-04-28T17:13:53Z

-        ),
+        explicit_packages=resolved_explicit,
        external_packages=external_packages,
+        requested_packages=_requested_packages_from_metadata(


Is there extra work on the conda side that needs to be done so that these requested packages get represented in the history?

jezdez requested a review from a team as a code owner April 24, 2026 11:11

conda-bot added the cla-signed label Apr 24, 2026

jezdez mentioned this pull request Apr 24, 2026

CEP 37: loosen custom_metadata from dict[str, str] conda/ceps#163

Open

jezdez mentioned this pull request Apr 24, 2026

record environment intent in exported lockfiles #8

Open

Apply ruff-format fix for requested_specs one-liner

ef1cd43

soapy1 reviewed Apr 28, 2026

View reviewed changes

jezdez marked this pull request as draft May 5, 2026 13:22

Navigation Menu

Record environment intent in exported lockfiles - #132

Record environment intent in exported lockfiles#132
jezdez wants to merge 3 commits into
mainfrom
record-intent-issue-8

jezdez commented Apr 24, 2026

Uh oh!

jezdez commented Apr 27, 2026

Uh oh!

soapy1 Apr 28, 2026

Uh oh!

soapy1 Apr 28, 2026

Uh oh!

soapy1 Apr 28, 2026

Uh oh!

soapy1 Apr 28, 2026

Uh oh!

soapy1 Apr 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		from conda.common.path import PathType


		def requested_specs_from_prefix(prefix: PathType \| None) -> list[str]:

Conversation

jezdez commented Apr 24, 2026

Summary

Storage

Loader round-trip

Test coverage

References

Uh oh!

jezdez commented Apr 27, 2026

Uh oh!

soapy1 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

soapy1 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

soapy1 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

soapy1 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

soapy1 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants