|
| 1 | +# Dotted Dependency Selector Architecture |
| 2 | + |
| 3 | +Date: 2026-06-02 |
| 4 | +Branch: `codex/dotted-dependency-analysis` |
| 5 | +Status: locally verified on feature branch; PR/CI/release still pending. |
| 6 | + |
| 7 | +## Scope |
| 8 | + |
| 9 | +This document evaluates how `mcpp` should support dotted dependency selectors |
| 10 | +such as: |
| 11 | + |
| 12 | +```toml |
| 13 | +[dependencies] |
| 14 | +cmdline = "0.0.1" |
| 15 | +capi.lua = "0.0.3" |
| 16 | +compat.gtest = "1.15.2" |
| 17 | +imgui.core = "0.0.1" |
| 18 | +imgui.backend.glfw_opengl3 = "0.0.1" |
| 19 | +``` |
| 20 | + |
| 21 | +The goal is not only to make one manifest example pass. The goal is a coherent |
| 22 | +dependency model for a module-oriented package manager while preserving current |
| 23 | +manifests, xpkg compatibility, workspace behavior, and CLI ergonomics. |
| 24 | + |
| 25 | +Local verification environment is a separate concern. If the local shell picks |
| 26 | +an old `mcpp`, switch it explicitly through xlings: |
| 27 | + |
| 28 | +```bash |
| 29 | +xlings use mcpp 0.0.42 |
| 30 | +``` |
| 31 | + |
| 32 | +That fixes local tool selection. It does not define the dependency selector |
| 33 | +architecture. |
| 34 | + |
| 35 | +## Requirements |
| 36 | + |
| 37 | +1. A single `[dependencies]` table can express both package namespaces and |
| 38 | + multi-level package/module names. |
| 39 | +2. `mcpplibs` may be omitted by users as a priority alias, but it is not the |
| 40 | + only root namespace. |
| 41 | +3. Explicit namespaces such as `compat` still work. |
| 42 | +4. Multi-level names such as `imgui.backend.glfw_opengl3` are first-class. |
| 43 | +5. Root dependencies, dev/build dependencies, workspace dependencies, xpkg |
| 44 | + `mcpp.deps`, and CLI add/remove must share the same selector rules. |
| 45 | +6. Existing manifests and quoted legacy dotted keys remain accepted. |
| 46 | +7. The resolver must be deterministic in candidate ordering. Package lookup may |
| 47 | + then select the first available candidate from local/index metadata. |
| 48 | + |
| 49 | +## Current Code Findings |
| 50 | + |
| 51 | +The current implementation is close in some places but is not yet one |
| 52 | +architecture: |
| 53 | + |
| 54 | +- `src/libs/toml.cppm` already parses unquoted TOML dotted keys into nested |
| 55 | + tables. This is why `[dependencies] compat.gtest = "..."` can be distinguished |
| 56 | + from a quoted flat key. |
| 57 | +- `src/libs/toml.cppm` also tracks explicit tables. That distinction matters: |
| 58 | + `[dependencies.imgui] core = "..."` is not the same user intent as |
| 59 | + `[dependencies] imgui.core = "..."`. |
| 60 | +- `src/manifest.cppm` has namespace-aware logic for part of root dependency |
| 61 | + parsing, but other dependency surfaces still split strings differently. |
| 62 | +- Workspace dependency parsing and synthesized xpkg `mcpp.deps` parsing do not |
| 63 | + currently share the same selector model. |
| 64 | +- `src/pm/compat.cppm` already owns package lookup compatibility and index/file |
| 65 | + naming fallback logic. It is a reasonable temporary home for compatibility |
| 66 | + helpers, but canonical selector parsing should be conceptually separate from |
| 67 | + legacy fallback behavior. |
| 68 | +- `src/pm/package_fetcher.cppm` and `src/pm/resolver.cppm` already consume a |
| 69 | + structured package namespace plus package name. The missing piece is upstream |
| 70 | + normalization before dependencies reach those layers. |
| 71 | +- `src/pm/commands.cppm` still needs the same user selector rules for add/remove |
| 72 | + so CLI behavior matches manifest behavior. |
| 73 | + |
| 74 | +## Design Decision |
| 75 | + |
| 76 | +Use one canonical dependency selector resolver at manifest/command ingestion |
| 77 | +time. The resolver should produce ordered package coordinate candidates: |
| 78 | + |
| 79 | +```text |
| 80 | +candidate(namespace + shortName)[] + stableMapKey |
| 81 | +``` |
| 82 | + |
| 83 | +The important rule is that `mcpplibs` is an optional prefix with priority, not a |
| 84 | +forced root. `imgui`, `compat`, `mcpplibs`, and custom index namespaces are |
| 85 | +peer roots. For selector `a.b`, lookup should first try the omitted-mcpplibs |
| 86 | +candidate, then fall back to the literal peer-root candidate. |
| 87 | + |
| 88 | +This keeps the interpretation explicit and ordered. The resolver itself should |
| 89 | +not perform network work; package lookup can resolve the ordered candidates |
| 90 | +against already available index metadata. |
| 91 | + |
| 92 | +## Selector Semantics |
| 93 | + |
| 94 | +Omitted `mcpplibs` priority: |
| 95 | + |
| 96 | +| selector | candidate priority | stable map key | |
| 97 | +| --- | --- | --- | |
| 98 | +| `cmdline` | `mcpplibs/cmdline` | `cmdline` | |
| 99 | +| `capi.lua` | `mcpplibs.capi/lua`, then `capi/lua` | `capi.lua` | |
| 100 | +| `imgui.core` | `mcpplibs.imgui/core`, then `imgui/core` | `imgui.core` | |
| 101 | +| `imgui.backend.glfw_opengl3` | `mcpplibs.imgui.backend/glfw_opengl3`, then `imgui.backend/glfw_opengl3` | `imgui.backend.glfw_opengl3` | |
| 102 | + |
| 103 | +Fully explicit prefix and peer-root fallback: |
| 104 | + |
| 105 | +| selector | candidate priority | stable map key | |
| 106 | +| --- | --- | --- | |
| 107 | +| `mcpplibs.capi.lua` | `mcpplibs.capi/lua` | `mcpplibs.capi.lua` | |
| 108 | +| `compat.gtest` | `mcpplibs.compat/gtest`, then `compat/gtest` | `compat.gtest` | |
| 109 | + |
| 110 | +Explicit subtable: |
| 111 | + |
| 112 | +```toml |
| 113 | +[dependencies.compat] |
| 114 | +gtest = "1.15.2" |
| 115 | +``` |
| 116 | + |
| 117 | +This remains a direct explicit namespace form and should resolve to |
| 118 | +`compat/gtest`, not to an omitted-mcpplibs candidate. Explicit subtables are the |
| 119 | +clearest form for non-default or custom index namespaces when ambiguity matters. |
| 120 | + |
| 121 | +## Namespace Roots |
| 122 | + |
| 123 | +Peer namespace roots include: |
| 124 | + |
| 125 | +- `mcpplibs` |
| 126 | +- `compat` |
| 127 | +- `imgui` and future module family roots if they exist as package indexes |
| 128 | +- explicit names declared in `[indices]` |
| 129 | +- explicit dependency table roots such as `[dependencies.acme]` |
| 130 | + |
| 131 | +Unquoted dotted selectors in the single `[dependencies]` table do not create a |
| 132 | +new namespace unconditionally. They create an ordered lookup: |
| 133 | + |
| 134 | +```text |
| 135 | +a.b.c -> mcpplibs.a.b/c, then a.b/c |
| 136 | +``` |
| 137 | + |
| 138 | +This is the key rule: `mcpplibs` has priority because it may be omitted, but |
| 139 | +`a.b` remains a valid peer namespace fallback. |
| 140 | + |
| 141 | +For custom index names, the deterministic rule should be: |
| 142 | + |
| 143 | +- If an explicit subtable is used, treat the subtable name as the root. |
| 144 | +- If a single-table dotted selector is used, try the omitted-mcpplibs candidate |
| 145 | + first, then the literal peer-root candidate. |
| 146 | +- If users need to bypass priority matching, they can use an explicit subtable |
| 147 | + such as `[dependencies.compat]` or a fully explicit selector such as |
| 148 | + `mcpplibs.<name>...`. |
| 149 | + |
| 150 | +This avoids unordered probing while still giving users an escape hatch. |
| 151 | + |
| 152 | +## Resolver Shape |
| 153 | + |
| 154 | +Recommended API shape: |
| 155 | + |
| 156 | +```cpp |
| 157 | +struct DependencySelectorContext { |
| 158 | + std::unordered_set<std::string> explicitNamespaceRoots; |
| 159 | + std::string defaultNamespace = "mcpplibs"; |
| 160 | +}; |
| 161 | + |
| 162 | +struct DependencySelector { |
| 163 | + std::vector<PackageCoordinate> candidates; |
| 164 | + std::string stableMapKey; |
| 165 | + bool explicitRoot = false; |
| 166 | +}; |
| 167 | + |
| 168 | +DependencySelector resolve_dependency_selector( |
| 169 | + std::span<const std::string> segments, |
| 170 | + const DependencySelectorContext& context); |
| 171 | +``` |
| 172 | +
|
| 173 | +The important design point is not the exact C++ names. The important point is |
| 174 | +that all dependency entry points call one resolver instead of each splitting on |
| 175 | +`.` independently. |
| 176 | +
|
| 177 | +## Ingestion Points |
| 178 | +
|
| 179 | +Apply the resolver at the edges: |
| 180 | +
|
| 181 | +- `src/manifest.cppm`: root dependencies, dev-dependencies, build-dependencies. |
| 182 | +- `src/manifest.cppm`: workspace dependencies. |
| 183 | +- `src/manifest.cppm`: synthesized dependencies from Lua xpkg `mcpp.deps`. |
| 184 | +- `src/pm/commands.cppm`: `mcpp add`, `mcpp remove`, and related CLI parsing. |
| 185 | +- Documentation examples in `docs/05-mcpp-toml.md`. |
| 186 | +
|
| 187 | +After candidate generation, package fetch/build/resolution should select the |
| 188 | +first available structured package coordinate. That keeps user-facing selector |
| 189 | +syntax out of lower build layers while preserving the requested priority |
| 190 | +matching behavior. |
| 191 | +
|
| 192 | +## Compatibility Policy |
| 193 | +
|
| 194 | +Keep all existing supported forms: |
| 195 | +
|
| 196 | +- `[dependencies] cmdline = "..."` |
| 197 | +- `[dependencies.mcpplibs] cmdline = "..."` |
| 198 | +- `[dependencies.compat] gtest = "..."` |
| 199 | +- quoted `"mcpplibs.cmdline" = "..."` |
| 200 | +- quoted legacy dotted keys where currently accepted |
| 201 | +- path/git inline specs |
| 202 | +- visibility/use requirements from the existing dependency model |
| 203 | +
|
| 204 | +For one-segment omitted-mcpplibs dependencies, keep the stable map key as the |
| 205 | +old bare key (`cmdline`) to avoid unnecessary lockfile or update churn. For |
| 206 | +multi-segment selectors, preserving user spelling such as `imgui.core` is more |
| 207 | +appropriate than rewriting everything to `mcpplibs.imgui.core`, because the |
| 208 | +selector is an ordered match rather than a forced namespace rewrite. |
| 209 | +
|
| 210 | +Quoted flat dotted keys should remain compatibility input, not the primary new |
| 211 | +syntax. The canonical user-facing syntax should be unquoted TOML dotted keys or |
| 212 | +explicit dependency subtables. |
| 213 | +
|
| 214 | +## Alternatives Considered |
| 215 | +
|
| 216 | +### A. Canonical Selector Resolver |
| 217 | +
|
| 218 | +Recommended. It gives one deterministic rule set and keeps compatibility logic |
| 219 | +at the ingestion boundary. |
| 220 | +
|
| 221 | +### B. Patch Each Parser Site Independently |
| 222 | +
|
| 223 | +Rejected. It may fix the first failing test but preserves divergent behavior |
| 224 | +between root deps, workspace deps, xpkg deps, and CLI commands. |
| 225 | +
|
| 226 | +### C. Unordered Index-Probing Fallback |
| 227 | +
|
| 228 | +Rejected. Arbitrary probing that changes priority based on current index |
| 229 | +contents is fragile. Ordered candidate lookup is different: the selector has a |
| 230 | +fixed priority list, and package resolution selects the first candidate that is |
| 231 | +available. |
| 232 | +
|
| 233 | +## Verification Plan |
| 234 | +
|
| 235 | +Unit tests: |
| 236 | +
|
| 237 | +- Selector matrix in the PM layer. |
| 238 | +- Root dependency dotted selectors. |
| 239 | +- Dev/build dependency dotted selectors. |
| 240 | +- Workspace dependency dotted selectors. |
| 241 | +- Synthesized xpkg `mcpp.deps` dotted selectors. |
| 242 | +- CLI add/remove selector normalization. |
| 243 | +- Quoted legacy dotted key compatibility. |
| 244 | +- Explicit custom index namespace behavior. |
| 245 | +
|
| 246 | +End-to-end tests: |
| 247 | +
|
| 248 | +- A local-index package using `compat.gtest`. |
| 249 | +- A local-index package using `imgui.core`. |
| 250 | +- A local-index package using `imgui.backend.glfw_opengl3`. |
| 251 | +
|
| 252 | +Before running local tests, pin the shell to the intended mcpp version: |
| 253 | +
|
| 254 | +```bash |
| 255 | +xlings use mcpp 0.0.42 |
| 256 | +mcpp test |
| 257 | +``` |
| 258 | + |
| 259 | +## Implementation Progress |
| 260 | + |
| 261 | +- 2026-06-02: Added `mcpp.pm.dependency_selector` with ordered candidates. |
| 262 | +- 2026-06-02: Extended `DependencySpec` with ordered candidate coordinates. |
| 263 | +- 2026-06-02: Updated root dependencies, dev/build dependencies, workspace |
| 264 | + dependencies, and xpkg `mcpp.deps` synthesis to preserve dotted selector |
| 265 | + spelling while recording candidate priority. |
| 266 | +- 2026-06-02: Updated dependency resolution to select the first candidate whose |
| 267 | + strict canonical xpkg.lua entry exists. This avoids legacy fallback lookup |
| 268 | + accidentally treating `compat.gtest` as `mcpplibs.compat/gtest`. |
| 269 | +- 2026-06-02: Added unit coverage for selector candidates and manifest parsing. |
| 270 | +- 2026-06-02: Added e2e coverage for `imgui.core` falling back from |
| 271 | + `mcpplibs.imgui/core` to peer-root `imgui/core`. |
| 272 | +- 2026-06-02: Updated `mcpp add` so dotted input stays in the single |
| 273 | + `[dependencies]` table; `ns:name` remains the explicit subtable syntax. |
| 274 | +- 2026-06-02: Full unit tests and targeted local-index/preinstall e2e checks |
| 275 | + pass locally. |
| 276 | +- 2026-06-02: Bumped the pending release version to `0.0.43` and added the |
| 277 | + changelog entry for dotted dependency selectors. |
| 278 | + |
| 279 | +Current local checks: |
| 280 | + |
| 281 | +```bash |
| 282 | +xlings use mcpp 0.0.42 |
| 283 | +mcpp test -- --gtest_filter='DependencySelector.*:Manifest.DependenciesDottedSelectorPreservesUserKeyAndCandidates:Manifest.DependenciesNamespacedSubtableNestedDottedKeyIsCanonical:SynthesizeFromXpkgLua.DepsDottedSelectorsUseManifestRules:Manifest.WorkspaceDependenciesUseDottedSelectorRules' |
| 284 | +mcpp build |
| 285 | +MCPP=target/.../bin/mcpp bash tests/e2e/62_dotted_dependency_selector_priority.sh |
| 286 | +MCPP=target/.../bin/mcpp bash tests/e2e/12_add_command.sh |
| 287 | +MCPP=target/.../bin/mcpp bash tests/e2e/52_local_path_namespaced_index.sh |
| 288 | +MCPP=target/.../bin/mcpp bash tests/e2e/58_preinstall_mcpp_deps_for_hooks.sh |
| 289 | +``` |
| 290 | + |
| 291 | +## Resolved Decisions |
| 292 | + |
| 293 | +1. Declared custom `[indices]` roots do not turn single-table dotted selectors |
| 294 | + into explicit roots. Single-table selectors still use omitted-mcpplibs |
| 295 | + priority: try `mcpplibs.<path>` first, then `<peer-root>.<path>`. |
| 296 | +2. The selector resolver lives in `src/pm/dependency_selector.cppm`. |
| 297 | + `compat.cppm` remains focused on legacy dotted-key and xpkg filename |
| 298 | + compatibility. |
| 299 | +3. CLI add/remove preserve user spelling for dotted selectors. Explicit |
| 300 | + namespace subtables remain available through `ns:name`. |
0 commit comments