Skip to content

ci: emit per-board Kconfig dep graph as a release sidecar#2171

Merged
widgetii merged 1 commit into
masterfrom
feat/kconfig-graph
Jun 4, 2026
Merged

ci: emit per-board Kconfig dep graph as a release sidecar#2171
widgetii merged 1 commit into
masterfrom
feat/kconfig-graph

Conversation

@widgetii

@widgetii widgetii commented Jun 4, 2026

Copy link
Copy Markdown
Member

Summary

Adds `general/scripts/kconfig_graph.py` — a kconfiglib-based emitter that
walks the merged Buildroot tree, filters to currently-set + user-prompted
`BR2_PACKAGE_*` symbols, and writes two schema-versioned JSONs into
`IMAGES_DIR` alongside `sizes.json`:

  • `kconfig-graph.-.json` — per-symbol type / prompt /
    depends / selects / selected_by (direct only)
  • `kconfig-help.-.json` — per-symbol help text, split out

This feeds firmware-explorer v0.3's interactive configurator (separate PR)
which lets users toggle packages off, computes the cascade closure with
real Kconfig semantics, and downloads a defconfig fragment.

Why filter currently-set + user-prompted

  • Buildroot's full `Config.in` defines ~7100 `BR2_PACKAGE_*` symbols across
    all upstream packages. Shipping all of them per board produces ~3 MB per
    platform with no UX benefit — the configurator can't usefully help you
    enable a package whose dependencies aren't already on this board.
  • Non-prompted symbols (`_ARCH_SUPPORTS`, `SUPPORTS`, `HAS*`) are
    feature-detection Buildroot derives automatically; they aren't
    user-toggleable.
  • The currently-set ∧ prompted intersection is closed under Kconfig's own
    dep tracking: every symbol referenced from an emitted symbol's
    `depends_on` / `select` chain is also set, so cascade resolution in the
    explorer doesn't lose references.

Empirical on a fully-configured cv200lite tree:

Artefact Symbols Raw Gzipped
`kconfig-graph.hi3516cv200-lite.json` 48 (12 cascade-required) 14 KB 1.8 KB
`kconfig-help.hi3516cv200-lite.json` 47 entries 9 KB 3.8 KB

`selected_by` precision (subtle)

Kconfiglib's `rev_dep` expression includes symbols from both the select
source AND its condition (`select X if Y` puts Y in the rev_dep). For the
cascade UX we want direct selectors only — otherwise depended-on symbols
look like spurious "blockers". The emitter pre-builds a
`target_symbol → [direct selectors]` index by iterating `sym.selects`
across the whole symbol set, then filters to the currently-set ones.

Kconfig env contract

Mirrors `size_report.py`: `TARGET_DIR` / `BR2_OUTPUT_DIR` / `IMAGES_DIR` /
`OPENIPC_SOC_MODEL` / `OPENIPC_VARIANT`. Sets defaults for the env vars
Buildroot's Kconfig top-level expects (`BR2_BASE_DIR`, `BR2_EXTERNAL`,
`BR2_EXTERNAL_GENERAL_PATH`, `srctree`). Two non-obvious ones:

  • `CONFIG_=""` — Kconfiglib defaults to stripping a `CONFIG_` prefix
    from `.config` lines; Buildroot's symbols carry the `BR2_` prefix
    included in the symbol table, so any non-empty prefix here makes
    `load_config` silently skip every line as "malformed".
  • `BR2_SKIP_LEGACY=YES` — skip `Config.in.legacy` parsing. It only
    generates renamed/removed warnings that we don't surface.

Makefile + CI

  • New `kconfig-graph` target mirrors `size-report:122-132`.
  • `make deps` appends `pip install --break-system-packages kconfiglib`
    (kconfiglib is a single-file pure-Python module, ~10k LoC, MIT).
  • `.github/workflows/build.yml` gets a "Build kconfig graph" step
    after the existing "Build size report". Best-effort: kconfiglib
    install or graph emit failure emits a `::warning::` and continues, so
    an upstream kconfiglib regression cannot sink a working build.
  • `${{env.KCONFIG}}` + `${{env.KHELP}}` added to all three upload
    steps with the same pattern that already promotes `SIZES`.

Tests

`general/scripts/tests/test_kconfig_graph.py` drives `build_report()`
against a synthetic Config.in fixture, asserting: type / prompt /
currently_set filtering; depends_on / selects / selected_by extraction;
direct-selector precision; help-text split. Run via
`python3 general/scripts/tests/test_kconfig_graph.py` — 2 unittest
cases, both green locally.

Cross-repo dependency

Once this lands and one nightly cycle attaches the JSON to release tags,
`OpenIPC/firmware-explorer` PR (separate) lights up the configurator tab.
Until then, that PR's prebuild logs "no kconfig assets found in any
retained tag" and the tab stays disabled with a tooltip — by design.

Test plan

  • Synthetic fixture tests: `python3 general/scripts/tests/test_kconfig_graph.py` — 2 passes
  • Real-tree smoke: `make kconfig-graph` against `output-cv200lite` — emits 48 symbols / 14 KB raw / 1.8 KB gzipped
  • Cascade correctness: BUSYBOX shows `selected_by: [BR2_INIT_BUSYBOX]`; ZLIB shows `selected_by: [BR2_PACKAGE_DROPBEAR_OPENIPC]` (one direct selector each, not the false-positive 40-element list from including condition expressions)
  • CI green on this branch
  • After merge: nightly attaches `kconfig-graph..json` + `kconfig-help..json` to release tags alongside `sizes.*.json`

🤖 Generated with Claude Code

Adds general/scripts/kconfig_graph.py — a kconfiglib-based emitter that walks
the merged Buildroot tree, filters to currently-set + user-prompted
BR2_PACKAGE_* symbols, and writes two schema-versioned JSONs into IMAGES_DIR
alongside sizes.json:

  kconfig-graph.<board>-<variant>.json   - per-symbol type/prompt/depends/
                                          selects/selected_by (direct only)
  kconfig-help.<board>-<variant>.json    - per-symbol help text, split out

This feeds firmware-explorer v0.3's interactive configurator (separate PR)
which lets users toggle packages off, computes the cascade closure with
real Kconfig semantics, and downloads a defconfig fragment.

Why filter currently-set + user-prompted?

  * Buildroot's full Config.in defines ~7100 BR2_PACKAGE_* symbols across
    all upstream packages. Shipping every symbol per board produces ~3 MB
    per platform with no UX benefit — the configurator can't usefully help
    you enable a package whose dependencies aren't already on this board.
  * Non-prompted symbols (*_ARCH_SUPPORTS, *_SUPPORTS, HAS_*) are feature
    detection Buildroot derives automatically; they aren't user-toggleable
    and would just be noise in the UI.
  * The currently-set ∧ prompted intersection is closed under Kconfig's
    own dep tracking: every symbol referenced from an emitted symbol's
    depends_on / select chain is also set, so cascade resolution in the
    explorer doesn't lose references.

  Empirical on a fully-configured cv200lite board: emits 48 symbols, 12
  cascade-required, **14 KB raw / 1.8 KB gzipped** per platform.

selected_by precision

  Kconfiglib's `rev_dep` expression collects symbols from both the select
  source AND its condition (`select X if Y` puts Y in the rev_dep). For
  the cascade UX we want direct selectors only — otherwise depended-on
  symbols look like spurious "blockers". The emitter pre-builds a
  target_symbol → [direct selectors] index by iterating
  `sym.selects` across the whole symbol set, then filters to the
  currently-set ones for the emitted `selected_by` list.

Kconfig env contract

  Mirrors size_report.py: TARGET_DIR / BR2_OUTPUT_DIR / IMAGES_DIR /
  OPENIPC_SOC_MODEL / OPENIPC_VARIANT. Sets defaults for the env vars
  Buildroot's Kconfig top-level expects (BR2_BASE_DIR, BR2_EXTERNAL,
  BR2_EXTERNAL_GENERAL_PATH, srctree). Two non-obvious ones:

    CONFIG_ = ""        Kconfiglib defaults to stripping a CONFIG_ prefix
                        from .config lines; Buildroot's symbols carry the
                        BR2_ prefix INCLUDED in the symbol table, so any
                        non-empty prefix here makes load_config silently
                        skip every line as "malformed".

    BR2_SKIP_LEGACY=YES Skip Config.in.legacy parsing. It only generates
                        renamed/removed warnings that we don't surface
                        in the configurator UI.

Makefile + CI

  New `kconfig-graph` target mirrors `size-report`. `make deps` gains
  `pip install --break-system-packages kconfiglib`. CI's Build kconfig
  graph step is best-effort: a kconfiglib install or graph emit failure
  emits a ::warning:: and continues, so an upstream kconfiglib regression
  cannot sink a working build. The KCONFIG + KHELP env vars are added
  to all three upload steps with the same pattern that already promotes
  SIZES.

Tests

  general/scripts/tests/test_kconfig_graph.py drives build_report()
  against a synthetic Config.in fixture, asserting: type / prompt /
  currently_set filtering; depends_on / selects / selected_by extraction;
  direct-selector precision; help-text split.  Run via
  `python3 general/scripts/tests/test_kconfig_graph.py`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@widgetii widgetii merged commit d7e89a8 into master Jun 4, 2026
103 checks passed
@widgetii widgetii deleted the feat/kconfig-graph branch June 19, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant