Skip to content

Conversation

@bjordiscollaku
Copy link
Contributor

@bjordiscollaku bjordiscollaku commented Dec 23, 2025

This change enhances the rootfs build flow by replacing the ISO/squashfs baseline with a deterministic, seed-driven debootstrap baseline. The update spans Baseline rootfs creation, Seed parsing + safety guarantees, Preprocessing refactor, CLI improvements, Config cleanup / removing ISO-era remnants, Package policy changes, and Generalize builder naming + artifacts + GRUB labeling.

What changed (high level)

Baseline rootfs creation

  • Replaces the previous Ubuntu ISO → 7z extract → locate *.squashfsunsquashfs → copy into rootfs/ pipeline with debootstrap.
  • Baseline rootfs is now generated from:
      - qcom-product.conf for DISTRO/CODENAME/ARCH (and optional apt settings), and
      - a seed file containing one package per line (supports # comments and blank lines).

Seed parsing + safety guarantees

  • Adds seed parsing to produce a debootstrap --include list.
  • Enforces “one package token per line” (fails fast on whitespace entries to avoid accidental multi-token installs).
  • Ensures a minimal required package set is always included (e.g., lsb-release, apt, sudo, systemd-sysv, etc.) so the existing chroot stages can run unchanged (e.g., apt install, user creation, systemd symlinks, etc.).

Preprocessing refactor

  • Consolidates baseline preparation into a single platform/distro agnostic function: image_preprocessing().
  • Removes the platform-specific preprocessing entry points (image_preproccessing_iot/compute/server) since baseline generation no longer depends on platform-specific ISO handling.

CLI improvements

  • Replaces legacy positional argument modes with explicit named options:
      - --product-conf <qcom-product.conf>
      - --seed <seed_file>
      - --kernel-package <kernel.deb>
      - --firmware <firmware.deb>
      - optional --overlay <package-manifest.json>
  • Adds --help usage output and strict validation for required inputs.

Config cleanup / removing ISO-era remnants

  • Removes reliance on ISO-era configuration such as BASE_IMAGE_URL / IMG_URL and related defaulting logic, since preprocessing is now strictly debootstrap-based.
  • Introduces optional qcom-product.conf knobs for bootstrap control:
      - APT_MIRROR (defaults to an Ubuntu ports mirror)
      - APT_COMPONENTS (defaults to main,universe)

Package policy changes

  • Drops implicit desktop-by-default installs:
      - Removes the fixed apt install -y ubuntu-desktop-minimal ... behavior from the chroot stage so package composition is fully defined by the seed and/or overlay manifest.
      - If a desktop image is desired, add ubuntu-desktop-minimal (and any other UI packages) to the seed or overlay manifest explicitly.
  • Installs a minimal networking stack in the Step 8 chroot stage:
      - Moves network-manager out of debootstrap --include to avoid observed debootstrap failures during “configure base packages” when using --variant=minbase.
      - Adds a dedicated chroot block that runs apt-get update and installs (with --no-install-recommends):
        - network-manager
        - wpasupplicant
        - iw
        - net-tools

Generalize builder naming + artifacts + GRUB labeling

  • Rename the rootfs builder to reflect that it is not Ubuntu-specific:
      - build-ubuntu-rootfs.shbuild-rootfs.sh
      - Header text and usage examples refreshed accordingly.
  • Align the default output name with the generic script:
      - ROOTFS_IMG: ubuntu.imgrootfs.img
  • Generalize GRUB labeling and remove chroot-side codename detection:
      - Use "${DISTRO} ${CODENAME}" for the GRUB menuentry title (rather than hardcoding “Ubuntu …”).
      - Pass DISTRO/CODENAME into the chroot via environment (from qcom-product.conf) instead of calling lsb_release inside the chroot.

Why this is better

  • Deterministic & reproducible: baseline comes from codename + seed rather than ISO contents and squashfs layout variations.
  • Less brittle: avoids hunting for minimal.squashfs / ISO layout drift across releases.
  • Cleaner interface: explicit flags + seed file makes builds easier to audit and automate.
  • Future-ready: preprocessing is no longer tied to a specific target platform and can evolve via product-conf inputs.

Notes / limitations

  • This update currently assumes ARCH=arm64 (host/target ARM64) and errors out otherwise.
  • User-facing interface change: callers/scripts that reference build-ubuntu-rootfs.sh or ubuntu.img must be updated (or add an out-of-tree compatibility wrapper/symlink if needed).

Example usage

./build-rootfs.sh \
  --product-conf qcom-product.conf \
  --seed seed_file \
  --kernel-package kernel.deb \
  --firmware firmware.deb \
  [--overlay package-manifest.json]

…otstrap rootfs

This change modernizes `build-ubuntu-rootfs.sh` by removing the ISO download/extract + squashfs unsquash baseline flow and replacing it with a deterministic, seed-driven `debootstrap` baseline rootfs build. The rest of the pipeline (kernel/firmware injection, chroot package installs, GRUB config, manifest capture, and image assembly) is intentionally preserved to keep behavior stable while making the baseline generation faster, simpler, and more reproducible.

**What changed (high level):**

* **Baseline rootfs creation**

  * Replaces the previous **Ubuntu ISO → 7z extract → locate ****`*.squashfs`**** → ****`unsquashfs`**** → copy into ****`rootfs/`** pipeline with **`debootstrap`**.
  * Baseline rootfs is now generated from:

    * `qcom-product.conf` for **DISTRO/CODENAME/ARCH** (and optional apt settings), and
    * a **seed file** containing one package per line (supports `#` comments and blank lines).

* **New seed parsing + safety guarantees**

  * Adds seed parsing to produce a `debootstrap --include` list.
  * Enforces **“one package token per line”** (fails fast on whitespace entries to avoid accidental multi-token installs).
  * Ensures a **minimal required package set** is always included (e.g., `lsb-release`, `apt`, `sudo`, `systemd-sysv`, etc.) so the existing **chroot stages run unchanged** (e.g., `lsb_release -sc`, `apt install`, user creation, systemd symlinks).

* **Preprocessing refactor**

  * Consolidates baseline preparation into a single **platform/distro agnostic** function: `image_preprocessing()`.
  * Removes the platform-specific preprocessing entry points (`image_preproccessing_iot/compute/server`) since baseline generation no longer depends on platform-specific ISO handling.

* **CLI improvements**

  * Replaces legacy positional argument modes with explicit named options for clarity and repeatability:

    * `--product-conf <qcom-product.conf>`
    * `--seed <seed_file>`
    * `--kernel-package <kernel.deb>`
    * `--firmware <firmware.deb>`
    * optional `--overlay <package-manifest.json>`
  * Adds a dedicated `--help`/usage output and strict validation for required inputs.

* **Config cleanup / removing ISO-era remnants**

  * Removes reliance on ISO-era configuration such as `BASE_IMAGE_URL` / `IMG_URL` and related defaulting logic, since preprocessing is now strictly `debootstrap`-based.
  * Introduces optional `qcom-product.conf` knobs for baseline bootstrap control:

    * `APT_MIRROR` (defaults to a sensible Ubuntu ports mirror)
    * `APT_COMPONENTS` (defaults to `main,universe`)

**What did NOT change (intentional):**

* Kernel + firmware `.deb` injection behavior is unchanged.
* Chroot install/config stages remain as-is (base package installs, user creation, GRUB config, manifest capture).
* Overlay/manifest processing (apt source injection + additional package installation) remains supported and **functionally unchanged**—it is only referenced by name to preserve the existing interface.

**Why this is better:**

* **Deterministic & reproducible**: baseline comes from codename + seed rather than ISO contents and squashfs layout variations.
* **Less brittle**: avoids hunting for `minimal.squashfs` / ISO layout drift across releases.
* **Cleaner interface**: explicit flags + seed file makes builds easier to audit and automate.
* **Future-ready**: preprocessing is no longer tied to a specific target platform and can evolve with product-conf inputs.

**Example usage:**
`./build-ubuntu-rootfs.sh --product-conf qcom-product.conf --seed seed_file --kernel-package kernel.deb --firmware firmware.deb [--overlay package-manifest.json]`


Signed-off-by: Bjordis Collaku <[email protected]>
Remove the fixed `apt install -y ubuntu-desktop-minimal network-manager iw net-tools` from Step 8 (chroot) so package selection is fully driven by the seed and/or overlay manifest, rather than being implicitly enforced by the script.

**What changed:**

* Deleted the Step 8 block that installed `ubuntu-desktop-minimal`, `network-manager`, `iw`, and `net-tools` during the chroot phase.

**Why:**

* Keeps the debootstrap workflow policy-free: the script builds the rootfs, while package composition is defined explicitly via inputs (seed/overlay).
* Avoids surprising “desktop by default” behavior when the baseline is intended to be minimal and reproducible.

**Notes:**

* If a desktop image is desired, add these packages to the seed file or overlay manifest instead.
* All other chroot configuration (services symlinks, kernel/firmware install, user setup, GRUB config, manifest capture) remains unchanged.

Signed-off-by: Bjordis Collaku <[email protected]>
Move installation of the Wi-Fi/networking tools out of the debootstrap baseline and into the Step 8 chroot stage to avoid debootstrap “Failure while configuring base packages” issues (observed when pulling in `network-manager` during `--variant=minbase`).

**Changes:**  
- Re-introduce a dedicated Step 8 block that:
  - runs `apt-get update`
  - installs a minimal networking set with `--no-install-recommends`:
    - `network-manager` (network control via `nmcli`)
    - `wpasupplicant` (Wi-Fi authentication backend)
    - `iw` (Wi-Fi device/scan tooling)
    - `net-tools` (legacy helpers like `ifconfig`, `netstat`)  
- Keep desktop meta-packages out of the flow (no `ubuntu-desktop-minimal`), preserving a lean baseline while still enabling practical Wi-Fi bring-up.

**Rationale:**  
- `debootstrap` + `minbase` is intentionally minimal and can be brittle when higher-level networking packages are pulled into `--include` during the bootstrap/configure phase. Installing these packages after the rootfs exists (in Step 8) is more reliable and keeps preprocessing deterministic.

Signed-off-by: Bjordis Collaku <[email protected]>
…tifacts

Rename the rootfs builder to reflect that it is not Ubuntu-specific:

- build-ubuntu-rootfs.sh -> build-rootfs.sh
- Refresh header text and usage examples accordingly

Align the default output name with the generic script:

- ROOTFS\_IMG: ubuntu.img -> rootfs.img

Generalize GRUB labeling and remove chroot-side codename detection:

- Use "${DISTRO} ${CODENAME}" for the menuentry title
- Pass DISTRO/CODENAME into the chroot via env (from qcom-product.conf) instead of calling lsb\_release inside the chroot

Notes:

- This is a user-facing interface change: callers/scripts that reference build-ubuntu-rootfs.sh or ubuntu.img must be updated (or add a compatibility wrapper/symlink out of tree if needed).

Signed-off-by: Bjordis Collaku <[email protected]>
@lool
Copy link

lool commented Dec 29, 2025

Hey –that's pretty cool! Instead of a delta definition and relying on existing images, this actually creates the images from packages.

Could we go one step further and use the same image building tool as the underlying distro? Ubuntu uses livecd-rootfs as a wrapper to the ubuntu-image snap and the live-build deb. If we could call livecd-rootfs directly, that would be neat, or we could call ubuntu-image. Image definitions are nice yaml files which we can easily extend. IMO seeds are a bit cumbersome to maintain for a downstream, but doable; to truly use seeds, we'd want to run germinate somewhere, but perhaps it's overkill and we can stick to standard seeds + extra packages?

For Debian, it's a similar toolset, but the image generation packages are non-trivially patched between Debian and Ubuntu. I've found the Debian Libre Live image building scripts to be quite neat wrappers:
https://gitlab.com/debdistutils/debian-libre/debian-libre-live#debian-libre-live-images

@bjordiscollaku bjordiscollaku changed the title rootfs: replace ISO/squashfs baseline with conf- and seed-driven deboostrap rootfs rootfs: replace ISO/squashfs baseline with conf- and seed-driven mmdebstrap rootfs Jan 8, 2026
@bjordiscollaku bjordiscollaku changed the title rootfs: replace ISO/squashfs baseline with conf- and seed-driven mmdebstrap rootfs rootfs: replace ISO/squashfs baseline with conf- and seed-driven debootstrap rootfs Jan 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants