Skip to content

i/statemachine: add manifest-v2 to align with livecd-rootfs format#288

Merged
upils merged 5 commits into
mainfrom
fhost/manifest
Jun 26, 2026
Merged

i/statemachine: add manifest-v2 to align with livecd-rootfs format#288
upils merged 5 commits into
mainfrom
fhost/manifest

Conversation

@fhost

@fhost fhost commented Feb 13, 2026

Copy link
Copy Markdown
Collaborator

Add a new artifact manifest-v2 for classic image with the same format as the one from livecd-rootfs.
Difference with the current manifest:

  • Use tab instead of space in the manifest
  • Add snaps to the manifest

@fhost fhost requested review from mwhudson and upils February 13, 2026 22:36
@fhost fhost force-pushed the fhost/manifest branch 3 times, most recently from 2031a11 to b3228f7 Compare February 16, 2026 14:43
@codecov

codecov Bot commented Feb 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.27778% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.47%. Comparing base (3533000) to head (654bf4b).
⚠️ Report is 15 commits behind head on main.

Files with missing lines Patch % Lines
internal/statemachine/helper.go 88.67% 3 Missing and 3 partials ⚠️
internal/helper/helper.go 83.33% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #288      +/-   ##
==========================================
- Coverage   93.49%   93.47%   -0.03%     
==========================================
  Files          19       19              
  Lines        3552     3602      +50     
==========================================
+ Hits         3321     3367      +46     
- Misses        151      153       +2     
- Partials       80       82       +2     
Flag Coverage Δ
unittests 93.47% <90.27%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fhost

fhost commented Feb 16, 2026

Copy link
Copy Markdown
Collaborator Author

(I'll do the coverage if you do accept the modification)

Comment thread internal/statemachine/helper.go Outdated
}

// GenerateClassicManifest generate the classic manifest file for the given rootfs
func GenerateClassicManifest(rootfs string, outputPath string, debug bool) error {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this need to be exported?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is used in classic_state.go.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's in the same package though? my Go is a bit rusty but I'm not that out of it yet!

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My Go is a bit too new, so I did not know it works like this :)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

Comment thread internal/statemachine/helper.go Outdated
@fhost fhost requested a review from mwhudson February 18, 2026 09:39

@mwhudson mwhudson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i do want you to unexport GenerateClassicManifest please

Comment thread internal/statemachine/helper.go Outdated
}

// GenerateClassicManifest generate the classic manifest file for the given rootfs
func GenerateClassicManifest(rootfs string, outputPath string, debug bool) error {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's in the same package though? my Go is a bit rusty but I'm not that out of it yet!

Comment thread internal/statemachine/helper.go Outdated
@fhost fhost requested a review from mwhudson February 19, 2026 09:39

@upils upils left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The manifest currently produced is very likely consumed by some tools by some people. So this change would be a breaking change due to the 2 changes you mention in the description. I would like to avoid that if possible.

I am more inclined to post-process the output than change it. We have unfortunately no way to express the format version so this will be a surprising change for users.

I assume the space->tab issue could be solved with a call to sed. Is that so or is that more complex?
Also, in the README, it is clearly stated that the manifest lists packages, the current proposition would change this definition.

I agree the snaps list is currently missing and that would be nice to extract it. What do you think of generating another separate "snaps.manifest" file, listing snaps (and maybe using the tab separator?)? A bit a thinking is needed to define how that would be expressed in the image definition file but we can discuss it.
With that, the post-process would also be trivial before calling sbom-generate.

Comment thread internal/statemachine/helper.go Outdated
@fhost

fhost commented Feb 25, 2026

Copy link
Copy Markdown
Collaborator Author

The manifest currently produced is very likely consumed by some tools by some people. So this change would be a breaking change due to the 2 changes you mention in the description. I would like to avoid that if possible.

It is what I'm afraid of.

I am more inclined to post-process the output than change it. We have unfortunately no way to express the format version so this will be a surprising change for users.

I assume the space->tab issue could be solved with a call to sed. Is that so or is that more complex? Also, in the README, it is clearly stated that the manifest lists packages, the current proposition would change this definition.

I guess the tab/space issue could be also solved on sbom side (i.e. accepting both). But we do need the snap list.

I agree the snaps list is currently missing and that would be nice to extract it. What do you think of generating another separate "snaps.manifest" file, listing snaps (and maybe using the tab separator?)? A bit a thinking is needed to define how that would be expressed in the image definition file but we can discuss it. With that, the post-process would also be trivial before calling sbom-generate.

Could we add another artifact entry and generates that manifest (tab + snap) if specified? Not sure about a name tho, as it is also a manifest ^^´

artifacts:
  img:
    - name: ubuntu-24.04-preinstalled-server-arm64+dragonwing.img
  manifest:
    name: ubuntu-24.04-preinstalled-server-arm64+dragonwing.manifest
  full-manifest:
    name: buntu-24.04-preinstalled-server-arm64+dragonwing-with-snap-and-tabs.manifest

@upils

upils commented Feb 25, 2026

Copy link
Copy Markdown
Collaborator

Could we add another artifact entry and generates that manifest (tab + snap) if specified?

This is another possibility but as you pointed out it might be tricky to find a good name (because the concept might be hard to express, which might indicate something is fishy). Side note: this is unlikely, but if we ever add another kind of object, the "full-manifest" name might be misleading or might see its definition change.

Another approach would be to add a new "manifest-v2" (name TBD here too) that would accept a list of "kinds" (package,snap, etc.). So users could express precisely what they want in the manifest. Drawbacks:

  • This might be giving too much freedom to users
  • There is no good default

@fhost

fhost commented Feb 26, 2026

Copy link
Copy Markdown
Collaborator Author

This is another possibility but as you pointed out it might be tricky to find a good name (because the concept might be hard to express, which might indicate something is fishy). Side note: this is unlikely, but if we ever add another kind of object, the "full-manifest" name might be misleading or might see its definition change.

The fishy part is that we have different tools that output different manfiests and we're trying to align them ^^´
And yes, "full-manifest" is not a good name. Maybe the "manifest-v2" is a good name. It's just another "format" for the manifest. And you're using "manifest" to get the legacy manifest or "manifest-v2" to get the "new" one.

Another approach would be to add a new "manifest-v2" (name TBD here too) that would accept a list of "kinds" (package,snap, etc.). So users could express precisely what they want in the manifest. Drawbacks:

* This might be giving too much freedom to users

* There is no good default

Yeah, the point is to have the same manifest format everywhere, so I don't think it is a good idea to give user customization.

@fhost fhost force-pushed the fhost/manifest branch 2 times, most recently from c726e9b to b937ee8 Compare May 28, 2026 16:32
@fhost

fhost commented May 28, 2026

Copy link
Copy Markdown
Collaborator Author

Added manifest-v2 (we can change the name later). This way it doesn't break anything.

And I don't understand what is failing from the tests, there is no FAIL: :(

@fhost fhost requested a review from upils May 28, 2026 16:39
@upils upils requested a review from Copilot May 29, 2026 13:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates classic image manifest generation to optionally produce a “v2” manifest that matches the livecd-rootfs format (tab-separated package entries and inclusion of snap seed entries).

Changes:

  • Add manifest-v2 as a new classic artifact and implement generateClassicManifestV2 to append snap seed info from seed.yaml.
  • Refactor classic manifest generation to use helper functions instead of inline chroot dpkg-query calls.
  • Extend tests/mocks to cover the v2 manifest output format.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
internal/statemachine/helper.go Adds classic manifest generator helpers, including v2 logic and snap seed parsing.
internal/statemachine/classic_states.go Updates generatePackageManifest() to generate v1 and/or v2 manifests based on artifacts.
internal/statemachine/classic.go Adds manifest-v2 artifact support when assembling classic artifact states.
internal/statemachine/classic_test.go Converts manifest testing into table-driven cases and adds v2 expectations (including snaps).
internal/statemachine/state_machine_test.go Extends the exec helper process to return tab-separated dpkg output for the v2 test case.
internal/imagedefinition/image_definition.go Adds ManifestV2 to the artifact schema (manifest-v2).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread internal/statemachine/helper.go Outdated
Comment thread internal/statemachine/helper.go Outdated
Comment thread internal/statemachine/classic.go Outdated
Comment thread internal/imagedefinition/image_definition.go Outdated
Comment thread internal/statemachine/classic_test.go
Comment thread internal/statemachine/classic_test.go Outdated
Comment thread internal/statemachine/state_machine_test.go
Comment thread internal/statemachine/helper.go Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Comment thread internal/statemachine/helper.go Outdated
Comment thread internal/imagedefinition/image_definition.go

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Comment thread internal/statemachine/helper.go
Comment thread internal/imagedefinition/README.rst
Comment thread internal/statemachine/classic_states.go
@fhost fhost changed the title i/statemachine: fix manifest generation for classic image i/statemachine: add manifest-v2 to align with livecd-rootfs format May 29, 2026
@fhost fhost force-pushed the fhost/manifest branch from 8dd99c7 to d56847f Compare May 29, 2026 15:24
@fhost

fhost commented May 29, 2026

Copy link
Copy Markdown
Collaborator Author

Fixed Copilot reviews, I still don't understand why short tests are failing?

@upils upils left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it LGTM, thanks @fhost!

See my comments for some suggestions. Can you also:

  • fix the linter errors
  • add a changelog entry
  • update the snapcraft.yaml conf

I was unable to reproduce the test failure locally. Let's see after pushing your fixes if this is still breaking.

Comment thread internal/statemachine/helper.go Outdated
Comment on lines +365 to +367
File string `yaml:"file"`
Name string `yaml:"name"`
Channel string `yaml:"channel"`

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: did you consider using https://github.com/canonical/snapd/blob/master/seed/seed.go#L50 instead of defining a new type here? This might make it easier to maintain over time.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No I did not, I'll take a look.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So my modification to use the Seed class is working, but that break the test as just having the seed file is probably not enough now.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how to test >.< I don't know what is required, but I guess at least the snaps in the seed file in the snaps/ directory. I'll test with snapd, but I don't think adding snapd into the repository for testing is a good idea... If the file is enough, maybe I should make an empty example snap?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I've just mocked the Seed directly!

@upils upils Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not expect such a big change when I suggested that, I thought unmarshaling into the Seed type from the snapd lib was going to be enough. It looks fine though and I guess that was the cleaner way to do it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was enough for running, not for testing ^^´

Comment thread internal/statemachine/helper.go Outdated
@fhost fhost force-pushed the fhost/manifest branch 2 times, most recently from 03fa94b to 7b88ce4 Compare June 1, 2026 23:17
@fhost fhost requested a review from Copilot June 12, 2026 15:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Comment thread debian/changelog Outdated
Comment thread internal/statemachine/helper.go
@fhost fhost requested a review from upils June 12, 2026 15:42
upils
upils previously approved these changes Jun 25, 2026

@upils upils left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I am waiting for the base on main and the rerun of the CI to see if unit tests pass.

fhost added 2 commits June 25, 2026 10:37
Make the classic image manifest follow the same format as the one from
livecd-rootfs.
Changes:
 * Use tab instead of space in the manifest
 * Add snaps to the manifest
@upils upils merged commit 27aa535 into main Jun 26, 2026
8 of 20 checks passed
@upils upils deleted the fhost/manifest branch June 26, 2026 06:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants