Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ImageCustomizer: Implement new MIC Overlays APIs. #10138

Merged
merged 20 commits into from
Aug 27, 2024

Conversation

liulanze
Copy link
Contributor

@liulanze liulanze commented Aug 13, 2024

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

  • The toolchain has been rebuilt successfully (or no changes were made to it)
  • The toolchain/worker package manifests are up-to-date
  • Any updated packages successfully build (or no packages were changed)
  • Packages depending on static components modified in this PR (Golang, *-static subpackages, etc.) have had their Release tag incremented.
  • Package tests (%check section) have been verified with RUN_CHECK=y for existing SPEC files, or added to new SPEC files
  • All package sources are available
  • cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
  • LICENSE-MAP files are up-to-date (./LICENSES-AND-NOTICES/SPECS/data/licenses.json, ./LICENSES-AND-NOTICES/SPECS/LICENSES-MAP.md, ./LICENSES-AND-NOTICES/SPECS/LICENSE-EXCEPTIONS.PHOTON)
  • All source files have up-to-date hashes in the *.signatures.json files
  • sudo make go-tidy-all and sudo make go-test-coverage pass
  • Documentation has been updated to match any changes to the build system
  • Ready to merge

Summary

What does the PR accomplish, why was it needed?

This PR restructures the MIC Overlays APIs, optimizing the process by transitioning from relying on the initrd module script for enabling overlays to leveraging the fstab file. The changes streamline the Overlay feature, making it more generic and easier for users to configure and use.

Does this affect the toolchain?

NO

Associated issues
Test Methodology
  • Local build:
  • Used overlays field from the config file:
  overlays:
    - lowerDirs:
      - /etc
      upperDir: /var/overlays/etc/upper
      workDir: /var/overlays/etc/work
      mountPoint: /etc
      isRootfsOverlay: true
      mountDependencies: 
      - /var
    - lowerDirs:
      - /media
      - /home
      upperDir: /overlays/media/upper
      workDir: /overlays/media/work
      mountPoint: /media
  • Print of df -h:
root [ /home/test ]# df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        4.0M     0  4.0M   0% /dev
tmpfs           851M     0  851M   0% /dev/shm
tmpfs           341M  620K  340M   1% /run
tmpfs           4.0M     0  4.0M   0% /sys/fs/cgroup
/dev/sda3       1.9G  380M  1.4G  22% /
/dev/sda4       2.0G  193M  1.7G  11% /var
overlay         2.0G  193M  1.7G  11% /etc
tmpfs           851M     0  851M   0% /tmp
overlay         1.9G  380M  1.4G  22% /media
/dev/sda2        89M   32M   51M  38% /boot
/dev/sda1       8.0M  2.3M  5.7M  29% /boot/efi
tmpfs           171M     0  171M   0% /run/user/1000
  • Print of fstab:
root [ /home/test ]# cat /etc/fstab
PARTUUID=a48afb5e-98d8-4910-ab67-ca790c6d8abc / ext4 defaults 0 1
PARTUUID=68e8263e-d932-4e98-87c1-f8e3e929a8e1 /boot ext4 defaults 0 2
PARTUUID=6282d628-3af5-4981-bd41-49f6057ab664 /boot/efi vfat umask=0077 0 2
PARTUUID=53b61c5f-4ebb-4ee7-847e-feb46ef5ffb9 /var ext4 defaults,x-initrd.mount 0 2
overlay /etc overlay lowerdir=/sysroot/etc,upperdir=/sysroot/var/overlays/etc/upper,workdir=/sysroot/var/overlays/etc/work,x-systemd.requires=/sysroot/var,x-initrd.mount,x-systemd.wanted-by=initrd-fs.target 0 0
overlay /media overlay lowerdir=/media:/home,upperdir=/overlays/media/upper,workdir=/overlays/media/work 0 0

@liulanze liulanze requested a review from a team as a code owner August 13, 2024 23:57
@liulanze liulanze self-assigned this Aug 13, 2024
@liulanze liulanze requested a review from cwize1 August 13, 2024 23:58
toolkit/tools/imagecustomizerapi/overlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
@liulanze liulanze force-pushed the user/lanzeliu/mic-overlay-new-apis branch from 8953e4a to f395218 Compare August 16, 2024 22:09
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizer/docs/configuration.md Outdated Show resolved Hide resolved
toolkit/tools/imagecustomizerapi/overlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeos.go Outdated Show resolved Hide resolved
toolkit/tools/pkg/imagecustomizerlib/customizeoverlay.go Outdated Show resolved Hide resolved
@liulanze liulanze requested a review from cwize1 August 20, 2024 22:51
@liulanze liulanze requested review from jiria and cwize1 August 26, 2024 16:48
if strings.Contains(path, " ") {
return fmt.Errorf("path (%s) contains spaces and is invalid", path)
// Check if the path contains spaces.
if strings.Contains(filePath, " ") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would probably be good to also exclude tabs, newlines, colons, and commas to help avoid the path values from messing up the fstab file's formatting.

strings.ContainsAny could be handy for this.

Also, if you could also use validatePath for MountPoint.Path that would be greatly appreciated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, please check.

}
}

if strings.Contains(o.MountOptions, " ") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would probably be good to also exclude tabs and newlines.

Also, if you could apply the same check to MountPoint.Options that would be greatly appreciated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added, please check.

@liulanze liulanze force-pushed the user/lanzeliu/mic-overlay-new-apis branch from 024c5ec to 8c58876 Compare August 27, 2024 22:00
@liulanze liulanze merged commit d47dc39 into 3.0-dev Aug 27, 2024
12 checks passed
@liulanze liulanze deleted the user/lanzeliu/mic-overlay-new-apis branch August 27, 2024 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants