composefs-finalize-staged fails when switching from image where directory in /etc is a symlink in staged image

[renner0e]

When I bootc switch from
ghcr.io/renner0e/aurora-composefs@sha256:8fee93a376b4aaa4bdfaf04788820772474a8f90f85804471502c9778b7e0154
to
ghcr.io/zirconium-dev/zirconium-hawaii@sha256:6df3ff8ea4057dea6bdde6ef91689b66a60af4086877e26860301b060a539819

the new deployment is staged but the system won't actually boot into the new deployment, when I troubleshooted this with bootc composefs-finalize-staged I get this error:

error: Computing diff: Getting modifications: Getting modifications: Directory entry "certs" is not a subdirectory

aurora-composefs: /etc/ssl/certs is a directory
zirconium-hawaii: /etc/ssl/certs is a symlink -> /etc/pki/ca-trust/extracted/pem/directory-hash

this is with bootc 1.15.2

[Johan-Liebert1]

Seems like an edge case in the etc-merge module

[Johan-Liebert1]

Had a go at this and the problem is pretty apparent but a bit complicated and we need to set up a standard on how it's done. For example, the easy case if directory was converted to a symlink and the original directory was not modified at all. In this case we can simply disregard the change.

But, if some modifications were made in the original directory, then we probably have to throw an error while merging as there's no way to merge without losing data. I believe this is what ostree does. We could also give an option to the user on what to do, but this'll probably need manual intervention.

[cgwalters]

One thing that would help the UX is to "eagerly" detect this situation at deploy time and error out then instead of only at finalization.

But, if some modifications were made in the original directory, then we probably have to throw an error while merging as there's no way to merge without losing data.

I think actually in most cases we could just copy over any added files to the new symlink directory target, right?

The inverse case gets more complex (replacing a directory with a symlink).

But yeah, in this specific case it would really help to allow easily finding and fixing it at pull time.

We may also need a clear and easy way to ignore diffs to /etc for specific subpaths.

[Johan-Liebert1]

I think actually in most cases we could just copy over any added files to the new symlink directory target, right?

we can't do this if the target isn't in etc. Usually won't happen, but definitely could

One thing that would help the UX is to "eagerly" detect this situation at deploy time and error out then instead of only at finalization.

yes, but it's a failure regardless. Wonder if we should have an .etc-ignore from which we pick up diffs to ignore