Ingestion of updated compat layer tarballs changed permissions on many directories

[bedroge]

After the ingestion of the tarballs from PR EESSI/compatibility-layer#164, many directories suddenly had very restrictive permissions, causing lots of errors on clients (the directories are only readable/accessible for the cvmfs user, not for the actual user using the EESSI stack):

$ ls -l /cvmfs/pilot.eessi-hpc.org/versions/2021.12/compat/linux/x86_64/usr
total 81
drwxrwxr-x  2 cvmfs cvmfs 20480 Jan 31 17:46 bin
drwx------ 87 cvmfs cvmfs 12288 Feb  1 08:17 include
drwxr-xr-x 14 cvmfs cvmfs  4096 Jan 31 17:40 lib
drwxr-xr-x 27 cvmfs cvmfs 20480 Jan 31 17:46 lib64
drwxr-xr-x  8 cvmfs cvmfs  4096 Dec  2  2021 libexec
drwxr-xr-x  4 cvmfs cvmfs  4096 Dec  1  2021 local
drwx------  2 cvmfs cvmfs  4096 Feb  1 08:17 sbin
drwxrwxr-x 65 cvmfs cvmfs  4096 Jan 31 17:42 share
drwx------  6 cvmfs cvmfs  4096 Feb  1 08:17 x86_64-pc-linux-gnu

[EESSI pilot 2021.12] $ vi
Error detected while processing /cvmfs/pilot.eessi-hpc.org/versions/2021.12/compat/linux/x86_64/etc/vim/vimrc:
line   92:
E484: Can't open file /cvmfs/pilot.eessi-hpc.org/versions/2021.12/compat/linux/x86_64/usr/share/vim/syntax/syntax.vim
E1187: Failed to source defaults.vim
Press ENTER or type command to continue

etc...

The directories in the tarball do have the right permissions, though, so something went wrong on the ingestion.

[bedroge]

Ingesting it to a new and empty repo does seem to work fine, so it looks like there is some weird permission issue with our pilot repo.

[bedroge]

Extracting the tarball with --no-same-owner --no-same-permissions (which is the default for regular users, but not for root) seems to solve the issue. Probably related to which user id was used for creating the files that are included in the tarball?

[bedroge]

Only --no-same-owner seems to be sufficient. I'll make a PR to update the ingestion script.

[bedroge]

My conclusion: only for compat layer updates we use a manual tar operation, in order to remove the older layer and extract the new one in a single transaction. Doing this as root will by default use --same-owner, which means that the user ids from the build host are set as owner of the files:

       --same-owner
              Try  extracting  files  with the same ownership as exists in the
              archive (default for superuser).
       --no-same-owner
              Extract files as yourself (default for ordinary users).

This made some sort of mess of the ownerships of several (sub)directories. I've fixed this using cvmfs_server catalog-chown command.

[bedroge]

While doing the actual fixes, it turns out that we actually do need both options to fix this properly, i.e. --no-same-owner --no-same-permissions.