Weird behaviour on Arch Linux 32: Asking for overwriting files with no changes

[VorpalBlade]

General description of the problem:

On ArchLinux32 aconfmgr will ask about overwriting and installing unchanged files for some reason.

For example:

: Applying configuration...
:: Installing priority files...
::: Installing /etc/group...
:::: Proceed? [Y/n/d] d
Files /etc/group and /tmp/aconfmgr-arvid/output/files//etc/group are identical

And so on:

::: Installing 44 new files.
:::: Proceed? [Y/n/d] d
:::: Installing the following new files:
* /etc/apparmor
* /etc/apparmor.d/local/usr.sbin.avahi-daemon
* /etc/apparmor.d/local/usr.sbin.dnsmasq
* /etc/apparmor/parser.conf
[...]

Again, these were just installed, so there should be no change.

If you have any suggestions where I can start to look into this I'd appreciate it! I realise debugging on Arch32 is probably not convenient for you, but I might need some hints about where to start digging at this.

Steps to reproduce the problem:

1. Set up ArchLinux32 on an old computer
2. Try to manage it with aconfmgr
3. ????

I'm using xfs instead of btrfs for the root file system on this computer, not sure if that is relevant.

Configuration:

# I have yet to figure out what causes it, but it seems premature to post my entire config, as that works perfectly on x86_64.

Expected result:

Normal behaviour, where it knows what an empty diff is.

Actual result:

Lots of "fake" diffs reported that aren't actually different.

Log:

: Collecting data...
:: Compiling user configuration...
::: Using configuration in .
::: Sourcing ./02-properties.sh...
::: Sourcing ./05-augtool-helpers.sh...
::: Sourcing ./05-ignore-helpers.sh...
::: Sourcing ./05-nm-network-helpers.sh...
::: Sourcing ./05-package-helpers.sh...
::: Sourcing ./05-systemd-helpers.sh...
::: Sourcing ./10-10-user-helpers.sh...
::: Sourcing ./10-ignores.sh...
::: Sourcing ./14-boot.sh...
::: Sourcing ./14-grub.sh...
[sudo] password for arvid: 
::: Sourcing ./15-base-network.sh...
::: Sourcing ./15-base-networkmanager.sh...
::: Sourcing ./15-base.sh...
::: Sourcing ./15-mkinitcpio.sh...
::: Sourcing ./15-pkgmgmt.sh...
::: Sourcing ./15-systemd.sh...
::: Sourcing ./16-bluetooth.sh...
::: Sourcing ./16-cups.sh...
::: Sourcing ./16-hardware.sh...
::: Sourcing ./17-fonts.sh...
::: Sourcing ./17-gui.sh...
::: Sourcing ./17-tools.sh...
::: Sourcing ./18-media.sh...
::: Sourcing ./19-archiving.sh...
::: Sourcing ./19-cadcam.sh...
::: Sourcing ./19-dev.sh...
::: Sourcing ./19-emulation.sh...
::: Sourcing ./19-games.sh...
::: Sourcing ./19-office.sh...
::: Sourcing ./19-online.sh...
::: Sourcing ./19-osm.sh...
::: Sourcing ./19-virt.sh...
::: Sourcing ./20-apparmor.sh...
::: Sourcing ./20-hardening.sh...
::: Sourcing ./30-sysconf.sh...
::: Sourcing ./31-networks.sh...
::: Sourcing ./31-system-specific.sh...
::: Sourcing ./32-work.sh...
::: Sourcing ./90-system-patches.sh...
::: Sourcing ./98-10-apply-users.sh...
::: Sourcing ./98-20-augtool-commit.sh...
::: Sourcing ./98-99-partial.sh...
::: Sourcing ./99-zz_wait.sh...
::: Done (249 native packages, 0 foreign packages, 170 files).
:: Inspecting system state...
::: Querying package list...
:::: Done.
::: Enumerating owned files...
:::: Done.
::: Searching for stray files...
:::: Done (164 stray files).
::: Cleaning up ignored files' directories...
:::: Done.
::: Searching for modified files...
:::: Done (0 modified files).
::: Reading file attributes...
:::: Reading file types...
:::: Reading file sizes...
:::: Reading file modes...
:::: Reading file owners...
:::: Reading file groups...
:::: Done.
::: Processing found files...
:::: Done.
::: Done (249 native packages, 0 foreign packages, 136 files).
:: Examining files...
::: Loading data...
:::: Done.
::: Comparing file data...
::: Only in config: /etc/apparmor
::: Only in config: /etc/apparmor.d/local/usr.sbin.avahi-daemon
::: Only in config: /etc/apparmor.d/local/usr.sbin.dnsmasq
::: Only in config: /etc/apparmor/parser.conf
::: Only in config: /etc/bluetooth
::: Only in config: /etc/bluetooth/main.conf
::: Only in config: /etc/chrony.conf
::: Only in config: /etc/conf.d
::: Only in config: /etc/conf.d/distccd
::: Only in config: /etc/conf.d/wireless-regdom
::: Only in config: /etc/default
::: Only in config: /etc/default/grub
::: Only in config: /etc/fstab
::: Only in config: /etc/fuse.conf
::: Only in config: /etc/group
::: Only in config: /etc/gshadow
::: Only in config: /etc/hosts
::: Only in config: /etc/locale.gen
::: Only in config: /etc/makepkg.conf
::: Only in config: /etc/mkinitcpio.conf
::: Only in config: /etc/nanorc
::: Only in config: /etc/nsswitch.conf
::: Only in config: /etc/pacman.conf
::: Only in config: /etc/pacman.d/mirrorlist
::: Only in config: /etc/papersize
::: Only in config: /etc/passwd
::: Only in config: /etc/security
::: Only in config: /etc/security/faillock.conf
::: Only in config: /etc/shadow
::: Only in config: /etc/shells
::: Only in config: /etc/ssh
::: Only in config: /etc/ssh/sshd_config
::: Only in config: /etc/sudoers
::: Only in config: /etc/systemd/journald.conf
::: Only in config: /etc/systemd/logind.conf
::: Only in config: /etc/updatedb.conf
::: Only in config: /etc/xdg
::: Only in config: /etc/xdg/autostart
::: Only in config: /etc/xdg/autostart/xfce4-power-manager.desktop
::: Only in config: /usr/share
::: Only in config: /usr/share/libalpm
::: Only in config: /usr/share/libalpm/hooks
::: Only in config: /usr/share/libalpm/hooks/05-etckeeper-pre-install.hook
::: Only in config: /usr/share/libalpm/hooks/zz-etckeeper-post-install.hook
::: Only in config: /usr/share/zsh
::: Only in config: /usr/share/zsh/functions
::: Only in config: /usr/share/zsh/functions/Completion
::: Only in config: /usr/share/zsh/functions/Completion/Linux
::: Only in config: /usr/share/zsh/functions/Completion/Linux/_modutils
::: Done (0 only in system, 0 changed, 49 only in config).
:: Examining file properties...
::: Loading data...
:::: Done.
::: Comparing file properties...
:::: Done.
::: Done (0 only in system, 0 changed, 0 only in config).
:: Done.
: Applying configuration...
:: Installing priority files...
::: Installing /etc/group...
:::: Proceed? [Y/n/d] d
Files /etc/group and /tmp/aconfmgr-arvid/output/files//etc/group are identical
:::: Proceed? [Y/n/d] 

:::: Done.
::: Installing /etc/makepkg.conf...
:::: Proceed? [Y/n/d] 

:::: Done.
::: Installing /etc/pacman.conf...
:::: Proceed? [Y/n/d] 

:::: Done.
::: Installing /etc/pacman.d/mirrorlist...
:::: Proceed? [Y/n/d] 

:::: Done.
::: Installing /etc/passwd...
:::: Proceed? [Y/n/d] 

:::: Done.
::: Done.
:: Configuring packages...
::: Done.
:: Configuring files...
::: Installing 44 new files.
:::: Proceed? [Y/n/d] d
:::: Installing the following new files:
* /etc/apparmor
* /etc/apparmor.d/local/usr.sbin.avahi-daemon
* /etc/apparmor.d/local/usr.sbin.dnsmasq
* /etc/apparmor/parser.conf
* /etc/bluetooth
* /etc/bluetooth/main.conf
* /etc/chrony.conf
* /etc/conf.d
* /etc/conf.d/distccd
* /etc/conf.d/wireless-regdom
* /etc/default
* /etc/default/grub
* /etc/fstab
* /etc/fuse.conf
* /etc/gshadow
* /etc/hosts
* /etc/locale.gen
* /etc/mkinitcpio.conf
* /etc/nanorc
* /etc/nsswitch.conf
* /etc/papersize
* /etc/security
* /etc/security/faillock.conf
* /etc/shadow
* /etc/shells
* /etc/ssh
* /etc/ssh/sshd_config
* /etc/sudoers
* /etc/systemd/journald.conf
* /etc/systemd/logind.conf
* /etc/updatedb.conf
* /etc/xdg
* /etc/xdg/autostart
* /etc/xdg/autostart/xfce4-power-manager.desktop
* /usr/share
* /usr/share/libalpm
* /usr/share/libalpm/hooks
* /usr/share/libalpm/hooks/05-etckeeper-pre-install.hook
* /usr/share/libalpm/hooks/zz-etckeeper-post-install.hook
* /usr/share/zsh
* /usr/share/zsh/functions
* /usr/share/zsh/functions/Completion
* /usr/share/zsh/functions/Completion/Linux
* /usr/share/zsh/functions/Completion/Linux/_modutils
:::: Proceed? [Y/n/d] n
:::: User abort

Additional context:

If you have any suggestions where I can start to look into this I'd appreciate it!

[CyberShadow]

Can you post the part of your configuration responsible for creating /etc/group (or some other file affected by this)?

I think there might be a quirk where if you specify the default mode ($default_file_mode, by default 644) explicitly instead of leaving it blank on file creation, the file will appear as modified to aconfmgr.

[VorpalBlade]

Sure, though this does not affect Arch64, and I don't specify the mode for most of the affected files. /etc/group is messy (built up with function, sorted at the end, etc). So lets do /etc/pacman.conf which is also affected by this:

I have a system with "feature-variables" that are either auto detected or from a case statement on the hostname. All of these are done in 02-properties.sh, and are named CUST_*, you will see them used below

IGNORED_PKGS=()

case $HOSTNAME in
	minerva)
		# New version breaks pyswip bindings, temporary workaround until upstream fixes this.
		IGNORED_PKGS+=(swi-prolog)
		;;
	*)
		;;
esac

f="$(GetPackageOriginalFile pacman /etc/pacman.conf)"
sed -i \
	-e '/^#Color/s/^#//' \
	-e '/^#ParallelDownloads/s/^#//' \
	"$f"

if (( ${#IGNORED_PKGS[@]} > 0 )); then
	oIFS="$IFS"
	IFS=$' \t\n'
	sed -i \
		-e '/^#IgnorePkg/s/^#//' \
		-e "s/IgnorePkg *=/IgnorePkg = ${IGNORED_PKGS[*]}/" \
		"$f"
	IFS="$oIFS"
fi

if [[ $CUST_USE_MULTILIB == 1 ]]; then
	sed -i \
		-e '/\[multilib\]/ { s/^#// ; n ; s/^#// }' \
		"$f"
fi

# Helper to add a file based repo to pacman.conf
# $1 Repo name
_add_repo() {
	local repo="$1"
	cat >> "$f" <<-EOF

		[${repo}]
		SigLevel = Optional TrustAll
		Server = file:///home/custompkgs/${repo}
	EOF
	CreateLink /etc/aurutils/pacman-${repo}.conf pacman-${CUST_PLATFORM}.conf
}

if [[ $CUST_PLATFORM == i686 ]]; then
	_add_repo custom-32
fi
if [[ $CUST_TASK_GAMES -gt 1 ]]; then
	_add_repo custom-games
fi
if [[ $HOSTNAME == athena ]]; then
	_add_repo custom-t480
fi
if [[ $HOSTNAME == dragon ]]; then
	_add_repo custom-r500
fi
if [[ $CUST_CPU == amd ]]; then
	_add_repo custom-zen
fi
# HACK: Arch32 hack to disable this repo, need to find a better way
if [[ $CUST_LAPTOP == 1 && $CUST_PLATFORM == x86_64 ]]; then
	_add_repo custom-laptop
fi
if [[ $CUST_IS_WORK == 1 ]]; then
	_add_repo custom-work
elif [[ $CUST_PLATFORM == x86_64 ]]; then
	# HACK: Another Arch32 hack to disable this repo, fix this too
	_add_repo custom-personal
fi
if [[ $CUST_TASK_ARCHIVING == 1 || $CUST_TASK_EMULATION == 1 ]]; then
	_add_repo custom-archiving
fi
if [[ $CUST_TASK_3D_SCAN == 1 || $CUST_TASK_CAD == 1 || $CUST_TASK_CAM == 1 ]]; then
	_add_repo custom-cadcam
fi
if [[ $CUST_TASK_VIDEO_EDITING == 1 ]]; then
	_add_repo custom-videoediting
fi
if [[ $CUST_TASK_OFFICE == 1 ]]; then
	_add_repo custom-office
fi
_add_repo custom-compat
if [[ $CUST_PLATFORM == x86_64 ]]; then
	# HACK: Arch32 strikes again
	_add_repo custom
fi

[CyberShadow]

Thanks! I wanted to see this line:

f="$(GetPackageOriginalFile pacman /etc/pacman.conf)"

So it's not the problem I was thinking about in my last post.

(built up with function, sorted at the end, etc)

Nice, you are doing it right 🙂👍

::: Only in config: /etc/pacman.conf

This seems to indicate that aconfmgr is not seeing the files being modified on the filesystem. Try running the command sudo paccheck --md5sum --files --file-properties --backup --noupgrade pacman on this system. The output should look something like this:

pacman: all files match database
pacman: '/etc/makepkg.conf' modification time mismatch (expected 2022-11-01 12:44:59)
pacman: '/etc/makepkg.conf' size mismatch (expected 6.10 K)
pacman: '/etc/pacman.conf' modification time mismatch (expected 2022-11-01 12:44:59)
pacman: '/etc/pacman.conf' size mismatch (expected 2.84 K)
pacman: '/etc/makepkg.conf' md5sum mismatch (expected 79fce043df7dfc676ae5ecb903762d8b)
pacman: '/etc/pacman.conf' md5sum mismatch (expected de541390e52468165b96511c4665bff4)

If the output looks good, it's possible that there's a problem with the loop aconfmgr uses to read the paccheck output, which is a little non-trivial because aconfmgr wants to show the progress.

[VorpalBlade]

$ paccheck --md5sum --files --file-properties --backup --noupgrade pacman
zsh: segmentation fault (core dumped)  paccheck --md5sum --files --file-properties --backup --noupgrade pacman

Well! That explains it, though it is unexpected. Checking with gdb it crashes somewhere in strnlen. Unfortunately upstream for pacutils is pretty dead in my experience. I'll look into this.

[VorpalBlade]

Arch32 bug report: https://bugs.archlinux32.org/index.php?do=details&task_id=317&string=pacutils
pacutils bug: andrewgregory/pacutils#64

[CyberShadow]

Aah, that's unfortunate. Is it this bug? andrewgregory/pacutils#50

[VorpalBlade]

Hm! That seems very similar indeed. Probably the same yes, since andrewgregory/pacutils#51 also adds -D_FILE_OFFSET_BITS=64

[CyberShadow]

I would be curious how other packages which depend / depended on pacutils deal with upstream abandonment, such as aurutils. Perhaps a community-maintained fork is in order? Perhaps one already exists?

[VorpalBlade]

Good question! pacutils does seem to work for now on 64-bit though, so perhaps few people have noticed yet? Not that many use 32-bit any more (and I only set that up because I needed something with an actual serial port, USB->serial wasn't good enough, and a 32-bit laptop was the only such thing I had).

[VorpalBlade]

Arch32 has a downstream fix now. I don't know about Arch Linux on ARM, though that may be 64-bit only? Feel free to close the bug here though.