podman quadlet install -r may cause broken generated systemd units

[simonbrauner]

Issue Description

When the quadlet generator fails to generate the unit successfully, it may leave corrupted units. The following problem I reported was caused by two generators running at the same time, which is not very likely to happen. Nevertheless, the proposed fix would make it less likely that there are corrupted units in general, for example, if the generator crashes mid run.

Running podman quadlet install -r a couple of times may produce broken systemd unit.

Steps to reproduce the issue

Example quadlet:

[Container]
Image=docker.io/library/postgres:17-alpine
ContainerName=pg17
PublishPort=54317:5432
Environment=POSTGRES_PASSWORD=postgres
[Service]
Restart=always
[Install]
WantedBy=default.target

Run the podman quadlet install -r command a couple of times and it will show this log

Jun 22 13:41:09 sbrauner-thinkpadt14gen6.tpb.csb systemd[3531]: Reloading finished in 160 ms.
Jun 22 13:41:09 sbrauner-thinkpadt14gen6.tpb.csb systemd[3531]: /run/user/4242740/systemd/generator/A.service:10: Invalid section header '[Service][Restart=aRestarEnvironmeEnvironment=PODMAN_SYSTEMDKillMode=KillMoExecStop=ExecStop=/usr/bin/podman rm -v -fExecStopPExecStopPost=-/usr/bin/podman rm -v -fDelegate=DeleType=notiTypNotifyAccNotifyAcSyslogIdeSyslogIdentExecStartExecStart=/usr/bin/podman run --name pg17 --replace --rm --cgroups=split --sdnotify=conmon -d --publish 54317:5432 --env POSTGRES_PASSWORD=postgres docker.io/library/postgres:1'
Jun 22 13:41:09 sbrauner-thinkpadt14gen6.tpb.csb systemd[50670]: quadlet-generator[50670]: Failed creating symlink /run/user/4242740/systemd/generator/default.target.wants/A.service: symlink ../A.service /run/user/4242740/systemd/generator/default.target.wants/A.service: file exists
Jun 22 13:41:08 sbrauner-thinkpadt14gen6.tpb.csb systemd[3531]: Reloading...
Jun 22 13:41:08 sbrauner-thinkpadt14gen6.tpb.csb systemd[3531]: Reload requested from client PID 50649 ('podman') (unit ptyxis-spawn-4148ae39-f8db-424a-a435-119e2e6a24d1.scope)...

And the output file will be corrupted

# Automatically generated by /usr/lib/systemd/user-generators/podman-user-generator
# 
[X-Container]
Image=docker.io/library/postgres:17-alpine
ContainerName=pg17
PublishPort=54317:5432
Environment=POSTGRES_PASSWORD=postgres

[Service][Restart=aRestarEnvironmeEnvironment=PODMAN_SYSTEMDKillMode=KillMoExecStop=ExecStop=/usr/bin/podman rm -v -fExecStopPExecStopPost=-/usr/bin/podman rm -v -fDelegate=DeleType=notiTypNotifyAccNotifyAcSyslogIdeSyslogIdentExecStartExecStart=/usr/bin/podman run --name pg17 --replace --rm --cgroups=split --sdnotify=conmon -d --publish 54317:5432 --env POSTGRES_PASSWORD=postgres docker.io/library/postgres:1
[Install][WantedBy=WantedBy=defaul
[Unit]
Wants=podWants=podman-user-wait-network-onlineAfter=podAfter=podman-user-wait-network-onlineSourcePatSourcePath=/home/sbrauner/.config/containers/systemd/A.cRequiresMRequiresMountsFor=%t/containers

Describe the results you received

$ cat /run/user/4242740/systemd/generator/default.target.wants/A.service
# Automatically generated by /usr/lib/systemd/user-generators/podman-user-generator
# 
[X-Container]
Image=docker.io/library/postgres:17-alpine
ContainerName=pg17
PublishPort=54317:5432
Environment=POSTGRES_PASSWORD=postgres

[Service][Restart=aRestarEnvironmeEnvironment=PODMAN_SYSTEMDKillMode=KillMoExecStop=ExecStop=/usr/bin/podman rm -v -fExecStopPExecStopPost=-/usr/bin/podman rm -v -fDelegate=DeleType=notiTypNotifyAccNotifyAcSyslogIdeSyslogIdentExecStartExecStart=/usr/bin/podman run --name pg17 --replace --rm --cgroups=split --sdnotify=conmon -d --publish 54317:5432 --env POSTGRES_PASSWORD=postgres docker.io/library/postgres:1
[Install][WantedBy=WantedBy=defaul
[Unit]
Wants=podWants=podman-user-wait-network-onlineAfter=podAfter=podman-user-wait-network-onlineSourcePatSourcePath=/home/sbrauner/.config/containers/systemd/A.cRequiresMRequiresMountsFor=%t/containers

Describe the results you expected

$ cat /run/user/4242740/systemd/generator/default.target.wants/A.service
# Automatically generated by /usr/lib/systemd/user-generators/podman-user-generator-original
# 
[X-Container]
Image=docker.io/library/postgres:17-alpine
ContainerName=pg17
PublishPort=54317:5432
Environment=POSTGRES_PASSWORD=postgres

[Service]
Restart=always
Environment=PODMAN_SYSTEMD_UNIT=%n
KillMode=mixed
ExecStop=/usr/bin/podman rm -v -f -i pg17
ExecStopPost=-/usr/bin/podman rm -v -f -i pg17
Delegate=yes
Type=notify
NotifyAccess=all
SyslogIdentifier=%N
ExecStart=/usr/bin/podman run --name pg17 --replace --rm --cgroups=split --sdnotify=conmon -d --publish 54317:5432 --env POSTGRES_PASSWORD=postgres docker.io/library/postgres:17-alpine

[Install]
WantedBy=default.target

[Unit]
Wants=podman-user-wait-network-online.service
After=podman-user-wait-network-online.service
SourcePath=/home/sbrauner/.config/containers/systemd/A.container
RequiresMountsFor=%t/containers

podman info output

host:
  arch: amd64
  buildahVersion: 1.44.0
  cdiSpecDirs:
  - /etc/cdi
  - /var/run/cdi
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.2.1-2.fc44.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.2.1, commit: '
  cpuUtilization:
    idlePercent: 94.05
    systemPercent: 1.34
    userPercent: 4.62
  cpus: 14
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: workstation
    version: "44"
  emulatedArchitectures:
  - linux/arm
  - linux/arm64
  - linux/arm64be
  - linux/loong64
  - linux/mips
  - linux/mips64
  - linux/ppc
  - linux/ppc64
  - linux/ppc64le
  - linux/riscv32
  - linux/riscv64
  - linux/s390x
  eventLogger: journald
  freeLocks: 2046
  hostname: sbrauner-thinkpadt14gen6.tpb.csb
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 4242740
      size: 1
    - container_id: 1
      host_id: 165536
      size: 165536
    uidmap:
    - container_id: 0
      host_id: 4242740
      size: 1
    - container_id: 1
      host_id: 165536
      size: 165536
  kernel: 7.0.12-201.fc44.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 9844387840
  memTotal: 33030959104
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    defaultNetwork: podman
    dns:
      package: aardvark-dns-2.0.0-1.20260622092550897134.main.5.gda6856f.fc44.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 2.0.0
    package: netavark-2.0.0-1.20260619111836814958.main.5.g8a649cd1.fc44.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 2.0.0
  ociRuntime:
    name: crun
    package: crun-1.28-1.20260617092403306714.main.6.gfe55c271.fc44.x86_64
    path: /usr/bin/crun
    version: |-
      crun version UNKNOWN
      commit: 2e9b9b5fa00dc90b9d80ce503b954f49f0f48b0a
      rundir: /run/user/4242740/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +JSON_C
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20260611.ga9c61ff-1.fc44.x86_64
    version: |
      pasta 0^20260611.ga9c61ff-1.fc44.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/4242740/podman/podman.sock
  rootlessNetworkCmd: pasta
  rootlessPortForwarder: rootlessport
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 1h 44m 51.00s (Approximately 0.04 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
store:
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /home/sbrauner/.local/share/containers/storage
  graphRootAllocated: 510389456896
  graphRootUsed: 28615192576
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 7
  runRoot: /run/user/4242740/containers
  transientStore: false
  volumePath: /home/sbrauner/.local/share/containers/storage/volumes
version:
  APIVersion: 6.1.0-dev
  BuildOrigin: 'Copr: rhcontainerbot/podman-next'
  Built: 1782086400
  BuiltTime: Mon Jun 22 02:00:00 2026
  GitCommit: c271cbdd1f92d4510810373cdc8a9cad61dc6d99
  GoVersion: go1.26.4-X:nodwarf5
  Os: linux
  OsArch: linux/amd64
  Version: 6.1.0-dev

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

Additional environment details

Additional information

Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting

[simonbrauner]

/assign

[simonbrauner]

For me, it turns out that I had two systemd generators running at the same time by mistake, so that explained why there were two writers to a file at the same time. Awaiting a discussion with another person with a similar bug.

But I suppose, in general, this could be solved by writing the unit to a temporary file and then renaming atomically, if there is a reasonable chance that the problem can occur even with one generator.

[Luap99]

@simonbrauner using a rename approach sounds good, if a single generator crashes in the middle of the write we also have a non complete file which seems bad.

[inknos]

+1 with moving on to this, just fyi, the related issue about install -r: #29013