Figure out why automated ingestion script wants to reingest a tarball #169
Description
See https://github.com/EESSI/staging/pull/486. This tarball was already ingested, but for some reason it was picked up again much later. This means that the script somehow decided to mark this as a new tarball in find_state:
https://github.com/EESSI/filesystem-layer/blob/main/scripts/automated_ingestion/eessitarball.py#L67
Log entries:
2023-11-03 19:22:42,837 - INFO - 2023.06/software/linux/x86_64/amd/zen2/1687960841/eessi-2023.06-software-linux-x86_64-amd-zen2-1687960841.tar.gz has already been ingested, skipping...
2023-11-03 19:42:52,132 - INFO - 2023.06/software/linux/x86_64/amd/zen2/1687946108/eessi-2023.06-software-linux-x86_64-amd-zen2-1687946108.tar.gz has already been ingested, skipping...
2023-11-03 19:42:58,251 - INFO - Found new tarball 2023.06/software/linux/x86_64/amd/zen2/1687960841/eessi-2023.06-software-linux-x86_64-amd-zen2-1687960841.tar.gz, downloading it...
2023-11-03 19:43:00,724 - INFO - Adding tarball's metadata to the "staged" folder of the git repository.
2023-11-03 20:00:04,561 - INFO - Found new tarball 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz, downloading it...
2023-11-03 20:00:04,810 - ERROR - Failed to download tarball 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.
2023-11-03 20:00:04,943 - ERROR - Failed to download metadata file 2021.12/compat/linux/aarch64/1675183498/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.meta.txt from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-aarch64-1675183498.tar.gz.meta.txt.
2023-11-03 20:00:04,943 - WARNING - Skipping this tarball...
2023-11-03 20:00:07,284 - INFO - Found new tarball 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz, downloading it...
2023-11-03 20:00:07,375 - ERROR - Failed to download tarball 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.
2023-11-03 20:00:07,444 - ERROR - Failed to download metadata file 2021.12/compat/linux/ppc64le/1675198951/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.meta.txt from eessi-staging to /software/tarballs/eessi-2021.12-compat-linux-ppc64le-1675198951.tar.gz.meta.txt.
2023-11-03 20:00:07,444 - WARNING - Skipping this tarball...
So half an hour earlier things were still find, but then something happened (GitHub issue perhaps?) that caused failures in assigning the right states to tarballs.