improve ingestion workflow

[boegel]

The current workflow triggers a lot of requests to GitHub, and does a lot of lookup operations in S3.

We could:

• restructure the S3 bucket so it's clear which tarballs have been ingested already:
  • by adding ingested and new folder below EESSI version for metadata files + a tarballs directory that has all the tarballs;
  • mainly because moving the tarballs seems to be quite time-consuming;
• clean up tarballs in S3 bucket more actively (or move them to S3 Glacier?);
• check out the PR branch to the staging repo to limit requests to the GitHub API;

[bedroge]

Just checked the current settings of this bucket, and currently we have the following:

Current version actions

Day 0
Objects uploaded

Day 30
Objects move to Standard-IA

Day 60
Objects move to Glacier Flexible Retrieval (formerly Glacier)

Day 180
Objects move to Glacier Deep Archive

Day 365
Objects expire