Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add package caching #6

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Add package caching #6

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1d43a6b
Add package caching
me-and Jan 9, 2023
8ad2694
Allow save-only/restore-only cache runs
me-and Jan 10, 2023
0066f67
Add much more comprehensive cache testing
me-and Jan 10, 2023
76f43ba
Add cache performance testing
me-and Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,3 +228,34 @@ jobs:
shell: bash
env:
SHELLOPTS: igncr

caching:
runs-on: windows-latest

name: 'Test use of the cache'

steps:
- run: git config --global core.autocrlf input
- uses: actions/checkout@v2
- name: Install Cygwin and bash-completion
uses: ./
with:
packages: bash-completion
package-cache-key: testing-cache
add-to-path: false
- name: Delete Cygwin installation and cache
run: |
Remove-Item -Force -Recurse C:\cygwin
Remove-Item -Force -Recurse C:\cygwin-packages
- name: Reinstall Cygwin only
uses: ./
with:
package-cache-key: testing-cache
add-to-path: false
- name: Find bash-completion in the package cache
shell: C:\cygwin\bin\bash.exe --noprofile --norc -e -o pipefail -o igncr {0}
run: |
for file in /cygdrive/c/cygwin-packages/*/*/*/bash-completion/bash-completion-*.tar.*; do
[[ -f "$file" ]] && exit 0 # File actually exists
done
exit 1 # No such downloaded file
17 changes: 17 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Parameters
| site | http://mirrors.kernel.org/sourceware/cygwin/ | Mirror site to install from
| check-sig | true | Whether to check the setup.ini signature
| add-to-path | true | Whether to add Cygwin's `/bin` directory to the system `PATH`
| package-cache-key | '' | The key to use for caching downloaded packages.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reasoning behind making this a string, rather than a boolean? I think that the cache-keys are scoped to the repository, so the only possible collision is with other cache action uses in the same workflow?

It would be nice to be able to make this on by default in future, without the user having to choose a key.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The store of caches is per-repository, and the actual restore behaviour is scoped even more tightly than that, with restrictions about what caches are allowed to be restored on what branch. But yes, this is something I debated; I went with letting the user decide mostly so they wouldn't be surprised by caches appearing with unexpected names.

Having been playing around with this a bit more, though, I think I'm going to suggest a different approach: hard-code the cache key name, but make the behaviour configurable with a cache-behaviour setting or similar, that takes values of disabled (get current cache-free behaviour), enabled (full new behaviour), or saveonly or restoreonly to do the obvious things. The latter are useful when – as with a lot of my cygport jobs – you need to run the action multiple times in the repository.

(As we discussed in #5, the ideal solution might be to swap from a composite action to a JS one, since that would allow post-run steps, which could be responsible for saving the cache after it has been used several times, but that's obviously a much bigger job!)


Line endings
------------
Expand Down Expand Up @@ -85,6 +86,22 @@ those executables directly in a `run:` in your workflow. Execute them via
Alternatively, putting e.g. `CYGWIN=winsymlinks:native` into the workflow's
environment works, since setup now honours that.

Caching
-------

If you're likely to do regular builds, you might want to store the packages
locally rather than needing to download them from the Cygwin mirrors on every
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure "locally" is the best word to use here.

build. Set `package-cache-key` to some string (e.g. `cygwin-package-cache`),
and the action will use [GitHub's dependency caching][0] to store downloaded
package files between runs.

[0]: https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

This has the effect of speeding up the run of the installation itself, at the
expense of taking slightly longer before and after the installation to check
and potentially update the cache. The installer will still check for updated
packages, and will download new packages if the cached ones are out of date

Mirrors and signatures
----------------------

Expand Down
43 changes: 43 additions & 0 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,33 @@ inputs:
description: Should Cygwin's bin directory be added to the system PATH?
required: false
default: true
package-cache-key:
description: Key prefix to use for package caches
required: false
default: ''

runs:
using: "composite"
steps:
- uses: actions/cache/restore@v3
if: inputs.package-cache-key != ''
with:
key: ${{ inputs.package-cache-key }}-${{ github.run_id }}-${{ github.run_attempt }}
path: C:\cygwin-packages
restore-keys:
${{ inputs.package-cache-key }}-${{ github.run_id }}-
${{ inputs.package-cache-key }}-

- if: inputs.package-cache-key != ''
working-directory: C:\
shell: bash
run: |
if [[ -d cygwin-packages ]]; then
find cygwin-packages -type f ! -name setup.ini -print0 |
sort -z |
xargs -0 b2sum >cygwin-packages-checksum.old
fi

- run: |
$platform = '${{ inputs.platform }}'
$platform = $platform -replace '^(x64|amd64)$', 'x86_64'
Expand Down Expand Up @@ -82,6 +105,26 @@ runs:
& C:\setup.exe $args | Out-Default
shell: powershell

- if: inputs.package-cache-key != ''
id: refresh-cache
working-directory: C:\
shell: bash
run: |
if [[ -d cygwin-packages ]]; then
find cygwin-packages -type f ! -name setup.ini -print0 |
sort -z |
xargs -0 b2sum >cygwin-packages-checksum.new
if ! diff cygwin-packages-checksum.old cygwin-packages-checksum.new; then
printf 'update_package_cache=YesPlease\n' >>"$GITHUB_OUTPUT"
fi
fi

- if: steps.refresh-cache.outputs.update_package_cache != ''
uses: actions/cache/save@v3
with:
key: ${{ inputs.package-cache-key }}-${{ github.run_id }}-${{ github.run_attempt }}
path: C:\cygwin-packages

- if: ${{ inputs.add-to-path == 'true' }}
run: echo "${{ inputs.install-dir }}\bin" >> $env:GITHUB_PATH
shell: powershell
Expand Down