Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add pigz for parallel decompression by containerd #2896

Merged
merged 1 commit into from
Mar 16, 2023

Conversation

ohkinozomu
Copy link
Contributor

@ohkinozomu ohkinozomu commented Mar 15, 2023

Issue number:

Closes #2886

Description of changes:

Let containerd use pigz (or more precisely, unpigz).
pigz doesn't have a LICENSE file (madler/pigz#89) and I decided to use README...

If there is no pigz, and the log level of containerd is set to debug, the following log will be output at the time of pull.

Mar 14 09:30:08 10.0.2.15 containerd[1175]: time="2023-03-14T09:30:08.609765577Z" level=debug msg="unpigz not found, falling back to go gzip" error="exec: \"unpigz\": executable file not found in $PATH"

I have confirmed that with this change that log disappears (i.e. containerd uses unpigz).

Here is a quick benchmark.

machine spec:
Google Compute Engine c3-highcpu-22 Ubuntu 22.04.2 (enable nested virtualization)

Command to execute:

ctr image pull docker.io/library/mysql:8.0.32

without pigz(commit b1ac8ef)

6.139457575s
7.236172524s
7.730825494s

with pigz(this commit)

5.842488681s
6.319545383s
5.777997267s

Testing done: yes

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

Copy link
Contributor

@bcressey bcressey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for picking this up!

Overall looks good, but needs a few fixes for some subtle cross-compiling and software licensing issues.

@ohkinozomu
Copy link
Contributor Author

Thanks for your review, I have fixed.

@ohkinozomu ohkinozomu requested a review from bcressey March 15, 2023 10:43
Copy link
Contributor

@stmcginnis stmcginnis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

Copy link
Contributor

@cbgbt cbgbt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for working on this! It looks good to me, just a minor nit.

@ohkinozomu ohkinozomu force-pushed the pigz branch 2 times, most recently from f558123 to 12f7bf5 Compare March 15, 2023 23:23
@ohkinozomu
Copy link
Contributor Author

Thanks to all reviewers!
Fix completed.

@bcressey
Copy link
Contributor

Nice work. Thanks for doing this, @ohkinozomu!

Copy link
Contributor

@stmcginnis stmcginnis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All comments addressed - looks great!

@stmcginnis stmcginnis merged commit 38d4f6a into bottlerocket-os:develop Mar 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

containerd with pigz
5 participants