Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

periodic errors exporting logs #999

Closed
BenTheElder opened this issue Oct 22, 2019 · 8 comments · Fixed by #1014, #1054 or #1066
Closed

periodic errors exporting logs #999

BenTheElder opened this issue Oct 22, 2019 · 8 comments · Fixed by #1014, #1054 or #1066
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Milestone

Comments

@BenTheElder
Copy link
Member

we're seeing errors tarring up log files for exporting the logs
/assign
/priority important-soon
/lifecycle active

xref previously: #949

@BenTheElder BenTheElder added the kind/bug Categorizes issue or PR as related to a bug. label Oct 22, 2019
@k8s-ci-robot k8s-ci-robot added lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Oct 22, 2019
@BenTheElder
Copy link
Member Author

having some difficulty triggering this now that i'm trying to fix it 😛

@BenTheElder
Copy link
Member Author

Got it once on the generic test PR, haven't gotten it again on the debug PR
https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/sigs.k8s.io_kind/648/pull-kind-conformance-parallel-1-13/1187394260584894464/

@BenTheElder BenTheElder added this to the v0.6.0 milestone Oct 26, 2019
@BenTheElder
Copy link
Member Author

#1014

@BenTheElder
Copy link
Member Author

reopening to keep an eye on this actually being fixed.

@BenTheElder
Copy link
Member Author

looking into this again today. I think I have a better idea what the root cause may be, finally getting back to verifying 🤞

@BenTheElder
Copy link
Member Author

ERROR: [command "docker exec --privileged kind-worker tar --hard-dereference -C /var/tmp/tmp.EqvsHf65G9 -chf - ." failed with error: write |1: broken pipe, command "docker exec --privileged kind-control-plane tar --hard-dereference -C /var/tmp/tmp.gLnoymvPuz -chf - ." failed with error: exit status 1]

still a thing https://prow.k8s.io/view/gcs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_kind/1059/pull-kind-e2e-kubernetes/1191980418304315397

@BenTheElder BenTheElder reopened this Nov 6, 2019
This was referenced Nov 6, 2019
@BenTheElder
Copy link
Member Author

I can repro locally

@BenTheElder
Copy link
Member Author

so there's null bytes at the end. I think the go tar reader exits early when it's done reading through the tar contents but before the tar util is done writing more null or similar.

we probably just need to read and discard the end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug. lifecycle/active Indicates that an issue or PR is actively being worked on by a contributor. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release.
Projects
None yet
2 participants