What happened?
While reviewing the remaining Spark unit-test coverage in #609, I noticed that read_pod_logs() splits non-streaming log responses using logs.split("\n").
This produces unexpected results:
"".split("\n") == [""]
"line\n".split("\n") == ["line", ""]
"line 1\r\nline 2".split("\n") == ["line 1\r", "line 2"]As a result, empty logs are represented as one blank line, newline-terminated logs contain an extra empty entry, and CRLF responses retain the carriage-return character.
read_pod_logs() is shared by both Spark Connect session logs and Spark batch-job logs.
Relevant code: kubeflow/spark/backends/kubernetes/utils.py::read_pod_logs
What did you expect to happen?
Non-streaming log output should be returned as logical lines:
- Empty output should return no lines.
- A trailing newline should not create an additional empty entry.
- CRLF line endings should be handled correctly.
- Genuine blank lines inside the output should remain preserved.
Using splitlines() for the non-streaming response would provide this behavior. Unit tests should cover empty output, trailing newlines, CRLF output, and interior blank lines.
Environment
Kubernetes version:
Not applicable. The issue is in client-side parsing of the string returned by the Kubernetes Python client and does not require a running cluster to reproduce.
Kubeflow Trainer version:
Not applicable.
Kubeflow Python SDK version:
Current main branch, observed at commit 8ecc659abbb13d089ee1046f5d967d5ad5a4da11.
Relevant file: kubeflow/spark/backends/kubernetes/utils.py
What happened?
While reviewing the remaining Spark unit-test coverage in #609, I noticed that
read_pod_logs()splits non-streaming log responses usinglogs.split("\n").This produces unexpected results:
As a result, empty logs are represented as one blank line, newline-terminated logs contain an extra empty entry, and CRLF responses retain the carriage-return character.
read_pod_logs()is shared by both Spark Connect session logs and Spark batch-job logs.Relevant code:
kubeflow/spark/backends/kubernetes/utils.py::read_pod_logsWhat did you expect to happen?
Non-streaming log output should be returned as logical lines:
Using
splitlines()for the non-streaming response would provide this behavior. Unit tests should cover empty output, trailing newlines, CRLF output, and interior blank lines.Environment
Kubernetes version:
Not applicable. The issue is in client-side parsing of the string returned by the Kubernetes Python client and does not require a running cluster to reproduce.
Kubeflow Trainer version:
Not applicable.
Kubeflow Python SDK version:
Current
mainbranch, observed at commit8ecc659abbb13d089ee1046f5d967d5ad5a4da11.Relevant file:
kubeflow/spark/backends/kubernetes/utils.py