Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HealthCheck log output options #23900

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Honny1
Copy link
Member

@Honny1 Honny1 commented Sep 9, 2024

This PR creates three new flags that can affect the output of the HealtCheck log.

Currently, when a container is configured with HealthCheck, the output from the HealthCheck command is only logged to the container status file, which is accessible via podman inspect. It is also limited to the last five executions and the first 500 characters per execution.

This makes debugging past problems very difficult, since the only information available about the failure of the HealthCheck command is the generic healthcheck service failed record.

  • The --health-log-destination flag sets the destination of the HealthCheck log.

    • none: (default behavior) HealthCheckResults are stored in overlay containers. (For example: ./run/containers/storage/overlay-containers/<container-ID>/healthcheck.log)
    • directory: creates a log file named <container-ID>-healthcheck.log with JSON HealthCheckResults in the specified directory.
    • events_logger: The log will be written with logging mechanism set by events_logger.
  • The --health-max-log-count flag sets the maximum number of attempts in the HealthCheck log file.

    • A value of 0 indicates an infinite number of attempts in the log file.
    • The default value is 5 attempts in the log file.
  • The --health-max-log-size flag sets the maximum length of the log stored.

    • A value of 0 indicates an infinite log length.
    • The default value is 500 log characters.

Does this PR introduce a user-facing change?

Added --health-log-destination, --health-max-log-count and --health-max-log-size flags that affect HealtCheck log output.

Fixes: RHEL-24623

@openshift-ci openshift-ci bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/release-note-label-needed Enforce release-note requirement, even if just None labels Sep 9, 2024
Copy link
Contributor

openshift-ci bot commented Sep 9, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Honny1
Once this PR has been reviewed and has the lgtm label, please assign flouthoc for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@Honny1 Honny1 force-pushed the healthcheck-log branch 11 times, most recently from 331ce42 to ebadb8b Compare September 10, 2024 15:58
@containers containers deleted a comment from packit-as-a-service bot Sep 10, 2024
@Honny1 Honny1 force-pushed the healthcheck-log branch 15 times, most recently from ee1d02a to bf7e1cb Compare September 13, 2024 10:56
@Honny1 Honny1 force-pushed the healthcheck-log branch 5 times, most recently from 0ff253e to a242a59 Compare September 20, 2024 14:53
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@github-actions github-actions bot added the kind/api-change Change to remote API; merits scrutiny label Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@Honny1 Honny1 force-pushed the healthcheck-log branch 3 times, most recently from 2ed163e to 01f4bdf Compare September 20, 2024 19:33
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@containers containers deleted a comment from packit-as-a-service bot Sep 20, 2024
@Honny1 Honny1 force-pushed the healthcheck-log branch 3 times, most recently from 00eecee to b70c402 Compare September 20, 2024 23:24
…nation flags

These flags can affect the output of the HealtCheck log. Currently, when a container is configured with HealthCheck, the output from the HealthCheck command is only logged to the container status file, which is accessible via `podman inspect`.
It is also limited to the last five executions and the first 500 characters per execution.

This makes debugging past problems very difficult, since the only information available about the failure of the HealthCheck command is the generic `healthcheck service failed` record.

- The `--health-log-destination` flag sets the destination of the HealthCheck log.
  - `none`: (default behavior) `HealthCheckResults` are stored in overlay containers. (For example: `$runroot/healthcheck.log`)
  - `directory`: creates a log file named `<container-ID>-healthcheck.log` with JSON `HealthCheckResults` in the specified directory.
  - `events_logger`: The log will be written with logging mechanism set by events_loggeri. It also saves the log to a default directory, for performance on a system with a large number of logs.

- The `--health-max-log-count` flag sets the maximum number of attempts in the HealthCheck log file.
  - A value of `0` indicates an infinite number of attempts in the log file.
  - The default value is `5` attempts in the log file.
- The `--health-max-log-size` flag sets the maximum length of the log stored.
  - A value of `0` indicates an infinite log length.
  - The default value is `500` log characters.

Add --health-max-log-count flag

Signed-off-by: Jan Rodák <[email protected]>

Add --health-max-log-size flag

Signed-off-by: Jan Rodák <[email protected]>

Add --health-log-destination flag

Signed-off-by: Jan Rodák <[email protected]>
Copy link

Cockpit tests failed for commit 8fae259. @martinpitt, @jelly, @mvollmer please check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/api-change Change to remote API; merits scrutiny release-note
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants