Skip to content

Commit 6ede09f

Browse files
committed
feat(redfish-system): add dedicated check for Redfish system health (#652)
Splits the Redfish monitoring into two plugins, one task each: * redfish-system: system-level health (manufacturer, model, hostname, processors, BIOS, power state, indicator LED, SKU, serial number) with the rolled-up Redfish `Status` as the check state. * redfish-drives: stays focused on drive and storage-controller health. Drops the systems_state aggregation so a system-level warning unrelated to storage (inlet temperature, indicator LED, etc.) no longer flips the check to WARN while all drives are healthy. The member line is trimmed to identification only (manufacturer, model, hostname, SKU, serial number) without a state label. Both plugins align their main() section comments to the example plugin (`# parse the command line`, `# fetch data`, `# init some vars`, `# analyze data`, `# build the message`, `# over and out`). Verified against a locally running DMTF Redfish-Mockup-Server configured with System Health=Warning and all drives healthy: redfish-drives now exits OK, redfish-system exits WARN. Closes #652.
1 parent eb173e0 commit 6ede09f

10 files changed

Lines changed: 614 additions & 38 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Monitoring Plugins:
5252
* procs: add `--top N` to list the top N processes by CPU time and memory usage
5353
* procs: add `--lengthy` for extended `--top` table output
5454
* procs: add `--warning-cpu-percent` / `--critical-cpu-percent` for aggregated CPU usage of filtered processes
55+
* redfish-system: checks overall system health (processors, BIOS, power state, indicator LED, hostname, SKU, serial number) reported by a Redfish-compatible server, previously bundled into `redfish-drives` ([#652](https://github.com/Linuxfabrik/monitoring-plugins/issues/652))
5556
* scanrootkit: add 41 signatures for modern Linux rootkits, backdoors and implants (BPFDoor, Drovorub, Ebury, FontOnLake, Kaiji, Kobalos, perfctl, PUMAKIT, Reptile, Symbiote, Winnti for Linux, and more)
5657
* scanrootkit: findings now show the year the rootkit was first publicly disclosed when known
5758
* sensors-temperatures: add `--ignore` to filter sensors by regex ([#965](https://github.com/Linuxfabrik/monitoring-plugins/issues/965))
@@ -75,6 +76,11 @@ Assets:
7576
Build, CI/CD:
7677

7778
* Bump pinned `linuxfabrik-lib` dependency from 3.0.0 to 3.2.0
79+
* GRAFANA: expanded guide. Documents the asset layout (per-plugin YAML, shared folder, default and Icinga dashboards), keeps the pinned Grizzly version at v0.2.0 (dashboards have not been validated against later releases), updates the Service Account navigation path to current Grafana, and adds an outlook section on the planned migration to `grafanactl` (tracked in [#1062](https://github.com/Linuxfabrik/monitoring-plugins/issues/1062))
80+
* PLUGINS-KEYCLOAK, PLUGINS-MYSQL, PLUGINS-ROCKETCHAT, PLUGINS-WILDFLY: rewritten with a unified structure (Plugins in this group, Authentication, Common parameters, Service Sets in the Icinga Director Basket). Keycloak corrects the role terminology (`query-groups` is a client role of the `master-realm` client, not a realm role of `master`; verified against Keycloak 26.6) and reflects the modern Admin Console navigation. Rocket.Chat now separates the Enterprise Edition and Community Edition paths (custom roles are EE only)
81+
* ICINGA: rewritten around the full-basket workflow. Adds sections on what the basket ships (Host/Service Templates, Service Sets, Time Periods, Notifications, Data Lists, Datafields), tag-based service assignment, upstream-vs-fork positioning, onboarding a host, upgrade-with-`basket-compare`, and troubleshooting (PHP/MySQL size limits, UUID conflicts, non-default master zone). Contributor-facing `build-basket` details remain in CONTRIBUTING.md
82+
* README: split the FAQ section into FAQ (concepts, design decisions, capability questions) and Troubleshooting (runtime errors); both lists sorted alphabetically. New Troubleshooting entry for plugins exiting UNKNOWN with sudo errors that points at the sudoers drop-in
83+
* INSTALL: rewritten as a single source of truth with a per-distro TOC (Debian, RHEL, SLE, Ubuntu), Linux before Windows, recommended paths first. Package-manager, Ansible, source tarball, Git, Windows MSI, Windows ZIP and Windows-from-source paths are now all documented on one page. `repo.linuxfabrik.ch` and `download.linuxfabrik.ch` link to the docs site instead of duplicating the commands
7884
* Windows MSI no longer depends on an installed Icinga2 agent (install path unchanged: `ProgramFiles64Folder/ICINGA2/sbin/linuxfabrik`)
7985

8086

@@ -172,6 +178,7 @@ Monitoring Plugins:
172178
* notify-host-mail, notify-service-mail: Icinga logo renders inline again on hosts with long FQDNs ([#790](https://github.com/Linuxfabrik/monitoring-plugins/issues/790))
173179
* ntp-\*: prevent `TypeError` when comparing int and str
174180
* podman-stats: precise numeric values; aggregate perfdata includes block and network I/O totals
181+
* redfish-drives: a system-level warning unrelated to storage (such as an inlet temperature threshold) no longer flips the check to WARN while all drives are healthy. The system-level health is covered by `redfish-system` instead ([#652](https://github.com/Linuxfabrik/monitoring-plugins/issues/652))
175182
* rocketchat-stats: crash when reporting the user count
176183
* scanrootkit: a single malformed signature file no longer crashes the whole check
177184
* scanrootkit: directory-only rootkit signatures (e.g. KBeast `/usr/_h4x_`, Kaiji `/usr/bin/lib`) are now actually evaluated

‎check-plugins/redfish-drives/README.md‎

Lines changed: 12 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,21 +3,23 @@
33

44
## Overview
55

6-
Checks the state of all physical drives and storage media in a Redfish-compatible server via the Redfish API. Iterates over the Systems collection, fetches storage controllers and their attached drives, and reports health status, media type, protocol, capacity, and predicted media life left.
6+
Checks the state of all physical drives and their storage controllers in a Redfish-compatible server via the Redfish API. Iterates over the Systems collection, fetches storage controllers and their attached drives, and reports health status, media type, protocol, capacity, and predicted media life left.
77

88
**Important Notes:**
99

1010
* Tested on DELL iDRAC and DMTF Simulator
1111
* A check takes up to 10 seconds. Increasing runtime timeout to 30 seconds is recommended.
1212
* This check runs with both HTTP and HTTPS. It uses GET requests only.
1313
* No additional Python Redfish modules need to be installed.
14+
* System-level health (processors, BIOS, power, temperature, indicator LED, etc.) is deliberately not aggregated into the plugin state, so that a system-level warning unrelated to storage (such as an inlet temperature threshold) cannot mask or flip the drive status. Use `redfish-system` for system-level health monitoring.
1415

1516
**Data Collection:**
1617

1718
* Queries `/redfish/v1/Systems` to enumerate system members
1819
* For each member, queries the Storage collection and iterates over all storage controllers and attached drives
1920
* Uses HTTP Basic authentication if `--username` and `--password` are provided
2021
* Only evaluates drives and controllers in "Enabled" or "Quiesced" state
22+
* Aggregates only drive health and storage-controller health into the plugin state. The member identification line (manufacturer, model, hostname, SKU, serial number) is shown as context without a status label.
2123

2224

2325
## Fact Sheet
@@ -39,9 +41,12 @@ usage: redfish-drives [-h] [-V] [--always-ok] [--insecure] [--no-proxy]
3941
[--password PASSWORD] [--timeout TIMEOUT] [--url URL]
4042
[--username USERNAME]
4143
42-
Checks the state of all physical drives and storage media in a Redfish-
43-
compatible server via the Redfish API. Alerts when any drive reports a
44-
degraded or failed state.
44+
Checks the state of all physical drives and their storage controllers in a
45+
Redfish-compatible server via the Redfish API. Alerts when any drive or
46+
storage controller reports a degraded or failed state. System-level health
47+
(processors, BIOS, power, temperature, indicator LED, etc.) is deliberately
48+
ignored by this check so that a system warning unrelated to storage does not
49+
mask the drive status; use `redfish-system` for that.
4550
4651
options:
4752
-h, --help show this help message and exit
@@ -67,7 +72,7 @@ Output:
6772
```text
6873
Everything is ok, checked storage on 1 member.
6974
70-
Member: Dell Inc. PowerEdge R750, Processors: 2x Intel(R) Xeon(R) Gold 6354 CPU @ 3.00GHz (72 logical), BIOS: 1.1.3, Power: On, LED: Lit, SKU: ABCDEFG, SerNo: 1234567890ABCDE
75+
Member: Dell Inc. PowerEdge R750, HostName: web483, SKU: ABCDEFG, SerNo: 1234567890ABCDE
7176
7277
Disk ! Type ! Proto ! Manufacturer ! Model ! SerialNumber ! Size ! LifeLeft % ! State
7378
--------------------+------+-------+--------------+---------------+--------------+----------+------------+-------
@@ -88,11 +93,12 @@ CPU.1 ! CPU.1 ! CP
8893

8994
## States
9095

91-
* OK if all drives and controllers are healthy.
96+
* OK if all drives and storage controllers are healthy.
9297
* WARN if an enabled drive's health or health rollup state is "Warning".
9398
* WARN if an enabled controller's health or health rollup state is "Warning".
9499
* CRIT if an enabled drive's health or health rollup state is "Critical".
95100
* CRIT if an enabled controller's health or health rollup state is "Critical".
101+
* System-level health is NOT considered. Use `redfish-system` for that.
96102
* `--always-ok` suppresses all alerts and always returns OK.
97103

98104

‎check-plugins/redfish-drives/icingaweb2-module-director/redfish-drives.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@
104104
"tpl-service-generic"
105105
],
106106
"max_check_attempts": 5,
107-
"notes": "Checks the state of all physical drives and storage media in a Redfish-compatible server via the Redfish API. Alerts when any drive reports a degraded or failed state.",
107+
"notes": "Checks the state of all physical drives and their storage controllers in a Redfish-compatible server via the Redfish API. Alerts when any drive or storage controller reports a degraded or failed state. System-level health (processors, BIOS, power, temperature, indicator LED, etc.) is deliberately ignored by this check so that a system warning unrelated to storage does not mask the drive status; use `redfish-system` for that.",
108108
"notes_url": "https://linuxfabrik.github.io/monitoring-plugins/check-plugins/redfish-drives/",
109109
"object_name": "tpl-service-redfish-drives",
110110
"object_type": "template",

‎check-plugins/redfish-drives/redfish-drives‎

Lines changed: 19 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -22,10 +22,13 @@ import lib.url
2222
from lib.globals import STATE_CRIT, STATE_OK, STATE_UNKNOWN, STATE_WARN
2323

2424
__author__ = 'Linuxfabrik GmbH, Zurich/Switzerland'
25-
__version__ = '2026040801'
25+
__version__ = '2026042301'
2626

27-
DESCRIPTION = """Checks the state of all physical drives and storage media in a Redfish-compatible
28-
server via the Redfish API. Alerts when any drive reports a degraded or failed state."""
27+
DESCRIPTION = """Checks the state of all physical drives and their storage controllers in a
28+
Redfish-compatible server via the Redfish API. Alerts when any drive or storage controller
29+
reports a degraded or failed state. System-level health (processors, BIOS, power, temperature,
30+
indicator LED, etc.) is deliberately ignored by this check so that a system warning unrelated
31+
to storage does not mask the drive status; use `redfish-system` for that."""
2932

3033
DEFAULT_INSECURE = True
3134
DEFAULT_NO_PROXY = False
@@ -102,24 +105,21 @@ def parse_args():
102105
def main():
103106
"""The main function. This is where the magic happens."""
104107

105-
# parse the command line, exit with UNKNOWN if it fails
108+
# parse the command line
106109
try:
107110
args = parse_args()
108111
except SystemExit:
109112
sys.exit(STATE_UNKNOWN)
110113

114+
# fetch data
111115
base_url = args.URL
112116
if not args.URL.startswith('http'):
113117
lib.base.cu('--url parameter has to start with "http://" or https://".')
114-
115-
# Authorization (if needed)
116-
header = {}
117-
header['Accept'] = 'application/json'
118+
header = {'Accept': 'application/json'}
118119
if args.USERNAME and args.PASSWORD:
119120
auth = f'{args.USERNAME}:{args.PASSWORD}'
120121
encoded_auth = lib.txt.to_text(base64.b64encode(lib.txt.to_bytes(auth)))
121122
header['Authorization'] = f'Basic {encoded_auth}'
122-
123123
# "/redfish/v1/Systems"
124124
url = f'{base_url}/redfish/v1/Systems'
125125
result = lib.base.coe(
@@ -143,9 +143,11 @@ def main():
143143
msg = ''
144144
state = STATE_OK
145145
perfdata = ''
146-
147146
member_count = 0
148-
# fetch and analyze data by following the "Members" found
147+
148+
# analyze data: follow each "Member" link, aggregate drive and
149+
# storage-controller health into `state`. System-level health is
150+
# deliberately not aggregated (see `redfish-system` for that).
149151
for member in result.get('Members', []):
150152
# "/redfish/v1/Systems/437XR1138R2"
151153
url = f'{base_url}{member["@odata.id"]}'
@@ -162,34 +164,19 @@ def main():
162164
if systems['Status_State'] not in ['Enabled', 'Quiesced']:
163165
continue
164166
member_count += 1
165-
systems_state = lib.redfish.get_state(systems)
166-
state = lib.base.get_worst(state, systems_state)
167167

168-
169-
# build the message
168+
# build the message: only identify the member (manufacturer, model,
169+
# hostname, SKU, serial number). System-level health is intentionally
170+
# not aggregated here and not labelled with a state; use
171+
# `redfish-system` for that.
170172
msg += 'Member:'
171173
msg += f' {systems["Manufacturer"]}' if systems['Manufacturer'] else ''
172174
msg += f' {systems["Model"]}' if systems['Model'] else ''
173175
msg += ', '
174176
msg += f'HostName: {systems["HostName"]}, ' if systems['HostName'] else ''
175-
msg += f'Processors: {systems["ProcessorSummary_Count"]}x'
176-
msg += (
177-
f' {systems["ProcessorSummary_Model"]}'
178-
if systems['ProcessorSummary_Model']
179-
else ''
180-
)
181-
msg += (
182-
f' ({systems["ProcessorSummary_LogicalProcessorCount"]} logical)'
183-
if systems['ProcessorSummary_LogicalProcessorCount']
184-
else ''
185-
)
186-
msg += ', '
187-
msg += f'BIOS: {systems["BiosVersion"]}, '
188-
msg += f'Power: {systems["PowerState"]}, ' if systems['PowerState'] else ''
189-
msg += f'LED: {systems["IndicatorLED"]}, ' if systems['IndicatorLED'] else ''
190177
msg += f'SKU: {systems["SKU"]}, ' if systems['SKU'] else ''
191178
msg += f'SerNo: {systems["SerialNumber"]}, ' if systems['SerialNumber'] else ''
192-
msg = msg[:-2] + lib.base.state2str(systems_state, prefix=' ')
179+
msg = msg[:-2]
193180

194181
# get all available storage links for the member
195182
if not systems['Storage_@odata.id']:
@@ -289,6 +276,7 @@ def main():
289276

290277
msg += '\n\n'
291278

279+
# build the message
292280
members = lib.txt.pluralize('member', member_count)
293281
if state == STATE_CRIT:
294282
msg = (
Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,92 @@
1+
# Check redfish-system
2+
3+
## Overview
4+
5+
Checks the overall system health reported by a Redfish-compatible server via the Redfish API. Iterates over the Systems collection and reports each member's identification (manufacturer, model, hostname, SKU, serial number), its compute summary (processors, BIOS version, power state, indicator LED) and its rolled-up health status.
6+
7+
**Important Notes:**
8+
9+
* Tested on DELL iDRAC and DMTF Simulator
10+
* A check takes up to 10 seconds. Increasing runtime timeout to 30 seconds is recommended.
11+
* This check runs with both HTTP and HTTPS. It uses GET requests only.
12+
* No additional Python Redfish modules need to be installed.
13+
* This check reports system-level health only. For drive and storage-controller monitoring use `redfish-drives`.
14+
15+
**Data Collection:**
16+
17+
* Queries `/redfish/v1/Systems` to enumerate system members
18+
* For each member in "Enabled" or "Quiesced" state, reads the `Status` block (including `HealthRollup`) and the compute summary
19+
* Uses HTTP Basic authentication if `--username` and `--password` are provided
20+
21+
22+
## Fact Sheet
23+
24+
| Fact | Value |
25+
|----|-----|
26+
| Check Plugin Download | <https://github.com/Linuxfabrik/monitoring-plugins/tree/main/check-plugins/redfish-system> |
27+
| Nagios/Icinga Check Name | `check_redfish_system` |
28+
| Check Interval Recommendation | Every 5 minutes |
29+
| Can be called without parameters | Yes |
30+
| Runs on | Cross-platform |
31+
| Compiled for Windows | No |
32+
33+
34+
## Help
35+
36+
```text
37+
usage: redfish-system [-h] [-V] [--always-ok] [--insecure] [--no-proxy]
38+
[--password PASSWORD] [--timeout TIMEOUT] [--url URL]
39+
[--username USERNAME]
40+
41+
Checks the overall system health reported by a Redfish-compatible server via
42+
the Redfish API. Reports every enabled system member with its identification
43+
(manufacturer, model, hostname, SKU, serial number), compute summary
44+
(processors, BIOS version, power state, indicator LED) and rolled-up health
45+
status, and alerts whenever any system's status leaves `OK`. Use `redfish-
46+
drives` for drive- and storage-controller-specific monitoring.
47+
48+
options:
49+
-h, --help show this help message and exit
50+
-V, --version show program's version number and exit
51+
--always-ok Always returns OK.
52+
--insecure This option explicitly allows insecure SSL connections.
53+
--no-proxy Do not use a proxy.
54+
--password PASSWORD Redfish API password.
55+
--timeout TIMEOUT Network timeout in seconds. Default: 8 (seconds)
56+
--url URL Redfish API URL. Default: https://localhost:5000
57+
--username USERNAME Redfish API username.
58+
```
59+
60+
61+
## Usage Examples
62+
63+
```bash
64+
./redfish-system --url https://bmc --username redfish-monitoring --password 'linuxfabrik'
65+
```
66+
67+
Output:
68+
69+
```text
70+
Checked system health on 1 member. There are warnings.
71+
72+
Member: Dell Inc. PowerEdge R750, HostName: web483, Processors: 2x Intel(R) Xeon(R) Gold 6354 CPU @ 3.00GHz (72 logical), BIOS: 1.1.3, Power: On, LED: Lit, SKU: ABCDEFG, SerNo: 1234567890ABCDE [WARNING]
73+
```
74+
75+
76+
## States
77+
78+
* OK if every enabled system member reports `Status.Health` (or `Status.HealthRollup`) as "OK".
79+
* WARN if any enabled system member reports `Status.Health` (or `Status.HealthRollup`) as "Warning".
80+
* CRIT if any enabled system member reports `Status.Health` (or `Status.HealthRollup`) as "Critical".
81+
* `--always-ok` suppresses all alerts and always returns OK.
82+
83+
84+
## Perfdata / Metrics
85+
86+
There is no perfdata.
87+
88+
89+
## Credits, License
90+
91+
* Authors: [Linuxfabrik GmbH, Zurich](https://www.linuxfabrik.ch)
92+
* License: The Unlicense, see [LICENSE file](https://unlicense.org/).

0 commit comments

Comments
 (0)