Skip to content

feat: support configuring a logrus.Entry on server.Configuration - #597

Open
dobby-coder[bot] wants to merge 10 commits into
masterfrom
feat/server-config-logger-entry
Open

feat: support configuring a logrus.Entry on server.Configuration#597
dobby-coder[bot] wants to merge 10 commits into
masterfrom
feat/server-config-logger-entry

Conversation

@dobby-coder

@dobby-coder dobby-coder Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Closes #232

What

server.Configuration only accepted a *logrus.Logger, so callers could not attach persistent fields (e.g. WithField("lib", "irma") to tag IRMA-library logs) without a hacky decorator formatter.

This adds an optional LoggerEntry *logrus.Entry field and routes all server logging through a *logrus.Entry, so any persistent fields a caller attaches appear on every log line the server emits.

Chosen API shape (please confirm)

A new optional field on server.Configuration:

// Custom logger entry. Takes precedence over Logger and allows attaching
// persistent fields (e.g. LoggerEntry.WithField("lib", "irma")) that are
// then included on every log line emitted by the server.
LoggerEntry *logrus.Entry `json:"-"`

Resolution happens in Configuration.Check (precedence is backward-compatible):

Set by caller Result
LoggerEntry takes precedence; Logger is derived from LoggerEntry.Logger (a default logger is attached if the entry has none)
only Logger an entry is derived automatically via logrus.NewEntry(Logger)
neither a logger and entry are built from Verbose/Quiet/LogJSON (unchanged default behaviour)

After Check, conf.Logger and conf.LoggerEntry always both exist and share the same underlying *logrus.Logger.

To actually surface the persistent fields, the server and irmaserver packages (and the keyshare/myirma consumers of the package-level logger) now log through the resolved entry rather than the bare logger. Level checks (IsLevelEnabled) stay on the underlying *logrus.Logger, and the exported package-level server.Logger is kept working alongside a new exported server.LoggerEntry.

This keeps the existing Logger *logrus.Logger field fully working, so it is a pure additive, backward-compatible API change.

Alternative considered

Replacing Logger outright with LoggerEntry (as the issue phrased it) would break every existing caller that sets Logger. The additive approach above delivers the same capability without a breaking change. Happy to adjust the field name or precedence if maintainers prefer a different shape.

Testing

  • New unit test TestConfigurationResolveLogger (irma/server/conf_logger_test.go) covers all resolution branches and asserts persistent fields flow through to emitted log lines (no docker needed).
  • gofmt -l . clean, go vet ./... clean, go build ./... clean.
  • go test ./irma/server/ passes; the docker-free irmaserver session-store tests pass. The Postgres/Redis/SMTP-backed suites run in CI.

Kept as a draft because this is a public-API addition and the field name / precedence is a maintainer call.


Review follow-up (2026-06-29)

Addressed the CI failures, the GitHub security comments and the review nits:

  • CodeQL CWE-312 (clear-text logging of sensitive information) on irma/server/api.go. Request-header logging is filtered through a constant allowlist of header names (Accept*, Content-Type, Content-Length, User-Agent, X-Forwarded-*, X-Real-Ip); headers outside it are dropped entirely. On top of that, the Copilot Autofix commit (b7c9b6e) redacts the values of the allowlisted headers to <redacted> rather than logging them, so the log records which known-safe headers a client sent without emitting any header value. Because the logged values are now constant strings, CodeQL no longer sees user-controlled header data flowing into the log sink (the earlier denylist + common.SanitizeForLog could not clear it: SanitizeForLog only strips CR/LF — a CWE-117 barrier — and is not a CWE-312 barrier, and CodeQL cannot prove a denylist is complete). Redaction also removes the residual CWE-117 surface from header values. url, message and from remain sanitized via common.SanitizeForLog. The autofix did not update the tests, so they were corrected to assert the redaction behaviour: TestFilterHeaders, TestFilterHeadersDropsUnlistedHeaders, and TestFilterHeadersRedactsValues (replacing the former …SanitizesValues).
  • Changelog CI. Added an ### Added entry under Unreleased for the LoggerEntry feature.
  • Nit: requestorserver logged through the bare Logger. All requestorserver log sites (server.go, conf.go) now emit via conf.LoggerEntry, so persistent fields appear on the operator-visible daemon log lines (session start, TLS, auth warnings) too.

Accepted limitation (library logs)

resolveLogger still calls irma.SetLogger(conf.Logger) with the bare *logrus.Logger, so log lines from the irma/gabi/common/revocation libraries do not carry the persistent fields set on LoggerEntry. Threading an entry through irma.SetLogger would require changing the logger type of the external gabi package (and common/revocation), i.e. a cross-repo API change that is out of scope for this PR. The server-side path requested in #232 is fully delivered; the library-log tagging case is documented here as a follow-up for maintainers.

server.Configuration only accepted a *logrus.Logger, so callers could not
attach persistent fields (e.g. WithField to tag IRMA-library logs) without a
decorator formatter.

Add an optional LoggerEntry *logrus.Entry field, resolved in Configuration.Check
in a backward-compatible way:
- LoggerEntry set: it takes precedence and Logger is derived from its .Logger.
- only Logger set: an entry is derived from it automatically.
- neither set: a logger and entry are built from Verbose/Quiet/LogJSON.

All server logging (the server and irmaserver packages, plus the keyshare/
myirma consumers of the package-level logger) now flows through the resolved
*logrus.Entry, so any persistent fields the caller attached appear on every log
line. Level checks stay on the underlying *logrus.Logger.

Closes #232

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dobby-coder
dobby-coder Bot requested a review from rubenhensen June 28, 2026 23:19
Comment thread irma/server/api.go Fixed
Comment thread irma/server/api.go Fixed

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rules + review gatekeeper — sign-off (non-blocking)

Verdict: sign-off / no blocking issues. Posted as COMMENT only because GitHub blocks APPROVE on a self-authored PR — this is not withholding approval. The two notes below are advisory scope items for maintainers, not change requests. Leaving the PR as a draft on purpose: issue #232 asked for a draft so maintainers can confirm the public-API shape.

The per-rule compliance sweep (all binding rules + irmago repo notes, checked per-rule) found no rule violations: conventional-commit title, tests present (TestConfigurationResolveLogger), no ST1005/race/nil-deref patterns, gofmt/vet clean. The LoggerEntry API addition is backward-compatible and the implementation is sound.

Two advisory scope notes for maintainers (neither is a bug; both are about how far the feature reaches):

1. Library logs do not carry the persistent fields. resolveLogger calls irma.SetLogger(conf.Logger) with the bare *logrus.Logger, so log lines from the irma/gabi/common/revocation libraries do not carry fields attached to LoggerEntry. Issue #232 gave "WithField to tag IRMA-library logs" as a motivating example; the core ask (an entry-based server.Configuration logging path) IS delivered, but irma.SetLogger only accepts a *logrus.Logger, so the library-log tagging case isn't reached. Worth a maintainer decision: extend the irma package with an entry-aware setter, or document this as an accepted limitation in the PR body. (Inline note on irma/server/conf.go:192.)

2. requestorserver still logs through the bare Logger. The irma server daemon HTTP layer (irma/server/requestorserver/server.go + conf.go) still emits via s.conf.Logger / conf.Logger at ~24 sites (e.g. server.go:105,127,262,346,540; conf.go:203,313) rather than the resolved LoggerEntry, so persistent fields are absent on some operator-visible lines (session start, TLS, auth warnings). This file is not in the diff, so it's noted here rather than inline. A follow-up routing these through the entry would make tagging consistent server-wide.

Neither note blocks merge. Signing off; maintainers can confirm the API shape and decide on the two scope items.

Comment thread irma/server/conf.go
}
Logger = conf.Logger
LoggerEntry = conf.LoggerEntry
irma.SetLogger(conf.Logger)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking scope note: irma.SetLogger(conf.Logger) passes the bare *logrus.Logger, so irma/gabi/common/revocation library log lines won't carry the persistent fields set on LoggerEntry. Issue #232's motivating example was tagging IRMA-library logs; the server-side path is fully delivered, but the library path would need an entry-aware setter in the irma package. A maintainer call — either extend irma or note it as an accepted limitation in the PR body.

@rubenhensen

Copy link
Copy Markdown
Contributor

/dobby fix the nits, the CI and the comments from github security

@dobby-coder

dobby-coder Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

On it! I'll address the nits, CI failures, and security comments on this PR and get it cleaned up.

dobby-coder Bot and others added 2 commits June 29, 2026 09:47
…rEntry, add changelog

- Sanitize user-controlled HTTP request header values and the request body
  before they are written to trace logs, fixing the CodeQL "log entries created
  from user input" / "clear-text logging" findings on LogRequest. Sensitive
  headers stay redacted; remaining values now pass through common.SanitizeForLog
  (the same barrier already applied to url/from).
- Route requestorserver logging (server.go, conf.go) through conf.LoggerEntry so
  persistent fields configured on Configuration.LoggerEntry appear on the
  operator-visible daemon log lines too.
- Add a CHANGELOG.md "Added" entry for the LoggerEntry feature (fixes the
  changelog CI check).
- Add TestFilterHeadersSanitizesValues covering CR/LF stripping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add Proxy-Authorization, X-Api-Key, Api-Key, X-Csrf-Token and X-Xsrf-Token
to the denylist of HTTP headers whose values are redacted before request
logging, hardening the CWE-312 (clear-text logging of sensitive data) path
flagged by CodeQL. Extends TestFilterHeaders and updates the changelog.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rules + review gatekeeper — verdict: REQUEST CHANGES (cycle 2)

Posted as a COMMENT only because GitHub blocks REQUEST_CHANGES on a self-authored PR (dobby-coder[bot]). Treat this as blocking — it is not a sign-off. Sending the PR back to the fix stage.

The per-rule compliance sweep (all binding rules + the irmago repo notes, checked per-rule via sub-agents) plus the code review converge on one blocking issue: the PR's stated goal — "fix the nits, the CI and the comments from github security" — is not yet met, because CI is still RED.

CodeQL is still failing on the fix commit (e28fdfa). The analyze job on the current HEAD completed and still reports 1 high-severity alertClear-text logging of sensitive information (CWE-312) — at irma/server/api.go:468 (the LoggerEntry.WithFields(fields).Tracef("=> request") sink in LogRequest). This is not stale: the run on e28fdfa itself fails, so the GitHub-security comments the PR set out to clear are live, not leftovers.

Why this PR's approach doesn't clear it: common.SanitizeForLog only strips CR/LF. That satisfies the CWE-117 log-injection query but is not a barrier for CWE-312 — stripping newlines doesn't make a header value non-sensitive. The name-keyed sensitiveHeaders denylist does redact the listed headers at runtime (TestFilterHeaders proves it, so this is not a confirmed runtime leak), but CodeQL can't prove denylist completeness and keeps tainting the whole filterHeaders(headers)fields["headers"] → log flow.

To turn the check green, the PR Dobby must change approach (pick one):

  1. Log only an allowlist of constant, known-safe header names instead of filtering a denylist; or
  2. Drop the headers field from request logs entirely; or
  3. If the maintainers accept the denylist as sufficient, dismiss the alert as a false positive in the code-scanning dashboard (a maintainer action, not a code change).

Rule sweep — what passed

Conventional-commit title (feat: …) ✓ · tests present and on-point (TestConfigurationResolveLogger, expanded TestFilterHeaders, new TestFilterHeadersSanitizesValues) ✓ · no modern-loop / ST1005 / nil-deref regressions (the deleteExpired nil-guard at sessions.go:288 is intact) ✓ · gofmt/vet clean, LoggerEntry addition is backward-compatible. The LoggerEntry design itself is sound — the only thing standing between this PR and sign-off is the red CodeQL check above.

Once CI is green, this is good to merge.

Comment thread irma/server/api.go
fields["from"] = common.SanitizeForLog(from)
}
Logger.WithFields(fields).Tracef("=> request")
LoggerEntry.WithFields(fields).Tracef("=> request")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI blocker — CodeQL CWE-312 still live here. The analyze job on HEAD e28fdfa completed and still reports 1 high alert at this LoggerEntry.WithFields(fields).Tracef sink: Clear-text logging of sensitive information. The PR's fix (expanded sensitiveHeaders denylist + common.SanitizeForLog) does not clear it: SanitizeForLog only strips CR/LF, which satisfies the CWE-117 log-injection query but is not a barrier for CWE-312 — stripping newlines doesn't make a header value non-sensitive. CodeQL can't prove the name-keyed denylist is complete and keeps tainting filterHeaders(headers)fields["headers"] → this log call.

To make the check green, switch to one of: (1) log only an allowlist of constant safe header names, (2) drop the headers field from request logs, or (3) have a maintainer dismiss the alert as a false positive in the code-scanning dashboard. A denylist + SanitizeForLog will not turn it green.

The CodeQL CWE-312 (clear-text logging of sensitive information) alert at
LogRequest stayed live with the denylist approach: redacting known-sensitive
header names plus SanitizeForLog only strips CR/LF (a CWE-117 barrier), and
CodeQL cannot prove a denylist is complete, so it kept tainting the whole
headers map flowing into the log.

Switch filterHeaders to an allowlist of constant, known-safe header names
(Accept*, Content-Type, Content-Length, User-Agent, X-Forwarded-*, X-Real-Ip).
Headers outside the allowlist are dropped entirely, so no credential- or
session-carrying header value can ever reach the logs. Allowlisted values are
still run through SanitizeForLog to keep the log-injection barrier.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@dobby-coder

dobby-coder Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

🧦 Dobby kept getting interrupted: the repo/PR/issue changed while Dobby was working, even after 2 attempts to catch up. Latest changes detected:

Dobby is backing off to avoid posting stale output — this one needs manual review.

@rubenhensen

Copy link
Copy Markdown
Contributor

/dobby try again, fix the nits, the CI and the comments from github security

@dobby-coder

dobby-coder Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

On it, @rubenhensen! I'll get started on fixing the nits, CI failures, and security comments on this PR now.

rubenhensen and others added 4 commits June 29, 2026 14:06
…f sensitive information'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
The Copilot Autofix commit (b7c9b6e) redacts allowlisted request-header
values to <redacted> instead of logging them, which clears the CodeQL
CWE-312 alert but left the api_test.go assertions checking the previous
value-passthrough behaviour, so the test suite would fail.

Update TestFilterHeaders / TestFilterHeadersDropsUnlistedHeaders to expect
the <redacted> placeholders, and replace TestFilterHeadersSanitizesValues
with TestFilterHeadersRedactsValues (values are now redacted, not merely
CR/LF-sanitized, so no header value or log-injection surface remains).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The request-log header handling now restricts headers to a constant
allowlist of known-safe names and redacts their values, rather than
filtering a denylist of sensitive names. Update the CHANGELOG entry to
match the implemented behaviour.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
keyshareserver, myirmaserver and the keyshare tasks runner still log via
conf.Logger (*logrus.Logger), so 'All server-side logging' overclaimed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rules + review gatekeeper — verdict: SIGN-OFF (cycle 3, loop limit reached)

Posted as a COMMENT only because GitHub blocks APPROVE on a self-authored PR (dobby-coder[bot]). This is a sign-off, not withholding approval.

Both the cycle-2 blocker and the cycle-3 nit are now cleared:

1. CodeQL blocker (cycle 2) — RESOLVED. The header-logging rewrite switched from a sensitive-header denylist to a constant allowlist (loggableHeaders) with every value redacted to <redacted>. CodeQL can now prove no user-controlled/credential header value reaches a log sink, so the CWE-312 (clear-text logging) and CWE-117 (log injection) alerts at irma/server/api.go are gone. On HEAD the CodeQL, analyze, build, lint, changelog and both integration checks are green (only the test matrix job was still finishing). The earlier github-advanced-security inline comments were posted against the superseded commit e28fdfa.

2. CHANGELOG accuracy nit (Review Dobby 2) — FIXED INLINE this cycle. The entry claimed "All server-side logging now flows through a *logrus.Entry", but keyshare/keyshareserver, keyshare/myirmaserver and keyshare/tasks/tasks.go still log via conf.Logger (*logrus.Logger). Rather than loop a one-line doc fix back through the fix stage, Dobby scoped the sentence to the three packages actually converted (server, irmaserver, requestorserver) and pushed it directly (commit e0c1993).

Rule sweep — clean

Checked against the binding rule set + the irmago repo notes: conventional-commit title (feat:/docs:) ✓ · tests present and on-point (TestConfigurationResolveLogger, expanded TestFilterHeaders, new TestFilterHeadersDropsUnlistedHeaders / TestFilterHeadersRedactsValues) ✓ · deleteExpired nil-guard intact ✓ · no ST1005 / modern-loop / nil-deref regressions · the LoggerEntry addition is backward-compatible (an entry is derived from a bare Logger automatically). No rule violations.

Staying a draft on purpose

Per the cycle-1 note, this PR is intentionally left as a draft so maintainers can confirm the public server.Configuration.LoggerEntry API shape against issue #232 before it ships — not flipping it to ready. The remaining scope notes (library logs via irma.SetLogger, and the keyshare/myirma/tasks packages still on the bare Logger) are accepted follow-ups, not blockers.

This is the 3rd gatekeeper iteration, so the fix loop limit has been reached; Dobby is signing off here rather than cycling again. The PR is clean and ready for maintainer review of the API shape.

@dobby-coder

dobby-coder Bot commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

Cycle 3 done — all green. The CodeQL CWE-312/CWE-117 blocker from cycle 2 is resolved: the sensitive-header denylist is gone, replaced by a constant allowlist (loggableHeaders) that redacts all values to <redacted>. I also fixed the CHANGELOG accuracy nit inline (e0c1993) — the claim about logging flows now scopes correctly to server/irmaserver/requestorserver rather than sweeping in keyshareserver/myirmaserver/tasks.

Submitted a sign-off review (GitHub 422s on APPROVE for self-authored PRs, so it's a comment). PR is deliberately left in draft — before merge, worth a quick check with maintainers that the public server.Configuration.LoggerEntry API shape aligns with what issue #232 intended.

dobby-coder Bot added a commit that referenced this pull request Jun 29, 2026
filterHeaders previously kept allowlisted header values (only CR/LF-sanitized
via SanitizeForLog). CodeQL's clear-text-logging query (go/clear-text-logging,
CWE-312) does not treat SanitizeForLog as a barrier — stripping CR/LF does not
make a value non-sensitive — so the header-value -> filterHeaders ->
fields["headers"] -> log flow stayed tainted and the check was red (confirmed by
sibling PR #597).

Stop logging any header value: allowlisted headers are now recorded with the
constant marker "<present>", every other header with "[redacted]". Logging only
constants breaks the taint flow at the source, so CWE-312 clears, while the
fail-closed allowlist still distinguishes known-safe headers from unanticipated
ones in the log. Update TestFilterHeaders/TestFilterHeadersCaseInsensitive and
the CHANGELOG to match.

Also reword the stale comment before smtp.SendMail in keyshare/email.go: the
SendMail path no longer routes through SanitizeForLog — envelope recipients come
from buildRecipients (mail.ParseAddressList) and the To: header from
sanitizeEmailHeaderValue.

Addresses review feedback on PR #586.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@rubenhensen
rubenhensen marked this pull request as ready for review July 21, 2026 12:20
@rubenhensen

Copy link
Copy Markdown
Contributor

/dobby fix the conflicts

@dobby-coder

dobby-coder Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Got it — I'm on it. I'll take a look at PR #597 and resolve the merge conflicts.

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: approve

Rules + review gatekeeper — sign-off (conflict-fix re-review)

Posted as COMMENT with an explicit VERDICT: approve because GitHub blocks APPROVE on a self-authored PR (dobby-coder[bot]). This is a sign-off, not withholding approval.

The /dobby fix the conflicts request has been handled. Re-reviewed the resolved 883d132 HEAD:

  • CHANGELOG merge resolved cleanly — master's reorganised Unreleased section is intact, with the LoggerEntry ### Added entry folded in and the security-header line updated to the allowlist+redaction wording. No conflict markers, no dropped or duplicated entries.
  • Not a no-op — the diff against base is 14 files (+275/−106); the LoggerEntry feature is fully present, so this is a real change to merge (not an empty PR to close).
  • Rule sweep clean — per-rule check over the Go/CHANGELOG-relevant rules (Go struct-equality trap, conflict-resolution no-op, correctness). resolveLogger sets both Logger and LoggerEntry non-nil on every branch and shares one underlying logger; level checks stay on *logrus.Logger while emission uses the entry; filterHeaders allowlist+redaction leaks no header value. No struct ==/!= traps (the tests' require.Same is an intentional pointer-identity assertion).
  • Review findings: none carried forward.

Conventional-commit title ✓, Closes #232 ✓. No blocking issues — signing off and flipping out of draft.

@dobby-coder

dobby-coder Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

Done — the conflicts on irmago#597 are resolved. The CHANGELOG merge is clean (master's reorganised Unreleased section is intact, the LoggerEntry entry and security-header wording are folded in correctly, no dropped entries or leftover markers), and the diff is a real 14-file change against base.

I re-ran the rule/review sweep too — nothing came up (resolveLogger and filterHeaders both check out). Posted a sign-off review comment with VERDICT: approve (GitHub won't let me self-approve) and took it out of draft, so it's ready for a maintainer to merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Configure logrus.Entry instead of Logger for better customization

2 participants