Skip to content

feat(privacy): add watcher-side privacy filter (drop/redact before send) - #135

Merged
ErikBjare merged 6 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/privacy-filter-watcher-side
Oct 14, 2026
Merged

ErikBjare merged 6 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/privacy-filter-watcher-side

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Summary

Adds a client-side privacy filter that drops or redacts window events before they are sent to aw-server, so sensitive data never leaves the machine at all.

This mirrors the server-side privacy_filters engine that landed in aw-server-rust#600, allowing users to enforce the same rules at both layers. Previously the watcher only supported dropping titles via exclude_title/exclude_titles (redact to "excluded"). This PR adds explicit drop and redact actions with user-configurable regex patterns and replacement strings.

What changed

  • aw_watcher_window/privacy_filter.py (new) — pure Python, no new deps:
    • compile_privacy_rules(raw) — validates and compiles regex patterns; invalid regexes and unknown actions are skipped with an error log rather than crashing
    • apply_privacy_filters(window, rules) — returns None (drop) or a filtered copy (redact); never mutates the input dict
  • aw_watcher_window/config.py — exposes privacy_filter_rules from [[aw-watcher-window.privacy_filter]] TOML array of tables
  • aw_watcher_window/main.py — privacy filter runs first in transform_window; a None return skips the heartbeat; heartbeat_loop threads the rules through
  • tests/test_privacy_filter.py (new) — 23 tests covering compile, drop, redact, input immutability, edge cases
  • README.md — documents the new config section with examples

Config example

# Drop private-browsing windows entirely
[[aw-watcher-window.privacy_filter]]
pattern = "(?i)private browsing|incognito"
action  = "drop"

# Redact banking titles
[[aw-watcher-window.privacy_filter]]
pattern     = "(?i)bank|my account"
action      = "redact"
replacement = "REDACTED"   # optional; defaults to "excluded"

# Drop by app name (not title)
[[aw-watcher-window.privacy_filter]]
pattern = "(?i)signal"
field   = "app"
action  = "drop"

Rule fields: pattern (Python regex), field (default "title"), action ("drop" or "redact"), replacement (for redact; default "excluded").

Ordering

Privacy filter runs before Research Edition and exclude_title transforms. A "drop" rule exits immediately — subsequent rules are not evaluated for that event.

macOS note

The default swift strategy bypasses this Python transform (same as Research Edition). Use --strategy jxa or --strategy applescript to enable watcher-side privacy filtering on macOS.

Test plan

  • python3 -m pytest tests/test_privacy_filter.py -v — 23 passed
  • python3 -m pytest tests/test_main.py -v — 12 passed (no regressions)
  • Normal run with no config: zero behaviour change
  • drop rule matching title: heartbeat skipped
  • redact rule matching title: heartbeat sent with replacement value
  • Invalid regex in rule: rule skipped with error log, watcher continues

@greptile-apps

greptile-apps Bot commented Jul 31, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

Adds watcher-side privacy filtering before heartbeat transmission.

  • Compiles configured drop and redact rules while logging and skipping malformed rules.
  • Applies privacy filtering before existing window transforms and breaks heartbeat chains when events are dropped.
  • Refuses to use the incompatible macOS Swift strategy when privacy filtering is configured.
  • Documents the configuration and adds focused configuration, filtering, startup, and integration tests.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Read window event] --> B[Apply privacy-filter rules]
  B -->|Drop match| C[Skip heartbeat and break chain]
  B -->|Redact or no match| D[Apply app exclusion]
  D -->|Excluded| C
  D -->|Allowed| E[Apply remaining transforms]
  E --> F[Send heartbeat]
Loading

Reviews (3) · Last reviewed commit: "fix(tests): add missing exclude_apps and..." · Reviewed by Greptile

Comment thread aw_watcher_window/privacy_filter.py Outdated
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

@greptileai review

@ErikBjare

Copy link
Copy Markdown
Member

@TimeToBuildBob The watcher-side privacy filter should also be configurable via webui (non-technical user-facing central config) similar to the server-side privacy filter (but watcher-only config should also be supported, if e.g. server is untrusted).

@TimeToBuildBob
TimeToBuildBob force-pushed the feat/privacy-filter-watcher-side branch from cdf5c3b to 9176770 Compare August 17, 2026 18:33
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Good point. The TOML config in this PR covers the watcher-only path you described (useful when the server is untrusted or the user wants filtering before data ever leaves the machine).

For webui configurability, the shape I'd expect is:

  • aw-server exposes a config endpoint for watcher-side privacy rules (similar to how server-side privacy_filters are managed)
  • aw-webui adds a settings panel for configuring those rules
  • The watcher fetches its config from the server when available, falling back to TOML when disconnected or server is untrusted

That touches aw-server and aw-webui (separate repos), so it's naturally a follow-up rather than something that fits cleanly into this PR. Would you prefer I scope it here (which would expand the PR significantly) or open a follow-up issue to track it?

Also resolved the rebase conflict with master (#136, #137) — the branch is up to date now.

TimeToBuildBob added a commit to TimeToBuildBob/aw-watcher-window that referenced this pull request Aug 27, 2026
codeql-action v2 is deprecated as of 2025-01-10 (GitHub changelog). Update all
three action steps (init, autobuild, analyze) to v3. Also bump actions/checkout
from v3 to v4 while here.

This fixes the failing 'Analyze (python)' CI check on PR ActivityWatch#135.
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Still waiting for direction on scope — whether to include the webui-configurable path in this PR or track it as a follow-up. The watcher-only TOML config path is ready, CI green, and rebased. Happy to expand or split depending on your preference.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

CI-green and mergeable (Greptile 5/5) — waiting only on a maintainer click.

This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted.

@TimeToBuildBob

TimeToBuildBob commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor Author

🤖 AI code review

Safe to merge — 3 findings disposed (fixed in 1d68a6b; rejected)

Updated after inline dispositions on finding threads — this is the current state; the verdict below is frozen at review time and is kept as the historical record of that pass.

Finding disposition
Finding Severity State
aw_watcher_window/main.py:102 P1 fixed in 1d68a6b
aw_watcher_window/main.py:224 P1 rejected — The verification said: "the exclude_titles loop at main.py line 235-237 mutates the original current_window, but the fun
aw_watcher_window/main.py:103 P1 superseded by latest review (not reproduced)
aw_watcher_window/main.py:460 P1 rejected — Reviewer traces: 'privacy rule redacts app field, exclude_apps sees redacted value.' The code trace is correct, but the
aw_watcher_window/main.py:210 P1 resolved (no disposition on record)
aw_watcher_window/main.py:102 P2 rejected

Adds a watcher-side privacy filter that drops or redacts window events before they are sent to aw-server. New privacy_filter.py compiles and applies regex rules; config.py reads [[aw-watcher-window.privacy_filter]] TOML tables; main.py runs the filter first in transform_window and refuses to start under the macOS swift strategy when rules are configured. Adds tests for the filter, config parsing, and swift refusal, plus a README section and CodeQL action version bumps.

Not safe to merge — 2 P1 open

Confidence 2/5

2 findings · ❌ 2 P1 · 🔒 2 security

❌ P1 high · 🔒 security — aw_watcher_window/main.py:460

In transform_window, when privacy_filter_rules is non-empty, apply_privacy_filters returns a shallow copy of the window dict. The subsequent exclude_titles loop mutates that copy in place (current_window["title"] = "excluded"). This is fine for the normal path, but when research_category_map is not None, research_transform is called on the copy and the exclude_titles loop is skipped. The privacy filter's redaction already happened on the copy, so that is preserved. However, the real defect is in the interaction with exclude_apps: the privacy filter copy is made, then exclude_apps checks pattern.search(current_window.get("app", "")) on the copy. That is correct. The actual bug is that apply_privacy_filters returns the original window object when rules is empty (line 110: return window), but when rules is non-empty it returns a copy. In transform_window, when privacy_filter_rules is non-empty but no rule matches, the copy is used and exclude_titles mutates the copy. When privacy_filter_rules is empty, exclude_titles mutates the original current_window dict, which is the same object passed in. This inconsistency is not itself a bug because the caller does not reuse the dict. The real issue is that the privacy filter's redact action replaces the entire field value with the replacement string, but subsequent rules in the same apply_privacy_filters call operate on the redacted value. This is documented and tested. A more serious issue: the redact action replaces the field value, but if a later rule with field="title" and action="drop" matches the replacement string, the event is dropped. That is intended per the test. The actual defect I can trace: in main.py, the swift refusal gate checks raw_privacy_filter_rules, which is the list from config before compilation. If the user configures a rule with an invalid regex, compile_privacy_rules skips it, but raw_privacy_filter_rules is still non-empty, so the watcher refuses to start under swift. This is intentional fail-closed behavior. However, on non-swift strategies, an invalid rule is silently skipped, and the user's intent to filter is not honored — the watcher starts and sends events that the user wanted filtered. The README says invalid rules are skipped with an error log, but the user may not notice. This is a trade-off, not a bug. The real bug I found: in apply_privacy_filters, when a redact rule matches, it sets result[field] = rule["replacement"]. If the same field is matched by a later rule, the later rule sees the replacement value, not the original. This is intended per the test test_multiple_rules_redact_then_drop. But consider a drop rule that comes after a redact rule on a different field: the redact rule changes the title, and the drop rule checks the app field — no interaction. The actual defect is that the privacy filter runs before exclude_apps, but exclude_apps checks the app field on the copy. If a privacy rule redacts the app field (field="app", action="redact"), then exclude_apps sees the redacted app name, not the original. This could cause an app that should be excluded to pass through if the redaction removes the pattern. For example, exclude_apps=[re.compile("1Password")] and a privacy rule redacts app field to "REDACTED" when it matches "bank". If the app is "1Password Bank", the privacy rule redacts it to "REDACTED", and exclude_apps no longer matches, so the event is sent. This is a privacy leak: the user's exclude_apps guarantee is bypassed by the privacy filter's redaction of the app field. The ordering comment says privacy filter runs first so sensitive events never reach other transforms, but it also means exclude_apps runs on a potentially redacted app name. The fix is to run exclude_apps before the privacy filter, or to have the privacy filter not redact the app field in a way that hides the app name from exclude_apps. This is a real correctness bug in the privacy guarantee.

Move the exclude_apps loop before the privacy filter block, or have apply_privacy_filters skip redaction of the 'app' field when exclude_apps is configured.

How this was verified: Traced transform_window: privacy filter runs at line 461-464, then exclude_apps at line 468-470. If a privacy rule redacts the 'app' field, exclude_apps sees the redacted value. Example: exclude_apps=[re.compile('1Password')], privacy rule {field:'app', pattern:'bank', action:'redact', replacement:'REDACTED'}, window app='1Password Bank' -> privacy redacts app to 'REDACTED', exclude_apps pattern.search('REDACTED') is None, event is sent. This violates the documented exclude_apps privacy guarantee.

Consensus: 2/3 passes agreed — pass 2 looked and disagreed
Distinct keys: 1 (general)

❌ P1 high · 🔒 security — aw_watcher_window/main.py:210

The swift strategy refusal gate in main.py checks raw_privacy_filter_rules (the un-compiled list from config) and exits if it is truthy. However, the config parser in config.py always sets privacy_filter_rules to a list, defaulting to [] when the key is absent. The gate is placed after compile_privacy_rules, but it uses the raw list. This is intentional per the comment. The issue: if a user configures a single [aw-watcher-window.privacy_filter] table (not array of tables), config.py wraps it into a list, so the gate works. If the user configures an empty array privacy_filter = [], the gate does not fire, which is correct. If the user configures a non-table value like a string, config.py does list(privacy_filter_cfg or []) which would turn the string into a list of characters, making raw_privacy_filter_rules truthy and causing swift to refuse to start even though no valid rules exist. This is a minor fail-closed behavior, not a leak. More importantly, the gate only checks raw_privacy_filter_rules truthiness, not whether any rule actually compiled. If all rules are invalid, the watcher refuses to start under swift, which is fail-closed and intended. The defect: on non-macOS platforms or with jxa/applescript, invalid rules are silently skipped and the watcher runs without any privacy filtering, potentially leaking sensitive data the user intended to filter. The user gets only an error log per invalid rule, but the watcher continues. This is a fail-open behavior for invalid regexes on the Python path. The PR description says invalid regexes are skipped with an error log and the watcher continues, which is a deliberate design choice, but it means a typo in a pattern silently disables privacy protection. This is a security-relevant issue: a user who configures a drop rule for 'incognito' but mistypes the regex will have private browsing windows sent to the server. The severity is P1 because it defeats the privacy feature on a configuration error, and the user may not notice the error log.

Consider failing closed when any configured rule is invalid, or at least logging a prominent warning that privacy filtering is disabled due to invalid rules.

How this was verified: Checked compile_privacy_rules: invalid regexes are skipped with logger.error and the rule is not added to compiled. In main.py, privacy_filter_rules is compiled and passed to heartbeat_loop; if all rules are invalid, the list is empty and no filtering occurs. The swift gate uses raw rules, so it fails closed on macOS, but on Linux/Windows the watcher runs unfiltered.

Consensus: 2/3 passes agreed — pass 2 looked and disagreed
Distinct keys: 1 (general)

Files changed (9) — the diff as I read it
  • .github/workflows/codeql.yml — Bumps actions/checkout from v3 to v4 and codeql-action from v2 to v3.
  • README.md — Documents the new privacy_filter config section with drop/redact examples and a macOS swift-strategy warning.
  • aw_watcher_window/config.py — Adds parsing of privacy_filter config into privacy_filter_rules, wrapping a single TOML table into a list.
  • aw_watcher_window/main.py — Compiles privacy rules, gates swift startup on configured rules, threads rules through heartbeat_loop, and applies them first in transform_window.
  • aw_watcher_window/privacy_filter.py — New module with compile_privacy_rules and apply_privacy_filters for drop/redact regex rules.
  • tests/test_config.py — Adds tests for single-table and array privacy_filter config parsing.
  • tests/test_main.py — Adds tests for privacy filter interaction with exclude_titles and swift refusal to start.
  • tests/test_privacy_filter.py — New test file covering compile and apply behavior of the privacy filter.
  • tests/test_startup.py — Adds exclude_apps and privacy_filter_rules to the mocked parse_args namespace.
Previous review passes
commit score findings engine when
1bdacd80303b 2/5 2 llm 2026-09-16 08:52 UTC
1d68a6b12d6d 4/5 1 llm 2026-09-16 10:19 UTC
1d68a6b12d6d 3/5 1 llm 2026-10-09 02:44 UTC
ef4ac06d328f 5/5 0 llm 2026-10-09 04:04 UTC
ef4ac06d328f 5/5 0 llm 2026-10-09 05:33 UTC

Reviewed cd068ef4b51c · openrouter/deepseek/deepseek-v4-flash-0731 · llm engine · 235s · about this reviewer

Maintainer commands

@TimeToBuildBob review (own line) — fresh review · @TimeToBuildBob fix — a worker acts on the findings. Once per comment; 👀 = received.

Comment thread aw_watcher_window/main.py
Comment thread aw_watcher_window/main.py
Comment thread aw_watcher_window/main.py
Comment thread aw_watcher_window/main.py Outdated
TimeToBuildBob and others added 5 commits October 14, 2026 11:13
Adds a client-side privacy filter that drops or redacts window events
before they are sent to aw-server, so sensitive data never leaves the
machine at all.

New module `aw_watcher_window/privacy_filter.py`:
- `compile_privacy_rules(raw)` — validates and compiles regex patterns;
  invalid regexes and unknown actions are skipped with an error log
- `apply_privacy_filters(window, rules)` — returns None (drop) or a
  filtered copy (redact); never mutates the input dict

Config via `[[aw-watcher-window.privacy_filter]]` TOML tables:
  pattern     = "(?i)private browsing|incognito"
  action      = "drop"           # or "redact"
  field       = "title"          # optional; defaults to "title"
  replacement = "excluded"       # optional; used for redact action

Integration in main.py:
- Privacy filter runs before research and exclude_title transforms
- A None return from transform_window skips the heartbeat entirely
- heartbeat_loop passes privacy_filter_rules through

23 new tests in tests/test_privacy_filter.py; all existing tests pass.

Mirrors the server-side privacy_filters engine in aw-server-rust (#600)
so users can enforce the same rules at both the watcher and server layers.

macOS note: the swift strategy bypasses this Python transform; use
--strategy jxa or --strategy applescript to enable it on macOS.

Git-Session-Id: c4abdfa3-ec34-58ee-9600-b997fa16fbe8
Git-Session-Id: c4abdfa3-ec34-58ee-9600-b997fa16fbe8
codeql-action v2 is deprecated as of 2025-01-10 (GitHub changelog). Update all
three action steps (init, autobuild, analyze) to v3. Also bump actions/checkout
from v3 to v4 while here.

This fixes the failing 'Analyze (python)' CI check on PR ActivityWatch#135.

Git-Session-Id: c4abdfa3-ec34-58ee-9600-b997fa16fbe8
The macOS swift strategy sends heartbeats from a separate binary and
cannot apply privacy_filter rules. Refuse to start when those rules
are configured so titles are not leaked; use jxa or applescript.

Also wrap a single privacy_filter table (list(mapping) was silently
dropping the rule) and skip non-string field/replacement values.

Git-Session-Id: 106b67f2-1ed0-575b-af3a-59ab2170164e
…ompile

Git-Session-Id: 62d4746f-0cba-5722-ab4e-4aced59d086d
@TimeToBuildBob
TimeToBuildBob force-pushed the feat/privacy-filter-watcher-side branch from ef4ac06 to 070f15a Compare October 14, 2026 11:14
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Rebased onto master to resolve conflicts with #156 (exclude_apps, just merged).

Conflict resolution in main.py:

  • Privacy filter runs first (before exclude_apps), consistent with the PR's "filter before any other transform" design
  • exclude_apps still runs after, as a second privacy guarantee
  • Both use the _chain_broken_by_exclusion pulsetime=0 mechanism so the server can't extend an event across an excluded window

Also merged the new exclude_apps integration tests from #156 with the privacy filter tests — 66 passed, 1 skipped locally.

CI running on the new head.

…tup fixture

The parse_args mock in test_startup.py was missing two attributes that
main.py now accesses: args.exclude_apps (added with the --exclude-apps
flag) and args.privacy_filter_rules (added by the watcher-side privacy
filter). The missing exclude_apps caused an AttributeError that prevented
the watcher thread from starting, failing both startup integration tests.

privacy_filter_rules is already guarded with getattr() in main.py so it
wouldn't crash, but including it makes the fixture match the real args
object and avoids future surprises.

Git-Session-Id: a2106031-86a2-4f5c-b8dc-39b2cb86b17f
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

The startup-test failure is fixed in cd068ef: the mocked arguments were missing exclude_apps, causing the watcher thread to crash before recording started. The fixture now includes it and privacy_filter_rules.

All six checks on that head are now green, including Linux, Windows, macOS, and typecheck: https://github.com/ActivityWatch/aw-watcher-window/actions/runs/38556886980 . No additional code change or rerun was needed for this recovery dispatch.

@ErikBjare

Copy link
Copy Markdown
Member

This feels like we're duplicating or re-implementing the feature. It also doesn't address the fact that we want most user settings like this on the server. The only problem with these privacy filter settings is that they are themselves sensitive, at least if they are user-configurable and not picked from some public list. We've discussed this at length before in other issues. I've left comments and such about the design of the whole thing with these constraints. Also, not sure if that privacy filter engine that we have was even built into the Python server for parity.

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Valid concerns, and worth a direct answer on each.

Duplication: The server-side privacy filter (aw-server-rust) filters data after it arrives at the server. That's fine when the user owns and trusts the server. This PR targets the different case: untrusted server (employer-managed, shared, or a third-party host). In that case, data must be filtered before it ever leaves the machine — server-side filtering can't help there. That's the only reason to put it in the watcher.

Sensitivity of filter rules: This is the strongest argument for local TOML config, not against — if the rules are sensitive, they should never reach the server at all. A server-side setting API would expose them on the wire.

Server parity / duplication maintenance: Fair. If the Python server doesn't have a working privacy filter either, that's a gap. I don't know the current state there.

What I'd suggest: If the untrusted-server scenario is in scope for AW (and it seems like it should be, given the privacy-first ethos), this PR's approach is the right level to implement it. If it's not in scope, or if you'd prefer a different design (e.g., a standalone privacy proxy layer), I'm happy to close this and open a design issue instead.

One-word answer from you on direction would help: close (not the right design) vs. continue (this approach is fine, merge when ready).

Comment thread aw_watcher_window/main.py
Comment thread aw_watcher_window/main.py
@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Closing based on Erik's design feedback (2026-10-14): the watcher-side drop/redact approach duplicates what should live on the server, and user-configurable privacy rules are themselves sensitive. The right design is server-side filtering, per the existing discussion in related issues.

@ErikBjare ErikBjare reopened this Oct 14, 2026
@ErikBjare
ErikBjare merged commit d5b9c9c into ActivityWatch:master Oct 14, 2026
12 checks passed
@ErikBjare

Copy link
Copy Markdown
Member

@TimeToBuildBob Reopened and merged this after all, I think it's a good feature for the reasons you described, even if less used. However I want follow-up PRs improving the docstrings etc cross-referencing the other privacy filter approaches etc. Also should check that parity state in aw-server.

ErikBjare pushed a commit that referenced this pull request Oct 14, 2026
codeql-action v2 is deprecated as of 2025-01-10 (GitHub changelog). Update all
three action steps (init, autobuild, analyze) to v3. Also bump actions/checkout
from v3 to v4 while here.

This fixes the failing 'Analyze (python)' CI check on PR #135.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants