Webinterface - Cloudflare Workers#7
Open
koljasagorski wants to merge 28 commits into
Open
Conversation
- Wrap api_client.logout() in try/except inside a finally block so failures don't mask earlier errors and don't leak unhandled exceptions - Add InwxApiError exception and a _call() helper to deduplicate the repeated `code == 1000` API-response checks - Make file paths (domains list, CSV output, log file) and the per-call API delay configurable via environment variables; rate-limit between domain checks to avoid hammering the INWX API - Validate that the domains file exists before iterating - Unify logging: file handler + warning-level console handler, set up inside main() so importing the module has no side effects - Move logging.basicConfig out of module scope (was creating log.txt on import) and extract process_domain / print_summary helpers from main() - Bump dependencies to current versions (requests 2.32, urllib3 2.2, python-dotenv 1.0, certifi 2024.7, idna 3.7, etc.) - Move domains.txt to domains.txt.sample and gitignore the real file + generated artefacts (CSV, pycache, ruff/pytest caches) - Add pytest suite (load_domains, write_csv, _call success/failure, InwxApiError) with a stubbed INWX SDK so tests run without network - Add ruff + pytest configuration in pyproject.toml and a GitHub Actions CI workflow running on Python 3.10/3.11/3.12 - Flesh out README: start command, configuration table, dev setup https://claude.ai/code/session_01KJj6SRqa7UBbtCpReVjaeB
…tion-Z8Q8M Refactor INWX bot: robustness, configurability, tests + CI
Adds a TypeScript Cloudflare Worker variant of the INWX domain bot under cloudflare-worker/, alongside the existing Python CLI script. - Talks to the INWX JSON-RPC API directly via fetch (the requests-based Python SDK does not run on Workers); session cookie handled manually. - Runs on a Cron Trigger; domain list and results stored in Workers KV. - DRY_RUN defaults to "true" so it never registers domains until enabled. - Token-protected HTTP endpoints to manage the list, trigger runs, and download JSON/CSV results. - Optional 2FA (mobile TAN) via Web Crypto TOTP, verified against the RFC 6238 test vectors. - Optional Slack/Discord webhook notifications for noteworthy runs. - CI job added to typecheck the Worker; readme updated to describe both deployment options. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Add Cloudflare Worker deployment (scheduled, serverless)
Serves a self-contained dashboard at `/` that shows run status, per-domain results, lets you edit the domain list, trigger a check, and download the CSV. Data endpoints are reorganised under `/api/*`: - GET / -> dashboard HTML (strict nonce-based CSP, no external assets) - GET /api/status -> public status (dry-run mode, last run, last error) - GET/PUT /api/domains -> read / replace the domain list (token protected) - POST /api/run -> trigger a run (token protected; ?async=true supported) - GET /api/results.json|.csv -> last run results (token protected) The dashboard prompts for ADMIN_TOKEN (kept in sessionStorage) and renders all external data via safe DOM APIs (no innerHTML) to avoid XSS. README updated. Verified: tsc typecheck, client-script syntax check, and end-to-end requests against `wrangler dev` (HTML + CSP header, status, 401 without token, domain save/list, run error path, CSV, 404). https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Add web dashboard to the Cloudflare Worker
For every watched domain the bot now gathers registration metadata (registered / expires / last changed / status / registrar) and shows it in a new dashboard table. Expiry within 30 days is highlighted. Data sources (per domain, best first): - INWX `domain.info` for domains in the account — authoritative dates for any TLD (including .de). - RDAP (the JSON successor to WHOIS) for everything else. An explicit User-Agent is sent because some RDAP servers reject UA-less requests (403). Unregistered domains (RDAP 404) are reported as available. New endpoints (token protected): `GET /api/whois` and `POST /api/whois/refresh` (with `?async=true`). The data is stored in KV (`whois:latest.json`), refreshed on each cron run and via a dashboard button. `GET /api/status` now also reports `whoisLastRun`. Verified: tsc, client-script syntax/structure checks, and end-to-end against `wrangler dev` — RDAP returns real registration/expiry/updated data for a registered domain and flags an unregistered one as available. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
The committed wrangler.toml still carried the REPLACE_WITH_YOUR_KV_NAMESPACE_ID placeholder, which made the Cloudflare Workers Build (Git integration) fail with "KV namespace ... is not valid". Pin the real namespace id (an identifier, not a secret) so automatic deploys work. Actual secrets stay out of the repo. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Add WHOIS / domain-status table to the dashboard
Monitoring smarts for the Worker (KV-based, no new infrastructure): - Run history: each run is appended to a capped list (history:runs, last 50) and shown in a new dashboard "Verlauf" panel. New GET /api/history endpoint. - Per-domain state (state:domains) tracks the last action so notifications fire only when a domain's status actually CHANGES (newly available / bought / failed), instead of on every run. Fatal run errors are de-duplicated too. - Expiry alerts: as owned/registered domains approach renewal, a webhook alert is sent once per crossed threshold (EXPIRY_ALERT_DAYS, default 30,14,7,1), reset when the expiry date changes. - Best-effort KV lock prevents a manual run/whois-refresh from overlapping the scheduled run; /api/run and /api/whois/refresh return 409 when busy. Verified: tsc; unit tests of change-detection / threshold parsing / expiry crossing; and end-to-end against `wrangler dev` with a webhook capture — confirmed error-notify dedup (1 alert for 2 failing runs), an expiry alert firing once and de-duplicating on repeat, and history growing correctly. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Monitoring smarts: run history, change-based + expiry alerts, run lock
Domain entries are now per-domain config objects ({domain, mode, maxPrice?,
tags?, notes?}) instead of plain strings (plain strings / newline lists stay
supported and default to mode "auto"):
- mode "watch" only reports availability; "auto" registers when free unless
DRY_RUN is on.
- maxPrice skips an auto-purchase when the INWX price exceeds the budget;
domain.check now also returns the price (best-effort). When a price cannot be
determined and a budget is set, the purchase is skipped (safe default).
- POST /api/buy registers one available domain on explicit request; it ignores
DRY_RUN and the watch/auto mode (explicit user action) and is lock-guarded.
Dashboard: the domain list is now an editable table (domain / mode / max price /
tags), and the results table has a per-row "Kaufen" button (with confirmation)
for available domains.
Verified: tsc; dashboard client-script syntax/structure checks; unit tests of
every processDomain decision branch (never buys on watch / dry-run / over-budget
/ unknown-price-with-budget; buys otherwise); and end-to-end against
`wrangler dev` — domain-config round-trip incl. backward-compatible string/text
input, and /api/buy validation (400 on empty, graceful error without creds).
https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Per-domain modes (watch/auto), budget guard, and on-demand buy
Quality and operations hardening for the Worker: - Unit tests (vitest): 22 tests covering domain-config parsing, CSV escaping, run-change detection, expiry thresholds, RDAP parsing, and the RFC 6238 TOTP vectors. Pure helpers are re-exported from index.ts for testing. The CI "worker" job now runs `npm test` after the typecheck. - Wrangler bumped to v4 (removes the out-of-date warning; matches the Cloudflare Workers Build environment). Config is unchanged and verified via dev + deploy dry-run. - Telegram notifications: set TELEGRAM_BOT_TOKEN + TELEGRAM_CHAT_ID to receive alerts in Telegram in addition to (or instead of) the webhook. Notifications now fan out to every configured channel. - Security headers: dashboard CSP gains `frame-ancestors 'none'`, and responses send `X-Content-Type-Options: nosniff` (+ `Referrer-Policy: no-referrer` on the dashboard). - Dependabot config for npm (cloudflare-worker), pip, and github-actions. Verified: `npm ci` + typecheck + 22 tests green; wrangler v4 dev serves the dashboard with the new headers; notification fan-out confirmed against a local webhook receiver (with correct de-duplication). https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Quality & ops: unit tests + CI, Wrangler v4, Telegram, security headers, Dependabot
Final batch of the dashboard/ops UX:
- Runtime settings: dryRun / apiDelayMs / expiryAlertDays can be overridden via
KV (settings:config) and edited in a dashboard panel — no redeploy needed.
wrangler.toml [vars] remain the safe defaults; turning dry-run off prompts for
confirmation. New GET/PUT /api/settings; the run, whois and status paths now
read the effective settings.
- Ad-hoc quick check: POST /api/check ({domain} or {keyword,tlds[]}) and a
dashboard form report availability + price without touching the watch list.
- Audit log: token-authenticated mutating actions (run, buy, domains.save,
whois.refresh, settings.save, check) are recorded to a capped KV list
(audit:log), exposed via GET /api/audit and shown in the dashboard.
- Login throttling: protected endpoints count failed token attempts per client
IP and return 429 after a threshold (5-minute window); success clears the
counter. Fail-open on KV errors.
Verified: tsc; 29 unit tests (incl. new mergeSettings / expandCheckTargets);
dashboard syntax/structure checks; and end-to-end against `wrangler dev` —
settings override flips DRY_RUN at runtime (reflected in /api/status), audit log
records actions with IP, /api/check validates input + degrades without creds,
and throttling returns 429 after 10 bad attempts (other IPs unaffected).
https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Settings UI, ad-hoc check, audit log, login throttling
Auto-sync per always-keep-GitHub-in-sync policy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Polish and correctness fixes: - Manual buy now: confirmation dialog shows the price, the result row flips to "purchased" immediately, a notification is sent, and the stored results/state are updated so a reload stays consistent (previously a manual buy was silent and the table kept showing "available"). - domain.check's price is surfaced on every DomainStatus and shown in a new "Preis" column. - The per-domain "notes" field is now editable in the dashboard table (it existed in the model but was silently dropped by the UI). - Settings can be reset to the wrangler.toml defaults (DELETE /api/settings + dashboard button), and the panel warns that overrides persist across deploys. - Lock TTL raised 600s -> 1800s so a long run does not lose the lock mid-flight. - Stop tracking the accidentally committed `.wrangler/` cache (account info / build cache) and gitignore `.wrangler/` + `node_modules/` at the repo root. Verified: tsc; 29 unit tests; dashboard syntax/structure checks; and end-to-end against `wrangler dev` — notes round-trip, settings override + reset (status reflects the change and falls back to the env default), and audit logging. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Buy UX (price + notify + refresh), notes field, settings reset, cache cleanup
…rtbeat Addresses the subrequest-limit ceiling and registration/liveness gaps: - Availability is now checked in batches via a single domain.check call per ~30 domains (InwxClient.checkDomains) instead of one call per domain, so the run path stays well under Cloudflare's per-invocation subrequest limit. The ad-hoc /api/check uses the same batched path. - The cron is split into two triggers — one for the availability run (06:00), one for the WHOIS refresh (06:30) — so each invocation gets its own subrequest budget. The scheduled handler routes by event.cron (unknown cron values run both as a fallback). wrangler.toml ships both crons. - Registration options renewalMode / period / transferLock are applied to domain.create (defaults: AUTORENEW + transfer-locked), configurable via env vars, the dashboard settings panel, or PUT /api/settings. - Optional HEARTBEAT_URL is pinged after every cron invocation as a dead-man's-switch so a stalled cron is detected externally. - Lock TTL already raised in the previous PR; this keeps long batched runs safe. Verified: tsc; 32 unit tests (added chunk + registration-merge coverage); dashboard syntax/structure checks; and end-to-end against `wrangler dev --test-scheduled` — the run cron updates lastRun, the WHOIS cron updates whoisLastRun, each invocation pings the heartbeat, and the registration options round-trip through /api/settings. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Scale runs: batch domain.check, split cron, registration options, heartbeat
Channels and comfort improvements: - Email notifications via Resend: set RESEND_API_KEY + EMAIL_TO + EMAIL_FROM to also receive alerts by email. Notifications now fan out to every configured channel (webhook + Telegram + email), gated by a single hasNotifyChannel check. - Backup: GET /api/export downloads a JSON bundle of the domain list + settings override (with a Content-Disposition filename); POST /api/import restores it. A "Backup herunterladen" button is added to the dashboard; imports are audited. - Dashboard comfort: the results and WHOIS tables get a live filter input and click-to-sort column headers (numeric-aware), re-applied after each refresh. Verified: tsc; 32 unit tests; dashboard syntax/structure checks; and end-to-end against `wrangler dev` — export returns the bundle (+ download header), import restores domains and settings (confirmed via the API), and the action is audited. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Email notifications, backup export/import, table filter + sort
For ccTLDs that rdap.org doesn't cover (e.g. .de), RDAP returns 404 — which the WHOIS table previously misread as "available" for registered domains. Now: - For such ccTLDs the table reports availability as *unknown* instead of falsely "available" (RDAP stays authoritative for gTLDs). - Optional port-43 WHOIS fallback (WHOIS_PORT43="true", off by default) queries the registry over TCP (cloudflare:sockets) for the few mapped ccTLDs, parses registered/expires/updated/status, and detects "free" responses. It is time-boxed (5 s, with background socket close) so a blocked/non-responsive server can never stall the refresh — a hang found and fixed during testing. DENIC still never publishes registration/expiry; this mainly fills status + last-change, and many registries block datacenter IPs (then it yields nothing). Verified: tsc; 35 unit tests (added parseWhoisText for gTLD + DENIC formats and whoisServerFor); and end-to-end against `wrangler dev` — default keeps .de fast and "unknown" (not "available"), and with the flag on the refresh returns in ~5s (bounded by the timeout) instead of hanging. https://claude.ai/code/session_01KH9YazP7LmiLgvbA3NhW6Y
Opt-in port-43 WHOIS fallback; fix .de false-"available"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.