Skip to content

Bound GEDCOM parsing and isolate large-file workers #489

Description

@sodejm

Parent: #118
Origin: follow-up to the bounded GEDCOM intake shipped by PR #485.

Objective

Move large GEDCOM parsing and root-candidate discovery out of the Electron/FastAPI control process into a bounded worker process. The current 250,000-person admission limit remains the supported safety boundary until this issue's evidence passes.

Scope

  • Add a distinct process executor for GEDCOM inspection and candidate discovery.
  • Enforce byte, physical-line, record, nesting, xref, wall-time, memory, output, and concurrency limits before resource exhaustion.
  • Add cooperative cancellation plus watchdog termination for a stuck or over-budget worker.
  • Preserve the immutable source file and clean private staging after success, failure, cancellation, timeout, crash, or restart.
  • Return transport-neutral worker state and stable coded resource-limit errors without exposing records or scratch paths.

Acceptance criteria

  • Accepted boundary-size fictional files complete without blocking the Electron main process, renderer, or API control process.
  • Every one-over limit case fails deterministically with a documented stable code.
  • Crash, OOM, timeout, and cancellation injection leave inputs and existing outputs intact and clean staging on restart.
  • Provider none remains network-free and no GEDCOM payload, source path, or scratch path enters logs or telemetry.
  • Focused tests and canonical make test, make lint, make typecheck, make security, and desktop gates pass.
  • Architecture, threat-model, API, operational, user, and release documentation are updated or explicitly documented as unaffected.

Release placement

Milestone: 0.7.0 Genealogy Workflows. This is the first dependency in the large-file release sequence and blocks the disk-backed index work.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions