Skip to content

feat(signals): add updated_at to SignalScoutRun so post-create writes are observable #102977

Description

@andrewm4894

Problem

SignalScoutRun (products/signals/backend/models.py) has a created_at but no updated_at, yet five of its columns are written after the row is created:

Column Written by How
summary scout_harness/runner.py (_finalize_run_row, end of run) QuerySet.update(summary=...)
emitted_count, emitted_finding_ids scout_harness/tools/emit.py (_record_emit) run.save(update_fields=[...])
emitted_report_ids scout_report/persistence.py (_record_report_emit) run.save(update_fields=[...])
edited_report_ids scout_report/persistence.py (record_report_edit) run.save(update_fields=[...])
metadata scout_harness/derived_metadata.py (stamp_derived_metadata) run.save(update_fields=[...])

Nothing on the row records that any of these happened. Two consequences:

  • Any consumer that syncs the table incrementally has to key on created_at, and a created_at watermark never re-reads a row, so close-out writes that land after the row crossed the watermark are lost. The self-driving warehouse mirror hit exactly this: a material share of settled runs read with an empty summary, and the only fixes available were a re-read lookback window on one side and an hourly full refresh on the other. Both are workarounds for a missing column.
  • "When did this run last change?" has no answer on the row itself. Debugging a run that emitted late, or a metadata stamp that arrived after the summary, means correlating TaskRun.updated_at with events, because the bridge row carries no timestamp of its own.

SignalReport, SignalReportCheck, SignalScoutConfig and the other signals models with post-create writes all carry updated_at. SignalScoutRun is the exception.

Proposal

  1. Add updated_at to SignalScoutRun:

    updated_at = models.DateTimeField(auto_now=True, null=True)

    Nullable with no backfill so the AddField stays non-blocking on the populated table, the same reasoning emitted_count and emitted_report_ids carry on this model. Historical rows read NULL, which is honest: the column never observed them. Consumers that need a single timestamp use coalesce(updated_at, created_at).

  2. Make every writer advance it. auto_now only fires on Model.save() and is skipped when update_fields is passed without the field, so the field alone changes nothing today:

    • Override save() on the model to append "updated_at" to update_fields whenever a caller passes one. That covers the four save(update_fields=...) paths in one place and any future one.
    • The summary writer in runner.py uses QuerySet.update(); pass updated_at=timezone.now() alongside summary. Its comment explains why it is a targeted UPDATE, so keep that shape.
  3. Index it if the list endpoint or the harness ever filters on it. Not needed for the sync use case, which reads by watermark; leave the index out unless a query wants it.

  4. Tests. One test per write shape that proves updated_at moves: a save(update_fields=[...]) path and the .update() summary path. A test that a plain create() leaves updated_at equal to created_at within the same transaction is optional.

Out of scope

  • Changing what the row stores. TaskRun stays the source of status, timing and error; this only adds "last touched" to the bridge row.
  • Repointing the warehouse mirrors. Once the column has been live long enough to cover the rows a consumer cares about, syncs can key on updated_at and drop the lookback and full-refresh workarounds; that is a warehouse-side change and follows this.

Acceptance

  • SignalScoutRun.updated_at exists, migration is non-blocking (AddField with db_default, no backfill UPDATE).
  • Every post-create write in the table above advances updated_at, including the QuerySet.update() summary write.
  • hogli test products/signals green; the new tests fail on a model that has the field but not the save() override.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions