Problem
SignalScoutRun (products/signals/backend/models.py) has a created_at but no updated_at, yet five of its columns are written after the row is created:
| Column |
Written by |
How |
summary |
scout_harness/runner.py (_finalize_run_row, end of run) |
QuerySet.update(summary=...) |
emitted_count, emitted_finding_ids |
scout_harness/tools/emit.py (_record_emit) |
run.save(update_fields=[...]) |
emitted_report_ids |
scout_report/persistence.py (_record_report_emit) |
run.save(update_fields=[...]) |
edited_report_ids |
scout_report/persistence.py (record_report_edit) |
run.save(update_fields=[...]) |
metadata |
scout_harness/derived_metadata.py (stamp_derived_metadata) |
run.save(update_fields=[...]) |
Nothing on the row records that any of these happened. Two consequences:
- Any consumer that syncs the table incrementally has to key on
created_at, and a created_at watermark never re-reads a row, so close-out writes that land after the row crossed the watermark are lost. The self-driving warehouse mirror hit exactly this: a material share of settled runs read with an empty summary, and the only fixes available were a re-read lookback window on one side and an hourly full refresh on the other. Both are workarounds for a missing column.
- "When did this run last change?" has no answer on the row itself. Debugging a run that emitted late, or a metadata stamp that arrived after the summary, means correlating
TaskRun.updated_at with events, because the bridge row carries no timestamp of its own.
SignalReport, SignalReportCheck, SignalScoutConfig and the other signals models with post-create writes all carry updated_at. SignalScoutRun is the exception.
Proposal
-
Add updated_at to SignalScoutRun:
updated_at = models.DateTimeField(auto_now=True, null=True)
Nullable with no backfill so the AddField stays non-blocking on the populated table, the same reasoning emitted_count and emitted_report_ids carry on this model. Historical rows read NULL, which is honest: the column never observed them. Consumers that need a single timestamp use coalesce(updated_at, created_at).
-
Make every writer advance it. auto_now only fires on Model.save() and is skipped when update_fields is passed without the field, so the field alone changes nothing today:
- Override
save() on the model to append "updated_at" to update_fields whenever a caller passes one. That covers the four save(update_fields=...) paths in one place and any future one.
- The summary writer in
runner.py uses QuerySet.update(); pass updated_at=timezone.now() alongside summary. Its comment explains why it is a targeted UPDATE, so keep that shape.
-
Index it if the list endpoint or the harness ever filters on it. Not needed for the sync use case, which reads by watermark; leave the index out unless a query wants it.
-
Tests. One test per write shape that proves updated_at moves: a save(update_fields=[...]) path and the .update() summary path. A test that a plain create() leaves updated_at equal to created_at within the same transaction is optional.
Out of scope
- Changing what the row stores.
TaskRun stays the source of status, timing and error; this only adds "last touched" to the bridge row.
- Repointing the warehouse mirrors. Once the column has been live long enough to cover the rows a consumer cares about, syncs can key on
updated_at and drop the lookback and full-refresh workarounds; that is a warehouse-side change and follows this.
Acceptance
SignalScoutRun.updated_at exists, migration is non-blocking (AddField with db_default, no backfill UPDATE).
- Every post-create write in the table above advances
updated_at, including the QuerySet.update() summary write.
hogli test products/signals green; the new tests fail on a model that has the field but not the save() override.
Problem
SignalScoutRun(products/signals/backend/models.py) has acreated_atbut noupdated_at, yet five of its columns are written after the row is created:summaryscout_harness/runner.py(_finalize_run_row, end of run)QuerySet.update(summary=...)emitted_count,emitted_finding_idsscout_harness/tools/emit.py(_record_emit)run.save(update_fields=[...])emitted_report_idsscout_report/persistence.py(_record_report_emit)run.save(update_fields=[...])edited_report_idsscout_report/persistence.py(record_report_edit)run.save(update_fields=[...])metadatascout_harness/derived_metadata.py(stamp_derived_metadata)run.save(update_fields=[...])Nothing on the row records that any of these happened. Two consequences:
created_at, and acreated_atwatermark never re-reads a row, so close-out writes that land after the row crossed the watermark are lost. The self-driving warehouse mirror hit exactly this: a material share of settled runs read with an emptysummary, and the only fixes available were a re-read lookback window on one side and an hourly full refresh on the other. Both are workarounds for a missing column.TaskRun.updated_atwith events, because the bridge row carries no timestamp of its own.SignalReport,SignalReportCheck,SignalScoutConfigand the other signals models with post-create writes all carryupdated_at.SignalScoutRunis the exception.Proposal
Add
updated_attoSignalScoutRun:Nullable with no backfill so the
AddFieldstays non-blocking on the populated table, the same reasoningemitted_countandemitted_report_idscarry on this model. Historical rows read NULL, which is honest: the column never observed them. Consumers that need a single timestamp usecoalesce(updated_at, created_at).Make every writer advance it.
auto_nowonly fires onModel.save()and is skipped whenupdate_fieldsis passed without the field, so the field alone changes nothing today:save()on the model to append"updated_at"toupdate_fieldswhenever a caller passes one. That covers the foursave(update_fields=...)paths in one place and any future one.runner.pyusesQuerySet.update(); passupdated_at=timezone.now()alongsidesummary. Its comment explains why it is a targeted UPDATE, so keep that shape.Index it if the list endpoint or the harness ever filters on it. Not needed for the sync use case, which reads by watermark; leave the index out unless a query wants it.
Tests. One test per write shape that proves
updated_atmoves: asave(update_fields=[...])path and the.update()summary path. A test that a plaincreate()leavesupdated_atequal tocreated_atwithin the same transaction is optional.Out of scope
TaskRunstays the source of status, timing and error; this only adds "last touched" to the bridge row.updated_atand drop the lookback and full-refresh workarounds; that is a warehouse-side change and follows this.Acceptance
SignalScoutRun.updated_atexists, migration is non-blocking (AddFieldwithdb_default, no backfillUPDATE).updated_at, including theQuerySet.update()summary write.hogli test products/signalsgreen; the new tests fail on a model that has the field but not thesave()override.