Extend the radar loop from NOAA's 50 minutes to a configurable 24 hours - #35
Merged
Conversation
The radar loop is about to display a GIF it assembles from many fetches rather than the bytes of any single one, which breaks the assumption that the cached picture and the last HTTP response are the same object. Conditional revalidation and the 304 short-circuit now key off a dedicated _last_fetched_bytes, _async_fetch_image() takes an optional URL so a fallback path need not reimplement its error handling, and the recovery bookkeeping moves into _note_success() so a subclass can reuse it. No behaviour change: for every entity that exists today the two byte caches move in lockstep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
NOAA keeps only ten frames on the server, so a loop longer than its own fifty minutes cannot be downloaded -- it has to be accumulated. This adds the machinery: a per-radar-site directory of single-scan GIFs named by scan time, a sampler that picks evenly spaced frames by time rather than by position, and a Pillow encoder. Two things the encoder has to get right, both covered by tests that fail against the obvious implementation: Frames arrive separately palettised, and letting the GIF writer adopt the first frame's palette as the global table remaps every later frame against it. A palette built from frame one alone is just as wrong in a way that only shows up overnight: the loop starts on an empty radar, so the palette holds nothing but background, and the storm that arrives hours later is rendered as background too. The palette is therefore sampled across the whole window. Pillow also merges pixel-identical neighbouring frames and sums their delays, so a clear night legitimately encodes to fewer frames than it has sources. Playback time and the hold on the newest frame both survive that, so it stands as free compression. Nothing in the module imports Home Assistant, which is what lets its tests run against a real directory and a real Pillow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
A third options step asks how many hours of radar the loop should cover, defaulting to a full day and accepting 0 to keep serving NOAA's own fifty-minute animation and store nothing. The step writes all four option keys explicitly. Options replace the stored mapping rather than merging into it, so a key the final step forgets is a key silently dropped the next time anyone opens Configure -- there is a regression test pinning that down. The bounds are checked in Python as well as in the schema because the flow tests mock voluptuous wholesale, which would make a vol.Range accept anything. Initial setup is left at two steps: the loop writes frames into the configuration directory, and an upgrade should not start doing that to everyone who never opened the options page. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
The Radar Loop entity now polls the latest single scan rather than NOAA's finished animation, files each new one under its published time, and assembles the loop itself from a time-spaced sample of what it holds. The loop starts short after a fresh install and fills out over the following day; frames outlive restarts, so that happens once rather than on every reboot. Frames are identified by Last-Modified, which is when NOAA published the scan. That puts them on the real volume-scan cadence instead of our refresh boundary, and makes two refreshes that see the same scan resolve to the same file. Hashing the bytes would have been actively wrong: two scans of a clear sky are genuinely identical, so a quiet night would collapse to one frame and the loop would cut straight from clear to storm with no sense of time passing. Validators are now offered back only to the resource they came from. This entity fetches two URLs, and an ETag from the single frame must never be offered as a validator for the animation -- a server answering 304 to that would hand back the wrong image. Everything degrades to NOAA's own loop: too few frames yet, Pillow missing, assembly failing, a disk that will not take the frame. Every failure path leaves the displayed animation exactly as it was, and the loop_mode attribute reports which animation is actually on screen rather than which one was configured. Removing the integration deletes the frames, unless another entry shares the radar site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
Covers what the option does, that the loop fills in over time, where the frames live and roughly what they cost, and the two behaviour changes an existing user would otherwise be surprised by: the locally built loop is opaque where NOAA's is transparent, and 0 restores the old behaviour exactly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
The existing radar tests swap either the store or the encoder for a fake, which leaves the wiring between them untested -- and that wiring is where a 24-hour loop either works or quietly serves NOAA's fifty minutes forever. Two tests now drive the real entity over a real directory with a real Pillow: one watches the loop grow from NOAA's animation into a locally built one, the other restarts the entity over the same directory and checks the history is still there. Also pins the rule that cached validators are only offered back to the URL they came from, in both directions, plus the case it must not break: an entity with a single URL still revalidates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
A pre-merge review turned up fifteen issues; each fix below is pinned by a test that fails without it. Worst first, a time bomb in my own tests: the two end-to-end radar tests pinned their frame timestamps to a calendar date while the code prunes against the wall clock, so every frame would have fallen outside the window from the next day onward and CI would have gone red with no code change. Timestamps are now derived from the clock. Verified by running the suite with the clock two days, forty days and a year ahead. Correctness: - select_frames applied its window only after the "few enough frames, take them all" shortcut, so a buffer pruning had not caught up with built a loop out of the wrong day and then reported it as the configured window. - Pruning ran only when a new frame was stored. A radar site stuck on one scan for hours -- maintenance, an outage -- never pruned at all, which is also what the docs already claimed did not happen. - The prune had no upper bound on frame age. A frame dated into the future by a wrong clock is never reached by the cutoff, so it would outlive every real frame and strand the loop on a set that never updates. The file cap did not help: it trims the oldest, and those frames sort newest. - The oversize retry thinned with paths[::2], which drops the last frame whenever the count is even -- the normal case at 72. The animation ended, and held, on a scan at least twenty minutes stale. - The RADAR_LOOP_MIN_FRAMES floor was checked against every frame on disk but the loop is built from the sampled subset, so a window with a long outage could ship a four-frame loop in place of NOAA's ten. - The newest frame now sets the canonical size. Taking it from the oldest meant that after a NOAA product-size change, every new full-resolution scan was scaled back down to the superseded dimensions for a whole window. Regressions this PR had introduced: - The single validator slot meant the frame fetch and the fallback loop fetch evicted each other's ETags every cycle, so the fallback could never revalidate -- the scoping fix made 304s structurally unreachable rather than merely safe. Validators are now kept per resource. - Passing an explicit URL skipped cache busting, so the fallback loop lost the ?t= it had always had and could be served a stale CDN copy. Honesty of the exposed attributes: - frame_count and window_end came from the frames offered to the encoder rather than the ones it used, hiding exactly the shortfall the attributes exist to reveal. assemble_gif now returns the frames it encoded alongside the bytes. Housekeeping: - Switching forecast office, or setting the duration back to 0, orphaned the old radar site's frames forever: async_remove_entry only knows the site the entry ends on, and a store that is never constructed never prunes. Setup now sweeps directories no configured entry is collecting for, which is what makes the options screen's "store nothing" true. - The invalid-hours error hardcoded "between 0 and 24" while the step description interpolates the constant. Also replaces a sampler test that never reached the sampler -- its 16-frame fixture hit the take-them-all shortcut, so the tolerance logic it named could have been deleted without failing it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FticTYLC8japDARGL6mcLH
turbo5000c
marked this pull request as ready for review
August 23, 2026 21:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The Radar Loop image entity showed NOAA's ready-made animation, which is fixed at ten frames covering roughly 50 minutes. Only those ten frames exist on NOAA's server, so a longer loop cannot be downloaded — it has to be collected. This PR makes the entity collect one frame per refresh, file each under the time NOAA published it, and assemble the animation itself from an evenly time-spaced sample.
The window is a new options-flow step, defaulting to 24 hours. Setting it to
0restores the previous behaviour exactly: NOAA's own loop, proxied unchanged, with nothing written to disk.How it behaves
<config>/noaa_it_all/radar_frames/<RADAR_SITE>/— a few megabytes per radar site. Frames outside the window are pruned on every refresh. The directory is removed when the integration is deleted, when the entry switches to another office, and when the option is set back to0— unless another configured office still builds a loop from the same site.loop_modereports which animation is on screen (localorupstream), alongsideloop_hours,frame_count,window_startandwindow_end, so a loop quietly shorter than configured is visible from a template.Two things the encoder has to get right, both pinned by tests that fail against the obvious implementation:
Behaviour changes worth calling out
0.ETagfrom the single frame must never be sent as a validator for the animation.Not included: seeding the buffer from NOAA's ten-frame loop at first start. Its per-frame cadence is unpublished and varies by scan strategy, so every back-dated timestamp would be a guess polluting an otherwise exactly-timestamped buffer — for 50 minutes of history on a 24-hour feature that the upstream fallback already covers.
Review pass
A pre-merge review found fifteen issues, all fixed in
d83fa39. Each fix is pinned by a test that fails without it.The worst was a time bomb in this PR's own tests: the two end-to-end radar tests pinned frame timestamps to a calendar date while the code prunes against the wall clock, so from the next day every frame would have fallen outside the window and CI would have gone red with no code change. Timestamps now derive from the clock; verified by running the suite with the clock 2, 40 and 400 days ahead.
Correctness bugs that would otherwise have shipped:
select_framesapplied its window only after the "few enough frames, take them all" shortcut, so a buffer pruning had not caught up with built a loop from the wrong day and reported it as the configured window.paths[::2], which drops the last frame whenever the count is even — the normal case at 72 — so the animation ended, and held, on a stale scan.RADAR_LOOP_MIN_FRAMESfloor was checked against every frame on disk rather than the sampled subset that actually ships, so a window with a long outage could send a four-frame loop in place of NOAA's ten.Regressions introduced earlier in this PR and fixed in the same commit:
?t=it had before this PR.Plus:
frame_count/window_endreported frames offered to the encoder rather than used — hiding exactly the shortfall those attributes exist to reveal; switching office or setting hours to0orphaned the old site's frames forever, contradicting the options screen's "store nothing"; and the invalid-hours error hardcoded "between 0 and 24" while the step description interpolates the constant.Type
Checklist
Verification
strings.jsonandtranslations/en.jsonbyte-identical. Pillow added to the CI install line so the encoder tests actually run there rather than skipping.manifest.json— it is already a Home Assistant core requirement, and the module degrades to NOAA's loop if the import fails.Known follow-ups (not blocking)
{SITE}_0.gifis fetched twice per cycle — once by this entity, once by the base-reflectivity entity — plus a HEAD from the radar timestamp coordinator. Deduplicating means either coupling the two entities or adding a shared per-site fetch cache.