Extend the radar loop from NOAA's 50 minutes to a configurable 24 hours

.github/workflows/tests.yml

@@ -26,7 +26,7 @@ jobs:
       - name: Install dependencies
         run: |
           python -m pip install --upgrade pip
-          pip install pytest flake8 aiohttp voluptuous
+          pip install pytest flake8 aiohttp voluptuous pillow
 
       - name: Lint with flake8
         run: |

CHANGELOG.md

@@ -5,7 +5,60 @@ All notable changes to NOAA It All for Home Assistant will be documented in this
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [0.5.3] - Current
+## [0.6.0] - Current
+
+### Added
+- **The radar loop can now cover up to 24 hours instead of NOAA's fixed 50 minutes.** NOAA
+  publishes a ready-made animation at `{SITE}_loop.gif`, but it is fixed at ten frames covering
+  roughly 50 minutes, and only those ten frames exist on its server — so a longer loop cannot be
+  downloaded, it has to be collected. The Radar Loop entity now fetches the latest single scan on
+  each refresh, files it under the time NOAA published it, and assembles the animation itself from
+  an evenly time-spaced sample of what it holds.
+
+  Set the window under **Settings → Devices & Services → NOAA It All → Configure**, which gained a
+  third step. It defaults to **24 hours**; `0` restores the previous behaviour exactly — NOAA's own
+  loop, proxied unchanged, with nothing written to disk.
+
+  Things worth knowing:
+  - **The loop fills in over time.** A freshly configured loop is only as long as the history
+    collected so far and reaches its full length after that many hours of uptime. Below six frames
+    the card shows NOAA's own loop instead, so it is never blank and never worse than before.
+  - **Frames survive restarts**, stored as one small GIF per scan under
+    `<config>/noaa_it_all/radar_frames/<RADAR_SITE>/`. Budget a few megabytes per radar site.
+    Anything outside the window — or dated in the future by a wrong clock — is pruned on every
+    refresh. The directory is removed when the integration is deleted, when the entry is switched
+    to another forecast office, and when the option is set back to `0`, unless another configured
+    office is still building a loop from the same radar site.
+  - **The animation is larger than NOAA's**, and every open dashboard re-downloads it whenever it
+    changes. A 24-hour loop is capped at 72 frames (one every 20 minutes) and plays through in
+    about ten seconds; shorter windows are proportionally finer, with a 6-hour loop keeping roughly
+    one frame per scan.
+  - Frames are identified by `Last-Modified` — the time NOAA published the scan — which puts them
+    on the real volume-scan cadence rather than on our refresh boundary, and makes two refreshes
+    that see the same scan resolve to the same file. Hashing the image bytes would have been
+    actively wrong: two consecutive scans of a clear sky are genuinely identical, so a quiet night
+    would collapse into a single frame and the loop would cut straight from "clear" to "storm" with
+    no sense of time passing.
+  - Every failure — too few frames yet, Pillow missing, assembly failing, a disk that will not take
+    the frame — falls back to NOAA's own loop, and no failure path changes the picture already on
+    screen.
+- **The Radar Loop entity exposes what it is actually showing.** `loop_mode` is `local` when the
+  animation was built here and `upstream` when it is NOAA's, alongside `loop_hours`, `frame_count`,
+  `window_start` and `window_end` — so a loop quietly shorter than configured is visible from a
+  template rather than only from the logs.
+
+### Changed
+- **The locally built loop is opaque where NOAA's is transparent.** Source frames are transparent
+  overlays that each carry their own palette, and reconciling per-frame transparency across
+  differing palettes is the most reliable way to produce a psychedelic radar loop. Frames are
+  composited onto a solid black background before being combined. Cards that relied on the radar
+  loop being transparent over a custom background will see black instead; set the option to `0` to
+  keep NOAA's transparent animation.
+- **Cached image validators are only offered back to the URL they came from.** The radar loop
+  entity fetches two different resources, and an `ETag` from the single frame must never be sent as
+  a validator for the animation — a server answering `304` to that would hand back the wrong image.
+
+## [0.5.3] - Previous
 
 ### Fixed
 - **A network blip no longer blanks the NOAA image cards.** Every image entity's `async_image()`

CONFIGURATION.md

@@ -4,11 +4,12 @@ This guide provides detailed configuration examples for NOAA It All, including e
 
 ## Table of Contents
 1. [Installation Methods](#installation-methods)
-2. [Entity Configuration](#entity-configuration)
-3. [Device Grouping](#device-grouping)
-4. [Dashboard Card Examples](#dashboard-card-examples)
-5. [Automation Examples](#automation-examples)
-6. [Script Examples](#script-examples)
+2. [Options](#options)
+3. [Entity Configuration](#entity-configuration)
+4. [Device Grouping](#device-grouping)
+5. [Dashboard Card Examples](#dashboard-card-examples)
+6. [Automation Examples](#automation-examples)
+7. [Script Examples](#script-examples)
 
 ## Installation Methods
 
@@ -38,6 +39,74 @@ longitude: -117.1611
 > up. Remove the block and add the integration through **Settings → Devices & Services → Add
 > Integration → NOAA It All**.
 
+## Options
+
+**Settings** → **Devices & Services** → **NOAA It All** → **Configure**. The flow walks through
+latitude and longitude, then the forecast office, then the radar loop. Saving reloads the
+integration so the new values take effect immediately.
+
+| Option | Default | Notes |
+|---|---|---|
+| Latitude / Longitude | Home Assistant's Home location | Used for alerts, surf and aurora |
+| NWS Forecast Office | Nearest office to those coordinates | Determines the radar site |
+| Hours of radar history | `24` | Length of the Radar Loop animation, `0`–`24` |
+
+### Hours of radar history
+
+NOAA's own radar animation is fixed at about 50 minutes, and only its ten most recent frames exist
+on the server — there is no longer version to download. To show more than that, the integration
+saves one frame each time it refreshes and assembles the animation itself.
+
+```yaml
+# Stored in the config entry's options
+office_code: "SGX"
+latitude: 32.7157
+longitude: -117.1611
+radar_loop_hours: 24
+```
+
+- **`0`** — serve NOAA's own ~50 minute loop unchanged and store nothing on disk. This is how the
+  integration behaved before version 0.6.0.
+- **`1`–`24`** — build the loop locally over that window.
+
+What to expect when it is on:
+
+- **The loop fills in over time.** It starts at whatever history has been collected and reaches
+  full length after that many hours of uptime. Until there are at least six frames, the card falls
+  back to NOAA's own loop rather than showing a near-still image.
+- **Frames persist across restarts**, under `<config>/noaa_it_all/radar_frames/<RADAR_SITE>/`, one
+  small GIF per scan. Expect a few megabytes per radar site. Frames outside the window are deleted
+  on every refresh. The directory is removed when you delete the integration, when you switch the
+  entry to a different forecast office, or when you set this option back to `0` — unless another
+  configured office is still building a loop from the same radar site.
+- **Frame spacing follows the window.** The animation is capped at 72 frames and plays through in
+  about ten seconds, so a 24-hour loop steps every 20 minutes while a 6-hour loop keeps roughly one
+  frame per radar scan.
+- **The file is larger than NOAA's**, and every open dashboard re-downloads it whenever it changes.
+  On a wall tablet on a mobile connection, prefer a shorter window.
+
+While the buffer is still filling, each refresh makes two requests instead of one — the latest
+frame, plus NOAA's loop to display in the meantime. That stops once enough frames have been
+collected.
+
+The entity exposes what it is actually doing as attributes:
+
+| Attribute | Meaning |
+|---|---|
+| `loop_mode` | `local` when showing an animation built here, `upstream` when showing NOAA's |
+| `loop_hours` | The configured window |
+| `frame_count` | Frames in the animation currently being served |
+| `window_start` / `window_end` | Times of its oldest and newest frames |
+
+```yaml
+# Alert when the radar loop quietly falls back to NOAA's short animation
+template:
+  - binary_sensor:
+      - name: "Radar loop degraded"
+        state: >
+          {{ state_attr('image.noaa_ilm_weather_radar_loop', 'loop_mode') == 'upstream' }}
+```
+
 ## Entity Configuration
 
 ### Understanding Entity IDs

README.md

@@ -266,7 +266,17 @@ Visual representations of current conditions:
 
 **NOAA {OFFICE} Weather** (one per configured office):
 - **Radar Base Reflectivity** — Latest NEXRAD base reflectivity radar for your NWS office *(image.noaa_{office}_weather_radar_base_reflectivity)*
-- **Radar Loop** — Animated NEXRAD radar loop *(image.noaa_{office}_weather_radar_loop)*
+- **Radar Loop** — Animated NEXRAD radar loop, covering up to 24 hours *(image.noaa_{office}_weather_radar_loop)*
+
+> **Radar loop history**: NOAA's own radar animation is fixed at ten frames covering roughly 50 minutes, and only those ten frames exist on its server — a longer loop cannot simply be downloaded. So the integration collects one frame per refresh and assembles the animation itself. The window defaults to **24 hours** and is set under **Settings → Devices & Services → NOAA It All → Configure**; set it to `0` to serve NOAA's 50-minute loop unchanged and store nothing.
+>
+> Worth knowing before you turn it up:
+>
+> - **It fills in over time.** A newly configured loop starts at whatever history has been collected so far and reaches its full length after that many hours of uptime. Until there are enough frames, the card shows NOAA's own loop instead, so it is never blank.
+> - **Frames survive restarts.** They are stored under `<config>/noaa_it_all/radar_frames/<RADAR_SITE>/`, so a restart does not send the loop back to the beginning. Budget a few megabytes per radar site; the directory is deleted if you remove the integration.
+> - **The animation is bigger than NOAA's.** A 24-hour loop is sampled down to 72 frames (one every 20 minutes) and plays through in about ten seconds, but it is still a larger file that every open dashboard re-downloads whenever it changes. Shorter windows are proportionally finer: a 6-hour loop keeps one frame per scan.
+>
+> The entity's `loop_mode` attribute reports whether you are looking at a locally built animation (`local`) or NOAA's (`upstream`), alongside `frame_count`, `window_start` and `window_end`.
 
 > **Tip**: Image entities can be displayed on dashboards using the standard `picture-entity` or `picture-glance` cards.
 >

custom_components/noaa_it_all/__init__.py

@@ -11,8 +11,10 @@
     HURRICANE_COORDINATOR_KEY,
     HURRICANE_IMAGES_ADDED_KEY, HURRICANE_SENSORS_ADDED_KEY,
     OFFICE_RADAR_SITES, OFFICE_TIDE_STATIONS, OFFICE_BUOY_STATIONS,
+    RADAR_FRAME_DIR,
 )
 from .entry_config import resolve_entry_config
+from .radar_loop import RadarFrameStore
 from .coordinator import (
     SpaceWeatherCoordinator,
     HurricaneCoordinator,
@@ -187,6 +189,39 @@ async def async_unload_entry(hass: HomeAssistant, entry: ConfigEntry):
     return unload_ok
 
 
+async def async_remove_entry(hass: HomeAssistant, entry: ConfigEntry) -> None:
+    """Delete the radar frames this entry accumulated.
+
+    Home Assistant calls this when the integration is removed rather than
+    merely unloaded.  A day of radar is a few megabytes per site that nothing
+    else would ever clean up, and it sits in the configuration directory, so it
+    would otherwise ride along in every backup forever.
+    """
+    office_code = resolve_entry_config(entry).get(CONF_OFFICE_CODE)
+    radar_site = OFFICE_RADAR_SITES.get(office_code)
+    if not radar_site:
+        return
+
+    # Frames are keyed by radar site, and neighbouring offices can share one.
+    # Removing this entry must not take another entry's history with it.
+    for other in hass.config_entries.async_entries(DOMAIN):
+        if other.entry_id == entry.entry_id:
+            continue
+        other_office = resolve_entry_config(other).get(CONF_OFFICE_CODE)
+        if OFFICE_RADAR_SITES.get(other_office) == radar_site:
+            _LOGGER.debug(
+                "Keeping stored radar frames for %s; another entry still uses it",
+                radar_site,
+            )
+            return
+
+    store = RadarFrameStore(
+        hass, hass.config.path(DOMAIN, RADAR_FRAME_DIR), radar_site
+    )
+    await store.async_remove_all()
+    _LOGGER.info("Removed stored radar frames for %s", radar_site)
+
+
 # Legacy function for YAML setup
 def setup(hass, config):
     """Set up the NOAA component (legacy YAML support)."""

custom_components/noaa_it_all/config_flow.py

@@ -12,7 +12,10 @@
     CONF_LATITUDE,
     CONF_LONGITUDE,
     CONF_OFFICE_CODE,
+    CONF_RADAR_LOOP_HOURS,
+    DEFAULT_RADAR_LOOP_HOURS,
     OFFICE_COORDINATES,
+    RADAR_LOOP_MAX_HOURS,
 )
 from .entry_config import resolve_entry_config
 
@@ -248,6 +251,7 @@ def __init__(self):
         """
         self._latitude = None
         self._longitude = None
+        self._office_code = None
 
     async def async_step_init(self, user_input=None):
         """Step 1 of the options flow: latitude / longitude."""
@@ -323,14 +327,8 @@ async def async_step_office(self, user_input=None):
                     errors={CONF_OFFICE_CODE: "invalid_office"},
                     description_placeholders=self._office_placeholders(no_within_radius),
                 )
-            return self.async_create_entry(
-                title="",
-                data={
-                    CONF_OFFICE_CODE: office_code,
-                    CONF_LATITUDE: self._latitude,
-                    CONF_LONGITUDE: self._longitude,
-                },
-            )
+            self._office_code = office_code
+            return await self.async_step_radar()
 
         return self.async_show_form(
             step_id="office",
@@ -340,6 +338,56 @@ async def async_step_office(self, user_input=None):
             description_placeholders=self._office_placeholders(no_within_radius),
         )
 
+    async def async_step_radar(self, user_input=None):
+        """Step 3 of the options flow: how much radar history to keep.
+
+        NOAA's own animation covers about fifty minutes and cannot be made
+        longer, so anything beyond that is accumulated locally over time.  The
+        value is in hours, and 0 means "just serve NOAA's loop".
+        """
+        errors = {}
+        existing = resolve_entry_config(self.config_entry)
+        default_hours = existing.get(
+            CONF_RADAR_LOOP_HOURS, DEFAULT_RADAR_LOOP_HOURS
+        )
+
+        if user_input is not None:
+            hours = user_input.get(CONF_RADAR_LOOP_HOURS)
+            # Validated here rather than left to the schema alone: the flow
+            # tests mock voluptuous wholesale, so a vol.Range would be a
+            # MagicMock that accepts anything.
+            try:
+                hours = int(hours)
+            except (TypeError, ValueError):
+                hours = None
+            if hours is None or not 0 <= hours <= RADAR_LOOP_MAX_HOURS:
+                errors[CONF_RADAR_LOOP_HOURS] = "invalid_radar_hours"
+            else:
+                # Every key is listed explicitly.  Options replace the stored
+                # mapping wholesale rather than merging into it, so a key left
+                # out here is a key silently dropped on the next options edit.
+                return self.async_create_entry(
+                    title="",
+                    data={
+                        CONF_OFFICE_CODE: self._office_code,
+                        CONF_LATITUDE: self._latitude,
+                        CONF_LONGITUDE: self._longitude,
+                        CONF_RADAR_LOOP_HOURS: hours,
+                    },
+                )
+            default_hours = user_input.get(CONF_RADAR_LOOP_HOURS, default_hours)
+
+        return self.async_show_form(
+            step_id="radar",
+            data_schema=vol.Schema({
+                vol.Required(
+                    CONF_RADAR_LOOP_HOURS, default=default_hours
+                ): vol.All(vol.Coerce(int), vol.Range(min=0, max=RADAR_LOOP_MAX_HOURS)),
+            }),
+            errors=errors,
+            description_placeholders={"max_hours": str(RADAR_LOOP_MAX_HOURS)},
+        )
+
     def _office_placeholders(self, no_within_radius):
         """Build description placeholders for the office step."""
         return {

custom_components/noaa_it_all/const.py

@@ -57,6 +57,7 @@ def _manifest() -> dict:
 CONF_OFFICE_CODE = "office_code"
 CONF_LATITUDE = "latitude"
 CONF_LONGITUDE = "longitude"
+CONF_RADAR_LOOP_HOURS = "radar_loop_hours"
 
 # Default values
 DEFAULT_SCAN_INTERVAL = 10  # minutes
@@ -84,6 +85,51 @@ def _manifest() -> dict:
 IMAGE_FETCH_TIMEOUT = 20  # seconds
 IMAGE_MAX_BYTES = 20 * 1024 * 1024  # refuse absurd payloads rather than cache them
 
+# -------------------------------------------------------------------
+# Radar loop
+# -------------------------------------------------------------------
+# NOAA publishes a ready-made radar animation, but it is fixed at ten frames
+# covering roughly fifty minutes, which is long enough to see that it is
+# raining and too short to see where the rain came from.  NOAA also keeps only
+# those ten frames on the server, so a longer loop cannot be downloaded -- it
+# has to be accumulated here, one frame per refresh, and assembled locally.
+#
+# The window is measured in hours and 0 means "serve NOAA's own loop
+# unchanged", which is both the escape hatch and the behaviour every release
+# before this one had.
+DEFAULT_RADAR_LOOP_HOURS = 24
+RADAR_LOOP_MAX_HOURS = 24
+
+# The assembled animation is re-downloaded by every open dashboard each time it
+# changes, so frame count is a bandwidth and memory decision, not a fidelity
+# one.  Seventy-two frames spreads a 24-hour window over 20-minute steps: storm
+# motion stays legible, the GIF lands around 1-2 MB, and a cycle plays in about
+# ten seconds, which is roughly as long as anyone watches a loop.  Shorter
+# windows get proportionally finer steps from the same cap -- a six-hour loop
+# works out at one frame per five minutes, i.e. every scan NOAA publishes.
+RADAR_LOOP_MAX_FRAMES = 72
+RADAR_LOOP_MIN_FRAMES = 6  # below this the local loop is worse than NOAA's
+RADAR_LOOP_FRAME_MS = 120  # browsers clamp anything under ~20ms
+RADAR_LOOP_LAST_FRAME_MS = 1500  # hold on "now" so the loop reads as a loop
+RADAR_LOOP_MAX_BYTES = 8 * 1024 * 1024
+
+# Frames are composited onto an opaque background before being combined.  The
+# source frames are transparent overlays with a palette each, and reconciling
+# per-frame transparency across differing palettes is the single most reliable
+# way to produce a psychedelic radar loop.  Compositing removes the problem.
+RADAR_LOOP_BACKGROUND = (0, 0, 0)
+
+# Frames live in <config>/noaa_it_all/radar_frames/<SITE>/.  Every polled frame
+# inside the window is kept, not just the ones the current window displays, so
+# that changing the duration re-samples from real history instead of starting
+# over.  That is ~144 files, a few MB, per radar site.
+RADAR_FRAME_DIR = "radar_frames"
+RADAR_FRAME_MAX_FILES = 200  # backstop against a directory growing unbounded
+# A frame dated beyond now + this is the product of a wrong clock rather than a
+# scan we have not reached yet.  Ageing never reaches such a frame, so it is
+# discarded outright; the slack absorbs ordinary skew between us and NOAA.
+RADAR_FRAME_FUTURE_SLACK_MINUTES = 60
+
 # API endpoints
 NWS_SRF_URL = "https://forecast.weather.gov/product.php?site={office}&issuedby={office}&product=SRF&format=TXT"
 NWS_API_BASE = "https://api.weather.gov"