Rebuild the console, and stop backend routes from shipping unreachable - #67
Merged
Conversation
Three things the rebuilt console rendered but did not do. The tab strip. Layers / Find / Histogram / Info / More stayed mounted and clickable while a full-bleed app (Foundry, AI, Workflows, City, Country, Markets — six of the fourteen) had the whole body, so `bleed` had already removed the column they open into. Measured against a live Foundry: clicking Find set aria-selected="true" with zero `aside.csl2-panel` in the document. Five dead controls in the most prominent strip after the title bar. Asking for a panel now returns to the map, which is where the panel lives, and no tab claims the selected state while it cannot show anything. The Help menu has advertised "1-4 left panels" and every tab tooltip has named its key since the rebuild, with nothing listening; 1-5 are bound, and skip a keystroke aimed at a field. The action bar. Four literals: a filter button with no handler, the sentence "Showing all contact types across the current view" hardcoded so it stayed true of nothing the moment a filter was set, a Clear selection that cleared nothing, and an accent-filled primary "Add to filter path" for a capability that does not exist. It runs against useFilters and useSelection now, offering the two facets the filter evaluator actually understands, with the values and labels the histogram bars already use, so a chip here names exactly what a bar there counts. Both facets carry a `military` bucket, so a clause is named for the one it came from rather than announcing twice as "Military". Removable chips, and both verbs disabled with the reason in a tooltip when there is nothing to do. Colour schemes. Was a light/dark flag; is now seven complete palettes — Warm ink, Midnight, Slate, Night watch, High contrast, Daylight, Paper — picked from View, from Settings as swatches, or cycled on ⇧T. Each is a whole palette rather than a delta, because the light theme was a delta and inherited the dark `--hover`, so every hoverable row flashed near-black on white, and the scrollbar thumb stayed a white alpha that was invisible on it. contrast.test.ts holds all seven to AA on both bg-1 and bg-2, and asserts the registry, the CSS and the swatches agree, so a scheme cannot be added to one and forgotten by the others. Chrome that sits on the Cesium canvas or pins its own near-black panel — the scale bar, the cursor and centre readouts, the compass, the map toolbar, the analyst console, the workspace overlay — keeps the dark text ramp under the light schemes. It was taking dark text onto a black globe, which is a defect the light theme already had and the new light schemes would have doubled.
…lette
Three things the console still got wrong after the last pass, each found by
measuring rather than by reading the code.
The More parking lot. It held six surfaces and it was not insurance: five of
them — Investigation, Collab, News, Ground recon and the alert rail list — were
a SECOND copy of something already rendered at the address rehoming.test.ts
names for it, in the Graph app, in Reports, in Video and behind the alerts bell.
A duplicate mount is not a safety net, it is a second place to find a stale one.
Intel was the only genuinely unhomed surface, and it was unhomed because its
record was wrong: panels.ts called it a section of the dossier, and IntelPanel
takes no selection, so byHome could never place it and it fell through. It is
the fourth section of Info, with the AOI watch and the standing rollup it
belongs beside. The lot is gone; the escape hatch for an unrecorded rail item
stays. `time` is out of RIGHT_PANELS for the same reason in the other
direction — declared and never wired, because the TimeDock owns playback and
useTime holds no window a second surface could own without inventing one.
Dead controls, as a check rather than a memory. A live sweep of the running
console — React props, all fourteen apps, 460 to 770 interactive elements
each — found none left. liveControls.test.ts is the repeatable half: no
`<button>` under shell/ may render without a handler or a `disabled` that says
why, and nothing may print a keyboard hint that nothing binds. It was confirmed
to fail on a planted dead button before being kept.
Colour, measured in the browser instead of derived from the tokens. The token
guard cannot see a tint, a pinned surface or a glyph used as a swatch, and all
three were hiding failures:
- `text-white` on a solid fill, everywhere. Only true when the fill is dark,
and half of them are not: 2.14:1 on Night watch's amber accent, 2.34:1 on
High contrast's magenta, on the Pause button and the timeline cursor. Ink is
a property of the fill, so it is now --on-accent / --on-mag / --on-alert /
--on-ok / --on-warn, one per scheme.
- `text-accent` as body text, ~30 sites. --accent is chosen to HOLD ink, so it
measured 3.46:1 as ink. tailwind.config.js splits textColor from colors, so
bg-accent stays the fill and text-accent resolves to --accent-fg. --ok
gained the lightened tier the other three hues already had.
- The ramp is guaranteed on bg-1 and bg-2 only. On bg-3, on an accent tint and
on an --ok-bg tint, three live labels sat at 4.37 to 4.47.
- A faction swatch drawn as a `■` glyph inherited the text bar and failed it
at 1.70:1. It is a bordered chip now; the colour is data and did not change.
- The City viewport was `bg-black/40`, which is only dark when what is behind
it is. Over the light panel it composited to a mid grey.
Sweep after: 7 schemes x 17 surfaces = 119 passes, 42,478 rendered text nodes,
0 failures. contrast.test.ts grew the fill/ink and tint cases so the palette
cannot regress past the token layer again.
Palantir's design system is published as Blueprint, and this console is already calibrated against it: `$pt-button-height: 30px` is what pins the whole metric scale, and docs/mockups/console-2026-08/README.md records the ramp as "Blueprint 5.1.16 dark, with zero tokens outside it". That was prose. These four schemes make it a set something can test against. Blueprint dark blue3, on the substrate measured off the reference Blueprint turquoise turquoise3, so blue can stay a data colour Blueprint indigo indigo3, the hue already reserved for model output Blueprint light white panels on light-gray4, Blueprint's own arrangement The dark three do not just recolour the default. They sit on the panel surface the earlier mockup work measured and never shipped: dark-gray2, "Gotham's real 64% surface" per docs/mockups/gotham/gotham.css, one step LIGHTER than the default's dark-gray1, with cards on dark-gray3 above. That lighter substrate costs the muted tier its headroom — gray3 measures 4.35:1 on dark-gray3 — so the text ramp walks light-gray1 / gray5 / gray4 instead of gray5 / gray4 / gray3. Solved against the constraint, which is the same move gotham.css documents for the hover it could not take in-ramp. theme/blueprint.ts carries the 87 published swatches, extracted from @blueprintjs/colors@5.1.16 with the recipe in its header rather than transcribed from a docs page. The guard rejects any token in a scheme marked `blueprint: true` that is not one of them, with two exemptions it names and tokens.css justifies: --hover, and the selection magenta, which is welded to the globe's selection polyline and is therefore data rather than chrome. It was confirmed to fail on a planted off-ramp value before being kept. They are named for the design system, not for the products built on it. Blueprint is what these palettes are, and a picker full of product names would imply an affiliation this project does not have. One real defect found on the way in. Every scheme's --hover is a token pairing that the live browser sweep structurally cannot check, because a hover state is not in the DOM until something is hovered — and Daylight's put the muted tier at 4.39:1 on every hoverable row in the console. It is 4.52:1 now, and contrast.test.ts checks the pairing for all eleven schemes so the blind spot is covered by the layer that can see it. Verified with a pointer actually driven over the rail: 13 rows change background per scheme, 11 schemes, worst measured 4.52. Plus 36 surfaces and 19,425 text nodes swept across the new four, 0 failures.
A theme that takes a design system's colour and leaves its type is half a theme, and type is the half carrying "native application". Blueprint names no webfont at all: $pt-font-family asks for the platform UI font and takes what it gets, which is why a Blueprint app looks native on every OS. The four bp-* schemes now carry that stack verbatim, vendored into theme/blueprint.ts from the same package version as the ramp, minus $icons16-family — that entry exists so inline Blueprint icons resolve and this console does not ship that icon font, so carrying it would be a request nothing can satisfy. Measured in the browser rather than inferred: the default schemes paint with Inter, the Blueprint schemes with Ubuntu, and the same live tab label relays out from 80.61px to 77.05px. It is a real switch, not a fallback chain that changes nothing. Two corrections to how this was checked, both of which had me briefly believing something false. document.fonts.check() is not an availability probe. It answered true for Inter, IBM Plex Sans, Cantarell, Segoe UI and every other family asked, including ones fc-list reports zero faces for. The probe that works measures glyphs: render the sample in `"<family>", <sentinel>` and compare against the sentinel alone, with two different sentinels and a nonsense family as the control. Under that test IBM Plex Sans really is absent and Inter really is present — the opposite of what the mockup README has claimed since the rebuild, and the browser is the thing doing the painting. The first hover measurement reported all eleven schemes failing AA. That was the measurement: it read the tint's hue and dropped its alpha, comparing text against a colour never painted at strength. Composited properly, and with a pointer actually driven across the rail so 13 rows change background per scheme, the worst case is 4.52:1 and nothing fails. A font change moves metrics, so the sweep now also flags any box whose content overflows it without a stated policy. 32 surfaces, 18,781 nodes: 0 contrast failures, 0 overflowing elements, 0 page-level horizontal overflow. contrast.test.ts compares each bp-* block against the vendored stack token-for-token, including --font-label, since the README records Palantir as using one refined grotesque throughout rather than a condensed second face. Confirmed to fail on a stack drifted by one entry before being kept.
Three things the console did that a Gotham-level product cannot. The globe was framed by a hardcoded 20 Mm eye altitude. At this window's frustum that puts the Earth's disk across 56% of the map height and leaves the rest black, which is the single largest reason the console read as empty at first paint. globalAltitude() solves the altitude from the frustum instead: half-angle = asin(R / (R + alt)), aspect read off the canvas rather than frustum.aspectRatio, which on the first paint has not necessarily been sized to the laid-out canvas yet. The disk now spans 90% of the shorter axis at any window shape, and camera.test.ts checks the fill across four of them. The right column said "Nothing selected" in 384px for the whole session until something was clicked. WorldPanel stands in that place instead and answers what an analyst opens the console with: contacts on the map, split by provenance, the largest layers by name, how many sources are reporting and which one has the oldest fix. Every number is read from the live data sources and the feed store. And the layers list could not tell the truth about itself. A row that was off, a row that had answered zero and a row that had never loaded all rendered the same grey nothing, which is 58 of 64 rows. rowState() separates them, because a quiet source is information and a source that never loaded is a fault. The larger change is provenance. Every OSINT globe fuses ADS-B, AIS and a news firehose and presents the three as peers; two of them are machines reporting themselves and the third is somebody's assertion, and once they share a list the whole surface inherits the credibility of the weakest source. registry/ provenance.ts tiers all 56 layers as sensor, registry, filing or claim, the tier is marked on every row and totalled in the right column, and the Layers panel filters on it with claim OFF by default. GDELT armed events was on by default and is not any more: it is wire copy, however well coded, and it was sitting on the map beside the transponders with nothing distinguishing them. ACLED and UCDP are tiered as claims for the same reason, which is not a criticism of their coding but of what they code from. provenance.test.ts fails if a registered layer has no tier, if a tier entry names no layer, if a claim layer is on by default, or if claims ever outgrow the observed sources by more than a third. Plan of record for the rest: docs/plan-99-2026-08.md.
Windward, Kpler and Pole Star charge five figures a year to answer one question: is the ship I am looking at designated. The list is published by the US Treasury, free, with no key, and it names vessels by IMO, MMSI and call sign and aircraft by tail number. What is sold is not the data. It is the join. app/intel/sanctions.py loads the OFAC SDN CSV and turns it into the identifier maps a live feed can be joined on. The identifiers that matter are not columns: IMO and MMSI live in free-text remarks as "Vessel Registration Identification IMO 9187629; MMSI 572469210", and for an aircraft row the tail number IS the name field. Measured against today's file: 19,181 rows, 1,524 vessels, 344 aircraft, and 2,078 IMO, 789 MMSI, 920 call-sign and 344 tail keys to join on. /api/sanctions/vessels and /aircraft run that join against the live AIS and ADS-B snapshots and return GeoJSON. First live run: 125 designated hulls in a snapshot of 33,692, 28 of them matched on IMO or MMSI, clustered where you would expect them to be, in the Gulf of Finland off Primorsk and Ust-Luga. Aircraft returned an honest zero. Every match carries the identifier it rested on and a confidence, because an IMO hit and a name hit are not the same claim and a screening tool that blurs them is how an innocent ship gets flagged. A name match is never reported as exact, a name that hits several designated hulls says so rather than picking one, and a miss on /lookup is an explicit "we looked and found nothing" with the identifiers that were tried, never an empty body. The features keep the contact's own id and kind, so a designated tanker still opens as a vessel with its full dossier. Drawing is dispatched on a new style_kind property instead, which is what lets one feature be drawn as one thing and resolved as another. Both endpoints do their scan off the event loop. The vessel scan walks ~34k hulls and the map polls it on a timer; it must never be the reason the aircraft snapshot is late. OFAC only. The EU consolidated list, UK OFSI and the UN Security Council list are the same shape of work and are not loaded, so /summary says so rather than letting an absence read as a clean bill of health. Backend baseline 2164 -> 2172.
Every model call in this codebase is one-shot and serial to its caller. That is right for a selection brief and useless for the other half of the work, which is pushing a pile of raw documents through a model until they are structured facts. Done one at a time, a few hundred notices is an hour. app/llm_pool.py is a bounded pool over the existing completion path: order preserved, one bad item resolving to null in its slot instead of taking the batch down, and the queue living here where it can be measured rather than inside the model server where it cannot. POST /api/ai/batch exposes it and always returns the measured stats beside the answers, because a batch job whose throughput nobody can see is a batch job nobody can size. The width is the engine's width, read from wherever the engine got it: --parallel for llama.cpp, OLLAMA_NUM_PARALLEL for Ollama, LLM_POOL_WIDTH to override. A pool wider than the server does not go faster, it just moves the queue somewhere unobservable. tools/perf/llm_pool_sweep.py measures it instead of asserting it, and two things fell out of the first real run. The harness itself was wrong first. It measured whichever cloud backend the tier ladder picked, at a concurrency limit that has nothing to do with this box. It now pins the run to the local engine and restores the toggle afterwards. Then the pool was wrong. The Ollama fallback was 1 when OLLAMA_NUM_PARALLEL is unset, reasoning that an unset variable should not be assumed away. Measured: declared slots 1, and width 8 still finished the same 12 documents in 4.3 s against 32.0 s at width 1. A 7.4x speedup, zero failures, 12/12 parsed at every width. Ollama auto-sizes, so the fallback is 4. The full table, including the non-linearity (1 to 2 is 4.2x, 2 to 8 is another 1.8x) and the p95 climbing while p50 does not, is docs/perf-results-llm-pool-2026-08-05.md. llama.cpp is not in that table because its sidecar was not enable-able here, and an unmeasured default is recorded as unmeasured. Also: the sanctions screen now runs for every hull and every airframe, not only the ones that arrived on the sanctions layer, and it sits FIRST in the dossier. The first build put it under the profile, the actions and the pattern of life, which made the single fact that changes what an analyst does with a contact the last thing they would ever see. Verified live against NORD, IMO 9025778, moored at Ust-Luga: designated, matched on IMO, UKRAINE-EO13685. Backend baseline 2172 -> 2179.
…me match cannot The screen was OFAC only, and it said so, which was honest and still answered a narrower question than an analyst asks. Two more lists load now, each with its own parser because each hides its identifiers somewhere different. UK OFSI is a CSV behind two header rows carrying 19,761 entries, of which 81 are ships, and the hull identifiers are labelled spans inside free text: `(IMO number):7408873 (Flag of ship):North Korea`. The UN Security Council list is XML, 1,011 entries, and has no vessel element at all, so it contributes names and nothing joinable by hull. Merged live: 39,953 designations, 2,078 IMO keys, 789 MMSI, 920 call signs, 344 tails, and 15 hulls carried by both OFAC and the UK, which is exactly the fact the old index could not express. That is why every identifier map is now key to a LIST of designations. "OFAC and the UK" is a materially stronger statement than "OFAC", and the first version could only ever report whichever list happened to load first. A list that fails to fetch is named in the response, because a source that quietly dropped out must never read as a source that found nothing. The EU FSD export is deliberately not loaded, on a measurement rather than a guess: 25.7 MB for 6,225 entries whose only subject types are person and entity, 70 occurrences of "IMO" in the whole file, and no vessel element. It would broaden the name screen and add nothing joinable by hull. EU vessel designations live in the regulation annexes, not in that export. And the map layer now defaults to exact matches only, which is the more important half of this change. With name candidates in, the live run returned 335 hulls of which 294 matched only on a name: hull names are short, common and reused. A pin on a map carries no confidence label at a glance, it just reads as "this ship is sanctioned", so the layer shows only what an identifier vouches for. Measured after: 20 hulls, 6 by IMO and 14 by MMSI, every one of them a real designated tanker, with the 38 held-back candidates counted in the note and one query parameter away. The per-contact card still shows candidates, because there the caveat is next to the claim. Backend baseline 2179 -> 2183.
…y can be checked at all W1 promised a check and did not have one, which is the kind of debt that turns into a claim nobody ever verified. tools/perf/console_frame_check.mjs measures what only a browser can settle: at 1920x1080 the Earth's disk fills 0.878 of the map frame at 11,651 km on a 1242x940 canvas, the right dock renders 653 characters with nothing selected, 27 visible layer rows carry a tier mark, and there are no console errors. The fill is computed from the LIVE camera height and canvas size rather than from the constant the code was written with, so a regression that leaves globalAltitude() correct and simply stops calling it still fails here. The fourth claim, that a row distinguishes off from pending from empty from live, was in that check and had to come out. It failed, and it failed for the right reason: at the moment of the run every row happened to be either off or live, so the assertion was measuring what the upstreams had done in the last ten seconds rather than anything about the code. That distinction is deterministic in rowState, so it is guarded in useLayerCounts.test.ts, including the case the whole thing exists for: a source that answered zero and a source that has never answered both sum to zero through rowCount and must not share a state.
…ered The operator's first complaint was "a lot of old stuff not yet ported in", and nothing in this repo could answer it. shell/rehoming.test.ts asserts each re-homed address renders, in jsdom, with mocked data, which is the right guard for that contract and blind to the failure actually being described: a surface that mounts, paints its chrome and then carries nothing. tools/perf/app_reachability_check.mjs opens all 14 apps and all 4 left panels against the live backend and reports rendered characters per surface. It is a report, not a gate: the floor for "enough content" is a judgement, and a check that guessed it would be the same lie one level up. Apps are driven through the appView store's DEV handle rather than the launcher, so a broken launcher cannot hide a broken app behind it. Every one of them renders. explorer 22,145 characters down to workflows 1,119, with Info 893, Layers 585, Histogram 538 and Find 246 on the panels. The two thinnest, workflows and city, are where to look next. The more useful result is what it settled about the three failing requests. Two are not defects: /api/targets/board 401s because the board is deliberately local-first and treats the server row as an optional sync (state/targetBoard.ts:271), and /api/collab/shared-notes is the same shape on a keyless deployment. Only 404 /api/ontology/object/control:workspace is a real miss. Without the report all three read as breakage. Two fixes the report needed on itself: the panel tabs carry a count badge, so Info's accessible name is "Info4" and a role+name regex silently matched nothing and reported all four panels as missing; and appView had no DEV handle, which every other store in the console already has.
…ut it The map toolbar was three runs of unlabelled icons separated by a hairline. The redesign spec's own §2 says Gotham labels its toolbar groups, and says why: an icon with a tooltip is exactly the reachable-but-invisible pattern two persona waves kept reporting. A hairline says these three are different. It does not say what any of them is for. They are Tools, Draw and View now, in the same 40px column the buttons already use, so naming them costs the map no width. What this does NOT do, recorded rather than left to be discovered: that same spec's structural correction #2 says Gaia's map toolbar is a short horizontal cluster at the top-left of the map, about 30px tall, not a vertical rail down one side. Relocating it collides with the category legend that already owns the top-left and moves the anchor for the measure and area popovers, so it is a separate change with its own verification rather than a side effect of adding three labels. Also checked while here, because the operator's complaint pointed at them: Find reports 246 rendered characters in the reachability report and is not thin, it is idle. It is 432 lines with type and window facets, distance ranking and coordinate parsing, and 246 characters is what its empty state weighs before anyone has typed. Same for Series, which is wired to MetricsPanel and the archive card. Neither is unported.
… the news A country going off the internet is normally reported as a story, which makes it a claim, dated whenever a journalist noticed. It is not a claim. The BGP table is a machine reporting on itself: when a state orders a shutdown the announced prefixes for that country stop being announced, and RIPE's route collectors keep a public, keyless record of it. /api/cyber/routing reads country-resource-stats for a 51-country watchlist and reports each one against ITS OWN recent median, which is the only part of this that is analysis. An absolute prefix count says nothing: Iran carries 8,428 and South Sudan 91, and neither number is interesting. A count that is 15% below where that country has sat for a fortnight is. Verified live: all 51 countries reached, largest deviation Israel at 4.0%, everything classed none, which is the correct answer for a quiet day and the answer a threshold tuned to look busy would not have given. Three refusals, each of which is how this kind of readout goes wrong: Collector churn is not a finding. RIS counts wobble a percent or two on their own, so the floor is 5%. A threshold that fired on churn would cry wolf daily and be ignored inside a week. A country with fewer than five days of history gets baseline null and severity unknown, not a percentage. A number computed against a made-up normal is worse than no number. And the resolution is stated rather than glossed. RIS gives this daily, so a drop here is at best a day old and a shutdown contained inside one day can be invisible. The route says so in its own note and the layer inherits it. Nothing here claims to be live, because it is not. The map layer asks for min_drop=5, so it carries findings rather than 51 green dots, and the tile is slate rather than red: the colour means "measured against normal", and the severity lives in the property where it can be read. Backend baseline 2183 -> 2190.
The plan of record now carries its own state: eight commits, the evidence for each claim, the three things the wave got wrong before a measurement caught them, and an explicit list of what has not been started. W3 is the least advanced of the five waves and IRIS FDSN is a dead endpoint now, so anyone planning against §3 needs to re-source that row rather than discover it.
Both of this wave's new capabilities shipped as layers that are off by default, which is the right default and the wrong ending. A sanctions join that only reports once someone has found and enabled a layer they did not know existed converts nobody. Finding six of the persona study is titled "built-but- unreachable capability keeps not converting", and shipping two more of them would have been the same mistake with better data behind it. The right dock now carries a Findings section that queries both analytics directly, whether or not the layer is on, and puts the count where the operator is already looking. Enabling the layer is one click from the number. Live at first paint: 25 designated hulls under way, 0 countries below normal routing. A zero here is explained rather than left to be read as a shrug. "None of the vessels reporting right now is on OFAC, UK or UN by IMO or MMSI" and "every watched country is announcing its usual prefixes" are results. A source that failed to answer renders an em dash and says "source unavailable, so this is not a zero", because the two must not look the same.
Two routers were inserted next to their alphabetical neighbours by hand and ruff's isort rule disagreed about where routing and sanctions belong relative to the rest. verify.sh was red on this alone; everything else in it was green.
The sanctions join answers "is this hull designated". The question straight after it is always "and who owns it", and no free source answers that on its own: it is split across a registry, filings, procurement and the designation lists. Each quarter is public and keyless. None of them is joined to the others, and the join is what the commercial product is. /api/org/resolve queries GLEIF, SEC EDGAR full text, USAspending and the three designation lists concurrently. Verified live on SOVCOMFLOT: designated by OFAC SDN and UK OFSI under RUSSIA-EO14024 and UKRAINE-EO13662, two LEI records both carrying a LAPSED registration, ten SEC filings that mention it, and zero federal awards. The lapsed registrations are the kind of thing this exists to surface: an entity that stopped renewing its LEI, which is what tends to follow a designation. Every source reports whether it answered, on every response. A resolution that silently drops EDGAR and returns "no filings" is a claim, not an absence, and the card renders the two differently: "SEC EDGAR answered and had nothing" against "SEC EDGAR did not answer, so this is not a zero". Organisation search is substring where hull search stays tight. OFAC lists "PJSC SOVCOMFLOT" and an operator types "Sovcomflot"; an exact fold answers "not designated", which is wrong and confident. The same substring join on a ship called EBANO would light up half the list, so search_names is deliberately a separate function from match_vessel, and it refuses a query under three characters rather than matching most of 15,000 folded names. Where it lives was decided by a measurement, not by symmetry. The obvious home was the vessel dossier, hanging off the registered owner. OFAC carries an owner on almost none of the hulls that are actually under way right now, so that mount would have been unreachable in practice. It lives in Find, where somebody typing a company name already is. The dossier mount stays for the day a source carries an owner, and renders nothing until then. Backend baseline 2190 -> 2195.
…r than the reading Launch Library 2, NOAA SWPC and NOAA CO-OPS. All keyless, none of them narrative, and each answers something the console could not. Launches put a scheduled, geolocated event on the map for the first time: 40 of them at their pads, from a week back to sixty days out, verified live against Vandenberg, Wenchang, Taiyuan, Cape Canaveral and Guiana. The first version ordered descending from a lower bound and led with four Ariane flights in 2035 to 2039 carrying a TBD status, which is a manifest nobody is watching. It is bounded at both ends now. Space weather goes in Info rather than on the map, because Kp and the GOES X-ray class have no position. They are in the console because both sit UPSTREAM of feeds already on it: Kp degrades GNSS accuracy, which is exactly what the jamming layer infers degradation from. A quiet reading is the context that stops a degraded feed being read as an event, and the panel says so in those words. Live: Kp 0.0 quiet, X-ray B3.6. Surge is the one worth arguing about. A water level is not a finding; a water level against the tide that was predicted years ago is. /api/oceans/surge subtracts the prediction for the SAME MINUTE from the observation, and that qualifier is the whole route: the prediction series runs on a six-minute grid and keeps going past the last observation, so taking the last of each subtracts two different moments and reports the tide's own slope as surge. A station with no matching minute is dropped rather than guessed. Live: 15 of 15 stations, largest residual +0.230 m at Baltimore. Rate limiting is handled as a fact rather than a zero. Launch Library answers a throttled caller with a 200 carrying no results key; rendering that as "no launches" would be a confident lie about a manifest that is never empty, so the route reports reached=false and says why. That is ten keyless primary sources added this wave, tabulated in the plan with the live evidence for each. Backend baseline 2195 -> 2200, verify green.
…as missing The provenance tier is on every layer row, in the right dock's rollup and in the Layers filter. It was not on the contact itself, which made the dossier the one surface in the console where an observation and an assertion look identical, and the dossier is the surface an analyst reads before deciding anything. The join is not direct: a contact carries the SOURCE that saw it, `digitraffic` or `adsb` or `kystdatahuset`, never a layer id. tierOfSources matches those strings against the registry and takes the WEAKEST tier where a contact names several, which is the same rule the Layers rows use and for the same reason: rounding a fused contact up to its strongest input is the laundering the tier exists to stop. A source it does not recognise yields no tier rather than an assumed one. Verified live against SOLARIS, IMO 9333436, a tanker at Ust-Luga: Tier T0 · Sensor, source digitraffic, and directly below it a designation by OFAC SDN under UKRAINE-EO13662 and RUSSIA-EO14024 matched on IMO. Two tiers, both stated, on one contact.
…el on it
The tier was stated in four places and actionable in none of them. It is a
cross-filter facet now: the Histogram panel carries Provenance beside Aircraft
category and Flag, clicking T0 sets a clause, and the map dims everything a
sensor did not report. Live at 1920x1080 with the sanctions, launches and GDELT
layers on: 25,847 contacts, T0 Sensor 25,319, T1 Registry 76, T3 Claim 452, and
clicking T0 produces {facet: tier, value: sensor, mode: only}.
Two things had to be true for that to work, and neither was.
The tier is a property of the SOURCE and is not in any feature's property bag,
so the walk had nothing to count and the adapter had nothing to filter on. Both
now read it from the DataSource name, which is the layer id, and that is the only
place in either loop where the two meet.
And the first version of the facet counted nothing from the layer it mattered
most for. The tally gates on aircraft/vessel, and the walk skips any entity whose
property bag is empty, which is every billboard-only feature. GDELT was the only
claim-tier layer on screen, all 452 of its entities were exactly that shape, and
the histogram built to expose claim-tier sources could not see it. tallyTier runs
before both gates, because a contact's provenance does not depend on whether its
property bag was readable.
An unknown tier stays unknown throughout: it is not bucketed, and it cannot
satisfy an "only sensor" clause. Unknown provenance is not sensor provenance.
verify green, web 716 -> 721.
…y do The tier model demotes claim-tier sources rather than deleting them, and §0 of the plan says exactly why: a claim's one legitimate use is corroboration. Given something an instrument observed, does a claim exist near it in space and time, and how long after. That is a measurement ABOUT the claim, not a use of it, and until now it was a sentence in a document with nothing behind it. /api/intel/corroborate is that measurement. Verified live: 38 claim-tier reports within 60 km of Kyiv out of 1,500 in the window, the nearest 0.4 km away; and 0 within 60 km of a point in the mid-Pacific out of the same 1,500. The card reports 0 for an aircraft over Moldova as "nothing claim-tier was published within 50 km of this position in the window. That is a result, not a verdict on the observation." The wording is the feature. Proximity is not aboutness: a shelling report 12 km from a tanker at anchor may have nothing to do with the tanker. So the field is `nearby` and never `matching`, the heading counts reports near a position and never reports about a contact, and the note says so on every response and in the card. What the number IS good for is the lag, which is signed on purpose: a claim forty minutes after a transponder anomaly and one three days before it are different kinds of evidence. A dead claim feed is a 502, not an empty list. "Nobody said anything" and "we could not ask" are different answers, and the second dressed as the first is how an absence of evidence turns into evidence of absence. One real defect the tests found before anyone else could. GDELT dates events to the day as YYYYMMDD, and the first parser handed that to time.mktime, which NORMALISES rather than rejects: a garbage stamp of month 13 day 45 came back as a confident timestamp four months in the future, and would have produced a confident lag from nothing. The ranges are checked explicitly now. Backend baseline 2200 -> 2207, verify green.
Records the five acceptance criteria and the evidence for each, and corrects the remaining-work list: W5 is part done rather than not started (the histogram is a working cross-filter and now carries Provenance), and two §3 rows are known-bad and need re-sourcing rather than planning against — IRIS FDSN answers 410 since SAGE moved, and the Copernicus STAC row names a collection the v1 host returns nothing for.
Sixteen new backend routes shipped last week with no way into the product. A feed reaches the operator one of two ways: it carries coordinates and is a map layer, or it answers a question and is a row in a panel. Everything the wave added is now one or the other. Nineteen geo routes become registered layers with a provenance tier and a catalog row: Ukraine air-raid alerts (both relays), Meteoalarm, FEMA, SPC storm reports, DeepState's fire/radiation/report overlays, SatNOGS passes and stations, radiosondes, KiwiSDR receivers, mineral sites, OSM military land use, Wikimapia, the two privacy-filtered ADS-B feeds, and the two full airport gazetteers. Each draws a category tile dispatched on its own kind, so an oblast under alert is red and a clear one is slate. The rest have no position, so they get a Sources tab beside Investigate's Lookup: KEV, Shodan, GLEIF, CourtListener, UNHCR, WorldPop, HDX, Telegram, both GDELT endpoints, the four ADS-B identifier lookups, FR24, SatNOGS transmitters, tinyGS, the imagery and buildings manifests, splat search, and the source catalog itself. One generic runner, not eighteen bespoke views; a source that earns a real reading graduates out of the list. Overpass and Wikimapia now answer an empty collection with no bbox instead of 422, because above the LOD altitude the map deliberately sends none. Two guards hold the contract from both sides: the backend asserts every path is mounted, the frontend asserts every path is either a layer endpoint or a panel row.
Probed all forty-odd routes against their live upstreams. Six were reading a shape the upstream does not serve, and each failed quietly enough to look like "nothing is happening right now". DeepState's fire feed is a positional triple, [lat, lon, weight], not a list of objects; reading it as objects raised AttributeError and the route 500'd. Its history feed carries no lat/lon field at all — the position is the map pin in the description link, which is where the editor put it, so the entry is geolocated after all: 195 advance reports, newest first, markup stripped. The KiwiSDR public list is behind a CAPTCHA for every non-browser client, so the receiver list now comes from the dyatlov mirror that regenerates it: 852 stations. It is a JS literal with a trailing comma and a "(lat, lon)" string, and every failure path returns an empty FeatureCollection rather than a bare dict, because this route backs a map layer. MRDS has no JSON export and its WFS refuses application/json for the layer, so the GML is parsed with ElementTree: 1000 sites in an Oklahoma box. Meteoalarm has no JSON feed either, and a warning carries a NUTS3 region and no coordinates — so it answers a list and moves off the map into the Sources panel, rather than being plotted at a centroid it never claimed. HDX 406s our own User-Agent and nothing else; three UAs, one variable. GDELT's throttle and tinyGS's hang now say so, because an empty list reads as "no coverage", which is a different and wrong answer. Fixtures for all five wire formats, trimmed from today's live responses.
siren.pp.ua moved: /api/v1/states is a 404 and /api/states now sits behind Cloudflare, so the layer put a 502 in the browser console every thirty seconds. v3 answers a bare request, and it reports only the regions with an ACTIVE alert — the opposite of the primary relay, which lists all 25 oblasts with a boolean — so presence in the response is the alert. A second relay of the same alerts goes quiet when it is down rather than erroring: the primary layer still carries the picture, and an error badge on a map that is missing nothing is a false alarm about a false alarm.
Three defects the browser found that the guards could not. The DeepState layer ids began with `conflict.`, which is the prefix the compositor dispatches on, so all three went through the GDELT area adapter and were merged into 0.1-degree cells: 4904 fire points drew as 860, 195 reports as 62, and every one of them wore a conflict glyph instead of its own. They are named for their source now, and each feature is its own contact again. Clicking any of those contacts asked /api/entity for a kind the enrichment table has never heard of, so a new-layer selection put a 404 in the console and got no dossier. The table is keyed by the id prefix a feed MINTS, not by its ontology kind, which is why the entries read ds_rad and osm_mil. The Ukrainian oblasts were keyed by their Cyrillic names, which cannot survive the ASCII-only feed id regex — so every oblast contact was unresolvable by construction. They carry a slug now. Baseline 2207 to 2232.
Apple's geo service hands its own clients a session out of a public manifest and then signs every tile URL with it, so this is keyless in the same sense the rest of the basemaps are: no account, no key, no quota. The backend reads the manifest for the token and for style 7's data version, mints a session id, and AES-signs each tile; the browser only ever sees /tiles/apple and never carries a token. A tile with a stale version answers 410, so the version comes from the manifest and a 401/403/410 re-reads it once rather than leaving the basemap blank until a restart. Non-commercial: Apple's ToS is not a redistribution licence, so an entitled request is refused with 451 rather than quietly served, the way the CARTO and EOX sources are already handled. The picker itself was unreachable. It lived in CommandBar, which the console shell replaced with TitleBar, so every basemap but the default had no address in this shell — nine of them, including the two proxied stacks. The options move to their own module and the View menu is generated from them, for the same reason the colour schemes are. Verified over San Francisco: Apple's own capture, the street grid and Alcatraz sharp under the AIS labels. Baseline 2232 to 2236.
Two separate reports, one commit each would have been better, but they share a verify run. The menus opened all along. The dropdown rendered inside the header, and the header clips its overflow — that is what stops the control row growing a scrollbar — so a 792px menu painted three pixels of itself inside a 40px box. Every menu highlighted, set aria-expanded, and looked dead. Both dropdowns (the menu bar and the app launcher) are portalled to the body and positioned against their trigger's rect; the outside-click test learns that the popup is no longer inside the bar's subtree, or the next click would close nothing. The stale aircraft are a coverage problem, so this adds coverage rather than hiding the age. Flightradar24's own map feed is keyless — no account, no token — and it is fused ADS-B, MLAT and satellite, so it sees aircraft no ground-only readsb mirror can. It answers a bbox and caps every response at 1500 rows, so the world is walked as a 102-box grid with the busy quarters split; one pull is 3151 aircraft in 3.7 s at p50 4 s, p90 9 s. Folded into the existing freshest-observation-wins union it moved the whole snapshot's p90 from 96 s to 34 s and the share under 30 s from 86.0% to 89.6%, measured either side of the restart. Every field in that feed is an array index into an undocumented response, so each one is pinned by name against a captured row — a shifted index is silent and would fly aircraft at their vertical rate. Baseline 2236 to 2244.
The tier shipped gated on `adsb_sidecar_only`, and this box sets that flag, so it had never run once — the freshness numbers reported for it were normal variance, not the tier. That flag exists to shed the multi-MB aircraft.json MIRRORS on a CPU-starved box, which is a different load from ~100 small requests with a trivial parse, so the tier is appended after that filter and is governed only by its own flag. With it genuinely in the union, measured on this box against the same snapshot route: 14285 aircraft to 19665, p90 fix age 96 s to 20 s, share under 30 s 86.0% to 91.9%. Spot-checked contacts that were 543 s and 874 s stale now answer at 75 s, which is the honest arithmetic — a contact's reported age is FR24's own position age plus the age of the slice it arrived in, so the pull cadence is the floor and 20 s is where the request rate against somebody else's map backend stops being neighbourly. The suite turns the tier off: the feed tests stub the readsb fetch, this tier does not go through that stub, and 2000 live aircraft landing in the slice store mid-assertion is a test that passes for the wrong reason.
The bbox form is the cheap one, which is the whole point. Measured against the live API by reading x-rate-limit-remaining either side of each call: a 2x2 degree box costs 1 credit, a 10x10 box costs 2, the whole world costs 4, out of 400 a day. So the unauthenticated API affords ~400 boxes a day where it affords 100 world pulls, and this backend was spending its budget on the expensive form. Which boxes is the interesting half. A fixed grid would spend most of the budget re-observing sky three other tiers already carry at one second. Instead the ranking reads the snapshot we are currently serving, finds the cells holding the most contacts whose fixes have gone old, and buys those — worst first. Live on this box: one credit on 50N 2W, a cell holding 20 stale contacts, returned 131 aircraft and left 274 credits for the day. The budget is read, never assumed: every response states what is left, that number gates the next call, forty credits are reserved for operator lookups, and a 429 stops the tier for an hour. A state vector's own time_position becomes its seen_pos, so the tier competes honestly in the freshest-wins union and a vector with no position time loses rather than winning with a fabricated `now`. /api/status/perf now carries both new tiers, because "is it actually running" is a question I got wrong once already this session by inferring instead of asking. Snapshot on this box across the wave: 14285 aircraft at p90 96 s, to 20000 (the route cap) at p90 17.5 s, with 92.7% of fixes under 30 s. Baseline 2244 to 2252.
The credits are counted per source IP, so the daily global pull and the bbox gap filler were always spending the same 400. Neither knew it, which meant each discovered the other's spend by being refused — and the expensive shape wins that race, because four credits land in one call. Now every anonymous response reports what is left into the one budget object, and the global pull defers when the shared pool is near its reserve rather than spending four boxes' worth of sky on itself and leaving the cheap tier dead until midnight. With OAuth credentials the pool is separate and larger, so the check does not apply. Live after the restart: three feed slices, the bbox tier among them, one credit on 50N 2W for 126 aircraft, 264 left for the day. Baseline 2252 to 2254.
The backend had grown 373 routes and 87 of them had no caller anywhere in apps/web/src, so the capability existed and nothing in the product could reach it. The dark set was the analysis layer: the whole intel.py analytics cluster (density, jamming, anomalies, deception, emitter, baseline, incident-history, area bundle, aircraft and vessel queries, loaded AOIs, feed health), all four routing modes, the SAR vessel sweep, imagery availability and tasking providers, backend timing and provenance, the movement diff, and a dozen reference feeds. These go to the existing address for a route with no map position, the Sources panel under Investigate, rather than a new app. That panel already states the rule: a source that earns a real reading graduates out of the directory to the surface that reads it, so a row is the honest first home and a bespoke panel is what to build when one earns it. The 36 single-source OSINT connectors get one row with a connector picker instead of 36 near-identical rows; they were already reachable through the investigate fan-out that composes them server-side. The guard is the durable part. SourcesPanel.test.ts held a hand-written list of one wave's routes, which cannot catch the next wave. routeCoverage.test.ts walks every @router decorator in apps/api/app/routes and fails on any route that is neither called from apps/web/src nor listed in its EXEMPT map with a reason. Twelve exceptions are recorded: liveness probes, POST and agent-driven side effects, an image proxy consumed as a URL, one alias, and one route built from a template literal the search cannot see. A stale EXEMPT key fails too, so the list cannot outlive its routes. Also in here, from the same working tree: - Apple Maps satellite tiles ask for size=2 (512px retina) and stitch four z+1 children below the Esri zoom cap, so the imagery is sharp instead of upscaled. Boot pre-warms z0-4 (341 tiles) after bootstrapping the signing session, and a concurrency gate holds bursts at 24 so a cold pan does not trip throttling. - The ADS-B mirror interval drops 5s to 3s. Stale fixes are what make tracked aircraft jump, and the mirrors tolerate the faster poll. - Selection tracks use ArcType.NONE for aircraft at altitude and keep GEODESIC only for ground-clamped vessels. Segments are short enough that curvature is invisible, and NONE avoids the subdivision-density shift that made the trail jump between zoom levels. - Icon display range widens to 80,000 km so contacts stay drawn at full zoom-out. - The Apple tile prefetch in GlobeCanvas goes through apiFetch, so it carries auth like every other backend call, and two ruff findings in main.py and adsb.py are cleared. Both were failing scripts/verify.sh. Verified: 38 of the 39 newly addressed routes answer 200 against :8000 (/api/audit 401s to bare curl by design, it is auth-gated and apiFetch carries the key), a browser walkthrough ran the situation, anomalies, timing, provenance, SAR sweep, route candidates and DNS connector rows with real bodies rendered, and scripts/verify.sh is green at 2255 passed + 2 skipped.
AndrewCTF
force-pushed
the
backend-route-coverage
branch
from
August 8, 2026 03:28
1b8840b to
0f8563f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything on
gotham-console-mockupafter #65, ending with a guard thatstops backend capability from shipping where nobody can reach it. 31 commits
off
master.Console chrome
Colour schemes landed with a browser-checked palette, including four
Blueprint schemes held to Palantir's published ramp by a check, typeface
included. Every surface has one address, the chrome that highlighted and did
nothing now runs, window menus are visible, toolbar groups are named, and
what is still not Gotham about it is stated plainly rather than papered
over.
Provenance and honesty
Provenance became a question the operator can ask of the map rather than a
label printed on it: per tier, how many contacts it saw and how many only it
saw. A claim may corroborate an observation and nothing more. Sanctions
screening joins three lists onto contacts that are moving right now, and the
map stops asserting what a name match cannot support. The dossier states the
tier, which was the one place a contact did not carry it.
Feeds and egress
The mega-ledger feeds got an address an operator can reach and then got made
to answer with data instead of an empty envelope. A fourth ADS-B tier
landed; FR24 was allowed to actually run; the anonymous OpenSky budget is now
spent as bboxes over the sky we cover worst, with both anonymous callers
drawing from one budget. Apple Maps satellite is on the globe, at retina
resolution with stitching below the Esri zoom cap and a pre-warmed z0-4.
Route coverage (the last commit)
373 backend routes, 87 with no caller anywhere in
apps/web/src. The dark setwas the analysis layer: the whole
intel.pyanalytics cluster, all fourrouting modes, the SAR vessel sweep, imagery availability and tasking, backend
timing and provenance, the movement diff, and a dozen reference feeds. They go
to the existing address for a route with no map position, the Sources panel
under Investigate, rather than a new app.
routeCoverage.test.tsis the durable part: it walks every@routerdecorator in
apps/api/app/routesand fails on any route that is neithercalled from
apps/web/srcnor listed in itsEXEMPTmap with a reason. Astale
EXEMPTkey fails too, so the list cannot outlive its routes.Verification
bash scripts/verify.shgreen: typecheck, lint, web unit, and2255 passed + 2 skipped on the backend suite.
:8000(
/api/audit401s to bare curl by design — it is auth-gated andapiFetchcarries the key).
provenance, SAR sweep, route-candidates and DNS-connector rows with real
bodies rendered.