Skip to content

Commit 1794cf1

Browse files
committed
feat: add fast live processing previews
Run previews in the existing worker runtime to avoid a second Pyodide startup while keeping temporary results isolated from workspace state. Assisted-by: GPT-5.6 Sol
1 parent a2d52e4 commit 1794cf1

31 files changed

Lines changed: 2978 additions & 200 deletions

‎CHANGELOG.md‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,12 +6,22 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)
66

77
## [Unreleased]
88

9+
### Added
10+
11+
- **Optional processing previews:** compatible signal and image processings
12+
can now render a temporary result while their parameters are edited. The
13+
preview is disabled by default, reuses the already-loaded computation worker
14+
without changing the workspace, supports bounded numeric sliders and
15+
image-profile geometry, and is discarded on Cancel or Escape; OK still
16+
performs the normal processing from the original source.
17+
918
### Changed
1019

1120
- **Guidata compatibility:** browser runtimes now require guidata 3.15 and use
1221
its native async DataSet backend and JSON Schema exporter. The former
1322
backend backport has been removed; a focused compatibility patch preserves
14-
the `FloatArrayItem` sizing hints required by the browser array editor.
23+
the `FloatArrayItem` sizing and numeric-parity hints required by browser
24+
editors.
1525

1626
## [0.9.0] - 2026-08-11
1727

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ The latest release is deployed automatically to GitHub Pages. Open the link in a
2121
DataLab Web mirrors a large portion of the desktop application surface:
2222

2323
- **Signal & image panels** — 1D curves and 2D arrays with a rich set of synthetic generators, full Plotly visualisation, cross-hair markers, contrast adjustment, cross profiles and stats area tools.
24-
- **Processing & analysis** — operations, transforms, filters, fitting, FFT/PSD, stability analyses, measurements and profile extraction, exposed automatically through the menu bar by introspecting Sigima's catalog.
24+
- **Processing & analysis** — operations, transforms, filters, fitting, FFT/PSD, stability analyses, measurements and profile extraction, exposed automatically through the menu bar by introspecting Sigima's catalog. Compatible parameterised processings offer an optional non-publishing live preview before anything is added to the workspace.
2525
- **ROI & object tree** — segment / rectangular / circular / polygonal regions of interest, plus a multi-group workspace with drag & drop, metadata editor, statistics card and computation history.
2626
- **Macros & notebooks** — embedded Python editor and multi-tab notebook panel, each running in dedicated Web Workers, with `.ipynb` import/export and bidirectional macro ⇄ notebook conversion. See [doc/notebooks.md](doc/notebooks.md).
2727
- **Plugins** — Qt-compatible `PluginBase` API: the same plugin source runs in DataLab desktop and DataLab Web. See [doc/plugins.md](doc/plugins.md).

‎doc/architecture.md‎

Lines changed: 34 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,14 @@ flowchart TB
3333
Actions["Action registry<br/>(src/actions)"]
3434
DBridge["DialogBridge.tsx"]
3535
RT["DataLabRuntime (TS)<br/>(src/runtime/runtime.ts)"]
36+
Preview["ProcessingPreviewController.ts<br/>bounded preview queue"]
3637
Py0["Pyodide #0 — main<br/>bootstrap.py · processor.py<br/>dlw_*.py · Sigima · numpy · scipy"]
3738
3839
UI -->|dispatch| Actions
3940
Actions -->|typed calls| RT
4041
RT -->|runPython| Py0
42+
UI --> Preview
43+
Preview -->|previewFeature| RT
4144
Py0 -->|dialog requests| DBridge
4245
DBridge --> UI
4346
end
@@ -68,6 +71,9 @@ Key facts:
6871
- **Each macro / notebook runs in its own Web Worker** with its own
6972
Pyodide instance. Workers never touch the main object model directly:
7073
they call back to the main runtime through `proxyBridge`.
74+
- **Processing preview reuses the already-loaded computation runtime**. Python
75+
deep-copies the source, computes a temporary plotting payload, and never
76+
publishes the copy or result to the main object model.
7177
- **The host page (optional)** controls an embedded DataLab-Web through
7278
the same RPC vocabulary, but transported by `remoteBridge` over
7379
`window.postMessage`.
@@ -83,7 +89,7 @@ right.
8389
flowchart LR
8490
L1["<b>L1 — UI layer</b><br/>(React components)<br/><br/>src/components/<br/>src/App.tsx<br/>src/main.tsx<br/><br/><i>Purely presentational.<br/>No Pyodide imports.</i>"]
8591
L2["<b>L2 — Orchestration</b><br/><br/>src/actions/<br/>(registry, menu builder)<br/>src/macros/<br/>src/notebook/<br/>src/plugins/<br/>src/aiassistant/<br/>src/preferences/"]
86-
L3["<b>L3 — Runtime / bridge</b><br/>(TypeScript)<br/><br/>RuntimeApi.ts (façade)<br/>runtime.ts<br/>(DataLabRuntime + types)<br/>WorkerRuntimeProxy.ts · kernelWorker.ts<br/>runtimeMode.ts · workerProtocol.ts<br/>RuntimeContext.tsx<br/>WorkspaceContext.tsx<br/>MacroRuntime.ts<br/>proxyBridge.ts · remoteBridge.ts<br/>macroWorker.ts · notebookWorker.ts<br/>src/storage/ (OPFS spill stores)"]
92+
L3["<b>L3 — Runtime / bridge</b><br/>(TypeScript)<br/><br/>RuntimeApi.ts (façade)<br/>runtime.ts<br/>(DataLabRuntime + types)<br/>WorkerRuntimeProxy.ts · kernelWorker.ts<br/>runtimeMode.ts · workerProtocol.ts<br/>ProcessingPreviewController.ts<br/>RuntimeContext.tsx<br/>WorkspaceContext.tsx<br/>MacroRuntime.ts<br/>proxyBridge.ts · remoteBridge.ts<br/>macroWorker.ts · notebookWorker.ts<br/>src/storage/ (OPFS spill stores)"]
8793
L4["<b>L4 — Python kernel</b><br/>(loaded into Pyodide)<br/><br/>bootstrap.py<br/>processor.py<br/>dlw_main.py<br/>dlw_wheels.py<br/>dlw_plugins.py<br/>dlw_applications.py<br/>dlw_h5browser.py<br/>dlw_interactive_fit.py<br/>dlw_title_format.py<br/>notebook_display.py<br/>macro_proxy.py<br/>_guidata_*_shim.py"]
8894
L5["<b>L5 — Computation engine</b><br/><br/>Sigima<br/>+ numpy · scipy<br/>+ scikit-image<br/>+ h5py · pandas …<br/><br/><i>Installed via micropip<br/>on first load.</i>"]
8995
@@ -128,9 +134,12 @@ Presentational React. Notable components:
128134
metadata / stats / history.
129135
- `SignalPlot.tsx`, `ImagePlot.tsx`, `MultiImagePlot.tsx`,
130136
`CentralViewSwitcher.tsx` — Plotly-based central plot area.
131-
- `DataSetDialog.tsx` + `DataSetForm/` — **auto-generated parameter
137+
- `DataSetDialog.tsx`, `ProcessingDataSetDialog.tsx` + `DataSetForm/` —
138+
**auto-generated parameter
132139
dialogs** from guidata DataSet JSON schemas (do not hand-write a form
133140
unless the auto path genuinely cannot express it).
141+
- `ProcessingPreview.tsx` + `ProcessingPreviewPlot.tsx` — optional,
142+
non-publishing signal/image result preview for eligible processing dialogs.
134143
- `DialogBridge.tsx` — single React entry-point that **receives dialog
135144
requests from Python** via `bootstrap.set_dialog_bridge()` and routes
136145
them to the appropriate React dialog component.
@@ -233,7 +242,9 @@ flowchart LR
233242
MR["MacroRuntime.ts<br/>spawns workers on demand<br/>keeps a warm standby<br/>postMessage glue"]
234243
PB2["proxyBridge.ts<br/>whitelisted command router"]
235244
RT3["DataLabRuntime"]
245+
PC["ProcessingPreviewController.ts<br/>one active + latest pending"]
236246
PB2 --> RT3
247+
PC -->|previewFeature| RT3
237248
end
238249
239250
subgraph Workers2["Web Workers"]
@@ -261,6 +272,19 @@ call. Workers cannot invoke arbitrary methods — only the ones explicitly
261272
listed (add_signal, add_image, list/get/delete object, add/set object
262273
from pickle, add_group, get/set selection, etc.).
263274

275+
`ProcessingPreviewController` coordinates requests per `RuntimeApi`: one
276+
computation may run while a single pending slot retains only the latest
277+
parameters. `bootstrap.preview_feature` deep-copies the source and delegates
278+
to the same `BaseProcessor.apply` path as publication, then returns only a
279+
binary plotting payload. It never inserts the temporary object into `_MODEL`,
280+
updates processing history, or dirties the workspace.
281+
282+
Disabling preview, Cancel, Escape, or dialog unmount drops pending work and
283+
invalidates late results. A Python call that has already started is allowed to
284+
finish; it may delay the next serialised runtime call. Preview controls are
285+
therefore exposed only for the default worker-hosted runtime and hidden in the
286+
`runtime=main` fallback, where computation would block the UI thread.
287+
264288
#### Remote control bridge
265289

266290
```mermaid
@@ -457,12 +481,14 @@ compatibility independently of the application version.
457481
```text
458482
1. User clicks Processing › <feature> in MenuBar
459483
2. action.run(ctx) (from src/actions/registry.ts)
460-
3. If params needed → DataSetDialog (auto-generated from JSON schema)
461-
4. runtime.applyFeature(featureId, selectedOids, paramValues)
462-
5. pyodide.runPython → processor.apply_*(feature, oids, params)
463-
6. Sigima computes; new SignalObj/ImageObj added to _MODEL
464-
7. bootstrap returns new oids; UI refreshes ObjectTree + plots
465-
8. WorkspaceContext is marked dirty
484+
3. If params needed → ProcessingDataSetDialog (auto-generated from JSON schema)
485+
4. Optional Preview → runtime.previewFeature(featureId, sourceOid, paramValues)
486+
5. bootstrap deep-copies the source and returns a private plotting payload
487+
6. OK invalidates preview and calls runtime.applyFeature(featureId, selectedOids, paramValues)
488+
7. pyodide.runPython → processor.apply_*(feature, oids, params)
489+
8. Sigima computes; new SignalObj/ImageObj added to _MODEL
490+
9. bootstrap returns new oids; UI refreshes ObjectTree + plots
491+
10. WorkspaceContext is marked dirty
466492
```
467493

468494
### 5.2 Running a macro

‎doc/userguide/computation-engine.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,27 @@ Each processing's parameters are described by a
3333
reads that schema and renders the dialog automatically — the same data
3434
model that the desktop application uses to draw its Qt forms.
3535

36+
Eligible bounded numeric parameters also get a slider beside the exact
37+
numeric input. The slider never replaces typed entry and is omitted when the
38+
schema does not provide safe finite bounds.
39+
40+
Compatible one-input/one-output processings expose an optional **Preview**
41+
checkbox, initially off. Enabling it reuses the already-loaded computation
42+
runtime to process a private copy of the source; it does not add an object,
43+
alter history, or mark the workspace as modified. Edits are debounced while
44+
typing and sampled while dragging a slider, with an immediate refresh when the
45+
slider is released. If several objects were selected, **Preview source**
46+
chooses the representative input without changing the selection or the final
47+
batch operation.
48+
49+
**Cancel** and **Esc** discard the temporary result and any waiting refresh.
50+
An already-running computation finishes in the background and its late result
51+
is ignored. **OK** runs the normal processing path from the original source
52+
and current parameters. Preview is hidden in the `runtime=main` fallback to
53+
avoid running this interactive work on the UI thread. The Processing tab in
54+
the object side panel retains its explicit **Apply** and **Reset** controls;
55+
editing that tab never republishes a result automatically.
56+
3657
## What the desktop docs do **not** cover
3758

3859
- The **runtime constraints** of the browser (memory limits, OPFS,

‎src/App.tsx‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ import {
9090
import { MULTI_SIGNAL_LIMIT, useSelectionView } from "./hooks/useSelectionView";
9191
import { useSignalAxisGroups } from "./hooks/useSignalAxisGroups";
9292
import { DataSetDialog } from "./components/DataSetDialog";
93+
import { ProcessingDataSetDialog } from "./components/ProcessingDataSetDialog";
9394
import type { ProfileFeatureId } from "./components/ProfileDefinitionDialog";
9495
import { OperandPicker } from "./components/OperandPicker";
9596
import { HelpDialog, type HelpView } from "./components/HelpDialog";
@@ -389,7 +390,7 @@ function usePersistedSignalLayoutMode(): [
389390
}
390391

391392
export default function App() {
392-
const { runtime, status, message, error } = useRuntime();
393+
const { runtime, status, message, error, previewAvailable } = useRuntime();
393394
const workspace = useWorkspace();
394395
const confirm = useConfirm();
395396
const notify = useMessage();
@@ -5129,9 +5130,13 @@ export default function App() {
51295130
/>
51305131
)}
51315132
{pending && runtime && (
5132-
<DataSetDialog
5133+
<ProcessingDataSetDialog
51335134
title={pending.feature.label.replace(/\u2026$/, "")}
51345135
payload={pending.schema!}
5136+
runtime={runtime}
5137+
previewAvailable={previewAvailable}
5138+
feature={pending.feature}
5139+
sourceIds={pending.sourceIds}
51355140
resolveChoices={(itemName, currentValues) =>
51365141
runtime.resolveFeatureChoices(
51375142
pending.feature.id,
@@ -5159,6 +5164,10 @@ export default function App() {
51595164
<Suspense fallback={null}>
51605165
<ProfileDefinitionDialog
51615166
title={pendingProfile.feature.label.replace(/\u2026$/, "")}
5167+
runtime={runtime}
5168+
previewAvailable={previewAvailable}
5169+
feature={pendingProfile.feature}
5170+
sourceIds={pendingProfile.sourceIds}
51625171
featureId={
51635172
pendingProfile.feature.id.replace(
51645173
/^image:/,
@@ -5172,13 +5181,15 @@ export default function App() {
51725181
pendingProfile.feature.id,
51735182
itemName,
51745183
currentValues,
5184+
pendingProfile.sourceIds[0],
51755185
)
51765186
}
51775187
resolveCallbacks={(itemName, currentValues) =>
51785188
runtime.resolveFeatureCallbacks(
51795189
pendingProfile.feature.id,
51805190
itemName,
51815191
currentValues,
5192+
pendingProfile.sourceIds[0],
51825193
)
51835194
}
51845195
resolveActive={(currentValues) =>

0 commit comments

Comments
 (0)