Skip to content

feat: 5505 Harden Python custom logic block sandbox & extend libraries#5873

Open
nikolay-zezin wants to merge 7 commits intohashgraph:developfrom
nikolay-zezin:feat/5505
Open

feat: 5505 Harden Python custom logic block sandbox & extend libraries#5873
nikolay-zezin wants to merge 7 commits intohashgraph:developfrom
nikolay-zezin:feat/5505

Conversation

@nikolay-zezin
Copy link
Copy Markdown

@nikolay-zezin nikolay-zezin commented Mar 24, 2026

This PR hardens the Python custom logic block sandbox and extends supported Python libraries. It enforces input-data-only operations, prevents process initiation, restricts output to block channels, and adds an experimental Docker container isolation mode with native CPython.

  • Add scikit-learn, xarray, geopandas, pint to supported Python libraries (Extend the Python libraries supported by Guardian #5504)
  • Block JavaScript bridge, pyodide.http, os.system/exec/spawn, subprocess.run/Popen, socket
    networking functions in Pyodide worker
  • Clear os.environ secrets, block importlib.reload, guard builtins.import via closure
  • Add PEP 451 import hook blocking cffi, _posixsubprocess
  • Remove unnecessary libraries: duckdb, sqlalchemy, bokeh, altair, cartopy, seaborn
  • Add Pyodide warmup at service startup to pre-cache packages
  • Add shared package list in python-packages.ts (single source of truth)
  • Add experimental Docker mode (PYTHON_SANDBOX_MODE=docker) with native CPython 3.12
  • Add Docker container security: --network=none, --cap-drop=ALL, --read-only, --user=1001,
    --security-opt=no-new-privileges
  • Add CPython entrypoint with smart JSON serializer for numpy/pandas/datetime types
  • Add rasterio, rioxarray support in Docker mode (GDAL-dependent, unavailable in WASM)
  • Add safeResolve/safeReject guards to prevent double promise settlement
  • Add pendingDones array with Promise.all to prevent losing in-flight done() work

Related issue(s):

Fixes #5505
Fixes #5504

Notes for reviewer:

Two execution modes controlled by PYTHON_SANDBOX_MODE env var:

  • pyodide (default) — Pyodide/WASM in Worker Thread, Python-level sandbox restrictions
  • docker (experimental) — native CPython 3.12 in ephemeral Docker container, OS-level isolation

Docker mode requires building the sandbox image:
docker buildx build -t guardian/python-sandbox:latest policy-service/docker/python-sandbox

Known Pyodide-mode limitations (documented, mitigated by Docker mode):

  • Python introspection bypasses (closure, gc.get_objects()) — fundamental to
    application-level sandboxing
  • ctypes not blocked (pandas depends on it, harmless in WASM)
  • open() not blocked (breaks all libraries, WASM virtual FS mitigates)

@nikolay-zezin nikolay-zezin requested review from a team as code owners March 24, 2026 09:23
@Pyatakov Pyatakov self-assigned this Mar 25, 2026
@nikolay-zezin nikolay-zezin requested review from a team as code owners March 27, 2026 08:45
@nikolay-zezin
feat: 5504 Add scikit-learn, xarray, geopandas to Python custom logic…
b8f5e2a 
… block

Extend supported Python libraries per issue hashgraph#5504.

Added libraries:
- scikit-learn: machine learning (classification, regression, clustering)
- xarray: labeled multi-dimensional arrays for climate/environmental data
- geopandas: geospatial DataFrames with geometry and spatial operations

Already available (no install needed):
- calendar, datetime, collections, math, copy: Python built-ins
- dateutil, six: transitive dependencies of pandas

Not available in Pyodide (WASM):
- rasterio: depends on GDAL (C/C++ library not compiled to WASM)
- rioxarray: depends on rasterio
Workaround: pre-process raster data outside the block (e.g. convert
GeoTIFF to CSV/JSON) and pass as input documents.

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
feat: 5505 Harden Python custom logic block sandbox
239b3ae 
- Replace js module with restricted stub (blocks from js import fetch
  and all JS bridge access, survives re-import attempts)
- Replace pyodide.http with restricted stub (prevents re-import)
- Block all os.exec*/os.spawn*/os.system/os.popen functions
- Block subprocess.run/call/check_call/check_output/Popen
  (module remains importable for library compatibility)
- Install sys.meta_path import hook to prevent bypassing module
  restrictions via __import__ or importlib
- Remove unnecessary libraries: duckdb, sqlalchemy, bokeh, altair,
  cartopy, seaborn (matplotlib remains as transitive dep of networkx)

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
feat: experimental Docker container isolation for Python custom logic…
0e0e8b8 
… block

Add Docker-based sandbox for Python code execution in custom logic
blocks. Set PYTHON_SANDBOX_MODE=docker to enable (default is Pyodide
worker for backward compatibility).

Container security (no resource limits — matching develop):
- --network=none, --cap-drop=ALL, --security-opt=no-new-privileges
- --read-only, --user=1001:1001 (non-root)
- --name=python-sandbox-<uuid> (named for cleanup)
- --log-driver=none, --pull=never
- --tmpfs /tmp:rw,noexec,nosuid,size=64m
- Image name validation (regex)

Defense-in-depth Python sandbox (both paths):
- js/pyodide.http stubs + import hook
- builtins.__import__ guarded via closure (hides _original_import)
- os.system/exec*/spawn*/popen, subprocess.run/call/Popen blocked
- os.environ cleared, importlib.reload blocked
- ctypes/cffi/_posixsubprocess import blocked
- processLine checks settled before firing callbacks

Pyodide worker improvements:
- Timeout (PYTHON_SANDBOX_TIMEOUT_MS, default 120s)
- worker.on('exit') rejects on non-zero exit code
- safeResolve/safeReject prevent double settlement
- disposeTables() called on all exit/error paths

Docker worker: promise-only errors, settled guards in all callback
paths, processLine helper, stdin error handling, done(final=true)
tracking, package load failure reporting, non-blocking cleanup.

Bug fixes: debug field (data.message->data.result), command injection,
disposeTables in error paths, __globals__ bypass, pint removed.

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
docs: update Python implementation guide with new libraries and sandb…
254c528 
…ox security

Update supported libraries list: add scikit-learn, xarray, geopandas.
Document removed libraries (duckdb, sqlalchemy, pint, bokeh, altair,
cartopy, seaborn) with reasons. Add sandbox security section covering
blocked operations and execution modes (Pyodide default, Docker
experimental). Document built-in modules and transitive dependencies.

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
feat: replace Pyodide/WASM with CPython in Docker sandbox, harden Pyo…
e1ee1be 
…dide worker

Docker sandbox: replace Node.js + Pyodide (WASM) with native CPython
3.12. Same JSON stdin/stdout protocol — zero changes in host-side
Docker worker. Benefits: <1s startup (was 30-60s), ~300MB memory (was
2-4GB), native speed, rasterio/rioxarray now available.

Pyodide worker hardening:
- Block socket networking functions (socket.socket, create_connection,
  getaddrinfo, gethostbyname, etc.)
- Update import hook to PEP 451 API (find_spec)
- Extract shared package list to python-packages.json

Both paths:
- Accumulate pendingDone promises via array + Promise.all (fixes race
  where multiple done() calls could lose in-flight work)
- Smart JSON serializer for numpy/pandas/datetime types in CPython
- Fix DockerCallbacks.onDone type to Promise<void> | void
- Remove unused traceback import

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
docs: comprehensive Python implementation guide with Docker mode and …
4de29b4 
…security

Update python-implementation-in-guardian.md with:
- Library versions for all installed packages
- Docker-only libraries (rasterio, rioxarray)
- Full Docker mode documentation (setup, benefits, security flags)
- Execution modes comparison (Pyodide vs Docker)
- Sandbox security details for both modes
- Vulnerability comparison table
- Configuration reference

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
@nikolay-zezin
fix: prevent unhandled promise rejection when done() throws
2f60c6d 
Add .catch(safeReject) to pendingDones promises so that errors from
done() (e.g. invalid output schema) are caught instead of becoming
unhandled promise rejections that crash the process.

In develop, these errors are caught by the worker message handler's
try/catch → reject(). With pendingDones pattern, the promise could
reject after the exit handler already resolved, causing a crash.

Signed-off-by: nikolay-zezin <Nikolay.Zezin@waveaccess.global>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rbarker-dev rbarker-dev Awaiting requested review from rbarker-dev rbarker-dev was automatically assigned from hashgraph/platform-ci

@gregscullard gregscullard Awaiting requested review from gregscullard gregscullard was automatically assigned from hashgraph/developer-advocates

Assignees

@Pyatakov Pyatakov

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nikolay-zezin @Pyatakov