Obelisk

A deterministic workflow engine built on the WASM Component Model.

Warning

Pre-release: Expect changes in CLI, gRPC, WIT, and database schema.

Obelisk in Action

The Stargazers app features:

• A webhook listening to GitHub star events.
• Activities for interacting with Turso DB, OpenAI, and GitHub.
• A workflow orchestrating the activities.

What's Included

• Obelisk Runtime: A single binary executing deterministic workflows, activities, and webhook endpoints, persisting steps in execution log using SQLite.
• Control Interfaces:
  • CLI: Manage executions via the obelisk command.
  • gRPC API: Programmatic interaction.
  • Web UI: View and submit function executions, execution logs, WIT definitions, and a time-traveling debugger showing backtraces and sources of recorded events.

Core Principles

• Replayable Workflows: Deterministic execution ensures reliable recovery, debugging, and auditing.
• Resilient Activities: Automatic retries on errors and timeouts, with persistent input and results.
• Schema-first design with end-to-end type safety: Uses WASM Component Model and WIT IDL for generating API bindings.

Use Cases

• Periodic Tasks: Automate periodic checks with complex logic using regular code.
• Background Jobs: Offload tasks with built-in error handling and retries.
• Batch jobs: Manage large-scale tasks like faas deployments.
• End-to-End Testing: Automate tests with detailed logs of each step.

Key Features

WASI Activities

• Activities must be idempotent (retriable). This contract must be fulfilled by the activity itself.
• Support for HTTP requests via WASI 0.2 HTTP client.
• Automatic retries on errors, timeouts, and panics (WASM traps).
• Persistent execution results.

Deterministic Workflows

• Runtime guaranteed determinism, fully replayable with persistent execution log and thus crash resilient.
• Automatic retries on failures.
• Support for spawning child executions with structured concurrency.
• Distributed sagas (planned).

WASI Webhook Endpoints

• Mounted as a URL path, serving HTTP traffic.
• Support for spawning child executions.

Work Stealing Executor

• Concurrency limits and customizable retry handling.

Installation

Supported Platforms

• Linux x64, arm64 (musl, glibc v2.35+, NixOS)
• MacOS x64, arm64

Pre-built Binary

curl -L --tlsv1.2 -sSf https://raw.githubusercontent.com/obeli-sk/obelisk/main/download.sh | bash

Or use cargo-binstall:

cargo binstall obelisk

Docker

docker run --net=host getobelisk/obelisk

From Source

Requires protoc.

cargo install --locked obelisk

Using Nix:

nix run github:obeli-sk/obelisk/latest

Getting Started

Check out the Getting Started Guide for details.

Start the Server

obelisk server run

CLI Usage

obelisk client component list
# Call fibonacci(10) activity from the workflow 500 times in series.
obelisk client execution submit testing:fibo-workflow/workflow.fiboa '[10, 500]' --follow

Web UI

Visit localhost:8080 to manage components, functions and execution history.

Contributing

This project has a roadmap and features are added in a certain order. Discuss features via GitHub Discussions before contributing. A Contributor License Agreement is required.

Development

Set up dependencies via Nix:

nix develop

Or manually install dependencies (see dev-deps.txt).

Run the program:

cargo run --release

Running Tests

./scripts/test.sh

Project Licensing Information

This project (all files and folders except as noted below) is licensed under the GNU Affero General Public License version 3.

Subfolder Exceptions

The following subfolders are licensed under the MIT License:

• wit/ – see LICENSE-MIT
• toml/ – see LICENSE-MIT
• proto/ – see LICENSE-MIT

Generated WIT Files

The runtime includes functionality that may generate new "extension" WIT files. These generated files are typically based on a combination of user-provided WIT files and the base WIT definitions found in the MIT-licensed wit/ directory.

Users are free to use, modify, and distribute these generated WIT files under the terms of the MIT License, for example, to allow other WASM components to interact via these extended interfaces.