Skip to content

Commit e27e93b

Browse files
committed
clean-up proto changes
Signed-off-by: Filinto Duran <[email protected]>
1 parent 6aab54d commit e27e93b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+13310
-599
lines changed

.pre-commit-config.yaml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# Pre-commit configuration for durabletask-python
2+
repos:
3+
- repo: https://github.com/pre-commit/pre-commit-hooks
4+
rev: v4.4.0
5+
hooks:
6+
- id: trailing-whitespace
7+
- id: end-of-file-fixer
8+
- id: check-yaml
9+
- id: check-added-large-files
10+
- id: check-merge-conflict
11+
12+
- repo: https://github.com/astral-sh/ruff-pre-commit
13+
rev: v0.2.2
14+
hooks:
15+
- id: ruff
16+
args: [--fix, --exit-non-zero-on-fix]
17+
- id: ruff-format
18+
19+
- repo: https://github.com/pre-commit/mirrors-mypy
20+
rev: v1.8.0
21+
hooks:
22+
- id: mypy
23+
additional_dependencies: [types-protobuf]
24+
args: [--config-file=mypy.ini]
25+
files: ^durabletask/
26+
exclude: ^durabletask/internal/.*_pb2\.py$
27+
28+
- repo: local
29+
hooks:
30+
- id: pytest-asyncio
31+
name: Run asyncio tests
32+
entry: python -m pytest tests/aio/ -q
33+
language: system
34+
pass_filenames: false
35+
stages: [pre-push] # Only run on git push, not every commit
36+
37+
38+

README.md

Lines changed: 116 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -118,15 +118,15 @@ Orchestrations can start child orchestrations using the `call_sub_orchestrator`
118118

119119
Orchestrations can wait for external events using the `wait_for_external_event` API. External events are useful for implementing human interaction patterns, such as waiting for a user to approve an order before continuing.
120120

121-
### Continue-as-new (TODO)
121+
### Continue-as-new
122122

123123
Orchestrations can be continued as new using the `continue_as_new` API. This API allows an orchestration to restart itself from scratch, optionally with a new input.
124124

125125
### Suspend, resume, and terminate
126126

127127
Orchestrations can be suspended using the `suspend_orchestration` client API and will remain suspended until resumed using the `resume_orchestration` client API. A suspended orchestration will stop processing new events, but will continue to buffer any that happen to arrive until resumed, ensuring that no data is lost. An orchestration can also be terminated using the `terminate_orchestration` client API. Terminated orchestrations will stop processing new events and will discard any buffered events.
128128

129-
### Retry policies (TODO)
129+
### Retry policies
130130

131131
Orchestrations can specify retry policies for activities and sub-orchestrations. These policies control how many times and how frequently an activity or sub-orchestration will be retried in the event of a transient error.
132132

@@ -191,6 +191,120 @@ To run the E2E tests, run the following command from the project root:
191191
make test-e2e
192192
```
193193

194+
### Configuration
195+
196+
The SDK connects to a Durable Task sidecar. By default it uses `localhost:4001`. You can override via environment variables (checked in order):
197+
198+
- `DURABLETASK_GRPC_ENDPOINT` (e.g., `localhost:4001`, `grpcs://host:443`)
199+
- `DURABLETASK_GRPC_HOST` and `DURABLETASK_GRPC_PORT`
200+
- `TASKHUB_GRPC_ENDPOINT` (legacy)
201+
202+
Example (common ports: 4001 for DurableTask-Go emulator, 50001 for Dapr sidecar):
203+
204+
```sh
205+
export DURABLETASK_GRPC_ENDPOINT=localhost:4001
206+
# or
207+
export DURABLETASK_GRPC_ENDPOINT=localhost:50001
208+
```
209+
210+
### Async authoring compatibility
211+
212+
You can author orchestrators with `async def` using `add_async_orchestrator`, which provides awaitables for activities, timers, external events, and when_all/any:
213+
214+
```python
215+
from durabletask.worker import TaskHubGrpcWorker
216+
217+
async def my_orch(ctx, input):
218+
r1 = await ctx.activity("act1", input=input)
219+
await ctx.sleep(1)
220+
r2 = await ctx.activity("act2", input=r1)
221+
return r2
222+
223+
with TaskHubGrpcWorker() as worker:
224+
worker.add_async_orchestrator(my_orch, name="my_orch", sandbox_mode="off")
225+
```
226+
227+
Optional sandbox mode (`best_effort` or `strict`) patches `asyncio.sleep`, `random`, `uuid.uuid4`, and `time.time` within the workflow step to deterministic equivalents. This is best-effort and not a correctness guarantee.
228+
229+
In `strict` mode, `asyncio.create_task` is blocked inside workflows to preserve determinism and will raise a `RuntimeError` if used.
230+
231+
#### Async patterns
232+
233+
- Activities:
234+
```python
235+
result = await ctx.activity("process", input={"x": 1})
236+
```
237+
238+
- Timers:
239+
```python
240+
await ctx.sleep(1.5) # seconds or timedelta
241+
```
242+
243+
- External events:
244+
```python
245+
val = await ctx.wait_for_external_event("approval")
246+
```
247+
248+
- Concurrency:
249+
```python
250+
t1 = ctx.activity("a"); t2 = ctx.activity("b")
251+
await ctx.when_all([t1, t2])
252+
winner = await ctx.when_any([ctx.wait_for_external_event("x"), ctx.sleep(5)])
253+
```
254+
255+
- Sub-orchestrations (function reference or registered name):
256+
```python
257+
out = await ctx.sub_orchestrator(child_fn, input=payload)
258+
# or: out = await ctx.sub_orchestrator("child", input=payload)
259+
```
260+
261+
- Deterministic utilities:
262+
```python
263+
now = ctx.now(); rid = ctx.random().random(); uid = ctx.uuid4()
264+
```
265+
266+
#### Worker readiness
267+
268+
When starting a worker and scheduling immediately, wait for the connection to the sidecar to be established:
269+
270+
```python
271+
with TaskHubGrpcWorker() as worker:
272+
worker.add_async_orchestrator(my_orch, name="my_orch")
273+
worker.start()
274+
worker.wait_for_ready(timeout=5)
275+
# Now safe to schedule
276+
```
277+
278+
#### Suspension & termination
279+
280+
- `ctx.is_suspended` reflects suspension state during replay/processing.
281+
- Suspend pauses progress without raising inside async orchestrators.
282+
- Terminate completes with `TERMINATED` status; use client APIs to terminate/resume.
283+
- Only new events are buffered while suspended; replay events continue to apply to rebuild local state deterministically.
284+
285+
### Tracing and context propagation
286+
287+
The SDK surfaces W3C tracing context provided by the sidecar:
288+
289+
- Orchestrations: `ctx.trace_parent`, `ctx.trace_state`, and `ctx.orchestration_span_id` are available on `OrchestrationContext` (and on `AsyncWorkflowContext`).
290+
- Activities: `ctx.trace_parent` and `ctx.trace_state` are available on `ActivityContext`.
291+
292+
Propagate tracing to external systems (e.g., HTTP):
293+
294+
```python
295+
def activity(ctx, payload):
296+
headers = {
297+
"traceparent": ctx.trace_parent or "",
298+
"tracestate": ctx.trace_state or "",
299+
}
300+
# requests.post(url, headers=headers, json=payload)
301+
return "ok"
302+
```
303+
304+
Notes:
305+
- The sidecar controls inbound `traceparent`/`tracestate`. App code can append vendor entries to `tracestate` for outbound calls but cannot currently alter the sidecar’s propagation for downstream Durable operations.
306+
- Configure the sidecar endpoint with `DURABLETASK_GRPC_ENDPOINT` (e.g., `127.0.0.1:56178`).
307+
194308
## Contributing
195309

196310
This project welcomes contributions and suggestions. Most contributions require you to agree to a

durabletask/__init__.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,9 @@
55

66

77
PACKAGE_NAME = "durabletask"
8+
9+
# Public async exports (import directly from durabletask.aio)
10+
from durabletask.aio import ( # noqa: F401
11+
AsyncWorkflowContext,
12+
CoroutineOrchestratorRunner,
13+
)

durabletask/aio/__init__.py

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
"""Async workflow primitives (aio namespace).
2+
3+
This package contains the async implementation previously under
4+
`durabletask.asyncio`, now moved to `durabletask.aio` for naming
5+
consistency.
6+
"""
7+
8+
# Deterministic utilities
9+
from durabletask.deterministic import (
10+
DeterminismSeed,
11+
DeterministicContextMixin,
12+
derive_seed,
13+
deterministic_random,
14+
deterministic_uuid4,
15+
)
16+
17+
# Awaitable classes
18+
from .awaitables import (
19+
ActivityAwaitable,
20+
AwaitableBase,
21+
ExternalEventAwaitable,
22+
SleepAwaitable,
23+
SubOrchestratorAwaitable,
24+
SwallowExceptionAwaitable,
25+
TimeoutAwaitable,
26+
WhenAllAwaitable,
27+
WhenAnyAwaitable,
28+
WhenAnyResultAwaitable,
29+
gather,
30+
)
31+
32+
# Compatibility protocol (core functionality only)
33+
from .compatibility import OrchestrationContextProtocol, ensure_compatibility
34+
35+
# Core context and driver
36+
from .context import AsyncWorkflowContext
37+
from .driver import CoroutineOrchestratorRunner, WorkflowFunction
38+
39+
# Sandbox and error handling
40+
from .errors import (
41+
AsyncWorkflowError,
42+
NonDeterminismWarning,
43+
SandboxViolationError,
44+
WorkflowTimeoutError,
45+
WorkflowValidationError,
46+
)
47+
from .sandbox import (
48+
SandboxMode,
49+
_NonDeterminismDetector,
50+
sandbox_best_effort,
51+
sandbox_off,
52+
sandbox_scope,
53+
sandbox_strict,
54+
)
55+
56+
__all__ = [
57+
# Core classes
58+
"AsyncWorkflowContext",
59+
"CoroutineOrchestratorRunner",
60+
"WorkflowFunction",
61+
62+
# Deterministic utilities
63+
"DeterministicContextMixin",
64+
"DeterminismSeed",
65+
"derive_seed",
66+
"deterministic_random",
67+
"deterministic_uuid4",
68+
69+
# Awaitable classes
70+
"AwaitableBase",
71+
"ActivityAwaitable",
72+
"SubOrchestratorAwaitable",
73+
"SleepAwaitable",
74+
"ExternalEventAwaitable",
75+
"WhenAllAwaitable",
76+
"WhenAnyAwaitable",
77+
"WhenAnyResultAwaitable",
78+
"TimeoutAwaitable",
79+
"SwallowExceptionAwaitable",
80+
"gather",
81+
82+
# Sandbox and utilities
83+
"sandbox_scope",
84+
"SandboxMode",
85+
"sandbox_off",
86+
"sandbox_best_effort",
87+
"sandbox_strict",
88+
"_NonDeterminismDetector",
89+
90+
# Compatibility protocol
91+
"OrchestrationContextProtocol",
92+
"ensure_compatibility",
93+
94+
# Exceptions
95+
"AsyncWorkflowError",
96+
"NonDeterminismWarning",
97+
"WorkflowTimeoutError",
98+
"WorkflowValidationError",
99+
"SandboxViolationError",
100+
]
101+
102+

0 commit comments

Comments
 (0)