feat(dynaflow): built-in node library and graph execution pipeline (#283)

[shivamm-gupta]

Summary

Closes #283

This PR implements the built-in node library and graph execution pipeline for dynaflow under Milestone P1 (Orchestration Engine). It delivers 9 standard, typed node primitives with ReactFlow canvas metadata compatibility, a global node registry with palette export, and a directed execution pipeline (SimpleGraph).

Architecture & Components

1. Base Schema & Node Abstractions (dynavec.nodes.base):

   • NodePort: Typed input and output connection ports (string, number, boolean, array, object, any) with defaults and requirement flags.
   • NodeSchema: Node metadata describing category, inputs, outputs, config schemas, and ReactFlow-compatible canvas export via to_reactflow_dict().
   • Node: Abstract base class providing instance identity, config management, execute(inputs), and canvas node serialization via to_dict().

2. 9 Standard Built-in Nodes (dynavec.nodes.builtin):

   • RetrieverNode: Vector database retrieval against dynavec index, producing structured document lists and concatenated context blocks.
   • LLMNode: Language model invocation supporting ChatModel instances or callable functions with automatic grounding context integration and system prompts.
   • ToolNode: Execution of callable functions, tools, and MCP actions with automatic JSON argument deserialization and status tracking.
   • RouterNode: Dynamic branch routing via custom predicate functions or explicit route overrides.
   • CodeNode: Execution of custom Python transform functions.
   • ApprovalNode: Human-in-the-loop review and approval checkpoints.
   • LoopNode: Iterative map processing over item arrays with configurable max_iterations.
   • WebhookNode: Event ingestion entrypoint for external HTTP webhook triggers.
   • SubGraphNode: Modular encapsulation of nested graphs or sub-workflows.

3. Node Registry (dynavec.nodes.registry):

   • NodeRegistry: Registry pattern allowing node class registration, schema introspection, and full ReactFlow component palette export via node_registry.export_canvas_palette().

4. Pipeline Execution Graph (dynavec.nodes.graph):

   • Edge: Directed connection supporting source/target port binding and dictionary key mapping.
   • SimpleGraph: Directed pipeline that threads state across nodes in topological order.

Validation & Testing

• Added tests/test_nodes.py with 15 comprehensive unit tests:
  • Node registration and canvas palette export validation.
  • Serialization testing for ports, schemas, and ReactFlow nodes.
  • Independent execution tests for all 9 built-in nodes.
  • End-to-end multi-step pipeline execution: RetrieverNode -> LLMNode -> ToolNode.
  • Edge and invalid node error handling.
• Lint and formatting validated with ruff check and ruff format.

[codeforstartups]

Reviewed — a big, well-structured step for dynaflow (covers #282 node schema + #283 built-in nodes). 🎯

Node system — NodePort / NodeSchema (serializable, ReactFlow-compatible shape) + a Node ABC, with the full built-in set: Retriever, LLM, Tool, Router, Code, Approval, Loop, Webhook, and SubGraph — plus a NodeRegistry palette. The LLMNode works with either a ChatModel or a plain callable, and ToolNode handles both dict and JSON-string arguments with error capture.

Graph pipeline — SimpleGraph with typed Edges, add_node/add_edge, run(), and validation errors for malformed graphs — a solid first execution pass that the full engine (#277–#281) can build on.

15 tests cover every node type, port/schema round-trip, the registry palette, validation errors, and an end-to-end retriever→llm→tool pipeline. All 4 CI checks green incl. strict mypy. Merging — excellent work @shivamm-gupta!