Skip to content

Conversation

@ChenZiHong-Gavin
Copy link
Collaborator

@ChenZiHong-Gavin ChenZiHong-Gavin commented Nov 6, 2025

This PR adds a lightweight orchestration engine to GraphGen, turning the former hard-coded, sequential pipeline into a configurable, concurrent DAG.
Key changes:

  • Introduce Engine (topological sort + thread-pool execution) and @op decorator to declare tasks and dependencies.
  • Replace generate.py with run.py; it loads the DAG from pipeline sections in YAML configs and executes it.
  • Split the old generate() method into two atomic ops: partition and generate, letting the engine handle their dependency.
  • Update all config files to the new pipeline: syntax and rename shell scripts from generate_.sh to run_.sh.
  • Add unit tests for cyclic-dependency detection and correct DAG order.

@ChenZiHong-Gavin ChenZiHong-Gavin marked this pull request as ready for review November 6, 2025 08:14
@ChenZiHong-Gavin ChenZiHong-Gavin merged commit 519dfef into main Nov 7, 2025
3 checks passed
@ChenZiHong-Gavin ChenZiHong-Gavin deleted the feature/orchestration-operation branch November 7, 2025 08:27
@ChenZiHong-Gavin ChenZiHong-Gavin mentioned this pull request Nov 18, 2025
31 tasks
CHERRY-ui8 pushed a commit to CHERRY-ui8/GraphGen that referenced this pull request Dec 17, 2025
…estration-operation

feat: add orchestration engine for GraphGen and tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants