Support Separate Process for Reward Computation via Inter-Process Communication #146
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement Inter-Process Communication (IPC) Module Between GFlowNet and Reward Function
This PR introduces an inter-process communication (IPC) module that enables reward computation in a separate process from GFlowNet.
Implementation Details
The implementation includes three key classes (
gflownet.utils.communication):IPCModule– A core module for communication between GFlowNet and the reward function.IPCTask– A GFlowNet task withIPCModule.RewardModule– A reward function withIPCModule.Supported IPC Methods
The current implementation supports three IPC methods:
NetworkIPC– Uses a TCP/IP protocol (without file system). Recommended.FileSystemIPC– Uses the file system with serialized data. Recommended.FileSystemIPC_CSV– Uses the file system (csv format). It may be useful for non-Python reward functions or human feedback integration.User can setup IPC at
config.communication.There are two example scripts:
src/gflownet/tasks/seh_frag_ipc.py,src/gflownet/tasks/seh_frag_moo_ipc.pyImpact on Existing Code
This PR introduces minimal changes to the existing codebase. The key differences compared to the latest trunk are:
<2.0.0) inpyproject.toml.communicationtoconfig.