Fix missing runtime dependencies (pyarrow, kuzu)

[mdjhacker]

Problem

Running the following command fails at runtime due to missing dependencies:

bash examples/generate/generate_cot_qa/generate_cot.sh

Error 1: missing pyarrow

ModuleNotFoundError: No module named 'pyarrow'
  File "ray/data/_internal/util.py", line 33, in <module>
    import pyarrow

ray.data requires pyarrow, but it is not explicitly declared or checked, causing an import-time crash.

---

Error 2: missing kuzu

ImportError: KuzuDB is not installed. Please install it via `pip install kuzu`.
  File "graphgen/models/storage/graph/kuzu_storage.py", line 32, in __post_init__

This error occurs during GraphStorageActor initialization, after part of the pipeline has already started, leading to Ray actor failure.

---

Fix

• Add pyarrow and kuzu to requirements.txt to resolve runtime import errors in ray.data and GraphStorageActor.

[gemini-code-assist]

Summary of Changes

Hello @mdjhacker, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves critical runtime failures by explicitly adding pyarrow and kuzu to the project's requirements.txt. Previously, these packages were implicitly required by core components like ray.data and GraphStorageActor, leading to ModuleNotFoundError and ImportError respectively. By declaring these dependencies, the PR ensures a stable and reproducible environment, preventing crashes and streamlining the setup process for users.

Highlights

• Dependency Resolution: Addressed two critical runtime errors caused by missing pyarrow and kuzu dependencies.
• Pyarrow Fix: Resolved ModuleNotFoundError for pyarrow which was required by ray.data but not explicitly declared.
• Kuzu Fix: Fixed ImportError for kuzu that occurred during GraphStorageActor initialization.
• Requirements Update: Included kuzu and pyarrow in requirements.txt to ensure all necessary packages are installed.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request correctly fixes runtime errors by adding missing kuzu and pyarrow dependencies to requirements.txt. My review focuses on improving dependency management practices. I've suggested pinning the versions of the new dependencies to ensure reproducible builds. I also provided recommendations for long-term improvements, such as using ray extras for pyarrow and considering extras_require in setup.py for optional storage backends.

[gemini-code-assist]

Thanks for adding the missing dependencies. To improve dependency management and ensure reproducible builds, I have a couple of suggestions:

1. Pin dependency versions: It's a best practice to pin dependency versions (e.g., kuzu==X.Y.Z) to prevent your application from breaking when a new version of a dependency is released. Please add version specifiers for kuzu and pyarrow.

2. Manage pyarrow via ray extras: pyarrow is a dependency of ray.data. The idiomatic way to manage this is to specify it as an extra for ray, like ray[data]==2.52.1. This automatically pulls in the correct pyarrow version. While the ray dependency line isn't in this diff, I recommend creating a follow-up task to make this change for better dependency management.

For this PR, please at least pin the versions of the newly added dependencies.

[ChenZiHong-Gavin]

Thanks, LGTM.