feat: Add Goose AI developer agent example (do not merge) #862
Conversation
- Add example demonstrating how to use Goose to test PyAirbyte - Include session file with test tasks for Goose - Add example Python test script that Goose can execute - Document installation, usage, and MCP integration - Provide comparison with Hercules approach Co-Authored-By: AJ Steers <[email protected]>
Original prompt from AJ Steers |
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This PyAirbyte VersionYou can test this version of PyAirbyte using the following: # Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1762924931-goose-ai-test-agent' pyairbyte --help
# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1762924931-goose-ai-test-agent'Helpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
Community SupportQuestions? Join the #pyairbyte channel in our Slack workspace. |
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
- Remove trailing whitespace - Collapse multi-line function arguments to single line - Fix formatting issues caught by CI Co-Authored-By: AJ Steers <[email protected]>
📝 WalkthroughWalkthroughThree new files were added to Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
examples/ai-test-agents/goose/README.md (4)
20-33: Consider using proper headings for installation options?The static analysis tools suggest using proper markdown headings instead of bold text for "Option 1", "Option 2", and "Option 3". This would improve document structure. Something like:
-**Option 1: Using Homebrew (macOS/Linux)** +#### Option 1: Using Homebrew (macOS/Linux)But honestly, the bold format works fine for a casual example doc like this. Wdyt?
53-59: Add language identifier to code block for better rendering?The code block at line 53 (the example prompt) doesn't have a language identifier. You could add
textormarkdownto improve rendering:-``` +```text Test PyAirbyte by:Static analysis flagged this, but it's pretty minor for example docs. Wdyt?
99-125: Add language identifiers to example task code blocks?The code blocks for Tasks 1-3 (lines 99, 111, 119) don't have language identifiers. Since these are natural language prompts, you could add
text:-``` +```text Write and run a Python script that:This applies to all three task examples. Static analysis flagged these, but they're pretty readable as-is. Wdyt?
131-146: Add language identifier to example output?The example session output block at line 131 could use a language tag like
textorconsolefor better rendering:-``` +```text 🪿 Goose: I'll help you test PyAirbyte. Let me create a test script...Minor formatting thing flagged by static analysis. Wdyt?
examples/ai-test-agents/goose/test_pyairbyte_session.md (1)
43-70: Tasks 3 & 4 not yet implemented in example_test.py?These are great test scenarios! I notice that
test_basic_source_connectorandtest_connector_discoveryimplement Tasks 1 & 2, but Tasks 3 (stream selection) and 4 (schema validation) don't have corresponding implementations in example_test.py yet.Since this is a "do not merge" bake-off PR, maybe that's intentional? Or would you like me to help generate implementations for these tasks?
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
examples/ai-test-agents/goose/README.md(1 hunks)examples/ai-test-agents/goose/example_test.py(1 hunks)examples/ai-test-agents/goose/test_pyairbyte_session.md(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
examples/ai-test-agents/goose/example_test.py (2)
airbyte/caches/util.py (1)
new_local_cache(43-81)airbyte/registry.py (1)
get_available_connectors(226-282)
🪛 LanguageTool
examples/ai-test-agents/goose/README.md
[style] ~152-~152: Consider using a different verb for a more formal wording.
Context: ...de 3. Error Handling: Can debug and fix issues it encounters during testing 4. ...
(FIX_RESOLVE)
🪛 markdownlint-cli2 (0.18.1)
examples/ai-test-agents/goose/README.md
20-20: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
25-25: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
30-30: Emphasis used instead of a heading
(MD036, no-emphasis-as-heading)
53-53: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
111-111: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
119-119: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
131-131: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (6)
- GitHub Check: Pytest (All, Python 3.11, Windows)
- GitHub Check: Pytest (All, Python 3.11, Ubuntu)
- GitHub Check: Pytest (All, Python 3.10, Windows)
- GitHub Check: Pytest (All, Python 3.10, Ubuntu)
- GitHub Check: Pytest (No Creds)
- GitHub Check: Pytest (Fast)
🔇 Additional comments (7)
examples/ai-test-agents/goose/example_test.py (2)
11-37: LGTM on the basic test flow!The test logic for creating a source, checking connection, reading to cache, and validating data looks solid. One small thing to verify: does the "source-faker" connector guarantee a "users" stream? It might be worth checking the available streams first or handling the case where the stream name differs. Wdyt?
59-79: LGTM on the test orchestration!The main function provides clear output and appropriate error handling by catching, printing, and re-raising exceptions. The sequential execution is fine for an example script.
examples/ai-test-agents/goose/README.md (2)
71-92: Verify the MCP tool count claim?Line 92 mentions "44+ MCP tools" - is this number current and accurate for PyAirbyte's MCP server? It might be worth verifying or making it more general like "access to PyAirbyte's MCP tools" to avoid the number becoming stale. Wdyt?
171-176: All documentation links are accessible, though the Airbyte URLs redirectThe verification results show all four links are reachable:
- Goose documentation and GitHub repository both return
200(directly accessible)- PyAirbyte documentation and MCP server docs both return
301(Moved Permanently with redirects)The links function correctly, but the Airbyte URLs redirect, which suggests they might not be the canonical endpoints. Would you consider checking if there are preferred/canonical URLs for the PyAirbyte documentation links, wdyt? This could make the documentation more future-proof.
examples/ai-test-agents/goose/test_pyairbyte_session.md (3)
11-27: Task 1 aligns well with example_test.py!The task description matches the
test_basic_source_connectorfunction nicely. Same note as for the Python file: might want to verify that "users" stream actually exists in source-faker, or handle it gracefully if the stream name differs. Wdyt?
29-41: Task 2 description looks good!The task description for connector discovery is clear and matches the intent of
test_connector_discovery. Just note that the implementation in example_test.py has the API usage bug I flagged there (wrong parameter name and return type handling). This description is fine though.
72-86: LGTM on success criteria and notes!The success criteria and guidance for Goose are clear and comprehensive. The emphasis on error handling, cleanup, and clear output messages is great for making this a useful example.
- Rename file from example_test.py to goose_demo.py - Rename functions from test_* to example_* - Prevents pytest from discovering and running example code as tests - Fixes CI test failure where example code was being executed Co-Authored-By: AJ Steers <[email protected]>
|
|
||
| from airbyte.registry import get_available_connectors | ||
|
|
||
| sources = get_available_connectors(connector_type="source") |
|
|
||
| print("Reading data into cache...") | ||
| cache = ab.new_local_cache() | ||
| result = source.read(cache) |
feat: Add Goose AI developer agent example (do not merge)
Summary
This PR adds an example demonstrating how to use Goose (an open-source AI developer agent from Block) to test and validate PyAirbyte functionality. This is the second part of a "bake off" comparison between Hercules and Goose AI test agents (see PR #861 for the Hercules implementation).
Key additions:
examples/ai-test-agents/goose/README.mdexample_test.py) that demonstrates PyAirbyte functionalitytest_pyairbyte_session.md) with predefined test tasks for GooseUnlike Hercules (which uses Gherkin format for predefined test scenarios), Goose provides a more flexible, conversational approach to testing with code generation and debugging capabilities.
Review & Testing Checklist for Human
python examples/ai-test-agents/goose/example_test.pyto ensure it executes successfully and demonstrates PyAirbyte functionality correctly.test_pyairbyte_session.mdfollows the correct format that Goose expects for session plans.Notes
Important: This PR contains only documentation and example code. The example script has not been tested in a live environment, so there may be bugs or incorrect API usage. The installation instructions and MCP configuration should be verified before considering this ready for merge.
This work was requested by AJ Steers (@aaronsteers, [email protected]) as part of an AI test agent "bake off" comparison.
Link to Devin run: https://app.devin.ai/sessions/0d255eb4e8c14c6a93cd7cbc5c8f9a72
Summary by CodeRabbit
New Features
Documentation