-
Notifications
You must be signed in to change notification settings - Fork 13
feat: restructure repository #40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
e25e44e
250c646
610a819
73328e1
f748111
423bc9b
f61fe4b
bb428c1
5734cd1
b0266d4
f52e68b
fc9f9fc
39edddf
6a14bb7
15c178c
3528977
73bddfe
c973014
ff5d71c
8d98f4e
313bd74
233837a
eee34af
6220636
5a02668
8e4b043
fbfcaae
d785039
9b4736d
564bd6e
5db88a5
d71bfd3
88896ab
e48cbe8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: CI - CrewAI Backend | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/crewai_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-crewai-backend.yml' | ||
| - '.github/workflows/quality-crewai-backend.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/crewai_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-crewai-backend.yml' | ||
| - '.github/workflows/quality-crewai-backend.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-crewai-backend.yml | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: CI - DSPy Backend | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/dspy_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-dspy-backend.yml' | ||
| - '.github/workflows/quality-dspy-backend.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/dspy_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-dspy-backend.yml' | ||
| - '.github/workflows/quality-dspy-backend.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-dspy-backend.yml | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: CI - LangChain Backend | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/langchain_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-langchain-backend.yml' | ||
| - '.github/workflows/quality-langchain-backend.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/langchain_backend/**' | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-langchain-backend.yml' | ||
| - '.github/workflows/quality-langchain-backend.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-langchain-backend.yml | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| name: CI - Mellea Integration Core | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-mellea-integration-core.yml' | ||
| - '.github/workflows/quality-mellea-integration-core.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/mellea-integration-core/**' | ||
| - '.github/workflows/ci-mellea-integration-core.yml' | ||
| - '.github/workflows/quality-mellea-integration-core.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-mellea-integration-core.yml | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| name: CI - Reqlib Package | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/reqlib_package/**' | ||
| - '.github/workflows/ci-reqlib-package.yml' | ||
| - '.github/workflows/quality-reqlib-package.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/reqlib_package/**' | ||
| - '.github/workflows/ci-reqlib-package.yml' | ||
| - '.github/workflows/quality-reqlib-package.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-reqlib-package.yml |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,169 @@ | ||
| name: CI - Subpackage Validation | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/**' | ||
| - '.github/workflows/ci-*.yml' | ||
| - '.github/workflows/quality-*.yml' | ||
| - '.github/workflows/ci-subpackage-validation.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/**' | ||
| - '.github/workflows/ci-*.yml' | ||
| - '.github/workflows/quality-*.yml' | ||
| - '.github/workflows/ci-subpackage-validation.yml' | ||
|
|
||
| jobs: | ||
| validate-subpackage-ci: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - name: Validate subpackage CI configurations | ||
| run: | | ||
| set -e | ||
|
|
||
| echo "=== Validating CI configurations for mellea_contribs subpackages ===" | ||
| echo "" | ||
|
|
||
| # Track validation status | ||
| all_valid=true | ||
| missing_ci=() | ||
| missing_quality=() | ||
| invalid_paths=() | ||
| missing_test_jobs=() | ||
|
|
||
| # Find all subpackages (directories with pyproject.toml) | ||
| subpackages=() | ||
| for dir in mellea_contribs/*/; do | ||
| if [ -f "${dir}pyproject.toml" ]; then | ||
| subpackage=$(basename "$dir") | ||
| subpackages+=("$subpackage") | ||
| fi | ||
| done | ||
|
|
||
| echo "Found ${#subpackages[@]} subpackages:" | ||
| printf ' - %s\n' "${subpackages[@]}" | ||
| echo "" | ||
|
|
||
| # Validate each subpackage | ||
| for subpackage in "${subpackages[@]}"; do | ||
| echo "Checking: $subpackage" | ||
|
|
||
| # Convert subpackage name to CI filename format | ||
| # e.g., langchain_backend -> ci-langchain-backend.yml | ||
| ci_name=$(echo "$subpackage" | sed 's/_/-/g') | ||
| ci_file=".github/workflows/ci-${ci_name}.yml" | ||
| quality_file=".github/workflows/quality-${ci_name}.yml" | ||
|
|
||
| # Check if CI file exists | ||
| if [ ! -f "$ci_file" ]; then | ||
| echo " ❌ Missing CI file: $ci_file" | ||
| missing_ci+=("$subpackage") | ||
| all_valid=false | ||
| continue | ||
| fi | ||
| echo " ✓ CI file exists: $ci_file" | ||
|
|
||
| # Check if quality workflow file exists | ||
| if [ ! -f "$quality_file" ]; then | ||
| echo " ❌ Missing quality workflow: $quality_file" | ||
| missing_quality+=("$subpackage") | ||
| all_valid=false | ||
| continue | ||
| fi | ||
| echo " ✓ Quality workflow exists: $quality_file" | ||
|
|
||
| # Validate CI file has correct path triggers | ||
| expected_path="mellea_contribs/${subpackage}/" | ||
| if ! grep -q "paths:" "$ci_file"; then | ||
| echo " ❌ CI file missing 'paths:' section" | ||
| invalid_paths+=("$subpackage") | ||
| all_valid=false | ||
| elif ! grep -q "'${expected_path}\*\*'" "$ci_file" && ! grep -q "\"${expected_path}\*\*\"" "$ci_file"; then | ||
| echo " ❌ CI file missing path trigger for: ${expected_path}**" | ||
| invalid_paths+=("$subpackage") | ||
| all_valid=false | ||
| else | ||
| echo " ✓ CI file has correct path triggers" | ||
| fi | ||
|
|
||
| # Validate CI file references quality workflow | ||
| if ! grep -q "uses: ./.github/workflows/quality-${ci_name}.yml" "$ci_file"; then | ||
| echo " ⚠️ Warning: CI file may not reference quality workflow correctly" | ||
| else | ||
| echo " ✓ CI file references quality workflow" | ||
| fi | ||
|
|
||
| # Validate quality workflow has test jobs | ||
| if ! grep -q "jobs:" "$quality_file"; then | ||
| echo " ❌ Quality workflow missing 'jobs:' section" | ||
| missing_test_jobs+=("$subpackage") | ||
| all_valid=false | ||
| elif ! grep -q "pytest" "$quality_file" && ! grep -q "test" "$quality_file"; then | ||
| echo " ⚠️ Warning: Quality workflow may not run tests (no 'pytest' or 'test' found)" | ||
| else | ||
| echo " ✓ Quality workflow has test jobs" | ||
| fi | ||
|
|
||
| echo "" | ||
| done | ||
|
|
||
| # Summary report | ||
| echo "=== Validation Summary ===" | ||
| echo "" | ||
|
|
||
| if [ "$all_valid" = true ]; then | ||
| echo "✅ All subpackages have valid CI configurations!" | ||
| exit 0 | ||
| else | ||
| echo "❌ Validation failed for one or more subpackages" | ||
| echo "" | ||
|
|
||
| if [ ${#missing_ci[@]} -gt 0 ]; then | ||
| echo "Subpackages missing CI files:" | ||
| printf ' - %s\n' "${missing_ci[@]}" | ||
| echo "" | ||
| fi | ||
|
|
||
| if [ ${#missing_quality[@]} -gt 0 ]; then | ||
| echo "Subpackages missing quality workflows:" | ||
| printf ' - %s\n' "${missing_quality[@]}" | ||
| echo "" | ||
| fi | ||
|
|
||
| if [ ${#invalid_paths[@]} -gt 0 ]; then | ||
| echo "Subpackages with invalid path triggers:" | ||
| printf ' - %s\n' "${invalid_paths[@]}" | ||
| echo "" | ||
| fi | ||
|
|
||
| if [ ${#missing_test_jobs[@]} -gt 0 ]; then | ||
| echo "Subpackages with missing test jobs:" | ||
| printf ' - %s\n' "${missing_test_jobs[@]}" | ||
| echo "" | ||
| fi | ||
|
|
||
| echo "Please ensure each subpackage has:" | ||
| echo " 1. A CI workflow file: .github/workflows/ci-<subpackage-name>.yml" | ||
| echo " 2. A quality workflow file: .github/workflows/quality-<subpackage-name>.yml" | ||
| echo " 3. Correct path triggers in the CI file" | ||
| echo " 4. Test jobs in the quality workflow" | ||
| echo "" | ||
| echo "Example CI file structure:" | ||
| echo " on:" | ||
| echo " pull_request:" | ||
| echo " paths:" | ||
| echo " - 'mellea_contribs/<subpackage>/**'" | ||
| echo " - '.github/workflows/ci-<subpackage>.yml'" | ||
| echo " - '.github/workflows/quality-<subpackage>.yml'" | ||
| echo " jobs:" | ||
| echo " test:" | ||
| echo " uses: ./.github/workflows/quality-<subpackage>.yml" | ||
|
|
||
| exit 1 | ||
| fi |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| name: CI - Tools Package | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'mellea_contribs/tools_package/**' | ||
| - '.github/workflows/ci-tools-package.yml' | ||
| - '.github/workflows/quality-tools-package.yml' | ||
| push: | ||
| branches: | ||
| - main | ||
| paths: | ||
| - 'mellea_contribs/tools_package/**' | ||
| - '.github/workflows/ci-tools-package.yml' | ||
| - '.github/workflows/quality-tools-package.yml' | ||
|
|
||
| jobs: | ||
| test: | ||
| uses: ./.github/workflows/quality-tools-package.yml |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,66 @@ | ||
| name: "Run CI" | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, reopened, synchronize] | ||
| workflow_dispatch: | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this would need to be run manually right? would this be more for auditing or do you see another purpose?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, this must be run manually. I just left it for convenience. It can be deleted. |
||
|
|
||
| jobs: | ||
| code-checks: | ||
| uses: ./.github/workflows/quality.yml | ||
| detect-changes: | ||
| runs-on: ubuntu-latest | ||
| outputs: | ||
| mellea-integration-core: ${{ steps.changes.outputs.mellea-integration-core }} | ||
| crewai-backend: ${{ steps.changes.outputs.crewai-backend }} | ||
| dspy-backend: ${{ steps.changes.outputs.dspy-backend }} | ||
| langchain-backend: ${{ steps.changes.outputs.langchain-backend }} | ||
| reqlib-package: ${{ steps.changes.outputs.reqlib-package }} | ||
| tools-package: ${{ steps.changes.outputs.tools-package }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
| - name: Detect changed files | ||
| id: changes | ||
| run: | | ||
| # Get the base and head commits | ||
| BASE=${{ github.event.pull_request.base.sha }} | ||
| HEAD=${{ github.event.pull_request.head.sha }} | ||
|
|
||
| # Get list of changed files | ||
| CHANGED_FILES=$(git diff --name-only $BASE...$HEAD) | ||
|
|
||
| # Check each package | ||
| echo "mellea-integration-core=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/mellea-integration-core/\|\.github/workflows/quality-mellea-integration-core\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
| echo "crewai-backend=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/crewai_backend/\|\.github/workflows/quality-crewai-backend\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
| echo "dspy-backend=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/dspy_backend/\|\.github/workflows/quality-dspy-backend\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
| echo "langchain-backend=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/langchain_backend/\|\.github/workflows/quality-langchain-backend\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
| echo "reqlib-package=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/reqlib_package/\|\.github/workflows/quality-reqlib-package\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
| echo "tools-package=$(echo "$CHANGED_FILES" | grep -q 'mellea_contribs/tools_package/\|\.github/workflows/quality-tools-package\.yml' && echo 'true' || echo 'false')" >> $GITHUB_OUTPUT | ||
|
|
||
| mellea-integration-core: | ||
| needs: detect-changes | ||
| uses: ./.github/workflows/quality-mellea-integration-core.yml | ||
| if: needs.detect-changes.outputs.mellea-integration-core == 'true' | ||
|
|
||
| crewai-backend: | ||
| needs: [detect-changes, mellea-integration-core] | ||
| uses: ./.github/workflows/quality-crewai-backend.yml | ||
| if: needs.detect-changes.outputs.crewai-backend == 'true' | ||
|
|
||
| dspy-backend: | ||
| needs: [detect-changes, mellea-integration-core] | ||
| uses: ./.github/workflows/quality-dspy-backend.yml | ||
| if: needs.detect-changes.outputs.dspy-backend == 'true' | ||
|
|
||
| langchain-backend: | ||
| needs: [detect-changes, mellea-integration-core] | ||
| uses: ./.github/workflows/quality-langchain-backend.yml | ||
| if: needs.detect-changes.outputs.langchain-backend == 'true' | ||
|
|
||
| reqlib-package: | ||
| needs: detect-changes | ||
| uses: ./.github/workflows/quality-reqlib-package.yml | ||
| if: needs.detect-changes.outputs.reqlib-package == 'true' | ||
|
|
||
| tools-package: | ||
| needs: detect-changes | ||
| uses: ./.github/workflows/quality-tools-package.yml | ||
| if: needs.detect-changes.outputs.tools-package == 'true' | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since all the backends depend on mellea-integration-core, there is the chance that if we change something in the core that it would break the backends... should we also trigger a run of the backends if the core changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's right. I'll add mellea-integration-core change in all integration paths.