Skip to content

Add Android e2e tests workflow#44

Closed
gricha wants to merge 4 commits into
mainfrom
android-e2e-ci
Closed

Add Android e2e tests workflow#44
gricha wants to merge 4 commits into
mainfrom
android-e2e-ci

Conversation

@gricha
Copy link
Copy Markdown
Owner

@gricha gricha commented Jan 8, 2026

Summary

  • Adds Android e2e test workflow to measure actual CI time cost
  • Tests full e2e with backend (Docker + agent) like web tests do
  • Uses KVM hardware acceleration on ubuntu-latest
  • Includes Gradle and AVD snapshot caching

What we're testing

  • Can Android emulator run on Linux with acceptable speed?
  • Is full e2e (with backend) feasible time-wise?
  • Compare with iOS tests (~45 min on macOS)

Configuration

  • API level 34 with google_apis
  • Maestro tests excluding chat (same as iOS for fair comparison)
  • Full backend: Docker image build, agent startup, test workspace creation

🤖 Generated with Claude Code

Tests full e2e with backend (like web tests) to compare with iOS.
Includes:
- KVM hardware acceleration
- Gradle/AVD caching
- Full backend startup with Docker
- Maestro tests on Android emulator

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Comment on lines +56 to +61
- name: Install root dependencies
run: bun install

- name: Install mobile dependencies
working-directory: mobile
run: bun install

This comment was marked as outdated.

adb exec-out screencap -p > /tmp/android-before-test.png || true

# Run Maestro tests (exclude chat tests for now to compare with iOS)
$HOME/.maestro/bin/maestro test mobile/.maestro/flows/ --exclude-tags=chat --format junit --output mobile/maestro-android-report.xml

This comment was marked as outdated.

@gricha
Copy link
Copy Markdown
Owner Author

gricha commented Jan 8, 2026

Closing - Android emulator consistently times out on GitHub Actions ubuntu-latest runners. KVM hardware acceleration appears to not work reliably.

Findings:

  • APK build: ~12-14 min (similar to iOS xcodebuild)
  • Emulator boot: times out every time despite KVM being enabled
  • API level 31 and 34 both fail
  • 600s boot timeout still not enough

Conclusions:

  • Android e2e in GitHub Actions requires either:
    1. macOS runners (expensive, defeats the purpose)
    2. BuildJet runners with better KVM support
    3. Maestro Cloud (no local backend testing)
    4. Self-hosted runners with proper virtualization

iOS on macOS-14 remains the more reliable option for now.

@gricha gricha closed this Jan 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant