Skip to content

feat: BDD test suite with playwright-bdd for UX regression testing - #25

Merged
NP-compete merged 1 commit into
mainfrom
sodutta/bdd-tests
Jun 1, 2026
Merged

feat: BDD test suite with playwright-bdd for UX regression testing#25
NP-compete merged 1 commit into
mainfrom
sodutta/bdd-tests

Conversation

@NP-compete

Copy link
Copy Markdown
Owner

Summary

  • Adds 36 Gherkin feature files with 162 scenarios (~394 generated Playwright tests) using playwright-bdd
  • Covers all 25 Studio UI pages, all 6 user roles, all major API endpoints, and visual regression baselines
  • Existing 91 Playwright tests are untouched — coexist via dual Playwright project config

What's covered

Domain Feature Files Scenarios
Auth & RBAC 4 Login (6 roles), nav visibility, API access control, API keys
Agents 6 Chat creation, deep agents, detail view, agent chat, deploy modal, templates
Memory 3 Short-term CRUD, long-term embeddings + search, compaction
Guardrails 2 Pipeline checks, rule management
Studio UI 15 Dashboard, Connectors, MCP, Models, Skills, Evaluations, Flow Builder, Marketplace, FinOps, Audit Explorer, Approvals, Build Hub, Settings (6 tabs), YAML Editor, Org Chart
Enterprise 3 Multi-tenancy, 5 compliance frameworks (SOC2/GDPR/HIPAA/ISO27001/EU AI Act), audit integrity
Cross-cutting 3 Command palette, theme toggle, visual regression baselines

Architecture

e2e/
├── features/          # 36 Gherkin .feature files (readable by product/QA)
├── steps/common/      # 6 reusable step definitions (auth, nav, api, form, table, visual)
├── steps/domain/      # 5 domain step definitions (agent, memory, guardrails, enterprise, studio)
├── fixtures/          # playwright-bdd base fixtures + page objects
├── tests/             # Existing Playwright tests (UNCHANGED)
└── playwright.config.ts  # Dual project: "bdd" + "existing"

How to run

cd e2e && npm install

npm test              # BDD tests only
npm run test:existing # Original tests only
npm run test:all      # Both together
npm run test:smoke    # Smoke subset (@smoke tagged)
npm run test:visual   # Create/update visual baselines

Test plan

  • npm install succeeds in e2e/
  • npx bddgen generates .features-gen/ without errors
  • npm run test:existing — existing 91 tests still pass
  • npm test — BDD scenarios run against running platform
  • npm run test:smoke — smoke subset runs independently
  • Visual baselines created on first run with --update-snapshots

…ression testing

Add 36 Gherkin feature files with ~162 scenarios (394 generated tests after
outline expansion) covering all 25 Studio pages, 6 user roles, API endpoints,
memory system, guardrails, enterprise features, and visual regression.

Infrastructure:
- playwright-bdd v8+ with createBdd() pattern
- Dual Playwright projects: "bdd" (new) and "existing" (untouched)
- npm scripts: test, test:existing, test:all, test:smoke, test:visual

Feature coverage by domain:
- Auth: login, RBAC navigation, RBAC API access, API keys
- Agents: chat creation, deep agents, detail view, agent chat, deploy modal
- Memory: short-term CRUD, long-term embeddings + search, compaction
- Guardrails: pipeline checks, rule management
- Studio UI: all 15 pages (dashboard through org-chart)
- Enterprise: multi-tenancy, 5 compliance frameworks, audit integrity
- Cross-cutting: command palette, theme toggle, visual regression baselines
@NP-compete
NP-compete merged commit 0ca1347 into main Jun 1, 2026
14 checks passed
@NP-compete
NP-compete deleted the sodutta/bdd-tests branch June 7, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant