Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
233 changes: 233 additions & 0 deletions .claude/skills/e2e-studio-tests/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
---
name: e2e-studio-tests
description: Run e2e tests in the Studio app. Use when asked to run e2e tests, run studio tests, playwright tests, or test the feature.
---

# E2E Studio Tests

Run Playwright end-to-end tests for the Studio application.

## Running Tests

Tests must be run from the `e2e/studio` directory:

```bash
cd e2e/studio && pnpm run e2e
```

### Run specific file

```bash
cd e2e/studio && pnpm run e2e -- features/cron-jobs.spec.ts
```

### Run with grep filter

```bash
cd e2e/studio && pnpm run e2e -- --grep "test name pattern"
```

### UI mode for debugging

```bash
cd e2e/studio && pnpm run e2e -- --ui
```

## Environment Setup

- Tests auto-start Supabase local containers via web server config
- Self-hosted mode (`IS_PLATFORM=false`) runs tests in parallel (3 workers)
- No manual setup needed for self-hosted tests

## Test File Structure

- Tests are in `e2e/studio/features/*.spec.ts`
- Use custom test utility: `import { test } from '../utils/test.js'`
- Test fixtures provide `page`, `ref`, and other helpers

## Common Patterns

Wait for elements with generous timeouts:

```typescript
await expect(locator).toBeVisible({ timeout: 30000 })
```

Add messages to expects for debugging:

```typescript
await expect(locator).toBeVisible({ timeout: 30000 }, 'Element should be visible after page load')
```

Use serial mode for tests sharing database state:

```typescript
test.describe.configure({ mode: 'serial' })
```

## Writing Robust Selectors

### Selector priority (best to worst)

1. **`getByRole` with accessible name** - Most robust, tests accessibility
```typescript
page.getByRole('button', { name: 'Save' })
page.getByRole('button', { name: 'Configure API privileges' })
```

2. **`getByTestId`** - Stable, explicit test hooks
```typescript
page.getByTestId('table-editor-side-panel')
```

3. **`getByText` with exact match** - Good for unique text
```typescript
page.getByText('Data API Access', { exact: true })
```

4. **`locator` with CSS** - Use sparingly, more fragile
```typescript
page.locator('[data-state="open"]')
```

### Patterns to avoid

- **XPath selectors** - Fragile to DOM changes
```typescript
// BAD
locator('xpath=ancestor::div[contains(@class, "space-y")]')
```

- **Parent traversal with `locator('..')`** - Breaks when structure changes
```typescript
// BAD
element.locator('..').getByRole('button')
```

- **Broad `filter({ hasText })` on generic elements** - May match multiple elements
```typescript
// BAD - popover may have more than one combobox
// Could consider scoping down the container or filtering the combobox more specifically
popover.getByRole('combobox')
```

### Add accessible labels to components

When a component lacks a good accessible name, add one in the source code:

```tsx
// In the React component
<Button aria-label="Configure API privileges">
<Settings />
</Button>
```

Then use it in tests:
```typescript
page.getByRole('button', { name: 'Configure API privileges' })
```

### Narrowing search scope

Scope selectors to specific containers to avoid matching wrong elements:

```typescript
// Good - scoped to side panel
const sidePanel = page.getByTestId('table-editor-side-panel')
const toggle = sidePanel.getByRole('switch')

// Good - find unique element, then scope from there
const popover = page.locator('[data-radix-popper-content-wrapper]')
const roleSection = popover.getByText('Anonymous (anon)', { exact: true })
```

## Avoiding `waitForTimeout`

Never use `waitForTimeout` - always wait for something specific:

```typescript
// BAD
await page.waitForTimeout(1000)

// GOOD - wait for UI element
await expect(page.getByText('Success')).toBeVisible()

// GOOD - wait for API response
const apiPromise = waitForApiResponse(page, 'pg-meta', ref, 'query?key=table-create')
await saveButton.click()
await apiPromise

// GOOD - wait for toast indicating operation complete
await expect(page.getByText('Table created successfully')).toBeVisible({ timeout: 15000 })
```

## Avoiding `force: true` on clicks

Instead of forcing clicks on hidden elements, make them visible first:

```typescript
// BAD
await menuButton.click({ force: true })

// GOOD - hover to reveal, then click
await tableRow.hover()
await expect(menuButton).toBeVisible()
await menuButton.click()
```

## Debugging

### View trace

```bash
cd e2e/studio && pnpm exec playwright show-trace <path-to-trace.zip>
```

### View HTML report

```bash
cd e2e/studio && pnpm exec playwright show-report
```

### Error context

Error context files are saved in the `test-results/` directory.

### Playwright MCP tools

Use Playwright MCP tools to inspect UI when debugging locally.

## CI vs Local Development

The key difference is **cold start vs warm state**:

### CI (cold start)

Tests run from a blank database slate. Each test run resets the database and starts fresh containers. Extensions like pg_cron are NOT enabled by default.

### Local dev with `pnpm dev:studio-local`

When debugging with a running dev server, the database may already have state from previous runs (extensions enabled, test data present).

## Handling Cold Start Bugs

Tests that work locally but fail in CI often have assumptions about existing state.

### Common issues

1. Extension not enabled (must enable in test setup)
2. Race conditions when parallel tests try to modify shared state (use `test.describe.configure({ mode: 'serial' })`)
3. Locators matching wrong elements because the page structure differs when state isn't set up

### Reproducing CI behavior locally

The test framework automatically resets the database when running `pnpm run e2e`. This matches CI behavior.

If using `pnpm dev:studio-local` for Playwright MCP debugging, remember the state differs from CI.

## Debugging Workflow for CI Failures

1. First, run the test locally with `pnpm run e2e -- features/<file>.spec.ts` (cold start)
2. Check error context in `test-results/` directory
3. If you need to inspect UI state, start `pnpm dev:studio-local` and use Playwright MCP tools
4. Remember: what you see in the dev server may have state that doesn't exist in CI
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ next-env.d.ts
.claude/*
!.claude/settings.json
!.claude/scripts/
!.claude/skills/
.claude/skills/me-*
CLAUDE.md

#include template .env file for docker-compose
Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
import type { PostgresExtension } from '@supabase/postgres-meta'
import { Database, ExternalLinkIcon, Plus } from 'lucide-react'
import { useEffect, useState } from 'react'
import { toast } from 'sonner'

import { DocsButton } from 'components/ui/DocsButton'
import { useDatabaseExtensionEnableMutation } from 'data/database-extensions/database-extension-enable-mutation'
import { useSchemasQuery } from 'data/database/schemas-query'
import { executeSql } from 'data/sql/execute-sql-query'
import { useIsOrioleDb, useSelectedProjectQuery } from 'hooks/misc/useSelectedProject'
import { useProtectedSchemas } from 'hooks/useProtectedSchemas'
import { DOCS_URL } from 'lib/constants'
import { Database, ExternalLinkIcon, Plus } from 'lucide-react'
import { useEffect, useState } from 'react'
import { toast } from 'sonner'
import {
AlertDescription_Shadcn_,
AlertTitle_Shadcn_,
Expand Down Expand Up @@ -108,6 +107,8 @@ const EnableExtensionModal = ({ visible, extension, onCancel }: EnableExtensionM
return undefined
}

const isLoading = fetchingSchemaInfo || isSchemasLoading

const validate = (values: any) => {
const errors: any = {}
if (values.schema === 'custom' && !values.name) errors.name = 'Required field'
Expand Down Expand Up @@ -170,7 +171,7 @@ const EnableExtensionModal = ({ visible, extension, onCancel }: EnableExtensionM
</Admonition>
)}

{fetchingSchemaInfo || isSchemasLoading ? (
{isLoading ? (
<div className="space-y-2">
<ShimmeringLoader />
<div className="w-3/4">
Expand Down Expand Up @@ -269,7 +270,7 @@ const EnableExtensionModal = ({ visible, extension, onCancel }: EnableExtensionM
<Button type="default" disabled={isEnabling} onClick={() => onCancel()}>
Cancel
</Button>
<Button htmlType="submit" disabled={isEnabling} loading={isEnabling}>
<Button htmlType="submit" disabled={isEnabling || isLoading} loading={isEnabling}>
Enable extension
</Button>
</Modal.Content>
Expand Down
Loading
Loading