Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions docs/src/api/class-browser.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,9 +267,6 @@ await browser.CloseAsync();
### option: Browser.newContext.storageStatePath = %%-csharp-java-context-option-storage-state-path-%%
* since: v1.9

### option: Browser.newContext.agent = %%-context-option-agent-%%
* since: v1.58

## async method: Browser.newPage
* since: v1.8
- returns: <[Page]>
Expand Down
48 changes: 35 additions & 13 deletions docs/src/api/class-page.md
Original file line number Diff line number Diff line change
Expand Up @@ -544,17 +544,6 @@ sequence of events is `request`, `response` and `requestfinished`.
Emitted when [response] status and headers are received for a request. For a successful response, the sequence of events
is `request`, `response` and `requestfinished`.

## event: Page.agentTurn
* since: v1.58
- argument: <[Object]>
- `role` <[string]>
- `message` <[string]>
- `usage` ?<[Object]>
- `inputTokens` <[int]>
- `outputTokens` <[int]>

Emitted when the agent makes a turn.

## event: Page.webSocket
* since: v1.9
- argument: <[WebSocket]>
Expand Down Expand Up @@ -720,9 +709,42 @@ current working directory.

Raw CSS content to be injected into frame.

## property: Page.agent
## async method: Page.agent
* since: v1.58
- type: <[PageAgent]>
- returns: <[PageAgent]>

Initialize page agent with the llm provider and cache.

### option: Page.agent.cache
* since: v1.58
- `cache` <[Object]>
- `cacheFile` ?<[string]> Cache file to use/generate code for performed actions into. Cache is not used if not specified (default).
- `cacheOutFile` ?<[string]> When specified, generated entries are written into the `cacheOutFile` instead of updating the `cacheFile`.

### option: Page.agent.maxTurns
* since: v1.58
- `maxTurns` <[int]>

Maximum number of agentic turns to take per call. Defaults to 10.

### option: Page.agent.maxTokens
* since: v1.58
- `maxTokens` ?<[int]>

### option: Page.agent.provider
* since: v1.58
- `provider` <[Object]>
- `api` <[PageAgentAPI]<"openai"|"openai-compatible"|"anthropic"|"google">> API to use.
- `apiEndpoint` ?<[string]> Endpoint to use if different from default.
- `apiKey` <[string]> API key for the LLM provider.
- `model` <[string]> Model identifier within the provider. Required in non-cache mode.

### option: Page.agent.secrets
* since: v1.58
- `secrets` ?<[Object]<[string], [string]>>

Secrets to hide from the LLM.


## async method: Page.bringToFront
* since: v1.8
Expand Down
21 changes: 18 additions & 3 deletions docs/src/api/class-pageagent.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,21 @@
# class: PageAgent
* since: v1.58

## event: PageAgent.turn
* since: v1.58
- argument: <[Object]>
- `role` <[string]>
- `message` <[string]>
- `usage` ?<[Object]>
- `inputTokens` <[int]>
- `outputTokens` <[int]>

Emitted when the agent makes a turn.

## async method: PageAgent.dispose
* since: v1.58

Dispose this agent.

## async method: PageAgent.expect
* since: v1.58
Expand All @@ -10,7 +25,7 @@ Expect certain condition to be met.
**Usage**

```js
await page.agent.expect('"0 items" to be reported');
await agent.expect('"0 items" to be reported');
```

### param: PageAgent.expect.expectation
Expand All @@ -36,7 +51,7 @@ Extract information from the page using the agentic loop, return it in a given Z
**Usage**

```js
await page.agent.extract('List of items in the cart', z.object({
await agent.extract('List of items in the cart', z.object({
title: z.string().describe('Item title to extract'),
price: z.string().describe('Item price to extract'),
}).array());
Expand Down Expand Up @@ -69,7 +84,7 @@ Perform action using agentic loop.
**Usage**

```js
await page.agent.perform('Click submit button');
await agent.perform('Click submit button');
```

### param: PageAgent.perform.task
Expand Down
37 changes: 0 additions & 37 deletions docs/src/api/params.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,40 +370,6 @@ It makes the execution of the tests non-deterministic.
Emulates consistent window screen size available inside web page via `window.screen`. Is only used when the
[`option: viewport`] is set.

## context-option-agent
- `agent` <[Object]>
- `api` ?<[string]> API to use, `openapi`, `google` or `anthropic`. Required in non-cache mode.
- `apiEndpoint` ?<[string]> Endpoint to use if different from default.
- `apiKey` ?<[string]> API key for the LLM provider.
- `model` ?<[string]> Model identifier within the provider. Required in non-cache mode.
- `cacheFile` ?<[string]> Cache file to use/generate code for performed actions into. Cache is not used if not specified (default).
- `cacheOutFile` ?<[string]> When specified, generated entries are written into the `cacheOutFile` instead of updating the `cacheFile`.
- `secrets` ?<[Object]<[string], [string]>> Secrets to hide from the LLM.
- `maxTurns` ?<[int]> Maximum number of agentic turns to take per call. Defaults to 10.
- `maxTokens` ?<[int]> Maximum number of tokens to consume per call. The agentic loop will stop after input + output tokens exceed this value. Defaults on unlimited.

Agent settings for [`property: Page.agent`].

## page-agent-api
* since: v1.58
- `api` <[string]>

API to use, `openapi`, `google` or `anthropic`. Required in non-cache mode.

## page-agent-api-endpoint
* since: v1.58
- `apiEndpoint` <[string]>

Endpoint to use if different from default.

## page-agent-api-key
* since: v1.58
- `apiKey` <[string]>

API key for the LLM provider.

API version if relevant.

## page-agent-cache-key
* since: v1.58
- `cacheKey` <[string]>
Expand All @@ -425,9 +391,6 @@ Defaults to context-wide value specified in `agent` property.
Maximum number of agentic turns during this call, defaults to context-wide value specified in `agent` property.

## page-agent-call-options-v1.58
- %%-page-agent-api-%%
- %%-page-agent-api-key-%%
- %%-page-agent-api-endpoint-%%
- %%-page-agent-cache-key-%%
- %%-page-agent-max-tokens-%%
- %%-page-agent-max-turns-%%
Expand Down
4 changes: 4 additions & 0 deletions docs/src/test-api/class-fixtures.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ Given the test above, Playwright Test will set up the `page` fixture before runn

Playwright Test comes with builtin fixtures listed below, and you can add your own fixtures as well. Playwright Test also [provides options][TestOptions] to configure [`property: Fixtures.browser`], [`property: Fixtures.context`] and [`property: Fixtures.page`].

## property: Fixtures.agent
* since: v1.58
- type: <[PageAgent]>

## property: Fixtures.browser
* since: v1.10
- type: <[Browser]>
Expand Down
2 changes: 1 addition & 1 deletion docs/src/test-api/class-fullconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Base directory for all relative paths used in the reporters.
* since: v1.58
- type: <['RunAgentsMode]<"all"|"missing"|"none">>

Whether to run LLM agent for [`property: Page.agent`]:
Whether to run LLM agent for [PageAgent]:
* "all" disregards existing cache and performs all actions via LLM
* "missing" only performs actions that don't have generated cache actions
* "none" does not talk to LLM at all, relies on the cached actions (default)
Expand Down
2 changes: 1 addition & 1 deletion docs/src/test-api/class-testconfig.md
Original file line number Diff line number Diff line change
Expand Up @@ -518,7 +518,7 @@ export default defineConfig({
* since: v1.58
- type: ?<['RunAgentsMode]<"all"|"missing"|"none">>

Whether to run LLM agent for [`property: Page.agent`]:
Whether to run LLM agent for [PageAgent]:
* "all" disregards existing cache and performs all actions via LLM
* "missing" only performs actions that don't have generated cache actions
* "none" does not talk to LLM at all, relies on the cached actions (default)
Expand Down
6 changes: 4 additions & 2 deletions docs/src/test-api/class-testoptions.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,12 @@ export default defineConfig({
});
```

## property: TestOptions.agent
## property: TestOptions.agentOptions
* since: v1.58
- type: <[Object]>
- `provider` ?<[string]> LLM provider to use. Required in non-cache mode.
- `api` ?<[string]> LLM provider to use. Required in non-cache mode.
- `apiKey` ?<[string]> Key for the LLM provider.
- `apiEndpoint` ?<[string]> LLM provider endpoint.
- `model` ?<[string]> Model identifier within the provider. Required in non-cache mode.
- `cachePathTemplate` ?<[string]> Cache file template to use/generate code for performed actions into.
- `maxTurns` ?<[int]> Maximum number of agentic turns to take per call. Defaults to 10.
Expand Down
6 changes: 6 additions & 0 deletions examples/todomvc/tests/fixtures.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ import { test as baseTest } from '@playwright/test';
export { expect } from '@playwright/test';

export const test = baseTest.extend({
agentOptions: {
api: 'anthropic',
apiKey: process.env.AZURE_SONNET_API_KEY!,
apiEndpoint: process.env.AZURE_SONNET_ENDPOINT!,
model: 'claude-sonnet-4-5',
},
page: async ({ page }, use) => {
await page.goto('https://demo.playwright.dev/todomvc');
await use(page);
Expand Down
Original file line number Diff line number Diff line change
@@ -1,27 +1,18 @@
import { test } from '../../fixtures';

test.use({
agent: {
api: 'anthropic',
apiKey: process.env.AZURE_SONNET_API_KEY!,
apiEndpoint: process.env.AZURE_SONNET_ENDPOINT!,
model: 'claude-sonnet-4-5',
}
});

test('should complete multiple todos', async ({ page }) => {
await page.agent.expect(`The page loads with an empty todo list`);
test('should complete multiple todos', async ({ agent }) => {
await agent.expect(`The page loads with an empty todo list`);

await page.agent.perform(`Add three todos: 'Buy milk', 'Walk dog', 'Finish report'`);
await page.agent.expect(`All three todos are visible`);
await page.agent.expect(`Counter shows '3 items left'`);
await agent.perform(`Add three todos: 'Buy milk', 'Walk dog', 'Finish report'`);
await agent.expect(`All three todos are visible`);
await agent.expect(`Counter shows '3 items left'`);

await page.agent.perform(`Complete the first todo by clicking its checkbox`);
await page.agent.expect(`First todo is marked as complete`);
await page.agent.expect(`Counter shows '2 items left'`);
await agent.perform(`Complete the first todo by clicking its checkbox`);
await agent.expect(`First todo is marked as complete`);
await agent.expect(`Counter shows '2 items left'`);

await page.agent.perform(`Complete the third todo by clicking its checkbox`);
await page.agent.expect(`Third todo is marked as complete`);
await page.agent.expect(`Counter shows '1 item left'`);
await page.agent.expect(`The 'Clear completed' button appears`);
await agent.perform(`Complete the third todo by clicking its checkbox`);
await agent.expect(`Third todo is marked as complete`);
await agent.expect(`Counter shows '1 item left'`);
await agent.expect(`The 'Clear completed' button appears`);
});
Original file line number Diff line number Diff line change
@@ -1,23 +1,14 @@
import { test } from '../../fixtures';

test.use({
agent: {
api: 'anthropic',
apiKey: process.env.AZURE_SONNET_API_KEY!,
apiEndpoint: process.env.AZURE_SONNET_ENDPOINT!,
model: 'claude-sonnet-4-5',
}
});

test('should complete single todo', async ({ page }) => {
await page.agent.expect(`The page loads with an empty todo list`);
test('should complete single todo', async ({ agent }) => {
await agent.expect(`The page loads with an empty todo list`);

await page.agent.perform(`Add a todo 'Buy groceries'`);
await page.agent.expect(`The todo appears as active`);
await page.agent.expect(`Counter shows '1 item left'`);
await agent.perform(`Add a todo 'Buy groceries'`);
await agent.expect(`The todo appears as active`);
await agent.expect(`Counter shows '1 item left'`);

await page.agent.perform(`Click the checkbox next to the todo`);
await page.agent.expect(`The checkbox is checked`);
await page.agent.expect(`Counter shows '0 items left'`);
await page.agent.expect(`The 'Clear completed' button appears in the footer`);
await agent.perform(`Click the checkbox next to the todo`);
await agent.expect(`The checkbox is checked`);
await agent.expect(`Counter shows '0 items left'`);
await agent.expect(`The 'Clear completed' button appears in the footer`);
});
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
import { test } from '../../fixtures';

test.use({
agent: {
api: 'anthropic',
apiKey: process.env.AZURE_SONNET_API_KEY!,
apiEndpoint: process.env.AZURE_SONNET_ENDPOINT!,
model: 'claude-sonnet-4-5',
}
});

test('should toggle all todos complete', async ({ page }) => {
await page.agent.expect(`The page loads with an empty todo list`);

await page.agent.perform(`Add three todos: 'Task 1', 'Task 2', 'Task 3'`);
await page.agent.expect(`All three todos are visible and active`);
await page.agent.expect(`Counter shows '3 items left'`);

await page.agent.perform(`Click the 'Mark all as complete' checkbox`);
await page.agent.expect(`All three todos are marked as complete`);
await page.agent.expect(`All checkboxes are checked`);
await page.agent.expect(`Counter shows '0 items left'`);
await page.agent.expect(`The 'Clear completed' button appears`);
test('should toggle all todos complete', async ({ agent }) => {
await agent.expect(`The page loads with an empty todo list`);

await agent.perform(`Add three todos: 'Task 1', 'Task 2', 'Task 3'`);
await agent.expect(`All three todos are visible and active`);
await agent.expect(`Counter shows '3 items left'`);

await agent.perform(`Click the 'Mark all as complete' checkbox`);
await agent.expect(`All three todos are marked as complete`);
await agent.expect(`All checkboxes are checked`);
await agent.expect(`Counter shows '0 items left'`);
await agent.expect(`The 'Clear completed' button appears`);
});
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
import { test } from '../../fixtures';

test.use({
agent: {
api: 'anthropic',
apiKey: process.env.AZURE_SONNET_API_KEY!,
apiEndpoint: process.env.AZURE_SONNET_ENDPOINT!,
model: 'claude-sonnet-4-5',
}
});

test('should toggle all todos incomplete', async ({ page }) => {
await page.agent.expect(`The page loads with an empty todo list`);

await page.agent.perform(`Add three todos: 'Task 1', 'Task 2', 'Task 3' and mark all as complete using the toggle all checkbox`);
await page.agent.expect(`All todos are marked as complete`);
await page.agent.expect(`Counter shows '0 items left'`);

await page.agent.perform(`Click the 'Mark all as complete' checkbox again`);
await page.agent.expect(`All todos are marked as active`);
await page.agent.expect(`All checkboxes are unchecked`);
await page.agent.expect(`Counter shows '3 items left'`);
await page.agent.expect(`The 'Clear completed' button disappears`);
test('should toggle all todos incomplete', async ({ agent }) => {
await agent.expect(`The page loads with an empty todo list`);

await agent.perform(`Add three todos: 'Task 1', 'Task 2', 'Task 3' and mark all as complete using the toggle all checkbox`);
await agent.expect(`All todos are marked as complete`);
await agent.expect(`Counter shows '0 items left'`);

await agent.perform(`Click the 'Mark all as complete' checkbox again`);
await agent.expect(`All todos are marked as active`);
await agent.expect(`All checkboxes are unchecked`);
await agent.expect(`Counter shows '3 items left'`);
await agent.expect(`The 'Clear completed' button disappears`);
});
Loading
Loading