Skip to content

docs(skills): split argent-create-flow into references and add argent-qa-flows - #730

Open
hubgan wants to merge 7 commits into
feat/flow-screen-and-idle-conditionsfrom
docs/create-flow-references-and-qa-flows
Open

docs(skills): split argent-create-flow into references and add argent-qa-flows#730
hubgan wants to merge 7 commits into
feat/flow-screen-and-idle-conditionsfrom
docs/create-flow-references-and-qa-flows

Conversation

@hubgan

@hubgan hubgan commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #728, which now sits directly on #574 rather than on the #715-#726 chain.

Because of that, guidance describing behaviour owned by that chain is not carried here: the recorder's refusal of a vacuous hidden and of await-screen-idle (#724), its cross-tree re-probe warning (#717), the per-step recorded line (#715), and type: failing on unconfirmed focus (#726). Each of those paragraphs states what this base actually does instead. They are worth restoring to the references when that chain lands.

argent-create-flow: ~5,000 words to ~900 plus three references

Every word of that skill loaded on every invocation, whether the task was replaying a two-step fragment or authoring a full regression test. The core skill now routes to three references read on demand:

  • live-authoring.md - recording a walkthrough
  • flow-yaml.md - the file format and selector vocabulary
  • reliability-and-recovery.md - what to do when a step or a replay goes wrong

Nothing is dropped in the split itself - it moves behind a router. (Separately, the claims listed above were rewritten to match this base.)

New skill: argent-qa-flows

Turning a test case, ticket, or acceptance criteria into a repeatable regression test is a different job from recording a path worth replaying, and it was being done by a skill that does not ask for acceptance criteria.

It orchestrates argent-create-flow as its engine, records the first walkthrough live, requires every requested screen and state to be proved with stable evidence, and completes only after the unchanged flow passes twice consecutively.

Proving a navigation

Spelled out here for the first time, and it is two element-level checks rather than a route read:

- await: { visible: { id: settings-screen } } # identity: WHICH screen
- await: { idle: true } # readiness: it stopped moving

Neither implies the other. A dropped tap leaves the source screen perfectly idle, and the destination's elements enter the tree while the transition is still animating over them. So both are required after every screen change, including the one launch: performs, which is where readiness matters most: launch waits for platform automation readiness, not for the app's own loading or splash.

The readiness half is a directive with no live tool behind it, so it joins snapshot: on the short list of insertions allowed during polish.

Routing

The three skills were being confused by name alone, so the rules now draw the line explicitly:

you want skill
a one-off interactive check, nothing saved argent-test-ui-flow
a saved, replayable path argent-create-flow
a saved test with acceptance criteria and two-pass proof argent-qa-flows

"Record a flow" also stops being ambiguous with screen recording, which is video.

One rule worth calling out

If you may record a flow or QA test, do not interact with the app first - start the recorder before the first launch or in-app action.

A path already walked cannot be recorded retroactively, and re-walking it was one of the more expensive mistakes in practice.

Test

The YAML frontmatter of every bundled skill is parsed in a test, so a malformed new skill fails here rather than at install time.

Review notes

Every concrete claim about await: { idle: true } was checked against the implementation: defaults, the parse-time rejection of a minStableMs that cannot fit inside timeout, the three-read minimum, the hard failure on timeout against the tool's soft settled: false, the absent assert form, and the warning when no screenshot could be read. All 44 anchored internal links resolve.

Duplication was cut where the same rule had accumulated in several files: the "never prove a screen with a shared header / tab bar / positional id" list went from five placements to two, and the "never persist await-screen-idle" rule from three to one.

Three items in argent-device-interact are restored here rather than lost with #727, which is closed: the keyboard tool serves every platform (not just iOS and Android), a hidden note that never matched is a failed check rather than a pass, and await-screen-idle gets a compact entry because three flow references now point at it.

Comment thread packages/argent-installer/test/skills-frontmatter.test.ts
Comment thread packages/skills/rules/argent.md Outdated

### Step directives
1. **Record the path live.** The first walkthrough _is_ the recording; never rehearse a path and reconstruct it afterward. [Live authoring](references/live-authoring.md) has the per-platform start order and the discover → echo → `flow-add-step` → inspect cycle.
2. **Record each check when its state appears** — immediately after the transition or outcome it proves, before the next action. An echo or raw `screenshot` is diagnostic context, not an executable verdict; a reviewed `snapshot:` baseline is for inherently pixel-level requirements. Record absence as a trio in order — `visible` on the selector, the action that removes it, then `hidden` on the same selector — or the recorder refuses it.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the agent record await/assert live?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it cannot.

flow-add-step only rewrites three things:

  • gesture-tap -> tap:,
  • restart-app -> launch:, flow-execute -> run:. A check is recorded as an
  • await-ui-element call, stored as a raw tool: step, and converted to await:/assert: during polish. assert: has no live tool at all.

The core skill never mentioned await-ui-element once, and rule 2 said "Record
each check" while listing snapshot:, which has no recorder form at all. Three
fixes:

  • rule 2 now names the route and drops snapshot:
  • rule 5 says "raw step" instead of "raw action", so the check conversion is clearly allowed, and names the three insertions that have no recorder form
  • rule 4 notes identity is recorded live while readiness is a polish insertion

Comment thread packages/skills/skills/argent-create-flow/SKILL.md Outdated
Comment thread packages/skills/skills/argent-create-flow/references/flow-yaml.md Outdated
Comment thread packages/skills/skills/argent-create-flow/references/reliability-and-recovery.md Outdated
Comment thread packages/skills/skills/argent-qa-flows/SKILL.md
Comment thread packages/skills/skills/argent-qa-flows/SKILL.md Outdated
Comment thread packages/skills/skills/argent-qa-flows/SKILL.md Outdated
@hubgan
hubgan force-pushed the docs/create-flow-references-and-qa-flows branch from 1fb15d0 to 29900cb Compare August 5, 2026 08:36
@hubgan
hubgan changed the base branch from feat/recorder-screen-identity-gate to feat/flow-screen-and-idle-conditions August 5, 2026 08:37
@hubgan
hubgan force-pushed the feat/flow-screen-and-idle-conditions branch from e1d9a9a to e680fb0 Compare August 5, 2026 10:15
@hubgan
hubgan force-pushed the docs/create-flow-references-and-qa-flows branch 2 times, most recently from b440ec7 to f158f6a Compare August 5, 2026 10:21
@hubgan
hubgan force-pushed the feat/flow-screen-and-idle-conditions branch from e680fb0 to 151b8f6 Compare August 5, 2026 10:38
@hubgan
hubgan force-pushed the docs/create-flow-references-and-qa-flows branch from f158f6a to eb6c8a5 Compare August 5, 2026 10:38
@hubgan
hubgan force-pushed the feat/flow-screen-and-idle-conditions branch from 78c898b to 5f376c3 Compare August 6, 2026 09:10
hubgan added 6 commits August 6, 2026 12:15
…-qa-flows

`argent-create-flow`'s SKILL.md had grown to roughly 5,000 words — every word of
it loaded on every invocation, whether the task was replaying a two-step
fragment or authoring a regression test. It is now ~900 words that route to
three references read on demand: live-authoring (recording a walkthrough),
flow-yaml (the file format and selector vocabulary), and
reliability-and-recovery (what to do when a step or a replay goes wrong).

`argent-qa-flows` is new: turning a test case, ticket, or acceptance criteria
into a repeatable regression test is a different job from recording a path
worth replaying. It orchestrates create-flow as its engine, records the first
walkthrough live, requires every requested screen and state to be proved with
stable evidence, and completes only after the unchanged flow passes twice
consecutively.

The routing rules gain the distinction the three skills now need, because they
were being confused with each other by name alone: a one-off interactive check
is argent-test-ui-flow, a saved replayable path is argent-create-flow, and a
saved test with acceptance criteria and two-pass proof is argent-qa-flows.
"Record a flow" also stops being ambiguous with screen recording, which is
video.

One new rule is worth calling out: if a flow or QA test may be recorded, do not
interact with the app first — start the recorder before the first launch. A path
already walked cannot be recorded retroactively, and re-walking it was one of
the more expensive mistakes in practice.

Proving a navigation is spelled out here for the first time, and it is now two
element-level checks rather than a route read: an `await:` on something that
exists ONLY on the destination, then `await: { idle: true }`. Neither implies
the other — a dropped tap leaves the source screen perfectly idle, and the
destination's elements enter the tree while the transition is still animating
over them — so both are required after every screen change. The readiness half
is a directive with no live tool behind it, so it joins `snapshot:` on the short
list of insertions allowed during polish.

A test parses the YAML frontmatter of every bundled skill, so a malformed new
one fails here rather than at install time.
`await: { idle: true }` no longer fails a run, so the references stop promising
that it does and start saying what to do with the warning instead. The gate is
still recorded after every navigation — it returns the moment the screen stops,
which is what keeps the following tap from resolving its target against a
transition still in flight.

The instruction the agent needs is the same everywhere: a ⚠ on a passing step is
a finding, not noise. Nothing in the report separates motion that is there by
design from a screen that never finished loading — a stuck spinner reads as
both — so go and look, disclose what was moving, and make sure the next action
rests on a stable element rather than on stillness. Live authoring says it where
the gate is added, so an agent who watched something animate expects the warning
before it appears; the recovery table gets a row for it; QA has to resolve and
report it before finishing, though it does not block the pass streak.

Also corrects an unrelated claim in the same QA file: the recorder does not
enforce the visible → action → hidden order. A `hidden` whose selector never
matched records as a clean pass, flagged only by a note in the tool result, so
the ordering is the author's to keep.
The references and the QA contract described the outcome as a "⚠", which is
only how the MCP client and the CLI render it. What an agent can actually act on
is a `warning` on a passing step, so say that instead.
…TOCs

The core skill told the agent to "record each check" without ever naming what
does the recording, and listed `snapshot:` among the things to record even
though it has no recorder form. Rule 2 now says a check is recorded as an
`await-ui-element` call that polish converts into `await:`/`assert:`; rule 5
says "raw step" rather than "raw action", so that conversion is unambiguously
allowed, and names the three insertions that have no recorder form; rule 4 notes
that identity is recorded live while readiness is inserted during polish.

Also from the review:

- a leading `run:` does not make a flow e2e, and on Chromium that classification
  is what boots the app;
- `within` is geometric, so a child overflowing its parent's frame does not
  match it however clearly it is that parent's child in the code;
- the polish table was missing `tool: gesture-rotate` -> `rotate:`;
- the coordinate fallback gate now ends at the app's source, and recommends
  adding a stable test id when the element genuinely has none;
- the check following a kept coordinate proves the outcome rather than the
  target, which the wording never said;
- the QA worked example gated no theme baseline and never restored one, so its
  second consecutive run started Dark and proved nothing the first did. It now
  asserts Light in setup and restores it at the end.

The three reference tables of contents were single lines of up to 490
characters joined by a separator used nowhere else in the skills tree, and are
lists now. Proactive recording and the precautionary rule in rules/argent.md
both stated the instruction ahead of the reason, and are reworded.
…imeout

`waitForIdle` returns ok carrying a warning on every timeout path: readiness is
not an acceptance criterion, and healthy screens often never stop moving. The
tool description still told the opposite story, so the one place an agent reads
about `idle` without loading a skill contradicted both the implementation and
the create-flow reference.

Only an unreadable or permanently empty tree stops a run, as an errored step,
and the description now says so.
…nces

The reference split was written against a create-flow SKILL.md that predates
`flow-execute`'s second flow source, so moving the body out of SKILL.md would
have dropped four things main documents today rather than relocating them.

Restored where the split puts that kind of detail, not back into SKILL.md:

- Flow YAML, composition: a `run:` target resolves against the directory of the
  flow file holding the step (so `../shared/login.yaml` reaches a sibling
  directory), and the `.yaml` suffix is optional.
- Live authoring, replay: `flow-execute` takes exactly one source — `name` or an
  absolute `flow_path` — and `run:` targets and baselines resolve on the tool
  SERVER's filesystem. `flow_path` needs agent and server co-located; a remote
  `name` call arrives as an upload of one YAML into a temp directory, so a
  composed or snapshotting flow fails there.
- Live authoring, Chromium: `restart-app` has no Chromium support and only
  successful calls are recorded, which is *why* a recorded Chromium flow is
  always a fragment and its launch is written in during polish.
@hubgan
hubgan force-pushed the docs/create-flow-references-and-qa-flows branch from bd3df10 to dd49a0f Compare August 6, 2026 10:21
@hubgan
hubgan marked this pull request as ready for review August 6, 2026 17:17
@hubgan
hubgan requested review from j-piasecki and latekvo August 6, 2026 17:18
argent-create-flow dropped its numbered sections, so the two "skill §10"
references no longer resolved. Link both to the Diagnose a replay failure
procedure in references/reliability-and-recovery.md instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants