docs(skills): split argent-create-flow into references and add argent-qa-flows - #730
Open
hubgan wants to merge 7 commits into
Open
Conversation
j-piasecki
reviewed
Aug 4, 2026
|
|
||
| ### Step directives | ||
| 1. **Record the path live.** The first walkthrough _is_ the recording; never rehearse a path and reconstruct it afterward. [Live authoring](references/live-authoring.md) has the per-platform start order and the discover → echo → `flow-add-step` → inspect cycle. | ||
| 2. **Record each check when its state appears** — immediately after the transition or outcome it proves, before the next action. An echo or raw `screenshot` is diagnostic context, not an executable verdict; a reviewed `snapshot:` baseline is for inherently pixel-level requirements. Record absence as a trio in order — `visible` on the selector, the action that removes it, then `hidden` on the same selector — or the recorder refuses it. |
Member
There was a problem hiding this comment.
Can the agent record await/assert live?
Collaborator
Author
There was a problem hiding this comment.
No it cannot.
flow-add-step only rewrites three things:
gesture-tap->tap:,restart-app->launch:,flow-execute->run:. A check is recorded as anawait-ui-elementcall, stored as a rawtool:step, and converted toawait:/assert:during polish.assert:has no live tool at all.
The core skill never mentioned await-ui-element once, and rule 2 said "Record
each check" while listing snapshot:, which has no recorder form at all. Three
fixes:
- rule 2 now names the route and drops
snapshot: - rule 5 says "raw step" instead of "raw action", so the check conversion is clearly allowed, and names the three insertions that have no recorder form
- rule 4 notes identity is recorded live while readiness is a polish insertion
hubgan
force-pushed
the
docs/create-flow-references-and-qa-flows
branch
from
August 5, 2026 08:36
1fb15d0 to
29900cb
Compare
hubgan
changed the base branch from
feat/recorder-screen-identity-gate
to
feat/flow-screen-and-idle-conditions
August 5, 2026 08:37
This was referenced Aug 5, 2026
hubgan
force-pushed
the
feat/flow-screen-and-idle-conditions
branch
from
August 5, 2026 10:15
e1d9a9a to
e680fb0
Compare
hubgan
force-pushed
the
docs/create-flow-references-and-qa-flows
branch
2 times, most recently
from
August 5, 2026 10:21
b440ec7 to
f158f6a
Compare
hubgan
force-pushed
the
feat/flow-screen-and-idle-conditions
branch
from
August 5, 2026 10:38
e680fb0 to
151b8f6
Compare
hubgan
force-pushed
the
docs/create-flow-references-and-qa-flows
branch
from
August 5, 2026 10:38
f158f6a to
eb6c8a5
Compare
hubgan
force-pushed
the
feat/flow-screen-and-idle-conditions
branch
from
August 6, 2026 09:10
78c898b to
5f376c3
Compare
…-qa-flows
`argent-create-flow`'s SKILL.md had grown to roughly 5,000 words — every word of
it loaded on every invocation, whether the task was replaying a two-step
fragment or authoring a regression test. It is now ~900 words that route to
three references read on demand: live-authoring (recording a walkthrough),
flow-yaml (the file format and selector vocabulary), and
reliability-and-recovery (what to do when a step or a replay goes wrong).
`argent-qa-flows` is new: turning a test case, ticket, or acceptance criteria
into a repeatable regression test is a different job from recording a path
worth replaying. It orchestrates create-flow as its engine, records the first
walkthrough live, requires every requested screen and state to be proved with
stable evidence, and completes only after the unchanged flow passes twice
consecutively.
The routing rules gain the distinction the three skills now need, because they
were being confused with each other by name alone: a one-off interactive check
is argent-test-ui-flow, a saved replayable path is argent-create-flow, and a
saved test with acceptance criteria and two-pass proof is argent-qa-flows.
"Record a flow" also stops being ambiguous with screen recording, which is
video.
One new rule is worth calling out: if a flow or QA test may be recorded, do not
interact with the app first — start the recorder before the first launch. A path
already walked cannot be recorded retroactively, and re-walking it was one of
the more expensive mistakes in practice.
Proving a navigation is spelled out here for the first time, and it is now two
element-level checks rather than a route read: an `await:` on something that
exists ONLY on the destination, then `await: { idle: true }`. Neither implies
the other — a dropped tap leaves the source screen perfectly idle, and the
destination's elements enter the tree while the transition is still animating
over them — so both are required after every screen change. The readiness half
is a directive with no live tool behind it, so it joins `snapshot:` on the short
list of insertions allowed during polish.
A test parses the YAML frontmatter of every bundled skill, so a malformed new
one fails here rather than at install time.
`await: { idle: true }` no longer fails a run, so the references stop promising
that it does and start saying what to do with the warning instead. The gate is
still recorded after every navigation — it returns the moment the screen stops,
which is what keeps the following tap from resolving its target against a
transition still in flight.
The instruction the agent needs is the same everywhere: a ⚠ on a passing step is
a finding, not noise. Nothing in the report separates motion that is there by
design from a screen that never finished loading — a stuck spinner reads as
both — so go and look, disclose what was moving, and make sure the next action
rests on a stable element rather than on stillness. Live authoring says it where
the gate is added, so an agent who watched something animate expects the warning
before it appears; the recovery table gets a row for it; QA has to resolve and
report it before finishing, though it does not block the pass streak.
Also corrects an unrelated claim in the same QA file: the recorder does not
enforce the visible → action → hidden order. A `hidden` whose selector never
matched records as a clean pass, flagged only by a note in the tool result, so
the ordering is the author's to keep.
The references and the QA contract described the outcome as a "⚠", which is only how the MCP client and the CLI render it. What an agent can actually act on is a `warning` on a passing step, so say that instead.
…TOCs The core skill told the agent to "record each check" without ever naming what does the recording, and listed `snapshot:` among the things to record even though it has no recorder form. Rule 2 now says a check is recorded as an `await-ui-element` call that polish converts into `await:`/`assert:`; rule 5 says "raw step" rather than "raw action", so that conversion is unambiguously allowed, and names the three insertions that have no recorder form; rule 4 notes that identity is recorded live while readiness is inserted during polish. Also from the review: - a leading `run:` does not make a flow e2e, and on Chromium that classification is what boots the app; - `within` is geometric, so a child overflowing its parent's frame does not match it however clearly it is that parent's child in the code; - the polish table was missing `tool: gesture-rotate` -> `rotate:`; - the coordinate fallback gate now ends at the app's source, and recommends adding a stable test id when the element genuinely has none; - the check following a kept coordinate proves the outcome rather than the target, which the wording never said; - the QA worked example gated no theme baseline and never restored one, so its second consecutive run started Dark and proved nothing the first did. It now asserts Light in setup and restores it at the end. The three reference tables of contents were single lines of up to 490 characters joined by a separator used nowhere else in the skills tree, and are lists now. Proactive recording and the precautionary rule in rules/argent.md both stated the instruction ahead of the reason, and are reworded.
…imeout `waitForIdle` returns ok carrying a warning on every timeout path: readiness is not an acceptance criterion, and healthy screens often never stop moving. The tool description still told the opposite story, so the one place an agent reads about `idle` without loading a skill contradicted both the implementation and the create-flow reference. Only an unreadable or permanently empty tree stops a run, as an errored step, and the description now says so.
…nces The reference split was written against a create-flow SKILL.md that predates `flow-execute`'s second flow source, so moving the body out of SKILL.md would have dropped four things main documents today rather than relocating them. Restored where the split puts that kind of detail, not back into SKILL.md: - Flow YAML, composition: a `run:` target resolves against the directory of the flow file holding the step (so `../shared/login.yaml` reaches a sibling directory), and the `.yaml` suffix is optional. - Live authoring, replay: `flow-execute` takes exactly one source — `name` or an absolute `flow_path` — and `run:` targets and baselines resolve on the tool SERVER's filesystem. `flow_path` needs agent and server co-located; a remote `name` call arrives as an upload of one YAML into a temp directory, so a composed or snapshotting flow fails there. - Live authoring, Chromium: `restart-app` has no Chromium support and only successful calls are recorded, which is *why* a recorded Chromium flow is always a fragment and its launch is written in during polish.
hubgan
force-pushed
the
docs/create-flow-references-and-qa-flows
branch
from
August 6, 2026 10:21
bd3df10 to
dd49a0f
Compare
hubgan
marked this pull request as ready for review
August 6, 2026 17:17
argent-create-flow dropped its numbered sections, so the two "skill §10" references no longer resolved. Link both to the Diagnose a replay failure procedure in references/reliability-and-recovery.md instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
argent-create-flow: ~5,000 words to ~900 plus three referencesEvery word of that skill loaded on every invocation, whether the task was replaying a two-step fragment or authoring a full regression test. The core skill now routes to three references read on demand:
live-authoring.md- recording a walkthroughflow-yaml.md- the file format and selector vocabularyreliability-and-recovery.md- what to do when a step or a replay goes wrongNothing is dropped in the split itself - it moves behind a router. (Separately, the claims listed above were rewritten to match this base.)
New skill:
argent-qa-flowsTurning a test case, ticket, or acceptance criteria into a repeatable regression test is a different job from recording a path worth replaying, and it was being done by a skill that does not ask for acceptance criteria.
It orchestrates
argent-create-flowas its engine, records the first walkthrough live, requires every requested screen and state to be proved with stable evidence, and completes only after the unchanged flow passes twice consecutively.Proving a navigation
Spelled out here for the first time, and it is two element-level checks rather than a route read:
Neither implies the other. A dropped tap leaves the source screen perfectly idle, and the destination's elements enter the tree while the transition is still animating over them. So both are required after every screen change, including the one
launch:performs, which is where readiness matters most: launch waits for platform automation readiness, not for the app's own loading or splash.The readiness half is a directive with no live tool behind it, so it joins
snapshot:on the short list of insertions allowed during polish.Routing
The three skills were being confused by name alone, so the rules now draw the line explicitly:
argent-test-ui-flowargent-create-flowargent-qa-flows"Record a flow" also stops being ambiguous with screen recording, which is video.
One rule worth calling out
A path already walked cannot be recorded retroactively, and re-walking it was one of the more expensive mistakes in practice.
Test
The YAML frontmatter of every bundled skill is parsed in a test, so a malformed new skill fails here rather than at install time.
Review notes
Every concrete claim about
await: { idle: true }was checked against the implementation: defaults, the parse-time rejection of aminStableMsthat cannot fit insidetimeout, the three-read minimum, the hard failure on timeout against the tool's softsettled: false, the absentassertform, and the warning when no screenshot could be read. All 44 anchored internal links resolve.Duplication was cut where the same rule had accumulated in several files: the "never prove a screen with a shared header / tab bar / positional id" list went from five placements to two, and the "never persist
await-screen-idle" rule from three to one.Three items in
argent-device-interactare restored here rather than lost with #727, which is closed: thekeyboardtool serves every platform (not just iOS and Android), ahiddennote that never matched is a failed check rather than a pass, andawait-screen-idlegets a compact entry because three flow references now point at it.