Skip to content

feat(tiktok-symphony): add Symphony Creative Studio image and video adapters#2159

Open
kienbb wants to merge 2 commits into
jackwener:mainfrom
kienbb:feat/tiktok-symphony-image-generation
Open

feat(tiktok-symphony): add Symphony Creative Studio image and video adapters#2159
kienbb wants to merge 2 commits into
jackwener:mainfrom
kienbb:feat/tiktok-symphony-image-generation

Conversation

@kienbb

@kienbb kienbb commented Jul 19, 2026

Copy link
Copy Markdown

Adds seven adapters for TikTok Symphony Creative Studio (ads.tiktok.com), covering
both the Image and the Video tab.

Command Access Description
credits read Credit balance, plan, signed-in account
generations (alias library) read List generated assets in the Library
jobs (alias queue) read Render progress for the Create feed
download <asset> read Save a generated asset to a local file
generate <prompt> write Generate images from a prompt + up to 4 reference images
generate-video <prompt> write Render a video clip
delete <asset> write Permanently delete a generation

Strategy note

Strategy: UI_SELECTOR   Contract: visible-ui
Evidence:
- anchors: fieldset[aria-label="File drop zone"] wraps the composer, which exposes
  data-chatbox-part = topbar | body>header | textarea | footer | drop-overlay
- auth source: existing browser session cookie (analyze reports no anti-bot, 0 auth failures)
- replay result: exercised live — model switch, reference upload, prompt entry and
  submit all confirmed against the real page

The site does have internal XHR, but the composer's data-chatbox-part anchors are a
user-visible contract and a steadier bet than unversioned internal endpoints, so this
stays on the UI ladder rung rather than climbing to PAGE_FETCH / INTERCEPT.

Two site-specific constraints worth flagging for reviewers

ks-* web components carry a version suffix. Elements are ks-button-1-1-1m,
ks-tag-1-1-1m, and so on. Every lookup matches on tag prefix
(tagName.startsWith('ks-button')) — hard-coding the full tag would break on the next
component version bump.

Reference upload cannot use page.uploadFiles. The page renders no
input[type=file] at all, and injecting one and driving it through CDP is refused
({"code":-32000,"message":"Not allowed"}). Attaching a reference therefore goes
through the composer's own drop zone: the file is read Node-side, rebuilt as a File
inside the page, and dispatched as a real drop. This only reuses a capability the
page already offers to any user dragging a file in.

Symphony is a client-rendered SPA whose header and composer hydrate after navigation
resolves, so every command polls via a shared waitForValue helper rather than
assuming the DOM is ready — that gap was the first thing to bite during development.

The Video tab

generate-video covers all three Video sub-apps. Each is its own subApp URL, so the
command navigates straight to the mode rather than clicking the mode dropdown - one
fewer fragile interaction, and the composer state is unambiguous from the first paint.

--mode sub-app references
text CreativeStudio/MiniApp/TextToVideo none
image CreativeStudio/MiniApp/ImageToVideo 1, or 2 with --frames first-last
reference CreativeStudio/ReferenceToVideo/ReferenceToVideo 1-4

--duration is 5 / 10 / 12 seconds and Video 1.5 Pro is the only model the tab
offers. There is no aspect-ratio control.

Four things about video that shaped the design

A clip costs one credit per second. 980 to 970 across a 10s render; a 5s clip costs
5. That is unlike image generation, which was free on this plan. A render refused by
moderation is charged too, so --duration is rejected rather than coerced - a silently
adjusted length would bill the caller for something they did not ask for.

Rendering takes tens of minutes. The card says "Check back in 5-10 minutes"; the
first clip measured took over an hour. --wait therefore defaults to false: the
command submits, returns status: generating, and jobs / generations pick the
result up later. --wait true is available and polls up to --timeout (default 5400s).

Losing the progress percentage is not completion. The lifecycle is 3% ... 98%,
then the percentage disappears, then "Almost there. We'll notify you when ready." for a
long stretch, and only then does the <video> mount. Treating "no percentage" as done
would report success ten-plus minutes early, so jobs reports four states and claims
ready only when an output is actually mounted:

status condition
failed the card carries an Error code:
ready an output element is mounted (a non-demo <video>, or an <img> on the asset CDN)
generating a percentage is showing
finishing neither - the "Almost there" tail, and also a finished card scrolled out of view

finishing is deliberately non-committal: it says "cannot confirm" instead of guessing.
No prose is parsed for any of this, since the UI is localized.

A render can be refused by moderation, and it looks like nothing went wrong. The
failure arrives as ordinary card text - no dialog, no HTTP error:

<prompt> | Video 1.5 Pro | This content may involve third-party intellectual property,
which violates our Community Guidelines. Try generating again. |
Task ID: 7664452848403234817 | Error code: 10001305

generate-video --wait true raises a CommandExecutionError carrying the code and task
id instead of waiting out the timeout, and jobs surfaces it in the error column.
Incidentally this is the only place the site exposes a real generation id - successful
cards have none.

Clips are not on the image CDN

image  https://p16-ad-site-sign-sg.tiktokcdn.com/ad-creative-sg/<assetId>~tplv-...image
clip   https://v16-ad-creative.tiktokcdn-row.com/<h>/<h>/video/tos/alisg/.../<tosId>/?...&vid=<videoId>

A clip tile renders a <video> with no poster on the asset host, no ad-creative-sg
path segment, and its identity is the vid query parameter. generations reported no
video at all before this change, and download would have fetched a poster image.
download now returns a real MP4 (verified content-type: video/mp4, ftyp isom
magic bytes).

Relatedly, generations now derives type from the element kind rather than the tile
badge. The badge is localized, and clip tiles carry the tool name ("Reference to video")
instead of a Video badge at all.

delete

Tile overflow button, then the menu row carrying ks-icon[name=delete], then the
confirm dialog's [part=confirmButton]. Both anchors are structural rather than
textual, which matters here: the labels are localized, and the menu differs by asset
kind
(an image offers Share / Generate video / Edit / Feedback / Delete; a clip offers
Open in editor / Feedback / Delete).

The command is irreversible, so it is dry-run by default - without --yes it only
locates the tile and reports status: dry-run. That also lets the fixture verify the
whole lookup path without destroying anything. After confirming, it waits for the tile
to leave the grid before reporting deleted, rather than trusting the click.

Worth knowing: the dialog says "If you delete this generation...", but deleting is
per-asset. Removing one image left its three siblings from the same run untouched.

Fixes to the commands already in this PR

Background windows never render, so the grid never loads. Library and feed tiles
mount lazily through IntersectionObserver. A background tab is not rendered, so
nothing intersects and the grid stays empty forever. Same command, same moment:
--window background gave 1 row, --window foreground gave 20. The reading commands
now set defaultWindowMode: 'foreground'. This had been masquerading as flaky
thumbnails for a while before the cause turned up.

__deepAll skipped the root node's own shadow root. It walked
root.querySelectorAll('*') and recursed into the children's shadow roots. A
ks-modal keeps its title and buttons in its own shadow root while its light DOM holds
only the body text, so the delete dialog looked like it had no controls at all.

generate could report a previous run's assets. It snapshotted every asset id on
the page before submitting and diffed afterwards. The feed lazy-loads, so assets from an
earlier generation arrived after the snapshot and looked brand new - reproduced across
three consecutive verify runs, each returning one to three ids belonging to the run
before. It now scopes to the card carrying its own prompt, still subtracts a snapshot
taken once the feed has settled, and confirms the reading is stable before returning.

Reference upload broke on files over roughly half a megabyte. A single ~1 MB base64
evaluate argument fails the bridge with sendCommand: max attempts exhausted after
about four seconds. The base64 now goes over in 128 KB chunks and is reassembled into
the File in page context.

The Library grid scrolls inside its own container. window.scrollY stays 0, so
page.scroll left the grid untouched and everything below the fold stayed unrendered.
The container is now located by shape (clientHeight / scrollHeight) rather than by
class name, which survives a restyle.

Identity

The DOM carries no generation id anywhere (only data-inspector hashes, which are
dev-tool artifacts). The only stable per-asset handle is the CDN path segment, so that
is what generations emits as assetId and what download consumes. Asset URLs are
signed and short-lived, so download reads the live URL out of the Library instead of
reconstructing it.

Verification

Verified end-to-end against a live account, with hand-tuned fixtures (patterns +
notEmpty + tightened rowCount) under ~/.opencli/sites/tiktok-symphony/verify/:

✓ tiktok-symphony/credits        ✓ tiktok-symphony/generations
✓ tiktok-symphony/download       ✓ tiktok-symphony/jobs
✓ tiktok-symphony/delete         ✓ tiktok-symphony/generate-video

generate is the one command opencli browser verify cannot run: the harness kills the
adapter after 30 seconds (execFileSync(..., { timeout: 30000 }) in cli.js) and an
image generation takes about 50. Run directly it returns four fresh assets every time;
its fixture is committed for whenever that cap is lifted.

All three video modes were submitted for real (text-to-video, image-to-video,
reference-to-video), one of which came back refused by moderation - which is how that
failure path got covered. The --wait completion path was then exercised against the
finished card's real DOM without spending further credits.

Values were checked against the page by eye, not just for "the adapter didn't throw":

  • credits matches the header (balance / plan / account).
  • download produces a real PNG (magic bytes 89 50 4E 47, ~1 MB).
  • generate was run with a synthetic reference image (blue background, gold circle).
    The returned outputs are 3D chibi renders that keep the blue background and gold
    circle
    , which confirms the reference genuinely conditions the model rather than
    being silently dropped into a text-only generation. Both models were exercised.

One bug this caught: the Library tile badge regex /^(Image|Video)\b/ never matches
"ImageDownload" (no word boundary between the two words), which silently produced a
type column that was null on every row while the command still "succeeded".

Checks

npx tsc --noEmit                    clean
opencli validate                    Errors: 0   (8 warnings, all pre-existing, none in this site)
npm run check:typed-error-lint      new=0
npm run check:silent-column-drop    new=0
bash scripts/check-doc-coverage.sh  174/174 documented
npx vitest run --project adapter clis/tiktok-symphony/   18 passed

npm test has 11 pre-existing failures on this commit's base (instagram, suno, twitter,
xiaoyuzhou); I confirmed they reproduce on a clean tree with these changes stashed, so
they are unrelated to this PR.

Notes for reviewers

  • generate is access: 'write' — each run creates real assets on the account. On the
    Basic plan the observed cost was 0 credits (balance unchanged, composer showed cost
    0), but that is plan-dependent and the assets are real.
  • generate-video is access: 'write' and always costs credits — one per second of
    clip, charged even when the render is refused.
  • delete is access: 'write' and irreversible; --yes is required for it to do
    anything, and the fixture exercises only the dry-run path.
  • generate deliberately reloads the composer before starting: submitting does not
    clear the prompt or references, so a reused tab would silently resend them. If stale
    references somehow survive the reload the command fails loudly rather than generating
    from unknown inputs.
  • Out-of-range limit / timeout / refs raise ArgumentError; there is no silent
    clamp anywhere, and plan / account return null rather than a string sentinel.

🤖 Generated with Claude Code

kienbb and others added 2 commits July 20, 2026 02:02
Add four UI adapters for TikTok Symphony Creative Studio image generation:

- credits: credit balance, plan and signed-in account
- generations: list generated assets in the Library (alias: library)
- download: save a generated asset to a local file
- generate: generate images from a prompt plus up to 4 reference images,
  selecting between the Nano Banana and Flux Kontext Max models

Strategy is UI throughout. The composer exposes stable data-chatbox-part
anchors (topbar / header / textarea / footer), which are a better contract
than the site's unversioned internal XHR.

Two site-specific constraints shaped the implementation:

- ks-* web components carry a version suffix (ks-button-1-1-1m), so every
  lookup matches on tag prefix rather than the full tag name.
- The page exposes no input[type=file] and CDP file injection is refused,
  so reference images are attached through the composer's own drop zone by
  building the File inside the page and dispatching a real drop.

Symphony is a client-rendered SPA that hydrates after navigation, so each
command polls for readiness instead of assuming the DOM is present.

All four verified end-to-end against a live account, including an eyeball
check that a reference image genuinely conditions the output rather than
being silently ignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the adapter from the Image tab to the Video tab, plus the two
commands that make a slow write loop usable: watching a render and
removing what it produced.

New commands
- generate-video (alias video): all three Video sub-apps — text-to-video,
  image-to-video (first frame, or first+last frame), reference-to-video.
  Each mode is its own `subApp` URL, so the command navigates straight to
  it instead of clicking the mode dropdown.
- jobs (alias queue): render progress for the Create feed.
- delete (alias rm): remove a generation. Irreversible, so it is dry-run
  by default and needs --yes.

Video specifics found while building this
- A clip costs one credit per second, and a rejected render is charged
  too, so --duration is rejected rather than coerced.
- Rendering runs for tens of minutes (over an hour observed) despite the
  page saying 5-10. --wait therefore defaults to false: submit, return
  `generating`, collect later with jobs/generations.
- Losing the progress percentage is NOT completion — the site sits on
  "Almost there..." well before the clip mounts. jobs reports a distinct
  `finishing` state rather than guessing `ready`.
- A render can be refused by moderation. It arrives as ordinary card text
  with a task id and an error code, with no dialog and no HTTP error, so
  generate-video raises it instead of waiting out the timeout.
- Clips are not on the image CDN: they are <video> elements on
  v16-ad-creative.tiktokcdn-row.com identified by the `vid` query param,
  with no assetId path segment. generations reported no video at all
  before this, and download would have fetched a poster.

Fixes to the existing commands
- Background windows never render, so the lazily mounted Library and feed
  tiles never intersect and the grid stays empty. The reading commands now
  default to a foreground window; this was the real cause behind what
  looked like flaky thumbnails.
- __deepAll skipped the root node's own shadow root. A ks-modal keeps its
  buttons there while its light DOM holds only body text, so a dialog
  looked like it had no controls.
- generate diffed page-wide asset ids against a pre-submit snapshot. The
  feed lazy-loads, so assets from an earlier run arrived after the
  snapshot and were reported as this run's output — reproduced across
  three consecutive runs. It now scopes to the card carrying its own
  prompt and settles the reading before returning.
- Reference upload transfers its base64 in chunks; a single ~1 MB
  evaluate argument exceeds what the browser bridge carries.
- The Library grid scrolls inside its own container, so page.scroll left
  it untouched and everything below the fold stayed unrendered.
- generations derives `type` from the element kind instead of the tile
  badge, which is localized and absent on clip tiles.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kienbb kienbb changed the title feat(tiktok-symphony): add Symphony Creative Studio image adapters feat(tiktok-symphony): add Symphony Creative Studio image and video adapters Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants