feat(tiktok-symphony): add Symphony Creative Studio image and video adapters#2159
Open
kienbb wants to merge 2 commits into
Open
feat(tiktok-symphony): add Symphony Creative Studio image and video adapters#2159kienbb wants to merge 2 commits into
kienbb wants to merge 2 commits into
Conversation
Add four UI adapters for TikTok Symphony Creative Studio image generation: - credits: credit balance, plan and signed-in account - generations: list generated assets in the Library (alias: library) - download: save a generated asset to a local file - generate: generate images from a prompt plus up to 4 reference images, selecting between the Nano Banana and Flux Kontext Max models Strategy is UI throughout. The composer exposes stable data-chatbox-part anchors (topbar / header / textarea / footer), which are a better contract than the site's unversioned internal XHR. Two site-specific constraints shaped the implementation: - ks-* web components carry a version suffix (ks-button-1-1-1m), so every lookup matches on tag prefix rather than the full tag name. - The page exposes no input[type=file] and CDP file injection is refused, so reference images are attached through the composer's own drop zone by building the File inside the page and dispatching a real drop. Symphony is a client-rendered SPA that hydrates after navigation, so each command polls for readiness instead of assuming the DOM is present. All four verified end-to-end against a live account, including an eyeball check that a reference image genuinely conditions the output rather than being silently ignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extends the adapter from the Image tab to the Video tab, plus the two commands that make a slow write loop usable: watching a render and removing what it produced. New commands - generate-video (alias video): all three Video sub-apps — text-to-video, image-to-video (first frame, or first+last frame), reference-to-video. Each mode is its own `subApp` URL, so the command navigates straight to it instead of clicking the mode dropdown. - jobs (alias queue): render progress for the Create feed. - delete (alias rm): remove a generation. Irreversible, so it is dry-run by default and needs --yes. Video specifics found while building this - A clip costs one credit per second, and a rejected render is charged too, so --duration is rejected rather than coerced. - Rendering runs for tens of minutes (over an hour observed) despite the page saying 5-10. --wait therefore defaults to false: submit, return `generating`, collect later with jobs/generations. - Losing the progress percentage is NOT completion — the site sits on "Almost there..." well before the clip mounts. jobs reports a distinct `finishing` state rather than guessing `ready`. - A render can be refused by moderation. It arrives as ordinary card text with a task id and an error code, with no dialog and no HTTP error, so generate-video raises it instead of waiting out the timeout. - Clips are not on the image CDN: they are <video> elements on v16-ad-creative.tiktokcdn-row.com identified by the `vid` query param, with no assetId path segment. generations reported no video at all before this, and download would have fetched a poster. Fixes to the existing commands - Background windows never render, so the lazily mounted Library and feed tiles never intersect and the grid stays empty. The reading commands now default to a foreground window; this was the real cause behind what looked like flaky thumbnails. - __deepAll skipped the root node's own shadow root. A ks-modal keeps its buttons there while its light DOM holds only body text, so a dialog looked like it had no controls. - generate diffed page-wide asset ids against a pre-submit snapshot. The feed lazy-loads, so assets from an earlier run arrived after the snapshot and were reported as this run's output — reproduced across three consecutive runs. It now scopes to the card carrying its own prompt and settles the reading before returning. - Reference upload transfers its base64 in chunks; a single ~1 MB evaluate argument exceeds what the browser bridge carries. - The Library grid scrolls inside its own container, so page.scroll left it untouched and everything below the fold stayed unrendered. - generations derives `type` from the element kind instead of the tile badge, which is localized and absent on clip tiles. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds seven adapters for TikTok Symphony Creative Studio (
ads.tiktok.com), coveringboth the Image and the Video tab.
creditsgenerations(aliaslibrary)jobs(aliasqueue)download <asset>generate <prompt>generate-video <prompt>delete <asset>Strategy note
The site does have internal XHR, but the composer's
data-chatbox-partanchors are auser-visible contract and a steadier bet than unversioned internal endpoints, so this
stays on the UI ladder rung rather than climbing to
PAGE_FETCH/INTERCEPT.Two site-specific constraints worth flagging for reviewers
ks-*web components carry a version suffix. Elements areks-button-1-1-1m,ks-tag-1-1-1m, and so on. Every lookup matches on tag prefix(
tagName.startsWith('ks-button')) — hard-coding the full tag would break on the nextcomponent version bump.
Reference upload cannot use
page.uploadFiles. The page renders noinput[type=file]at all, and injecting one and driving it through CDP is refused(
{"code":-32000,"message":"Not allowed"}). Attaching a reference therefore goesthrough the composer's own drop zone: the file is read Node-side, rebuilt as a
Fileinside the page, and dispatched as a real
drop. This only reuses a capability thepage already offers to any user dragging a file in.
Symphony is a client-rendered SPA whose header and composer hydrate after navigation
resolves, so every command polls via a shared
waitForValuehelper rather thanassuming the DOM is ready — that gap was the first thing to bite during development.
The Video tab
generate-videocovers all three Video sub-apps. Each is its ownsubAppURL, so thecommand navigates straight to the mode rather than clicking the mode dropdown - one
fewer fragile interaction, and the composer state is unambiguous from the first paint.
--modetextCreativeStudio/MiniApp/TextToVideoimageCreativeStudio/MiniApp/ImageToVideo--frames first-lastreferenceCreativeStudio/ReferenceToVideo/ReferenceToVideo--durationis5/10/12seconds andVideo 1.5 Prois the only model the taboffers. There is no aspect-ratio control.
Four things about video that shaped the design
A clip costs one credit per second. 980 to 970 across a 10s render; a 5s clip costs
5. That is unlike image generation, which was free on this plan. A render refused by
moderation is charged too, so
--durationis rejected rather than coerced - a silentlyadjusted length would bill the caller for something they did not ask for.
Rendering takes tens of minutes. The card says "Check back in 5-10 minutes"; the
first clip measured took over an hour.
--waittherefore defaults to false: thecommand submits, returns
status: generating, andjobs/generationspick theresult up later.
--wait trueis available and polls up to--timeout(default 5400s).Losing the progress percentage is not completion. The lifecycle is
3% ... 98%,then the percentage disappears, then "Almost there. We'll notify you when ready." for a
long stretch, and only then does the
<video>mount. Treating "no percentage" as donewould report success ten-plus minutes early, so
jobsreports four states and claimsreadyonly when an output is actually mounted:failedError code:ready<video>, or an<img>on the asset CDN)generatingfinishingfinishingis deliberately non-committal: it says "cannot confirm" instead of guessing.No prose is parsed for any of this, since the UI is localized.
A render can be refused by moderation, and it looks like nothing went wrong. The
failure arrives as ordinary card text - no dialog, no HTTP error:
generate-video --wait trueraises aCommandExecutionErrorcarrying the code and taskid instead of waiting out the timeout, and
jobssurfaces it in theerrorcolumn.Incidentally this is the only place the site exposes a real generation id - successful
cards have none.
Clips are not on the image CDN
A clip tile renders a
<video>with no poster on the asset host, noad-creative-sgpath segment, and its identity is the
vidquery parameter.generationsreported novideo at all before this change, and
downloadwould have fetched a poster image.downloadnow returns a real MP4 (verifiedcontent-type: video/mp4,ftyp isommagic bytes).
Relatedly,
generationsnow derivestypefrom the element kind rather than the tilebadge. The badge is localized, and clip tiles carry the tool name ("Reference to video")
instead of a
Videobadge at all.deleteTile overflow button, then the menu row carrying
ks-icon[name=delete], then theconfirm dialog's
[part=confirmButton]. Both anchors are structural rather thantextual, which matters here: the labels are localized, and the menu differs by asset
kind (an image offers Share / Generate video / Edit / Feedback / Delete; a clip offers
Open in editor / Feedback / Delete).
The command is irreversible, so it is dry-run by default - without
--yesit onlylocates the tile and reports
status: dry-run. That also lets the fixture verify thewhole lookup path without destroying anything. After confirming, it waits for the tile
to leave the grid before reporting
deleted, rather than trusting the click.Worth knowing: the dialog says "If you delete this generation...", but deleting is
per-asset. Removing one image left its three siblings from the same run untouched.
Fixes to the commands already in this PR
Background windows never render, so the grid never loads. Library and feed tiles
mount lazily through
IntersectionObserver. A background tab is not rendered, sonothing intersects and the grid stays empty forever. Same command, same moment:
--window backgroundgave 1 row,--window foregroundgave 20. The reading commandsnow set
defaultWindowMode: 'foreground'. This had been masquerading as flakythumbnails for a while before the cause turned up.
__deepAllskipped the root node's own shadow root. It walkedroot.querySelectorAll('*')and recursed into the children's shadow roots. Aks-modalkeeps its title and buttons in its own shadow root while its light DOM holdsonly the body text, so the delete dialog looked like it had no controls at all.
generatecould report a previous run's assets. It snapshotted every asset id onthe page before submitting and diffed afterwards. The feed lazy-loads, so assets from an
earlier generation arrived after the snapshot and looked brand new - reproduced across
three consecutive verify runs, each returning one to three ids belonging to the run
before. It now scopes to the card carrying its own prompt, still subtracts a snapshot
taken once the feed has settled, and confirms the reading is stable before returning.
Reference upload broke on files over roughly half a megabyte. A single ~1 MB base64
evaluateargument fails the bridge withsendCommand: max attempts exhaustedafterabout four seconds. The base64 now goes over in 128 KB chunks and is reassembled into
the
Filein page context.The Library grid scrolls inside its own container.
window.scrollYstays 0, sopage.scrollleft the grid untouched and everything below the fold stayed unrendered.The container is now located by shape (
clientHeight/scrollHeight) rather than byclass name, which survives a restyle.
Identity
The DOM carries no generation id anywhere (only
data-inspectorhashes, which aredev-tool artifacts). The only stable per-asset handle is the CDN path segment, so that
is what
generationsemits asassetIdand whatdownloadconsumes. Asset URLs aresigned and short-lived, so
downloadreads the live URL out of the Library instead ofreconstructing it.
Verification
Verified end-to-end against a live account, with hand-tuned fixtures (
patterns+notEmpty+ tightenedrowCount) under~/.opencli/sites/tiktok-symphony/verify/:generateis the one commandopencli browser verifycannot run: the harness kills theadapter after 30 seconds (
execFileSync(..., { timeout: 30000 })incli.js) and animage generation takes about 50. Run directly it returns four fresh assets every time;
its fixture is committed for whenever that cap is lifted.
All three video modes were submitted for real (text-to-video, image-to-video,
reference-to-video), one of which came back refused by moderation - which is how that
failure path got covered. The
--waitcompletion path was then exercised against thefinished card's real DOM without spending further credits.
Values were checked against the page by eye, not just for "the adapter didn't throw":
creditsmatches the header (balance / plan / account).downloadproduces a real PNG (magic bytes89 50 4E 47, ~1 MB).generatewas run with a synthetic reference image (blue background, gold circle).The returned outputs are 3D chibi renders that keep the blue background and gold
circle, which confirms the reference genuinely conditions the model rather than
being silently dropped into a text-only generation. Both models were exercised.
One bug this caught: the Library tile badge regex
/^(Image|Video)\b/never matches"ImageDownload"(no word boundary between the two words), which silently produced atypecolumn that wasnullon every row while the command still "succeeded".Checks
npm testhas 11 pre-existing failures on this commit's base (instagram, suno, twitter,xiaoyuzhou); I confirmed they reproduce on a clean tree with these changes stashed, so
they are unrelated to this PR.
Notes for reviewers
generateisaccess: 'write'— each run creates real assets on the account. On theBasic plan the observed cost was 0 credits (balance unchanged, composer showed cost
0), but that is plan-dependent and the assets are real.generate-videoisaccess: 'write'and always costs credits — one per second ofclip, charged even when the render is refused.
deleteisaccess: 'write'and irreversible;--yesis required for it to doanything, and the fixture exercises only the dry-run path.
generatedeliberately reloads the composer before starting: submitting does notclear the prompt or references, so a reused tab would silently resend them. If stale
references somehow survive the reload the command fails loudly rather than generating
from unknown inputs.
limit/timeout/refsraiseArgumentError; there is no silentclamp anywhere, and
plan/accountreturnnullrather than a string sentinel.🤖 Generated with Claude Code