Skip to content

fix(doubao): refresh history and detail extraction#1375

Open
lmmsoft wants to merge 10 commits into
jackwener:mainfrom
lmmsoft:fix/doubao-history-detail-dom
Open

fix(doubao): refresh history and detail extraction#1375
lmmsoft wants to merge 10 commits into
jackwener:mainfrom
lmmsoft:fix/doubao-history-detail-dom

Conversation

@lmmsoft

@lmmsoft lmmsoft commented May 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • update Doubao history extraction to support the current sidebar DOM (#flow_chat_sidebar, conversation_* anchors, and /chat/<id> links)
  • update Doubao detail extraction to support current message list/item classes, user send bubbles, and assistant .flow-markdown-body content
  • add regression coverage for the new sidebar and detail selectors

Verification

  • PATH=/Users/lmm333/.nvm/versions/node/v24.13.0/bin:$PATH npm exec vitest run --project adapter clis/doubao/utils.test.js clis/doubao/history.test.js clis/doubao/detail.test.js
  • PATH=/Users/lmm333/.nvm/versions/node/v24.13.0/bin:$PATH npm run typecheck
  • manually verified on logged-in Doubao browser session that opencli doubao history --limit 5 returns real conversations and opencli doubao detail <conversation_id> returns user/assistant messages

Copilot AI review requested due to automatic review settings May 6, 2026 18:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Doubao CLI adapter’s DOM-scraping scripts so history and detail can continue working with Doubao’s current sidebar and conversation detail markup.

Changes:

  • Broadened sidebar selectors in getConversationListScript() to handle current conversation anchors and dedupe results.
  • Broadened detail selectors in getConversationDetailScript() to handle current message containers/bubbles.
  • Added/extended unit tests to assert the updated selector coverage, and exported the scripts via __test__ for testing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
clis/doubao/utils.js Updates the in-page evaluation scripts for conversation list + detail extraction, and exports the new scripts for tests.
clis/doubao/utils.test.js Adds tests asserting the updated DOM selectors are present in the generated scripts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread clis/doubao/utils.js Outdated
Comment thread clis/doubao/utils.js Outdated
@lmmsoft
lmmsoft force-pushed the fix/doubao-history-detail-dom branch from ec5f3e3 to 02dd1ce Compare June 6, 2026 02:59
@lmmsoft

lmmsoft commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Updated this PR to the latest Doubao adapter fix. The new head replaces the old DOM-only patch with the recent_conv API-backed history path, refreshed detail extraction, and pc_pin_query_type pagination handling. Local verification: npm test -- clis/doubao/history.test.js clis/doubao/detail.test.js clis/doubao/utils.test.js (29 tests passing).

@lmmsoft lmmsoft changed the title fix(doubao): support current history and detail DOM fix(doubao): refresh history and detail extraction Jun 6, 2026
@lmmsoft

lmmsoft commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Added a generic opencli doubao download command in commit 33fdae9.\n\nWhat changed:\n- Extracts Doubao conversation media from window._ROUTER_DATA first, with DOM image fallback.\n- Supports image variants via --variant original|raw|preview|thumb.\n- Downloads into <output>/<conversation-id>/ with --limit and per-file --timeout controls for safer batch/backfill use.\n- Includes tests for command behavior and route-data extraction.\n\nLocal checks run:\n- npm test -- clis/doubao (4 files, 35 tests)\n- npm run build-manifest\n- npx tsx src/main.ts validate doubao/download\n\nNote: on my current machine, the OpenCLI browser bridge/doctor did not return within ~15s, so I could not complete a live browser download smoke in this environment.

@lmmsoft

lmmsoft commented Jun 6, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up update in c43244d: fixed a live-smoke issue where stale route data / generic DOM images could be mistaken for media from the requested Doubao conversation. The extractor now scopes route data to the target conversation and only traverses matched message lists; DOM image fallback excludes avatars, bot icons, intro art, placeholders, and small generic images.\n\nAdditional local validation:\n- PR branch: npm test -- clis/doubao (4 files, 35 tests)\n- PR branch: npx tsx src/main.ts validate doubao/download\n- Isolated latest OpenCLI 1.8.3 smoke environment: npm test -- clis/doubao (4 files, 38 tests), node dist/src/main.js validate doubao/download, node dist/src/main.js doctor OK\n- Live smoke success for unexpired Doubao conversation media: downloaded a real 1920x1440 PNG (2.0 MB)\n- Live smoke for expired signed image URL: returned failed / HTTP 403, as expected, without downloading stale placeholders.

@lmmsoft
lmmsoft force-pushed the fix/doubao-history-detail-dom branch from c43244d to 0c431d2 Compare June 6, 2026 06:45
@lmmsoft
lmmsoft force-pushed the fix/doubao-history-detail-dom branch from 0c431d2 to cb78a6b Compare July 23, 2026 17:22
@lmmsoft

lmmsoft commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this PR onto current main and revalidated against the live Doubao site.

What changed in the refresh:

  • rebased the existing history/detail/media work onto 5256711
  • kept pc_pin_query_type pagination state and raised the validated history ceiling from 1,000 to 5,000
  • replaced sentinel rows and silent numeric fallbacks with typed ArgumentError / EmptyResultError
  • declared all doubao download output columns (filename and url included)

Local checks:

  • npm test -- clis/doubao — 4 files, 42 tests passed
  • npm run build
  • npm run check:silent-column-drop — no new violations
  • npm run check:typed-error-lint — no new violations
  • node dist/src/main.js validate doubao — 12 commands, 0 errors/warnings

Live smoke on the logged-in account:

  • doubao history --limit 2000 -f json returned 1,514 real conversations
  • conversation 38434119145388290 was found at index 45
  • doubao detail 38434119145388290 -f json returned one complete User row and one complete Assistant row

The previous Ubuntu CI failure (doubao/download ... missing=[filename, url]) is fixed in 7ca93d7. The separate security-audit failure is the repository-wide js-yaml 4.2.0 advisory from the current base lockfile, not introduced by this PR.

Generated on: cmcc-i5
@lmmsoft

lmmsoft commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

CI follow-up: the manifest drift was fixed in lmmsoft@6e63f8b. All PR-relevant build/test/doc checks are now green on Ubuntu, macOS, Windows, Node, and Bun. The only remaining red check is the repository-wide security audit for js-yaml 4.2.0 in the current base lockfile; this PR does not change that dependency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants