Skip to content

fix(ths): rewrite hot-rank to use upstream API for full 100 ranks#2141

Closed
qian0817 wants to merge 1 commit into
jackwener:mainfrom
qian0817:fix/ths-hot-rank-api
Closed

fix(ths): rewrite hot-rank to use upstream API for full 100 ranks#2141
qian0817 wants to merge 1 commit into
jackwener:mainfrom
qian0817:fix/ths-hot-rank-api

Conversation

@qian0817

Copy link
Copy Markdown

Summary

Fixes #2140.

Rewrites the ths/hot-rank adapter to fetch data from the public hot_list JSON API instead of scraping the Swiper-lazy-loaded DOM. This makes --limit 100 return the full 100 entries with an authoritative 1..100 rank.

Background / Root Cause

The page renders the ranking inside a Swiper carousel that lazy-loads 20 ranks per slide. DOM scraping only ever sees the first ~20 names, so:

  • --limit 100 returned only ~68 entries.
  • Ranks past the first slide were wrong: the rank text printed on each card is a per-slide 1..20 that recycles across slides, so idx+1 / rank++ logic cannot reconstruct the real 1..100 order.

The data source itself is a public, unauthenticated JSON API that returns all 100 entries in one call with an authoritative order rank.

Changes

  • clis/ths/hot-rank.js — pure HTTP call to the upstream API:
    GET https://dq.10jqka.com.cn/fuyao/hot_list_data/out/hot_list/v1/stock?stock_type=a&type=hour&list_type=normal
    
    • strategy: PUBLIC, browser: false (same pattern as toutiao/hot).
    • Field mapping: orderrank, codesymbol, rise_and_fallchangePercent, rateheat (formatted as x.x万热度), tag.{popularity_tag, concept_tag}tags.
    • limit clamped to [1, 100].
    • Errors: CommandExecutionError on HTTP / malformed-JSON / non-zero status_code; EmptyResultError on an empty stock_list.
    • Fix the non-browser func signature to (kwargs, debug). The previous (page, kwargs) signature silently swallowed --limit for browser: false commands.
    • Add a symbol column, aligning with eastmoney/hot-rank / tdx/hot-rank.
  • clis/ths/hot-rank.test.js — mock global fetch; cover limit=100 dense 1..100 ranks, non-sorted upstream order, clamping, and all error paths. 9 tests pass.
  • cli-manifest.json — rebuilt (new strategy / browser / columns).

Verification

$ opencli ths hot-rank --limit 100 -f json
# count: 100, rank = 1..100 dense, all fields 100% populated

Edge cases tested: limit=1, default 20, limit=200 (clamped to 100), table/plain/json formats.

Notes

  • package-lock.json changes from npm install were intentionally excluded from this PR.

The ths hot-rank adapter scraped the DOM, but the page renders the list
inside a Swiper carousel that lazy-loads 20 ranks per slide. Without
swiping through all slides the DOM only exposes the first ~20 names, so
`--limit 100` returned ~68 entries and ranks past the first slide were
wrong (the printed rank is a per-slide 1-20 that recycles across slides).

Switch to the public hot_list JSON API the page itself consumes. It
returns all 100 entries in one call with an authoritative `order` rank
and needs no auth:

  GET https://dq.10jqka.com.cn/fuyao/hot_list_data/out/hot_list/v1/stock
      ?stock_type=a&type=hour&list_type=normal

- strategy PUBLIC + browser:false (pure HTTP, like toutiao/hot)
- map fields: order->rank, code->symbol, rise_and_fall->changePercent,
  rate->heat (formatted as "x.x万热度"), tag.{popularity_tag,concept_tag}->tags
- clamp limit into [1,100]; throw CommandExecutionError on HTTP/JSON/
  in-band status errors and EmptyResultError on an empty stock_list
- correct the non-browser func signature to (kwargs, debug); the prior
  (page, kwargs) signature silently swallowed --limit for browser:false
  commands
- rebuild cli-manifest.json (new strategy/browser/columns)
- add `symbol` column, aligning with eastmoney/tdx hot-rank adapters
- tests now mock global fetch and cover limit=100 dense 1..100 ranks,
  non-sorted upstream order, clamping, and all error paths
@qian0817 qian0817 closed this Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: ths hot-rank returns incomplete data and wrong ranks when limit > 20

1 participant