fix(ths): rewrite hot-rank to use upstream API for full 100 ranks#2141
Closed
qian0817 wants to merge 1 commit into
Closed
fix(ths): rewrite hot-rank to use upstream API for full 100 ranks#2141qian0817 wants to merge 1 commit into
qian0817 wants to merge 1 commit into
Conversation
The ths hot-rank adapter scraped the DOM, but the page renders the list inside a Swiper carousel that lazy-loads 20 ranks per slide. Without swiping through all slides the DOM only exposes the first ~20 names, so `--limit 100` returned ~68 entries and ranks past the first slide were wrong (the printed rank is a per-slide 1-20 that recycles across slides). Switch to the public hot_list JSON API the page itself consumes. It returns all 100 entries in one call with an authoritative `order` rank and needs no auth: GET https://dq.10jqka.com.cn/fuyao/hot_list_data/out/hot_list/v1/stock ?stock_type=a&type=hour&list_type=normal - strategy PUBLIC + browser:false (pure HTTP, like toutiao/hot) - map fields: order->rank, code->symbol, rise_and_fall->changePercent, rate->heat (formatted as "x.x万热度"), tag.{popularity_tag,concept_tag}->tags - clamp limit into [1,100]; throw CommandExecutionError on HTTP/JSON/ in-band status errors and EmptyResultError on an empty stock_list - correct the non-browser func signature to (kwargs, debug); the prior (page, kwargs) signature silently swallowed --limit for browser:false commands - rebuild cli-manifest.json (new strategy/browser/columns) - add `symbol` column, aligning with eastmoney/tdx hot-rank adapters - tests now mock global fetch and cover limit=100 dense 1..100 ranks, non-sorted upstream order, clamping, and all error paths
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #2140.
Rewrites the
ths/hot-rankadapter to fetch data from the public hot_list JSON API instead of scraping the Swiper-lazy-loaded DOM. This makes--limit 100return the full 100 entries with an authoritative1..100rank.Background / Root Cause
The page renders the ranking inside a Swiper carousel that lazy-loads 20 ranks per slide. DOM scraping only ever sees the first ~20 names, so:
--limit 100returned only ~68 entries.1..20that recycles across slides, soidx+1/rank++logic cannot reconstruct the real1..100order.The data source itself is a public, unauthenticated JSON API that returns all 100 entries in one call with an authoritative
orderrank.Changes
clis/ths/hot-rank.js— pure HTTP call to the upstream API:strategy: PUBLIC,browser: false(same pattern astoutiao/hot).order→rank,code→symbol,rise_and_fall→changePercent,rate→heat(formatted asx.x万热度),tag.{popularity_tag, concept_tag}→tags.limitclamped to[1, 100].CommandExecutionErroron HTTP / malformed-JSON / non-zerostatus_code;EmptyResultErroron an emptystock_list.funcsignature to(kwargs, debug). The previous(page, kwargs)signature silently swallowed--limitforbrowser: falsecommands.symbolcolumn, aligning witheastmoney/hot-rank/tdx/hot-rank.clis/ths/hot-rank.test.js— mock globalfetch; coverlimit=100dense1..100ranks, non-sorted upstreamorder, clamping, and all error paths. 9 tests pass.cli-manifest.json— rebuilt (newstrategy/browser/columns).Verification
Edge cases tested:
limit=1, default20,limit=200(clamped to 100), table/plain/json formats.Notes
package-lock.jsonchanges fromnpm installwere intentionally excluded from this PR.