SKILLSPECTOR_MAX_LLM_CONCURRENCY does not bound total in-flight LLM requests

[Mark2Mac]

Title: SKILLSPECTOR_MAX_LLM_CONCURRENCY does not bound total in-flight LLM requests

Summary

SKILLSPECTOR_MAX_LLM_CONCURRENCY creates a semaphore inside a single analyzer's batch
fan-out. The analyzers themselves are separate LangGraph nodes that run concurrently, so N
analyzers issue requests simultaneously no matter what the variable is set to.

Graph topology (graph.py):

for analyzer_id in ...:
    workflow.add_edge("build_context", analyzer_id)   # fan-out, parallel
    workflow.add_edge(analyzer_id, "meta_analyzer")

Semaphore scope (llm_analyzer_base.py):

if max_concurrency is None:
    max_concurrency = resolve_max_concurrency()
sem = asyncio.Semaphore(max_concurrency)          # per-analyzer, not global
results = await asyncio.gather(*[_process(b) for b in batches], ...)

The docstring says "Users on rate-limited endpoints can lower this" — which is the intent, and
it does not hold: lowering it to 1 still produces N simultaneous requests.

Reproduction

Endpoint: build.nvidia.com (integrate.api.nvidia.com/v1), free tier, model z-ai/glm-5.2.
Target: one small skill (a single SKILL.md), which triggers 4 LLM calls.

SKILLSPECTOR_MAX_LLM_CONCURRENCY	calls succeeded
4	1 / 4
2	1 / 4
1	1 / 4

The three failures are 429 in every case, and the timing shows them arriving together. The
setting has no observable effect because it is not the binding constraint.

Placing an external serializing proxy in front of the endpoint — one in-flight request at a
time, with a minimum interval — takes the same scan to 4 / 4 succeeded, llm_degraded: null, with the meta-analyzer applied. That is the behaviour users are trying to get from the
environment variable.

Why this matters

metadata.llm_degraded correctly reports the partial coverage, so the scan is honest about it.
But on a rate-limited endpoint the default path loses 3 of 4 analyzers per unit, and the one
knob offered to fix it cannot. Related: #303, #305.

Suggested fix

A process-wide semaphore rather than a per-invocation one — e.g. a module-level
asyncio.Semaphore created once from resolve_max_concurrency() and shared by all analyzer
nodes. That makes the documented behaviour of the variable true.

Optional but valuable: expose the LLM request timeout. create_openai_compatible_chat_model
hard-codes timeout: float | None = 120 with no environment override, which caps how much
back-off any external mitigation can absorb.

Environment

SkillSpector 2.5.1 (pinned) and main @ 2.9.5, Linux, Python 3.13, nv_build / OpenAI-compatible provider.