Repository navigation
Review follow-ups: credential leak, phantom tool reference, Node floor, locator edge cases - #5
Merged
Merged
Conversation
Follow-ups from a scan of the tool surface and session lifecycle. filterEnv stripped only four of the six env names config.ts reads credentials from, so a user who configured the server via TESTINGBOT_USERNAME / TESTINGBOT_ACCESS_KEY handed those straight to the appium-mcp child — contradicting the contract that the child receives credentials only baked into the injected hub URL. The list now lives in an exported CREDENTIAL_ENV_NAMES, with a test that fails if getConfig() grows a fallback the filter doesn't cover. tb_listSessions' agent-facing description pointed at tb_appiumEndpoint, which has never existed, inviting the agent to call a missing tool. It now points at appium_session_management (action 'list'), the tool that actually manages mobile sessions. A new test walks every registered description and fails on any tb_* reference that isn't registered. preflight() enforced Node 18 while package.json declares >=20. tb_navigate's waitUntil is implemented rather than ignored. It was passed by the integration tests and silently dropped, since the schema only accepted sessionId and url. navigateTo returns per the session's pageLoadStrategy, which on a page with slow third-party resources can leave the agent looking at a half-built DOM, so readyState is now confirmed directly: 'load' (complete), 'domcontentloaded' (interactive or better), or 'none' to skip. A timeout warns rather than failing the call — the navigation succeeded and the page is usually usable, so erroring would leave the agent with nothing to act on. SessionManager.closeAll() latched `shuttingDown` on permanently, so every later register() threw and the manager was silently dead. The flag is now held only for the duration of the sweep, and register() restarts the reaper that closeAll() stopped — otherwise a revived manager's sessions would idle until TestingBot's own reaper billed them out. Tool arguments are logged wholesale by the CallTool handler, so args.text (tb_type's payload, i.e. whatever gets typed into a login form), args.capabilities and args.remoteServerUrl are now redacted, along with the generic *.secret / *.token / *.accessKey names. Locator and key edge cases: a leading digit in an id needs CSS's hex escape rather than a backslash; by:"class" with "btn primary" means the compound selector .btn.primary, not one class of that name; and "+" is both the chord separator and a real key, so it was unpressable.
jochen-testingbot
changed the base branch from
fix/w3c-capabilities-and-integration-coverage
to
main
September 21, 2026 07:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-ups from the scan of
mainrequested after #4. Each item is independent.1. Credential leak into the appium-mcp child
filterEnvstrippedTESTINGBOT_KEY,TESTINGBOT_SECRET,TB_KEY,TB_SECRET— butconfig.tsalso acceptsTESTINGBOT_USERNAMEandTESTINGBOT_ACCESS_KEYas fallbacks. Anyone configured via those names handed their credentials straight to the child process, contradicting the documented contract that the child gets credentials only baked into the injected hub URL.The list is now an exported
CREDENTIAL_ENV_NAMES, with two tests: one asserting nothing leaks into the spawned env (while unrelated vars still pass through), and one that readsconfig.tsand fails ifgetConfig()grows a fallback the filter doesn't cover.2.
tb_appiumEndpointnever existedIt was named in
tb_listSessions' agent-facing description, so the agent was being told to call a tool that isn't registered.What it should use:
appium_session_managementwithaction: 'list'— that's the proxied tool that actually owns mobile sessions. The description now says so explicitly, and states that mobile sessions are not listed bytb_listSessions.Added
tests/tools/tool-references.test.ts, which walks every registered description and fails on anytb_*reference that isn't a registered tool. This class of bug can't recur silently.3. Stale comment in
register.tsClaimed a fallback tool is registered when the appium spawn fails; the catch block only logs. Comment now matches the code. (CLAUDE.md's version of this text was already accurate.)
4. Node floor
preflight()enforced 18;package.jsondeclares>=20. Now 20, with a test at 18.20.0 to keep the runtime gate andengines.nodefrom drifting apart.5.
waitUntil— implemented, not droppedIt was being passed by the integration tests and silently ignored, since
tb_navigate's schema only acceptedsessionIdandurl.Implemented, because the underlying problem is real:
navigateToreturns per the session'spageLoadStrategy, which on a page with slow third-party resources can leave the agent looking at a half-built DOM.tb_navigatenow confirmsdocument.readyStateitself:load(default) —completedomcontentloaded—interactiveor better, returns earlier on ad-heavy pagesnone— return as soon as the driver's call resolvesA
timeoutMs(default 30s) bounds the poll. Timeout warns rather than failing the call — the navigation succeeded and the page is usually usable, so erroring would leave the agent with nothing to act on. A probe that throws mid-navigation (destroyed execution context) is retried until the deadline rather than propagating. Five unit tests plus an integration case.7.
SessionManagerwas a one-shotcloseAll()latchedshuttingDownon permanently, so every laterregister()threw "SessionManager is shutting down" — a library host that tears the tool family down and brings it back up in the same process got a silently dead manager.The flag is now held only for the duration of the sweep (registrations during teardown are still rejected — tested), and
register()restarts the reapercloseAll()stopped. Without that restart a revived manager's sessions would idle until TestingBot's own reaper billed them out.Two existing tests asserted the old latching behaviour and have been rewritten to encode the new contract.
8. Secrets in logs
The CallTool handler logs
{ tool, args }wholesale.tb_type'stext— whatever gets typed into a login form — was not redacted. Addedargs.text,args.capabilities(appium caps carry app credentials),args.remoteServerUrl(embedskey:secret), plus generic*.secret/*.token/*.accessKey.Verified live:
{"args":{"sessionId":"x","value":"#pw","text":"[redacted]"}}.tests/lib/logger.test.tsalso asserts the non-secret fields stay readable — over-redacting makes the logs useless.Only reachable under
TESTINGBOT_DEBUG=trueor dev mode, since the default level iserror.9. Locator and key edge cases
#2fa-code→#\32 fa-code).by: "class"with"btn primary"means the compound selector.btn.primary. It was producing one escaped class literally namedbtn primary."+"is both the chord separator and a real key, so it was unpressable."+"and"Control++"now work.Verification
npm run buildpasses: lint, format, 156 unit tests (up from 120), tsc. Integration suites still skip cleanly without the gate flags.