feat(escalating): add live web tool detection and routing for requests#270
feat(escalating): add live web tool detection and routing for requests#270Zochory wants to merge 1 commit into
Conversation
fix(document_tools): improve content type handling for URL suffix extraction test(escalating): add test for URL fetch request forcing RLM escalation test(tools): add case insensitive content type test for URL suffix
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
Warning Review limit reached
More reviews will be available in 25 minutes and 36 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Repository UI (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces routing logic to escalate user requests requiring live web tools to the RLM path, utilizing regex matching for URLs and specific action/target keywords. It also registers fetch_document_text as a tool, ensures case-insensitive retrieval of the Content-Type header when determining file suffixes, and adds corresponding unit tests. Feedback focuses on improving the keyword matching regex to be order-independent and more efficient by splitting it into separate action and target patterns, updating the routing check accordingly, and adding tests for keyword-based routing.
| _LIVE_WEB_REQUEST_RE = re.compile( | ||
| r"\b(" | ||
| r"browse|download|fetch|open|read|retrieve|scrape|summari[sz]e" | ||
| r")\b.*\b(" | ||
| r"internet|online|page|pdf|site|url|web|website" | ||
| r")\b", | ||
| flags=re.IGNORECASE, | ||
| ) |
There was a problem hiding this comment.
The current regex _LIVE_WEB_REQUEST_RE enforces a strict order where the action verb (e.g., browse) must precede the target noun (e.g., website) due to the .* pattern. This means queries like "Is there a website I can browse?" or "pdf to summarize" will fail to match, even though they clearly require live web tools.
Additionally, using .* can introduce backtracking overhead on long inputs.
We can resolve this by splitting the pattern into two separate, simpler regexes for actions and targets, making the check completely order-independent and more efficient.
| _LIVE_WEB_REQUEST_RE = re.compile( | |
| r"\b(" | |
| r"browse|download|fetch|open|read|retrieve|scrape|summari[sz]e" | |
| r")\b.*\b(" | |
| r"internet|online|page|pdf|site|url|web|website" | |
| r")\b", | |
| flags=re.IGNORECASE, | |
| ) | |
| _LIVE_WEB_ACTION_RE = re.compile( | |
| r"\b(browse|download|fetch|open|read|retrieve|scrape|summari[sz]e)\b", | |
| flags=re.IGNORECASE, | |
| ) | |
| _LIVE_WEB_TARGET_RE = re.compile( | |
| r"\b(internet|online|page|pdf|site|url|web|website)\b", | |
| flags=re.IGNORECASE, | |
| ) |
| def _requires_live_web_tools(user_request: str) -> bool: | ||
| """Return whether a turn should skip lightweight chat and use web-capable tools.""" | ||
| if _LIVE_WEB_URL_RE.search(user_request): | ||
| return True | ||
| return bool(_LIVE_WEB_REQUEST_RE.search(user_request)) |
There was a problem hiding this comment.
Update _requires_live_web_tools to use the new order-independent _LIVE_WEB_ACTION_RE and _LIVE_WEB_TARGET_RE regexes.
| def _requires_live_web_tools(user_request: str) -> bool: | |
| """Return whether a turn should skip lightweight chat and use web-capable tools.""" | |
| if _LIVE_WEB_URL_RE.search(user_request): | |
| return True | |
| return bool(_LIVE_WEB_REQUEST_RE.search(user_request)) | |
| def _requires_live_web_tools(user_request: str) -> bool: | |
| """Return whether a turn should skip lightweight chat and use web-capable tools.""" | |
| if _LIVE_WEB_URL_RE.search(user_request): | |
| return True | |
| return bool(_LIVE_WEB_ACTION_RE.search(user_request) and _LIVE_WEB_TARGET_RE.search(user_request)) |
| def test_url_fetch_request_forces_rlm_before_lightweight_response(self) -> None: | ||
| module = _make_module() | ||
| _stub_respond(module, reasoning="I cannot browse the live web.", response="no web access") | ||
| rlm_pred = _FakePrediction(answer="fetched document") | ||
| module._rlm = MagicMock(return_value=rlm_pred) | ||
| _stub_summarize(module) | ||
|
|
||
| result = module(user_request="fetch https://arxiv.org/pdf/2512.24601 please", execution_mode="auto") | ||
|
|
||
| module.respond.assert_not_called() | ||
| module._rlm.assert_called_once() | ||
| assert getattr(result, "answer", None) == "fetched document" |
There was a problem hiding this comment.
The test only verifies routing when a URL is present in the request (which matches _LIVE_WEB_URL_RE). We should also add a test case to verify that keyword-based requests (e.g., "please browse the internet for me" or "pdf to summarize") successfully trigger the RLM path without a URL.
| def test_url_fetch_request_forces_rlm_before_lightweight_response(self) -> None: | |
| module = _make_module() | |
| _stub_respond(module, reasoning="I cannot browse the live web.", response="no web access") | |
| rlm_pred = _FakePrediction(answer="fetched document") | |
| module._rlm = MagicMock(return_value=rlm_pred) | |
| _stub_summarize(module) | |
| result = module(user_request="fetch https://arxiv.org/pdf/2512.24601 please", execution_mode="auto") | |
| module.respond.assert_not_called() | |
| module._rlm.assert_called_once() | |
| assert getattr(result, "answer", None) == "fetched document" | |
| def test_url_fetch_request_forces_rlm_before_lightweight_response(self) -> None: | |
| module = _make_module() | |
| _stub_respond(module, reasoning="I cannot browse the live web.", response="no web access") | |
| rlm_pred = _FakePrediction(answer="fetched document") | |
| module._rlm = MagicMock(return_value=rlm_pred) | |
| _stub_summarize(module) | |
| # Test URL-based routing | |
| result = module(user_request="fetch https://arxiv.org/pdf/2512.24601 please", execution_mode="auto") | |
| module.respond.assert_not_called() | |
| module._rlm.assert_called_once() | |
| assert getattr(result, "answer", None) == "fetched document" | |
| # Test keyword-based routing (order-independent) | |
| module.respond.reset_mock() | |
| module._rlm.reset_mock() | |
| result_kw = module(user_request="please browse the internet for me", execution_mode="auto") | |
| module.respond.assert_not_called() | |
| module._rlm.assert_called_once() | |
| assert getattr(result_kw, "answer", None) == "fetched document" |
1 participant
Summary
_requires_live_web_tools()inescalating.pyto detect live web requests via URL regex and action-keyword patterns, routing them directly to the RLM path before lightweightrespondis attemptedContent-Typeheader lookup indocument_tools.pyso PDF detection works regardless of server header casingfetch_document_textas a@tool_fnso it appears in the ReAct tool registry and is discoverable by agentsTest plan
test_url_fetch_request_forces_rlm_before_lightweight_response— verifies URL in request bypassesrespondand goes straight to RLMtest_suffix_from_url_uses_case_insensitive_content_type_for_pdf— validates lowercasecontent-typeheader is recognizedtest_phase3_tools_are_registered— confirmsfetch_document_textis now in the registered tool set🤖 Generated with Claude Code