Skip to content

feat: add Firecrawl web search provider#13262

Open
flamerged wants to merge 1 commit into
danny-avila:devfrom
flamerged:feat/firecrawl-web-search
Open

feat: add Firecrawl web search provider#13262
flamerged wants to merge 1 commit into
danny-avila:devfrom
flamerged:feat/firecrawl-web-search

Conversation

@flamerged
Copy link
Copy Markdown

@flamerged flamerged commented May 22, 2026

Target: danny-avila/LibreChat:dev
Branch: feat/firecrawl-web-search
Commit: 251a9be176eb276f6d3d6401541011f0038f35fe

Dependency PR:

  • Depends on agents PR: danny-avila/agents#192
  • Dependency branch: feature/firecrawl-search
  • Dependency target: main
  • Dependency commit: f6ae68013d93310729444a2b2d99d1040f896366
  • Merge/publish order: merge/publish the agents PR first, then bump @librechat/agents here before final merge if required by maintainers.

Why This Matters

Firecrawl gives LibreChat an AI-oriented web search option that can combine search discovery and content extraction through one provider. That is useful for teams that do not want to run a separate metasearch provider plus a separate scraper just to produce model-ready web context.

This does not replace SearXNG's privacy/metasearch niche. SearXNG remains valuable when users want a self-hosted metasearch engine aggregating many upstream search services. Firecrawl is valuable when the deployment wants one API focused on finding sources and returning clean, scrape-ready page content for agents.

References:

Summary

Adds Firecrawl as a selectable web search provider in LibreChat config/schema/auth/UI, including:

  • Firecrawl URL/version/key fields through auth submission
  • schema and runtime auth handling
  • English locale keys and examples
  • duplicate Firecrawl provider/scraper field handling
  • focused backend/frontend/schema tests

Tests

  • post-rebase branch status: clean, feat/firecrawl-web-search...origin/dev [ahead 1]
  • packages/api src/web/web.spec.ts: pass, 54 tests
  • client ApiKeyDialog.test.tsx: pass, 9 tests
  • packages/data-provider src/config.spec.ts: pass, 119 tests
  • packages/data-schemas src/app/web.spec.ts: pass, 15 tests
  • JSON/YAML parse checks: pass
  • git diff --check: pass
  • git diff --check HEAD^ HEAD: pass

Locale/docs Notes

LibreChat policy says feature PRs update only client/src/locales/en/translation.json; non-English app locale files are automated externally and fall back to English. .env.example and librechat.example.yaml document the Firecrawl config; external feature docs belong in LibreChat-AI/librechat.ai if maintainers want a docs PR.

Residual Risks

Runtime end-to-end Firecrawl search should be verified after the agents package publish/bump and with FIRECRAWL_API_KEY configured.

@flamerged flamerged marked this pull request as ready for review May 22, 2026 21:19
@danny-avila
Copy link
Copy Markdown
Owner

firecrawl is already supported

@flamerged
Copy link
Copy Markdown
Author

flamerged commented May 23, 2026

firecrawl is already supported

You are right that Firecrawl is already supported today as a scraper. I think this PR is a different integration point: it adds Firecrawl as a search provider option.

I checked the current dev branch, and LibreChat has the same provider split:

  • SearchProviders includes SERPER, SEARXNG, and TAVILY
  • ScraperProviders includes FIRECRAWL, SERPER, and TAVILY
  • the API key dialog renders Firecrawl under the Scraper dropdown, not under Search Provider

So today a deployment can use scraperProvider: "firecrawl", but cannot select searchProvider: "firecrawl". This PR is intended to wire Firecrawl's /search endpoint through LibreChat config/schema/auth/UI, paired with danny-avila/agents#192 for the runtime search-provider implementation.

If the preferred direction is not to expose Firecrawl as a search provider, that is totally fair; I just wanted to clarify that the existing Firecrawl support is scraper support, not search-provider support.

@danny-avila

@danny-avila danny-avila reopened this May 23, 2026
@danny-avila
Copy link
Copy Markdown
Owner

My mistake, I could've sworn we had firecrawl as a search provider since the firecrawl team itself has contributed code here. Will review as soon as I can

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants