--- title: "Web Scraping MCP Servers Compared: Firecrawl, Apify & More" description: "Install configs, tools and costs for the Firecrawl, Apify, Bright Data, Browserbase and Crawl4AI MCP servers, and which one to give your AI agent." canonical_url: "https://bestaiscraper.com/blog/web-scraping-mcp-servers" --- > Index of every page on this site: [llms.txt](https://bestaiscraper.com/llms.txt) An **MCP server** lets an AI client (Claude Code, Cursor, VS Code and others) call a scraping tool directly instead of you pasting page text into a chat. Five scraping vendors ship one that matters. They differ in what the agent can do, how you authenticate, and what a run costs, so this guide compares them from each vendor's own docs, fetched on 2026-11-04. ## Quick comparison
Server What the agent gets Hosted or local Auth Best for
Firecrawl Scrape, search, map, crawl, parse, browser interaction Both None (limited), OAuth, or API key General page-to-markdown and structured extraction
Apify Search and run any Actor from the Apify Store Both OAuth or API token Site-specific scrapers (social, maps, marketplaces)
Bright Data Search, unblocked scraping, 1,200+ pre-built scrapers, remote browser Both API token Protected sites that block ordinary clients
Browserbase A cloud browser driven by natural language (Stagehand) Both Browserbase API key Interactive pages: logins, forms, clicks
Crawl4AI Markdown, HTML, screenshot, PDF, JS execution, crawl Self-hosted server Your own server (secure by default since 0.9.0) Open-source, run on your own infrastructure
## Firecrawl MCP Firecrawl is the only one here that works with no account. Its docs list three ways in: keyless, browser sign-in, or an API key. **Keyless** (Streamable HTTP, no credential): ```bash claude mcp add --transport http firecrawl https://mcp.firecrawl.dev/v2/mcp ``` Keyless is rate limited and exposes `firecrawl_search`, `firecrawl_scrape` and `firecrawl_parse`. For the full tool set, add an API key as a bearer token on the same endpoint: ```json { "mcpServers": { "firecrawl": { "type": "http", "url": "https://mcp.firecrawl.dev/v2/mcp", "headers": { "Authorization": "Bearer " } } } } ``` With an authenticated connection the docs also list `firecrawl_map` (discover URLs), `firecrawl_crawl`, `firecrawl_agent` (multi-source research), `firecrawl_interact` (clicks and form fills) and `firecrawl_monitor_*` (change tracking), subject to plan and team policy. To run it yourself, the local server needs Node.js 22 or newer and `FIRECRAWL_API_KEY`; set `FIRECRAWL_API_URL` to point it at a self-hosted Firecrawl API: ```bash env FIRECRAWL_API_KEY=fc-YOUR-API-KEY npx -y firecrawl-mcp ``` Cost: calls use your Firecrawl credits. See the [Firecrawl pricing table](https://bestaiscraper.com/tools/firecrawl) for plans. [Try Firecrawl ↗](https://bestaiscraper.com/go/firecrawl) ## Apify MCP Apify's server is a gateway to the Apify Store: the agent searches for an Actor, reads its input schema and runs it. That makes it the broadest option for site-specific data such as Instagram or Google search results. Hosted, with OAuth sign-in on first connection: ```json { "mcpServers": { "apify": { "url": "https://mcp.apify.com" } } } ``` Local stdio, with an API token: ```json { "mcpServers": { "actors-mcp-server": { "command": "npx", "args": ["-y", "@apify/actors-mcp-server"], "env": { "APIFY_TOKEN": "YOUR_APIFY_TOKEN" } } } } ``` By default it loads the `actors` and `docs` tool categories plus the `apify/rag-web-browser` and `apify/web-fetch` Actors. You can narrow it with a URL parameter, for example `https://mcp.apify.com?tools=apify/instagram-scraper,apify/google-search-scraper`. The core tools are `search-actors`, `fetch-actor-details` and `call-actor`; `call-actor` waits up to 30 seconds by default and returns the run status and storage IDs, not the data itself, so the agent then reads the dataset. Full-permission Actors are deliberately excluded. Cost: you pay for the Actor runs you trigger, and each Actor sets its own price. Cap spend with `maxTotalChargeUsd` in `callOptions`. Details in [our Apify review](https://bestaiscraper.com/tools/apify). [Try Apify ↗](https://bestaiscraper.com/go/apify) ## Bright Data MCP Bright Data's README says its server exposes 32 tools: search (Bing and Google), page scraping as Markdown with bot detection, CAPTCHA solving and proxy rotation handled on each request, structured data from 1,200+ pre-built scrapers, and a remote browser. ```bash claude mcp add --transport http brightdata "https://mcp.brightdata.com/mcp?token=YOUR_API_TOKEN" ``` To run it locally, use `npx @brightdata/mcp` with `API_TOKEN` set. Trim the tool list with `groups=` or `tools=` URL parameters so the model isn't offered 32 tools at once, for example `&tools=search_engine,scrape_as_markdown`. Cost: the README states 5,000 free requests per month, then pay as you go at $1.50 per 1,000 search/scrape/extract results and $8 per GB for browser navigation. Check [Bright Data's MCP pricing page](https://brightdata.com/pricing/mcp-server) before relying on those numbers. See also [our Bright Data review](https://bestaiscraper.com/tools/bright-data). [Try Bright Data ↗](https://bestaiscraper.com/go/bright-data) ## Browserbase MCP Browserbase gives the agent a real cloud browser, built on Stagehand. Instead of fetching a page, the agent calls tools like `navigate`, `act` ("click the login button") and `extract`, and can start and reuse sessions. Use it when the data sits behind interaction. ```bash claude mcp add --transport http browserbase "https://mcp.browserbase.com/mcp?browserbaseApiKey=YOUR_BROWSERBASE_API_KEY" ``` The hosted endpoint is recommended in the docs; a local STDIO mode also exists. The API key goes in the URL, so treat that config like a secret and keep it out of version control. Cost is Browserbase's browser usage; see the [Browserbase page](https://bestaiscraper.com/tools/browserbase). [Try Browserbase ↗](https://bestaiscraper.com/go/browserbase) ## Crawl4AI MCP Crawl4AI is open source, so its MCP server comes from the Docker server you run yourself. It exposes SSE at `http://localhost:11235/mcp/sse` and WebSocket at `ws://localhost:11235/mcp/ws`: ```bash claude mcp add --transport sse c4ai-sse http://localhost:11235/mcp/sse ``` The documented tools are `md`, `html`, `screenshot`, `pdf`, `execute_js`, `crawl` and `ask`. The docs say versions 0.9.0 and later are secure by default, so read the self-hosting guide before exposing the port beyond localhost. There is no per-call fee, but you run and maintain the infrastructure yourself. Crawl4AI's site also advertises a hosted Crawl4AI Cloud with a free monthly credit allowance; we haven't reviewed it. More in [our Crawl4AI page](https://bestaiscraper.com/tools/crawl4ai). ## Which one should your agent use? - **Just need page text or search, and want zero setup:** Firecrawl keyless, then add a key when you hit the limit. - **Need data from a specific site that already has a scraper:** Apify. - **Pages block ordinary fetchers:** Bright Data. - **Need clicks, logins or forms:** Browserbase. - **No vendor, no per-call cost:** Crawl4AI, if you'll host it. Two practical tips regardless of vendor. Enable only the tools you need, because every tool description consumes context in each request. And treat scraped pages as untrusted input: a page can contain text written to manipulate your agent, so don't give a scraping-enabled agent tools that can send data out or spend money without confirmation. Want typed JSON rather than raw page text? Compare the options in our [best AI web scrapers](https://bestaiscraper.com/best/ai-web-scrapers) ranking, or try [the extraction playground](https://bestaiscraper.com/extract). ## Sitemap See the full [sitemap](https://bestaiscraper.com/sitemap.md) for all pages.