Self-Host Firecrawl: Docker Setup & What's Missing

Firecrawl is open source (AGPL-3.0), so you can run the API on your own machine or servers instead of paying for credits. This guide walks through the setup from Firecrawl's own self-hosting docs, shows what you give up, and says when the cloud plan is the sensible choice. Everything below comes from Firecrawl's docs and repository, fetched on 2026-11-05.

Self-host or use the cloud?

Firecrawl's own recommendation is blunt: self-host when source access or infrastructure control is worth the operational work, and otherwise start with Firecrawl Cloud. Self-hosting means you own upgrades, secrets, storage, monitoring, recovery and security.

What you need

Per the docs: Git, Docker Engine or Docker Desktop, Docker Compose v2 (docker compose), curl, and port 3002 free. Firecrawl does not publish a verified minimum host size for the stack, so size it by testing.

Set it up

The docs pin their quickstart to release v2.11.162. A different release can have a different Compose file, so check out an exact tag rather than main, and read that tag's docker-compose.yaml before you change anything.

git clone https://github.com/firecrawl/firecrawl.git
cd firecrawl
git checkout v2.11.162

Create a minimal .env in the repository root. This turns API authentication off, which is only acceptable on a trusted network:

cat > .env <<'EOF'
USE_DB_AUTHENTICATION=false
POSTGRES_USER=postgres
POSTGRES_PASSWORD=replace-with-at-least-32-random-characters
POSTGRES_DB=postgres
EOF

Keep POSTGRES_DB=postgres for this release, because the bundled pg_cron configuration targets that database. Do not commit the file, and don't treat apps/api/.env.example as a Compose file: it is for API development.

Build and start the stack:

docker compose up --build -d
docker compose ps --all

Warnings about unset optional variables are expected. The stack runs the API and workers, Playwright, Redis, RabbitMQ and PostgreSQL (plus FoundationDB services for an optional queue backend you can ignore). Only the API is published to the host, on port 3002.

Test that it works

The readiness endpoint only proves the API answers HTTP; it does not check Redis, PostgreSQL, RabbitMQ, Playwright or outbound access:

curl --fail --silent --show-error --max-time 5 http://localhost:3002/v0/health/readiness

Run a real scrape before you trust the deployment:

curl --fail-with-body --silent --show-error --max-time 75 \
  -X POST http://localhost:3002/v2/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://example.com", "formats": ["markdown"], "timeout": 60000}'

With authentication disabled, no API key or Authorization header is needed. If the scrape times out, check that the API and Playwright containers are running and can reach the internet (docker compose logs --tail=200 api playwright-service).

You can also point the Firecrawl MCP server at your instance by setting FIRECRAWL_API_URL for the local server.

What the self-hosted version lacks

Firecrawl documents these gaps for the default stack:

• No Fire-engine. Its advanced anti-bot behaviour is a separate service you would have to run and configure. If your targets block ordinary fetchers, read how to scrape without getting blocked first.
• No screenshots or page actions. The docs say neither the fetch nor the Playwright engine supports them without Fire-engine.
• No AI by default. LLM-backed extraction needs an OpenAI-compatible endpoint or Ollama, and you should test that path on its own.
• No Agent, Browser, interact or feedback features in the default stack. Use Firecrawl Cloud for those.

Before you expose it

The quickstart is explicitly "not a production architecture":

• Authentication. Setting one variable is not enough. The default API is unauthenticated, so add a complete authentication design, TLS and network policy before it leaves a trusted network.
• Persistence. The root Compose file defines no persistent volumes for PostgreSQL, Redis or RabbitMQ. Add storage, backups and a tested restore.
• Private ports. Keep PostgreSQL, Redis, RabbitMQ and worker ports unpublished.
• Queue admin UI. Off by default. Enable it only with a strong BULL_AUTH_KEY and restricted network access.
• Upgrades. Review the target release's Compose file before bumping the tag.

The licence

The main project is AGPL-3.0; the SDKs and some UI components are MIT. Using it internally is one thing, but if you embed a modified Firecrawl in a product other people use over a network, read the AGPL's terms with a lawyer first. Also note that Firecrawl respects robots.txt by default, and that you are responsible for each target site's terms.

When the cloud plan is cheaper

Self-hosting has no per-page fee, but it isn't free. Count the server (and any proxies or model provider), the time to patch and monitor it, and the work to replace what Cloud includes. Cloud is usually the better choice when:

• you scrape a modest volume, where a monthly plan (see the plan table for current prices) is likely cheaper than the time spent running the stack;
• your targets need anti-bot handling, screenshots or actions;
• you want Agent, Browser or Interact;
• nobody on the team wants to be on call for a scraping service.

Self-hosting makes sense when you scrape public, easy pages at high volume, need the data to stay on your infrastructure, or want to modify the source. If you are still comparing options, see how it stacks up against Crawl4AI, which is also open source, or browse the best AI web scrapers.

Try Firecrawl Cloud ↗