TL;DR
A plain fetch gets your agent blank shells and captcha walls on any modern site. The Firecrawl MCP gives Claude real web scraping: add the server, and pages arrive as clean markdown - JavaScript-rendered pages included. Scrape one URL, crawl a whole domain, or extract exact fields as JSON. This is how you turn 'the internet' into something Claude can actually read.
Why plain fetch fails
Most of the modern web renders with JavaScript or sits behind anti-bot friction. When an agent grabs a URL with a basic request, it gets an empty shell or a block page - not the content. Firecrawl loads the real rendered page, gets past the friction, and hands back clean markdown the model can use.
Add the Firecrawl MCP
Firecrawl connects to Claude as an MCP server. Add it to your MCP config with your Firecrawl key, restart, and Claude gains scraping tools it can call directly. Check the current Firecrawl docs for the exact config block, since the fields evolve.
The three things it does
- Scrape - point it at one URL and get that page back as clean markdown, JS-rendered content included.
- Crawl - point it at a domain and let it walk the pages, so you can pull a whole site or section at once.
- Extract - describe the exact fields you want and get structured JSON back, not a wall of text.
Extract exact fields as JSON
textUse Firecrawl to scrape [URL] and extract these fields as JSON:
title, price, and availability. Return only the JSON - no prose.Because the output is structured, you can feed it straight into the next step - a spreadsheet, a database, another prompt - without hand-cleaning. That is the difference between 'Claude can read a page' and 'Claude can build a pipeline'.
What to build with it
- 1A research agent that reads real pages instead of guessing from memory.
- 2A price or listing watcher that pulls structured fields on a schedule.
- 3A content pipeline that turns source pages into briefs, summaries, or datasets.
Common questions
Do I need to code to use Firecrawl with Claude?
No. Once the MCP server is added, you ask Claude in plain language to scrape or crawl a page and it calls the tool for you. The setup is a one-time config; using it is just conversation.
Can it get past JavaScript-heavy pages?
Yes - that is the point. Firecrawl loads the real rendered page rather than a raw request, so JS-rendered content comes back as clean markdown where a plain fetch would return an empty shell.
What is the difference between scrape, crawl, and extract?
Scrape is one page to markdown. Crawl walks a whole domain or section. Extract pulls the exact fields you name back as structured JSON. Reach for scrape for a single source, crawl for coverage, extract when you need data, not prose.
Is Firecrawl free?
It has a free tier and paid plans beyond it, and it needs an API key. Keep that key out of any code you commit - it is a spend surface like any paid credential. Check Firecrawl's current pricing for the limits.
Want the Firecrawl scraping setup, step by step?
Get the other 4 in the skills stack, plus 5,000+ builders - $9/mo, cancel anytime.
Join the Club