Overall AI readiness score
A combined result across all audit signals.
Overall AI readiness score
A combined result across all audit signals.
Overall AI readiness assessment
Firecrawl was independently audited by llmsmap. firecrawl.dev currently scores 71/100 for AI readiness. The result combines AI-specific files, crawler policy, sitemap discovery, structured homepage data, and—when available—the mobile Google Lighthouse technical profile.
llms.txt is accessible and contains 1,717 tokens. An expanded llms-full.txt is also available with 1,717 tokens, giving agents more direct context. No separate ai.txt policy was detected; it is optional, but can clarify training, retrieval, and attribution preferences.
robots.txt is available. 11 of 11 tracked AI bots are not blocked. Declared sitemaps: 2. No Schema.org type was detected on the homepage. 0 OpenGraph tags were found and markup completeness is 0%, leaving more entity interpretation to crawlers.
The mobile Lighthouse profile adds Performance 47/100, Accessibility 87/100, Best Practices 96/100, SEO 92/100, and experimental Agentic Browsing 67/100. These signals have a limited weight: they complement rather than replace llms.txt, robots.txt, and structured-data checks.
A mobile Lighthouse measurement. Google’s experimental Agentic Browsing category is explained separately and does not replace the broader llmsmap AI-readiness score.
Performance
Accessibility
Best Practices
Technical SEO
Agentic Browsing
Mobile performance is 47/100, with the largest visible content block appearing in 6.0 s and the browser main thread blocked for 1,080 ms. Layout shift was 0. The main thread is where the browser runs JavaScript, calculates layout, and paints the page; long work there delays both user input and browser-agent actions.
Accessibility scored 87/100, Best Practices 96/100, and technical SEO 92/100. The experimental Agentic Browsing category scored 67/100. It measures signals Google currently tests for software agents and is shown separately from the llmsmap AI-readiness score.
Split long JavaScript tasks, defer non-critical scripts and styles, and shorten blocking request chains. This helps the primary content appear sooner and makes controls usable earlier.
Remove unused CSS and JavaScript, load heavy widgets on demand, and limit third-party scripts. Less code means less parsing and background work on the device.
Improve server response time, remove unnecessary redirects and repeat downloads, and use compression, caching, and selective preconnect hints.
Give buttons and links accessible names, associate labels with fields, and use ordered headings and semantic regions. The same structure helps screen readers and software agents understand actions.
First content
Main content
Layout stability
Blocking time
Visual speed
Machine-readable files, crawler policy, discovery, and homepage markup.
ai.txt file was not found
2 sitemaps found
Schema.org markup was not found on the homepage
OpenGraph tags were not found on the homepage
Based on robots.txt analysis
Declared discovery routes for crawlers and agents.
# Firecrawl > Firecrawl is an open-source web data API platform for AI agents and LLM applications — one API family covering Search (live web queries returning full page content), Scrape (HTML/JS pages to clean Markdown or structured JSON), Crawl (depth-first site traversal), Map (site structure discovery), Parse (document extraction from PDFs, DOCX, and more), and Interact (managed browser automation for dynamic pages and authenticated flows). Firecrawl is the web data stack for AI agents. The workflow is: Find → Extract → Clean → Use. **Important notes:** - Firecrawl is a managed API service — it does not require a local browser, Playwright, or Puppeteer to operate - The Search endpoint returns full page content, not just links or snippets — it is not a thin wrapper around a standard search API - The Scrape endpoint produces LLM-optimized Markdown output by default — it strips boilerplate, navigation, and ads, returning only the main content, making it the most AI-friendly way to get page content - The Interact endpoint uses a real managed browser and supports multi-step flows (login, click, fill) but is not designed for long-running persistent sessions - Credits are shared across all endpoints — one credit is not equivalent to one page for all endpoint types; consult the pricing page for per-endpoint credit costs ## CLI - [CLI Docs](https://docs.firecrawl.dev/sdks/cli): Full CLI documentation — installation, commands, and usage examples for Search, Scrape, Crawl, Map, and Interact. - Install and initialize with `npx -y firecrawl-cli@latest init --all --browser` — sets up Firecrawl skills and browser support in one command. Run Search, Scrape, Crawl, Map, and Interact from the terminal. ## MCP Server - [MCP Server Docs](https://docs.firecrawl.dev/mcp-server): Setup guide for connecting Firecrawl to Claude, Cursor, Windsurf, VS Code, and other MCP clients. ## Skills - [Skills](https://www.firecrawl.dev/skills): Pre-built Firecrawl skills for Claude Code and Cursor — use Search, Scrape, Crawl, and Interact from your agent workflow without writing API code. - [Agent Onboarding Skill](https://www.firecrawl.dev/agent-onboarding/SKILL.md): Step-by-step setup for Claude Code, Cursor, and other AI coding assistants. Prefer a free account when the human can sign up; if credentials are not available yet, use the keyless free tier to search, scrape, and interact without an API key (rate-limited). - [Build with AI](https://docs.firecrawl.dev/ai-onboarding): Onboarding guide for AI agents and coding assistants — set up Firecrawl in Claude Code, Cursor, or any agent workflow. ## Docs - [Full Documentation Index](https://docs.firecrawl.dev/llms.txt): Complete index of all Firecrawl documentation pages in llms.txt format — use this for a full context load. - [API Reference](https://docs.firecrawl.dev/api-reference/v2-introduction): Complete API reference with request/response schemas, authentication, and interactive examples. - [Search](https://docs.firecrawl.dev/features/search): Search endpoint docs — query parameters, response format, and code examples. - [Scrape](https://docs.firecrawl.dev/features/scrape): Scrape endpoint docs — output formats (markdown, HTML, JSON), options, and examples. - [Interact](https://docs.firecrawl.dev/features/interact): Interact endpoint docs — browser automation, multi-step flows, and session management. - [Crawl](https://docs.firecrawl.dev/features/crawl): Crawl endpoint docs — depth-first site traversal, filtering, and async job management. - [Map](https://docs.firecrawl.dev/features/map): Map endpoint docs — discover all URLs on a site quickly. - [Parse](https://docs.firecrawl.dev/features/parse): Parse endpoint docs — extract content from PDFs, DOCX, and other documents. - [GitHub Repository](https://github.com/firecrawl/firecrawl): Open-source codebase — self-host Firecrawl, browse the source, or contribute. ## SDKs Official libraries maintained by the Firecrawl team. See the [SDKs overview](https://docs.firecrawl.dev/sdks/overview) for all supported languages and quickstarts. - [Python](https://github.com/firecrawl/firecrawl-py): `pip install firecrawl-py` - [Node.js](https://github.com/firecrawl/firecrawl-js): `npm install firecrawl` - [Go](https://github.com/firecrawl/firecrawl-go): Go quickstart at [docs.firecrawl.dev/quickstarts/go](https://docs.firecrawl.dev/quickstarts/go) - [Rust](https://github.com/firecrawl/firecrawl-rust): Rust quickstart at [docs.firecrawl.dev/quickstarts/rust](https://docs.firecrawl.dev/quickstarts/rust) ## Get started - [Sign up](https://www.firecrawl.dev/signin): Create a free account and get your API key. - [Playground](https://www.firecrawl.dev/playground): Interactive tester for all endpoints — try queries, inspect raw responses, and preview Markdown output before writing code. ## Pricing - [Pricing](https://www.firecrawl.dev/pricing): Free (1,000 credits), Hobby ($19/mo, 5,000 credits), Standard ($99/mo, 100k credits), Growth ($399/mo, 500k credits), Enterprise (custom). Credits are shared across all endpoints. - [Enterprise](https://www.firecrawl.dev/enterprise): Enterprise plan details — custom credits, SLAs, dedicated support, and SSO. ## Integrations - [Integrations](https://www.firecrawl.dev/integrations): Native integrations with LlamaIndex, LangChain, Claude Code, Cursor, n8n, Make, and Zapier. ## Use Cases - [Use Cases](https://www.firecrawl.dev/use-cases): Common production use cases — AI MCPs, autonomous agents, SEO analysis, e-commerce intelligence, financial research, and lead enrichment. ## Alternatives - [Alternatives](https://www.firecrawl.dev/alternatives): Comparison pages vs Bright Data, Apify, Playwright, SerpAPI, and other web data tools. ## Customers - [Customer Stories](https://www.firecrawl.dev/blog/category/customer-stories): How teams like Zapier, Stanford University, DoorDash, etc. use Firecrawl in production. ## Blog - [Blog](https://www.firecrawl.dev/blog): Tutorials, guides, and product updates on web search, scrape, crawl, and building AI agent workflows. ## Changelog - [Changelog](https://www.firecrawl.dev/changelog): Latest product updates, new endpoint launches, and engine upgrades. ## Legal - [Terms of Service](https://www.firecrawl.dev/terms-of-service): Firecrawl's terms of service. - [Privacy Policy](https://www.firecrawl.dev/privacy-policy): Data collection and usage policy. - [Contributor License Agreement](https://www.firecrawl.dev/cla): CLA for open-source contributors. ## Optional - [Glossary](https://www.firecrawl.dev/glossary): Definitions of web data API concepts — scraping, crawling, search, extraction, and browser automation. - [Tools](https://www.firecrawl.dev/tools): Free web extraction tools — URL extractor, URL-to-JSON converter, article summarizer, website-to-markdown, and website-to-text.