OpenSource-Hub

firecrawl

CLI ツール

firecrawl/firecrawl

大規模なウェブスクレイピング、検索、データ抽出のためのAPIおよびCLIツール。

概要

Firecrawlは、Webページの検索、スクレイピング、操作を行うための強力なAPIとCLIを提供します。任意のURLをクリーンなMarkdownまたは構造化JSONに変換できるため、LLMアプリケーションやAIエージェントに最適です。クロール、バッチスクレイピング、ページ操作機能をサポートしています。

README プレビュー

\n  \n  \n\n\n\n  \n    \n  \n  \n    \n  \n  \n    \n  \n  \n    \n  \n\n\n\n  \n    \n      \n    \n    \n      \n    \n    \n      \n    \n  \n\n\n---\n\n# **🔥 Firecrawl**\n\n**The API to search, scrape, and interact with the web at scale. 🔥** The web context API to find sources, extract content, and turn it into clean Markdown or structured data your agents can ship with. Open source and available as a [hosted service](https://firecrawl.dev/?ref=github).\n\n_Pst. Hey, you, join our stargazers :)_\n\n\n  \n\n\n---\n\n## Why Firecrawl?\n\n- **Industry-leading reliability**: Covers 96% of the web, including JS-heavy pages — no proxy headaches, just clean data ([see benchmarks](https://www.firecrawl.dev/blog/the-worlds-best-web-data-api-v25))\n- **Blazingly fast**: P95 latency of 3.4s across millions of pages, built for real-time agents and dynamic apps\n- **LLM-ready output**: Clean markdown, structured JSON, screenshots, and more — spend fewer tokens, build better AI apps\n- **We handle the hard stuff**: Rotating proxies, orchestration, rate limits, JS-blocked content, and more — zero configuration\n- **Agent ready**: Connect Firecrawl to any AI agent or MCP client with a single command\n- **Media parsing**: Parse and extract content from web-hosted PDFs, DOCX, and more\n- **Actions**: Click, scroll, write, wait, and press before extracting content\n- **Open source**: Developed transparently and collaboratively — [join our community](https://github.com/firecrawl/firecrawl)\n\n---\n\n## Feature Overview\n\n**Core Endpoints**\n\n| Feature | Description |\n|---------|-------------|\n| [**Search**](#search) | Search the web and get full page content from results |\n| [**Scrape**](#scrape) | Convert any URL to markdown, HTML, screenshots, or structured JSON |\n| [**Interact**](#interact) | Scrape a page, then interact with it using AI prompts or code |\n\n**More**\n\n| Feature | Description |\n|---------|-------------|\n| [**Agent**](#agent) | Automated data gatherin