AgentRouter
Sign in
FirecrawlFirecrawl

Crawl website pages

/crawl

Crawl a bounded set of pages on one website and return their content. The requested page limit reserves the maximum cost.

Finished calls, last 7 days
112
Success rate
99.1%
Median execution time
16.2s

Cancellations excluded. Execution times cover successful calls.

Use it with your agent

Set up https://www.agentrouter.to/SKILL.md, then use Firecrawl’s /crawl endpoint through AgentRouter.

Inputs

urlRequired
Public HTTP(S) URL. X/Twitter and LinkedIn URLs use separate provider tariffs and are excluded.
limitOptional
Maximum pages to crawl, up to 10000. The full maximum cost is reserved before submitting.
promptOptional
Describe which pages to crawl; the provider derives path filters.
maxDiscoveryDepthOptional
Maximum link depth from the starting URL; 0 reads only that page.
includePathsOptional
Path patterns to include.
excludePathsOptional
Path patterns to exclude.
sitemapOptional
Skip the sitemap, combine it with discovery, or use only the sitemap.
crawlEntireDomainOptional
Crawl the entire domain rather than only the starting path.
allowSubdomainsOptional
Allow crawling subdomains of the starting domain.
delayOptional
Seconds to wait between page requests.
scrapeOptionsOptional
object

Example input

{
  "url": "https://example.com",
  "limit": 1,
  "maxDiscoveryDepth": 0
}

Output

All completed page documents, fetched through authenticated result pagination. Complete output is limited to 12 MiB; larger results fail without returning partial data.