Built by Metorial, the integration platform for agentic AI.
Use AI to extract structured data from one or more web pages. Provide a natural language prompt and/or a JSON schema to define the desired output structure. Supports wildcard URLs (e.g., `example.com/*`) to extract data across an entire domain. Can optionally follow links outside the specified domain.
Rapidly retrieve all URLs associated with a website to get a clear overview of its structure. Returns up to 100,000 URLs. Use this to discover all pages on a site before deciding which ones to scrape or crawl.
Start a crawl job to recursively scrape an entire website and extract content from all pages. The crawl runs asynchronously and returns a job ID for tracking progress. Use this to extract content from multiple pages across a website. Configure page limits, URL filters, and depth to control scope.
Check the status of an agent job and retrieve the gathered data when complete.
Launch an AI agent that autonomously searches, navigates, and gathers data from websites. Describe what data you want and the agent handles the rest. The agent runs asynchronously — use the returned agentId to check status and retrieve results.
Check the status of an extract job and retrieve the extracted structured data when complete.
Check the status of a batch scrape job and retrieve the scraped page data when available.
Scrape a single URL and extract its content as markdown, HTML, screenshot, or structured data. Handles JavaScript-rendered pages and dynamic content automatically. Use this to get clean, readable content from any web page. Supports browser automation actions like clicking buttons, filling forms, and scrolling before extraction.
Check the status of a crawl job and retrieve the scraped page data. Returns the current progress and any available results. Also supports cancelling a running crawl job.
Scrape multiple URLs simultaneously. Submit an array of URLs and receive results asynchronously. Supports the same format and configuration options as single page scraping.
Search the internet for pages matching a query and optionally extract full content from the results. Combines search with scraping to get both search metadata and full page content in one request. Supports advanced query operators like exact phrases, site filters, and file type filters.