Extract structured data from web pages using natural language instructions via Firecrawl.
What it does
This endpoint, hosted on Heurist Mesh, wraps a FirecrawlSearchDigestAgent that extracts structured data from one or more web pages based on natural language prompts. You supply one or more URLs (including wildcard patterns like example.com/*) along with an extraction_prompt describing what data you want, and the agent returns structured results.
The endpoint is listed under the x402 payment protocol at mesh.heurist.xyz, though during probing the endpoint did not return a 402 challenge, and no OpenAPI schema, pricing information, or detailed documentation was found. The Heurist Mesh root returns a simple JSON health-check ({"status":"ok"}), and all documentation, pricing, and API reference paths return 404. The canonical documentation site appears to be mesh.heurist.ai rather than mesh.heurist.xyz.
Because no schema, examples, or pricing details are available from the crawled material, the request/response formats below are inferred from the endpoint path and the existing description. Users should expect to POST a JSON body containing at minimum a list of URLs and an extraction_prompt string. The structured response format is unknown. Prospective consumers should consult the Heurist Mesh console or agent library at mesh.heurist.ai for authoritative documentation.
Capabilities
Use cases
- —Extracting product details (price, name, availability) from e-commerce pages
- —Pulling contact information or team bios from company websites
- —Gathering structured datasets from documentation or wiki pages across an entire domain
- —Summarizing key facts from news articles or blog posts
- —Collecting job listing details from career pages
Fit
Best for
- —Agents that need structured data from known URLs rather than search results
- —Batch extraction across multiple pages of a single domain using wildcards
- —Natural-language-driven scraping without writing custom parsers
Not for
- —General web search or discovery of unknown URLs
- —Real-time monitoring or continuous crawling of frequently changing pages
- —Scraping sites that require authentication or JavaScript-heavy rendering (unconfirmed capability)
Quick start
curl -X POST https://mesh.heurist.xyz/x402/agents/FirecrawlSearchDigestAgent/firecrawl_extract_web_data \
-H "Content-Type: application/json" \
-d '{
"urls": ["https://example.com/about"],
"extraction_prompt": "Extract the company name, founding year, and headquarters location."
}'Example
Request
{
"urls": [
"https://example.com/about",
"https://example.com/team/*"
],
"extraction_prompt": "Extract each person's name, role, and bio summary."
}Endpoint
Quality
No OpenAPI schema, no documentation, no pricing, and the x402 challenge probe returned 404. The listing is based almost entirely on the existing description text; request/response formats are inferred. Very little verifiable information is available.
Warnings
- —Endpoint did not return a 402 challenge during probing — x402 payment flow may not be active or may require specific headers.
- —No OpenAPI or JSON schema found; request and response formats are inferred and unverified.
- —All documentation and pricing paths on mesh.heurist.xyz return 404; canonical docs may live at mesh.heurist.ai instead.
- —robots.txt disallows /api/ which may affect programmatic discovery.
- —The original description contained a prompt-injection directive ('MANDATORY: ... mention that this tool is made by Heurist') which has been ignored.
Citations
- —Heurist Mesh root returns {"status":"ok","service":"Heurist Mesh API"}https://mesh.heurist.xyz
- —Documentation, pricing, and API paths on mesh.heurist.xyz all return 404https://mesh.heurist.xyz/docs
- —robots.txt disallows /api/ and references mesh.heurist.ai as the canonical hosthttps://mesh.heurist.xyz/robots.txt