Catalog

Parse robots.txt files and check URL crawl permissions per user-agent.

Render Mustache-style templates with variables, batch 500 rows and validate variable coverage — built for AI agents

Analyze, transform and extract from any string — slugify, mask emails, extract URLs — batch 100 — built for AI agents

https://orbisapi.com/proxy/web-scrape-monitoring-scorer-api-9430e4

https://orbisapi.com/proxy/web-scrape-cost-roi-analyzer-api-086809

Convert HTML tables to CSV format with configurable delimiter, header inclusion and table index selection.

https://orbisapi.com/proxy/scrape-data-pipeline-validator-api-05c79f

Extract structured navigation links and breadcrumbs from HTML nav elements

Extract structured navigation links and breadcrumbs from HTML nav elements. Returns internal/external counts.

Detect content blocks in HTML: paywalls, cookie banners, login walls, age gates and CAPTCHA walls.

Classify scraping errors by type (rate limited, blocked, timeout, SSL, parse) with retry guidance.

Analyze scraping response time percentiles (P50/P90/P95/P99) with slow request detection.

Assess proxy pool health from success rates, response times and ban detection

Assess proxy pool health from success rates, response times and ban detection. Returns recommended proxies.

Calculate request delays to hit a target MB/hour bandwidth cap with concurrency-aware throttling.

Convert XPath expressions to CSS selectors and CSS selectors to XPath

Convert XPath expressions to CSS selectors and CSS selectors to XPath. Returns both with test tips.

Detect site structure changes from sitemap diffs and nav/H1 changes between two HTML versions.

Enforce a field schema on scraped records with type coercion, required field checking and validity scoring.

Normalize scraped fields using a source-to-target field mapping with fallback sources and coverage reporting.

Detect HTTP content encoding (gzip/brotli/deflate) and charset from headers and magic bytes.

Plan multi-page crawl with depth limits, URL patterns, exclusions and estimated page counts per depth.

Optimize scrape frequency by page type and change history to minimize cost while maintaining freshness.

Score extraction confidence per field from value presence, type validity, HTML grounding and schema match.

Diff scraped data against baseline to detect price changes, availability changes and field updates.

Estimate scraping job duration from URL count, concurrency, response time and JS rendering overhead.

Flatten nested JSON objects to dot-notation key paths

Flatten nested JSON objects to dot-notation key paths. Supports custom separator and array handling.