Catalog

Classify scraping errors by type (rate limited, blocked, timeout, SSL, parse) with retry guidance.

Analyze scraping response time percentiles (P50/P90/P95/P99) with slow request detection.

Assess proxy pool health from success rates, response times and ban detection

Assess proxy pool health from success rates, response times and ban detection. Returns recommended proxies.

Calculate request delays to hit a target MB/hour bandwidth cap with concurrency-aware throttling.

Convert XPath expressions to CSS selectors and CSS selectors to XPath

Convert XPath expressions to CSS selectors and CSS selectors to XPath. Returns both with test tips.

Detect site structure changes from sitemap diffs and nav/H1 changes between two HTML versions.

Enforce a field schema on scraped records with type coercion, required field checking and validity scoring.

Normalize scraped fields using a source-to-target field mapping with fallback sources and coverage reporting.

Detect HTTP content encoding (gzip/brotli/deflate) and charset from headers and magic bytes.

Plan multi-page crawl with depth limits, URL patterns, exclusions and estimated page counts per depth.

Optimize scrape frequency by page type and change history to minimize cost while maintaining freshness.

Score extraction confidence per field from value presence, type validity, HTML grounding and schema match.

Diff scraped data against baseline to detect price changes, availability changes and field updates.

Estimate scraping job duration from URL count, concurrency, response time and JS rendering overhead.

Flatten nested JSON objects to dot-notation key paths

Flatten nested JSON objects to dot-notation key paths. Supports custom separator and array handling.

Detect CSS selector drift between two HTML versions

Detect CSS selector drift between two HTML versions. Returns broken/changed selectors for each field.

Estimate total scraping bandwidth from URL count, page size and blocking/compression optimizations.

Compare API vs scraping approaches by scoring cost, rate limits, difficulty and legal risk factors.

Track scraping session success rate, block rate, timeout rate and health trend from request results.

Detect geo-blocking from HTTP 451, page content messages, Cloudflare country headers and redirect patterns.

Remove scripts, nav, footer, ads and sidebars from HTML to isolate main content.

Detect DOM changes between two HTML versions: price changes, availability changes and layout shifts.

https://orbisapi.com/proxy/scrape-checkpoint-planner-api-765c2e

Calculate total scraping cost per record from proxy, compute, CAPTCHA, storage and dev costs.