{"id":"5dd2b797-3885-4d57-979c-5a79765d41cd","shortId":"uYsPyP","kind":"skill","title":"Crawl4AI LLM-Friendly Web Crawler","tagline":"Run web crawling and scraping workflows with Crawl4AI, an open-source crawler built to produce LLM-ready markdown and structured extraction output. It supports async crawling, browser automation hooks, deep crawls, CLI usage, and Python-based data pipelines.","description":"# Crawl4AI LLM-Friendly Web Crawler\n\nRun web crawling and scraping workflows with Crawl4AI, an open-source crawler built to produce LLM-ready markdown and structured extraction output. It supports async crawling, browser automation hooks, deep crawls, CLI usage, and Python-based data pipelines.\n\n## Installation\n\nUse the upstream install or setup path that matches your environment:\n- pip install -U crawl4ai\n- pip install crawl4ai --pre\n- Crawl4AI offers flexible installation options to suit various use cases. You can install it as a Python package or use Docker.\n- pip install crawl4ai\n\nRequirements and caveats from upstream:\n- [![Python Version](https://img.shields.io/pypi/pyversions/crawl4ai)](https://pypi.org/project/crawl4ai/)\n- ✨ Previous v0.7.8: Stability & Bug Fix Release! 11 bug fixes addressing Docker API issues, LLM extraction improvements, URL handling fixes, and dependency updates. [Release notes →](https://github.com/unclecode/crawl4...\n- **Deploy anywhere**, zero keys, CLI and Docker, cloud friendly\n\nBasic usage or getting-started notes:\n- I made it open source for **availability**, anyone can use it without a gate. Now I’m building the platform for **affordability**, anyone can run serious crawls without breaking the bank. If that resonates, join in, s...\n- ## 🚀 Quick Start\n- # For pre release versions\n\n- Source: https://github.com/unclecode/crawl4ai\n- Extracted from upstream docs: https://raw.githubusercontent.com/unclecode/crawl4ai/HEAD/README.md\n\n## Source\n\n- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawl4ai-llm-friendly-web-crawler/)","tags":["crawl4ai","llm","friendly","web","crawler","skills","agentskillexchange","agent-skills","ai-agents","ai-tools","awesome-list","claude-code"],"capabilities":["skill","source-agentskillexchange","skill-crawl4ai-llm-friendly-web-crawler","topic-agent-skills","topic-ai-agents","topic-ai-tools","topic-awesome-list","topic-claude-code","topic-codex","topic-cursor","topic-llm","topic-mcp","topic-npx-skills","topic-openclaw","topic-skills-catalog"],"categories":["skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/agentskillexchange/skills/crawl4ai-llm-friendly-web-crawler","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add agentskillexchange/skills","source_repo":"https://github.com/agentskillexchange/skills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (1,597 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:09:58.941Z","embedding":null,"createdAt":"2026-05-18T13:15:54.628Z","updatedAt":"2026-05-18T19:09:58.941Z","lastSeenAt":"2026-05-18T19:09:58.941Z","tsv":"'/pypi/pyversions/crawl4ai)](https://pypi.org/project/crawl4ai/)':148 '/skills/crawl4ai-llm-friendly-web-crawler/)':252 '/unclecode/crawl4...':175 '/unclecode/crawl4ai':238 '/unclecode/crawl4ai/head/readme.md':245 '11':155 'address':158 'afford':213 'agent':247 'agentskillexchange.com':251 'agentskillexchange.com/skills/crawl4ai-llm-friendly-web-crawler/)':250 'anyon':199,214 'anywher':177 'api':160 'async':33,80 'autom':36,83 'avail':198 'bank':222 'base':45,92 'basic':185 'break':220 'browser':35,82 'bug':152,156 'build':209 'built':20,67 'case':124 'caveat':141 'cli':40,87,180 'cloud':183 'crawl':9,34,39,56,81,86,218 'crawl4ai':1,14,48,61,110,113,115,138 'crawler':6,19,53,66 'data':46,93 'deep':38,85 'depend':169 'deploy':176 'doc':242 'docker':135,159,182 'environ':106 'exchang':249 'extract':29,76,163,239 'fix':153,157,167 'flexibl':117 'friend':4,51,184 'gate':205 'get':189 'getting-start':188 'github.com':174,237 'github.com/unclecode/crawl4...':173 'github.com/unclecode/crawl4ai':236 'handl':166 'hook':37,84 'img.shields.io':147 'img.shields.io/pypi/pyversions/crawl4ai)](https://pypi.org/project/crawl4ai/)':146 'improv':164 'instal':95,99,108,112,118,127,137 'issu':161 'join':226 'key':179 'llm':3,24,50,71,162 'llm-friend':2,49 'llm-readi':23,70 'm':208 'made':193 'markdown':26,73 'match':104 'note':172,191 'offer':116 'open':17,64,195 'open-sourc':16,63 'option':119 'output':30,77 'packag':132 'path':102 'pip':107,111,136 'pipelin':47,94 'platform':211 'pre':114,232 'previous':149 'produc':22,69 'python':44,91,131,144 'python-bas':43,90 'quick':229 'raw.githubusercontent.com':244 'raw.githubusercontent.com/unclecode/crawl4ai/head/readme.md':243 'readi':25,72 'releas':154,171,233 'requir':139 'reson':225 'run':7,54,216 'scrape':11,58 'serious':217 'setup':101 'skill':248 'skill-crawl4ai-llm-friendly-web-crawler' 'sourc':18,65,196,235,246 'source-agentskillexchange' 'stabil':151 'start':190,230 'structur':28,75 'suit':121 'support':32,79 'topic-agent-skills' 'topic-ai-agents' 'topic-ai-tools' 'topic-awesome-list' 'topic-claude-code' 'topic-codex' 'topic-cursor' 'topic-llm' 'topic-mcp' 'topic-npx-skills' 'topic-openclaw' 'topic-skills-catalog' 'u':109 'updat':170 'upstream':98,143,241 'url':165 'usag':41,88,186 'use':96,123,134,201 'v0.7.8':150 'various':122 'version':145,234 'web':5,8,52,55 'without':203,219 'workflow':12,59 'zero':178","prices":[{"id":"57e4d4c7-4e1d-4050-9077-429124481ed0","listingId":"5dd2b797-3885-4d57-979c-5a79765d41cd","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"agentskillexchange","category":"skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:15:54.628Z"}],"sources":[{"listingId":"5dd2b797-3885-4d57-979c-5a79765d41cd","source":"github","sourceId":"agentskillexchange/skills/crawl4ai-llm-friendly-web-crawler","sourceUrl":"https://github.com/agentskillexchange/skills/tree/main/skills/crawl4ai-llm-friendly-web-crawler","isPrimary":false,"firstSeenAt":"2026-05-18T13:15:54.628Z","lastSeenAt":"2026-05-18T19:09:58.941Z"}],"details":{"listingId":"5dd2b797-3885-4d57-979c-5a79765d41cd","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"agentskillexchange","slug":"crawl4ai-llm-friendly-web-crawler","github":{"repo":"agentskillexchange/skills","stars":8,"topics":["agent-skills","ai-agents","ai-tools","awesome-list","claude-code","codex","cursor","llm","mcp","npx-skills","openclaw","skills-catalog"],"license":"mit","html_url":"https://github.com/agentskillexchange/skills","pushed_at":"2026-05-18T19:02:17Z","description":"The open catalog of AI agent skills — 2,000+ security-scanned skills for Claude Code, Cursor, Codex, and more.","skill_md_sha":"d6b7575944a06eb14b531515219eb628f45ea06c","skill_md_path":"skills/crawl4ai-llm-friendly-web-crawler/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/agentskillexchange/skills/tree/main/skills/crawl4ai-llm-friendly-web-crawler"},"layout":"multi","source":"github","category":"skills","frontmatter":{"name":"Crawl4AI LLM-Friendly Web Crawler","description":"Run web crawling and scraping workflows with Crawl4AI, an open-source crawler built to produce LLM-ready markdown and structured extraction output. It supports async crawling, browser automation hooks, deep crawls, CLI usage, and Python-based data pipelines."},"skills_sh_url":"https://skills.sh/agentskillexchange/skills/crawl4ai-llm-friendly-web-crawler"},"updatedAt":"2026-05-18T19:09:58.941Z"}}