{"id":"4f06c406-50da-4995-8598-dcbfe523f637","shortId":"URb2WZ","kind":"skill","title":"Crawlee Smart Crawler Agent","tagline":"Implements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data.","description":"# Crawlee Smart Crawler Agent\n\nImplements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data.\n\n## Installation\n\nUse the upstream install or setup path that matches your environment:\n- npx crawlee create my-crawler\n- npm start\n- npm install crawlee playwright\n- npm install crawlee@next\n\nRequirements and caveats from upstream:\n- Do you prefer 🐍 Python instead of JavaScript? [👉 Checkout Crawlee for Python 👈](https://github.com/apify/crawlee-python).\n- Crawlee requires **Node.js 16 or higher**.\n\nBasic usage or getting-started notes:\n- We recommend visiting the [Introduction tutorial](https://crawlee.dev/js/docs/introduction) in Crawlee documentation for more information.\n- ### With Crawlee CLI\n- The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.\n\n- Source: https://github.com/apify/crawlee\n- Extracted from upstream docs: https://raw.githubusercontent.com/apify/crawlee/HEAD/README.md\n\n## Documentation\n\n- https://crawlee.dev/\n\n## Source\n\n- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawlee-smart-crawler-agent/)","tags":["crawlee","smart","crawler","agent","skills","agentskillexchange","agent-skills","ai-agents","ai-tools","awesome-list","claude-code","codex"],"capabilities":["skill","source-agentskillexchange","skill-crawlee-smart-crawler-agent","topic-agent-skills","topic-ai-agents","topic-ai-tools","topic-awesome-list","topic-claude-code","topic-codex","topic-cursor","topic-llm","topic-mcp","topic-npx-skills","topic-openclaw","topic-skills-catalog"],"categories":["skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/agentskillexchange/skills/crawlee-smart-crawler-agent","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add agentskillexchange/skills","source_repo":"https://github.com/agentskillexchange/skills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (1,356 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:09:59.298Z","embedding":null,"createdAt":"2026-05-18T13:15:55.220Z","updatedAt":"2026-05-18T19:09:59.298Z","lastSeenAt":"2026-05-18T19:09:59.298Z","tsv":"'/apify/crawlee':182 '/apify/crawlee-python).':111 '/apify/crawlee/head/readme.md':189 '/js/docs/introduction)':133 '/skills/crawlee-smart-crawler-agent/)':198 '16':115 'adapt':14,46 'add':171 'agent':4,36,193 'agentskillexchange.com':197 'agentskillexchange.com/skills/crawlee-smart-crawler-agent/)':196 'anti':26,58 'anti-bot':25,57 'basic':118 'boilerpl':172 'bot':27,59 'caveat':95 'challeng':28,60 'checkout':105 'cheeriocrawl':18,50 'choos':157 'cli':142,155,163 'code':173 'crawl':8,40 'crawle':1,11,33,43,78,87,91,106,112,135,141,148,154 'crawlee.dev':132,191 'crawlee.dev/js/docs/introduction)':131 'crawler':3,35,82 'creat':79 'data':32,64 'depend':169 'doc':186 'document':136,190 'environ':76 'exampl':161 'exchang':195 'export':30,62 'extract':183 'fastest':144 'framework':12,44 'get':122,159 'getting-start':121 'github.com':110,181 'github.com/apify/crawlee':180 'github.com/apify/crawlee-python).':109 'handl':24,56 'higher':117 'implement':5,37 'inform':139 'instal':65,69,86,90,165 'instead':102 'intellig':6,38 'introduct':129 'javascript':104 'manag':21,53 'match':74 'my-crawl':80 'necessari':168 'next':92 'node.js':114 'note':124 'npm':83,85,89 'npx':77 'path':72 'play':177 'playwright':88 'playwrightcrawl':20,52 'prefer':100 'python':101,108 'queue':23,55 'raw.githubusercontent.com':188 'raw.githubusercontent.com/apify/crawlee/head/readme.md':187 'recommend':126 'request':15,22,47,54 'requir':93,113 'rout':16,48 'setup':71 'skill':194 'skill-crawlee-smart-crawler-agent' 'smart':2,34 'sourc':179,192 'source-agentskillexchange' 'start':84,123,160 'structur':31,63 'topic-agent-skills' 'topic-ai-agents' 'topic-ai-tools' 'topic-awesome-list' 'topic-claude-code' 'topic-codex' 'topic-cursor' 'topic-llm' 'topic-mcp' 'topic-npx-skills' 'topic-openclaw' 'topic-skills-catalog' 'tri':147 'tutori':130 'upstream':68,97,185 'usag':119 'use':9,41,66,152 'visit':127 'way':145 'web':7,39","prices":[{"id":"dfac908e-853b-4033-b4c4-84c5f6b58f7a","listingId":"4f06c406-50da-4995-8598-dcbfe523f637","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"agentskillexchange","category":"skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:15:55.220Z"}],"sources":[{"listingId":"4f06c406-50da-4995-8598-dcbfe523f637","source":"github","sourceId":"agentskillexchange/skills/crawlee-smart-crawler-agent","sourceUrl":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-smart-crawler-agent","isPrimary":false,"firstSeenAt":"2026-05-18T13:15:55.220Z","lastSeenAt":"2026-05-18T19:09:59.298Z"}],"details":{"listingId":"4f06c406-50da-4995-8598-dcbfe523f637","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"agentskillexchange","slug":"crawlee-smart-crawler-agent","github":{"repo":"agentskillexchange/skills","stars":8,"topics":["agent-skills","ai-agents","ai-tools","awesome-list","claude-code","codex","cursor","llm","mcp","npx-skills","openclaw","skills-catalog"],"license":"mit","html_url":"https://github.com/agentskillexchange/skills","pushed_at":"2026-05-18T19:02:17Z","description":"The open catalog of AI agent skills — 2,000+ security-scanned skills for Claude Code, Cursor, Codex, and more.","skill_md_sha":"00664d4f6af34186182b8108dc76fcf8f12bda0a","skill_md_path":"skills/crawlee-smart-crawler-agent/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-smart-crawler-agent"},"layout":"multi","source":"github","category":"skills","frontmatter":{"name":"Crawlee Smart Crawler Agent","description":"Implements intelligent web crawling using the Crawlee framework with adaptive request routing between CheerioCrawler and PlaywrightCrawler. Manages request queues, handles anti-bot challenges, and exports structured data."},"skills_sh_url":"https://skills.sh/agentskillexchange/skills/crawlee-smart-crawler-agent"},"updatedAt":"2026-05-18T19:09:59.298Z"}}