{"id":"b410ff3c-c5d1-426a-a6b7-ea6397c24029","shortId":"y6qD4u","kind":"skill","title":"Crawlee Web Crawling and Browser Automation Library","tagline":"Builds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages.","description":"# Crawlee Web Crawling and Browser Automation Library\n\nBuilds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages.\n\n## Installation\n\nUse the upstream install or setup path that matches your environment:\n- npx crawlee create my-crawler\n- npm start\n- npm install crawlee playwright\n- npm install crawlee@next\n\nRequirements and caveats from upstream:\n- Do you prefer 🐍 Python instead of JavaScript? [👉 Checkout Crawlee for Python 👈](https://github.com/apify/crawlee-python).\n- Crawlee requires **Node.js 16 or higher**.\n\nBasic usage or getting-started notes:\n- We recommend visiting the [Introduction tutorial](https://crawlee.dev/js/docs/introduction) in Crawlee documentation for more information.\n- ### With Crawlee CLI\n- The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.\n\n- Source: https://github.com/apify/crawlee\n- Extracted from upstream docs: https://raw.githubusercontent.com/apify/crawlee/HEAD/README.md\n\n## Source\n\n- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawlee-web-crawling-browser-automation-library/)","tags":["crawlee","web","crawling","browser","automation","library","skills","agentskillexchange","agent-skills","ai-agents","ai-tools","awesome-list"],"capabilities":["skill","source-agentskillexchange","skill-crawlee-web-crawling-browser-automation-library","topic-agent-skills","topic-ai-agents","topic-ai-tools","topic-awesome-list","topic-claude-code","topic-codex","topic-cursor","topic-llm","topic-mcp","topic-npx-skills","topic-openclaw","topic-skills-catalog"],"categories":["skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/agentskillexchange/skills/crawlee-web-crawling-browser-automation-library","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add agentskillexchange/skills","source_repo":"https://github.com/agentskillexchange/skills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (1,401 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:09:59.590Z","embedding":null,"createdAt":"2026-05-18T13:15:55.604Z","updatedAt":"2026-05-18T19:09:59.590Z","lastSeenAt":"2026-05-18T19:09:59.590Z","tsv":"'/apify/crawlee':206 '/apify/crawlee-python).':135 '/apify/crawlee/head/readme.md':213 '/js/docs/introduction)':157 '/skills/crawlee-web-crawling-browser-automation-library/)':220 '16':139 'across':40,84 'add':195 'agent':215 'agentskillexchange.com':219 'agentskillexchange.com/skills/crawlee-web-crawling-browser-automation-library/)':218 'apifi':15,59 'autom':6,23,50,67 'basic':142 'boilerpl':196 'browser':5,22,49,66 'build':8,52 'caveat':119 'checkout':129 'choos':181 'cli':166,179,187 'code':197 'collect':11,55 'crawl':3,20,33,47,64,77 'crawle':1,14,45,58,102,111,115,130,136,159,165,172,178 'crawlee.dev':156 'crawlee.dev/js/docs/introduction)':155 'crawler':106 'creat':103 'dataset':35,79 'depend':193 'doc':210 'document':160 'environ':100 'exampl':185 'exchang':217 'export':36,80 'extract':207 'fastest':168 'get':146,183 'getting-start':145 'github.com':134,205 'github.com/apify/crawlee':204 'github.com/apify/crawlee-python).':133 'higher':141 'inform':163 'instal':89,93,110,114,189 'instead':126 'introduct':153 'javascript':128 'larg':41,85 'librari':7,24,51,68 'manag':29,73 'match':98 'my-crawl':104 'necessari':192 'next':116 'node.js':138 'note':148 'npm':107,109,113 'npx':101 'open':18,62 'open-sourc':17,61 'page':44,88 'path':96 'pipelin':12,56 'play':201 'playwright':30,74,112 'prefer':124 'puppet':32,76 'python':125,132 'queue':28,72 'raw.githubusercontent.com':212 'raw.githubusercontent.com/apify/crawlee/head/readme.md':211 'recommend':150 'request':27,71 'requir':117,137 'resili':38,82 'scalabl':9,53 'scrape':39,83 'set':42,86 'setup':95 'skill':216 'skill-crawlee-web-crawling-browser-automation-library' 'sourc':19,63,203,214 'source-agentskillexchange' 'start':108,147,184 'structur':34,78 'topic-agent-skills' 'topic-ai-agents' 'topic-ai-tools' 'topic-awesome-list' 'topic-claude-code' 'topic-codex' 'topic-cursor' 'topic-llm' 'topic-mcp' 'topic-npx-skills' 'topic-openclaw' 'topic-skills-catalog' 'tri':171 'tutori':154 'upstream':92,121,209 'usag':143 'use':25,69,90,176 'visit':151 'way':169 'web':2,10,46,54","prices":[{"id":"2d29c5ea-4dc5-46b8-a8f2-a9612fd904a7","listingId":"b410ff3c-c5d1-426a-a6b7-ea6397c24029","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"agentskillexchange","category":"skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:15:55.604Z"}],"sources":[{"listingId":"b410ff3c-c5d1-426a-a6b7-ea6397c24029","source":"github","sourceId":"agentskillexchange/skills/crawlee-web-crawling-browser-automation-library","sourceUrl":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-web-crawling-browser-automation-library","isPrimary":false,"firstSeenAt":"2026-05-18T13:15:55.604Z","lastSeenAt":"2026-05-18T19:09:59.590Z"}],"details":{"listingId":"b410ff3c-c5d1-426a-a6b7-ea6397c24029","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"agentskillexchange","slug":"crawlee-web-crawling-browser-automation-library","github":{"repo":"agentskillexchange/skills","stars":8,"topics":["agent-skills","ai-agents","ai-tools","awesome-list","claude-code","codex","cursor","llm","mcp","npx-skills","openclaw","skills-catalog"],"license":"mit","html_url":"https://github.com/agentskillexchange/skills","pushed_at":"2026-05-18T19:02:17Z","description":"The open catalog of AI agent skills — 2,000+ security-scanned skills for Claude Code, Cursor, Codex, and more.","skill_md_sha":"0ba6b53801524aa553a641817f8df6be3d22908a","skill_md_path":"skills/crawlee-web-crawling-browser-automation-library/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-web-crawling-browser-automation-library"},"layout":"multi","source":"github","category":"skills","frontmatter":{"name":"Crawlee Web Crawling and Browser Automation Library","description":"Builds scalable web collection pipelines with Crawlee, Apify’s open-source crawling and browser automation library. Useful for request queue management, Playwright or Puppeteer crawling, structured dataset export, and resilient scraping across large sets of pages."},"skills_sh_url":"https://skills.sh/agentskillexchange/skills/crawlee-web-crawling-browser-automation-library"},"updatedAt":"2026-05-18T19:09:59.590Z"}}