{"id":"2ba63f38-20e5-4489-9714-b518f4181d19","shortId":"JTXYHP","kind":"skill","title":"Crawlee Web Crawling and Scraping Library by Apify","tagline":"Crawlee is Apify’s open source crawling and scraping framework for Node.js. It unifies HTTP scraping and browser automation, adds queues, storage, retries, proxies, and lets developers switch between Playwright, Puppeteer, Cheerio, and JSDOM without rebuilding the whole pipeline.","description":"# Crawlee Web Crawling and Scraping Library by Apify\n\nCrawlee is Apify’s open source crawling and scraping framework for Node.js. It unifies HTTP scraping and browser automation, adds queues, storage, retries, proxies, and lets developers switch between Playwright, Puppeteer, Cheerio, and JSDOM without rebuilding the whole pipeline.\n\n## Installation\n\nUse the upstream install or setup path that matches your environment:\n- npx crawlee create my-crawler\n- npm start\n- npm install crawlee playwright\n- npm install crawlee@next\n\nRequirements and caveats from upstream:\n- Do you prefer 🐍 Python instead of JavaScript? [👉 Checkout Crawlee for Python 👈](https://github.com/apify/crawlee-python).\n- Crawlee requires **Node.js 16 or higher**.\n\nBasic usage or getting-started notes:\n- We recommend visiting the [Introduction tutorial](https://crawlee.dev/js/docs/introduction) in Crawlee documentation for more information.\n- ### With Crawlee CLI\n- The fastest way to try Crawlee out is to use the **Crawlee CLI** and choose the **Getting started example**. The CLI will install all the necessary dependencies and add boilerplate code for you to play with.\n\n- Source: https://github.com/apify/crawlee\n- Extracted from upstream docs: https://raw.githubusercontent.com/apify/crawlee/HEAD/README.md\n\n## Documentation\n\n- https://crawlee.dev/\n\n## Source\n\n- [Agent Skill Exchange](https://agentskillexchange.com/skills/crawlee-web-crawling-and-scraping-library-by-apify/)","tags":["crawlee","web","crawling","and","scraping","library","apify","skills","agentskillexchange","agent-skills","ai-agents","ai-tools"],"capabilities":["skill","source-agentskillexchange","skill-crawlee-web-crawling-and-scraping-library-by-apify","topic-agent-skills","topic-ai-agents","topic-ai-tools","topic-awesome-list","topic-claude-code","topic-codex","topic-cursor","topic-llm","topic-mcp","topic-npx-skills","topic-openclaw","topic-skills-catalog"],"categories":["skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/agentskillexchange/skills/crawlee-web-crawling-and-scraping-library-by-apify","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add agentskillexchange/skills","source_repo":"https://github.com/agentskillexchange/skills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (1,461 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:09:59.400Z","embedding":null,"createdAt":"2026-05-18T13:15:55.352Z","updatedAt":"2026-05-18T19:09:59.400Z","lastSeenAt":"2026-05-18T19:09:59.400Z","tsv":"'/apify/crawlee':212 '/apify/crawlee-python).':141 '/apify/crawlee/head/readme.md':219 '/js/docs/introduction)':163 '/skills/crawlee-web-crawling-and-scraping-library-by-apify/)':228 '16':145 'add':28,75,201 'agent':223 'agentskillexchange.com':227 'agentskillexchange.com/skills/crawlee-web-crawling-and-scraping-library-by-apify/)':226 'apifi':8,11,55,58 'autom':27,74 'basic':148 'boilerpl':202 'browser':26,73 'caveat':125 'checkout':135 'cheerio':40,87 'choos':187 'cli':172,185,193 'code':203 'crawl':3,15,50,62 'crawle':1,9,48,56,108,117,121,136,142,165,171,178,184 'crawlee.dev':162,221 'crawlee.dev/js/docs/introduction)':161 'crawler':112 'creat':109 'depend':199 'develop':35,82 'doc':216 'document':166,220 'environ':106 'exampl':191 'exchang':225 'extract':213 'fastest':174 'framework':18,65 'get':152,189 'getting-start':151 'github.com':140,211 'github.com/apify/crawlee':210 'github.com/apify/crawlee-python).':139 'higher':147 'http':23,70 'inform':169 'instal':95,99,116,120,195 'instead':132 'introduct':159 'javascript':134 'jsdom':42,89 'let':34,81 'librari':6,53 'match':104 'my-crawl':110 'necessari':198 'next':122 'node.js':20,67,144 'note':154 'npm':113,115,119 'npx':107 'open':13,60 'path':102 'pipelin':47,94 'play':207 'playwright':38,85,118 'prefer':130 'proxi':32,79 'puppet':39,86 'python':131,138 'queue':29,76 'raw.githubusercontent.com':218 'raw.githubusercontent.com/apify/crawlee/head/readme.md':217 'rebuild':44,91 'recommend':156 'requir':123,143 'retri':31,78 'scrape':5,17,24,52,64,71 'setup':101 'skill':224 'skill-crawlee-web-crawling-and-scraping-library-by-apify' 'sourc':14,61,209,222 'source-agentskillexchange' 'start':114,153,190 'storag':30,77 'switch':36,83 'topic-agent-skills' 'topic-ai-agents' 'topic-ai-tools' 'topic-awesome-list' 'topic-claude-code' 'topic-codex' 'topic-cursor' 'topic-llm' 'topic-mcp' 'topic-npx-skills' 'topic-openclaw' 'topic-skills-catalog' 'tri':177 'tutori':160 'unifi':22,69 'upstream':98,127,215 'usag':149 'use':96,182 'visit':157 'way':175 'web':2,49 'whole':46,93 'without':43,90","prices":[{"id":"50a733f9-0f60-4bba-9c0c-d8c769ef91f7","listingId":"2ba63f38-20e5-4489-9714-b518f4181d19","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"agentskillexchange","category":"skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:15:55.352Z"}],"sources":[{"listingId":"2ba63f38-20e5-4489-9714-b518f4181d19","source":"github","sourceId":"agentskillexchange/skills/crawlee-web-crawling-and-scraping-library-by-apify","sourceUrl":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-web-crawling-and-scraping-library-by-apify","isPrimary":false,"firstSeenAt":"2026-05-18T13:15:55.352Z","lastSeenAt":"2026-05-18T19:09:59.400Z"}],"details":{"listingId":"2ba63f38-20e5-4489-9714-b518f4181d19","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"agentskillexchange","slug":"crawlee-web-crawling-and-scraping-library-by-apify","github":{"repo":"agentskillexchange/skills","stars":8,"topics":["agent-skills","ai-agents","ai-tools","awesome-list","claude-code","codex","cursor","llm","mcp","npx-skills","openclaw","skills-catalog"],"license":"mit","html_url":"https://github.com/agentskillexchange/skills","pushed_at":"2026-05-18T19:02:17Z","description":"The open catalog of AI agent skills — 2,000+ security-scanned skills for Claude Code, Cursor, Codex, and more.","skill_md_sha":"f08a56d5d6a1e9d773dfa2520a37576f97e92d72","skill_md_path":"skills/crawlee-web-crawling-and-scraping-library-by-apify/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/agentskillexchange/skills/tree/main/skills/crawlee-web-crawling-and-scraping-library-by-apify"},"layout":"multi","source":"github","category":"skills","frontmatter":{"name":"Crawlee Web Crawling and Scraping Library by Apify","description":"Crawlee is Apify’s open source crawling and scraping framework for Node.js. It unifies HTTP scraping and browser automation, adds queues, storage, retries, proxies, and lets developers switch between Playwright, Puppeteer, Cheerio, and JSDOM without rebuilding the whole pipeline."},"skills_sh_url":"https://skills.sh/agentskillexchange/skills/crawlee-web-crawling-and-scraping-library-by-apify"},"updatedAt":"2026-05-18T19:09:59.400Z"}}