{"id":"02e57b96-ea7a-4d33-9502-957b44cae386","shortId":"gWCmAR","kind":"skill","title":"seo-firecrawl","tagline":"Ad-hoc web scraping, site mapping, and full-site crawling via Firecrawl MCP. Returns raw HTML, parsed metadata (og:*, twitter:*, JSON-LD, canonical, robots), JS-rendered DOM, and screenshots that WebFetch cannot. Distinct from the SE Ranking skills (which give keyword/traffic/SER","description":"# Firecrawl Orchestrator\n\nA direct interface to Firecrawl MCP for tasks that fall outside the data-driven SE Ranking skills. Use when:\n\n- You need raw HTML, `` metadata, JSON-LD, or post-JS DOM that WebFetch's markdown conversion strips.\n- You need a list of all URLs on a domain without pulling each one.\n- You need to crawl a site and audit each page's metadata.\n- You need to search within a known domain.\n- A higher-level skill (`seo-page`, `seo-schema`, `seo-content-audit`, etc.) called you as a sub-step.\n\n## Prerequisites\n\n- **Required:** the `firecrawl-mcp` MCP server. If `mcp__firecrawl-mcp__firecrawl_scrape` is unavailable, abort with the install command — `bash extensions/firecrawl/install.sh` from this plugin repo, plus the firecrawl.dev signup URL (free tier 500 credits/month). Don't attempt fallbacks; this skill exists for the cases WebFetch can't cover.\n- User provides: a target URL or domain, plus optionally a mode (`scrape` / `map` / `crawl` / `search`). If mode unspecified, infer from input shape (single URL → `scrape`, single domain → `map`).\n\n## Process\n\n1. **Preflight.** Confirm `firecrawl-mcp` is connected. If not, surface the install command and stop.\n2. **Mode selection.** Resolve user intent into one of:\n - `scrape` — single URL, full data (default if user supplies one URL).\n - `map` — single domain, list of URLs only (cheap reconnaissance).\n - `crawl` — single domain, fetch each discovered page (expensive; require explicit confirm).\n - `search` — query within a domain.\n3. **Cost estimation + confirmation.**\n - `scrape` (1 credit), `map` (~0.5 credit per discovered URL — estimate using a `map` first if scope unclear), `crawl` (1 credit per page crawled), `search` (1 credit per result returned).\n - For `crawl` and for `map` of >50 expected URLs, surface the estimate and require explicit go-ahead before calling.\n - Always read remaining credits implicitly via Firecrawl's response metadata (`creditsUsed` / `creditsRemaining` in `metadata`).\n4. **Execute.** Call the matching `mcp__firecrawl-mcp__firecrawl_*` tool.\n - `scrape`: pass `formats: [\"markdown\", \"html\"]` by default (markdown for prose, html for `` + JSON-LD). Add `formats: [\"screenshot\"]` only if the deliverable visibly uses one. SPAs: pass `waitFor: 2000` (or a CSS selector) so the JS-rendered DOM is captured. Default `onlyMainContent: true` to drop nav/footer noise — override only on explicit request.\n - `map`: default `limit: 500` (hard cap). Pass `excludePaths: [\"/admin/*\", \"/api/*\", \"/wp-admin/*\", \"/feed/*\"]` as a sane default.\n - `crawl`: default `limit: 50` (default cap), hard cap `limit: 200`. Always pass `excludePaths` to prune. Poll `firecrawl_check_crawl_status` if the job returns asynchronously.\n - `search`: default `limit: 20`.\n5. **Parse + structure output.** Don't dump the raw API response. Per-mode:\n - `scrape` → `RAW.md` (markdown body), `META.md` (og / twitter / canonical / robots / headers + parsed JSON-LD `@type` list with hashes), `links.csv`, optional `screenshot.png`.\n - `map` → `URLS.md` with pattern-grouped list (e.g., `/blog/* — 128 (37%)`, `/products/* — 84 (24%)`), plus `urls.csv`.\n - `crawl` → folder per page under `pages/{slugified-url}/` with `RAW.md` + `META.md`, plus a top-level `INDEX.md` summarising every page (URL, status, key signals).\n - `search` → `MATCHES.md` with hit excerpts + URLs ranked by relevance.\n6. **Synthesise** `FIRECRAWL.md` at the root: target, mode, credits used, key findings (5 bullets max), open loops, recommended next skill.\n\n## Output format\n\nFolder `seo-firecrawl-{slug}-{YYYYMMDD}/`:\n\n### Mode = scrape\n```\nseo-firecrawl-{slug}-{YYYYMMDD}/\n├── RAW.md (markdown body)\n├── META.md (og / twitter / canonical / robots / headers + parsed JSON-LD)\n├── links.csv (every on the page)\n├── screenshot.png (optional; only if requested)\n└── FIRECRAWL.md (synthesis + handoff payload)\n```\n\n### Mode = map\n```\nseo-firecrawl-{slug}-{YYYYMMDD}/\n├── URLS.md (pattern-grouped URL list)\n├── urls.csv (every URL with discovery depth, if available)\n└── FIRECRAWL.md\n```\n\n### Mode = crawl\n```\nseo-firecrawl-{slug}-{YYYYMMDD}/\n├── INDEX.md (every page + status code + key signals)\n├── pages/\n│ ├── {slug-1}/RAW.md\n│ ├── {slug-1}/META.md\n│ ├── {slug-2}/RAW.md\n│ └── ...\n└── FIRECRAWL.md\n```\n\n### Mode = search\n```\nseo-firecrawl-{slug}-{YYYYMMDD}/\n├── MATCHES.md (hit excerpts + URLs ranked by relevance)\n└── FIRECRAWL.md\n```\n\n`FIRECRAWL.md` follows this shape:\n\n```markdown\n# Firecrawl: {target}\n\n> Run dated {YYYY-MM-DD} · Mode: {scrape | map | crawl | search} · Credits used: {n}\n\n## Summary\n\n{One-paragraph what-came-back. Example: \"Scraped https://example.com/article. og:title and og:image present, JSON-LD Article schema with author + datePublished. 12 outbound links. Page is server-rendered (no JS-render divergence). Robots: index,follow.\"}\n\n## Key findings\n\n1. {Finding anchored in concrete data}\n2. ...\n5. ...\n\n## Open loops\n\n- {What this run did NOT answer}\n- ...\n\n## Recommended next step\n\n{One of: `seo-page` (when a single URL was scraped and now wants performance analysis) | `seo-schema` (when JSON-LD audit needs follow-up generation) | `seo-technical-audit` (when crawl revealed broken pages) | `seo-content-audit` (when crawl produced a corpus to audit) | `seo-drift baseline` (when the user wants to track this URL over time) | \"this completes the user's ask\".}\n\n## Handoff payload\n\n- **Produced by:** seo-firecrawl\n- **Target:** {url or domain}\n- **Mode:** {scrape | map | crawl | search}\n- **Credits used:** {n}\n- **Key findings:** {5 bullets — e.g., \"twitter:card present (summary_large_image)\", \"JSON-LD types: Article + Organization + BreadcrumbList\", \"robots: index,follow\", \"canonical self-referencing\", \"404s: 0 of 50 pages crawled\"}\n- **Open loops:** {what this didn't answer}\n- **Recommended next skill:** {seo-page | seo-schema | seo-technical-audit | seo-content-audit | …} — {one-line why}\n```\n\n## Tips\n\n- **Free tier 500 cr/month.** `map` is 0.5 cr/URL; `scrape` is 1 cr each; `crawl` 1 cr/page. Surface cost up front; warn when a single run will eat >100 credits.\n- **Default `onlyMainContent: true`** for `scrape` to drop nav/footer noise. Override only if the user explicitly asks for full-page DOM.\n- **Use `waitFor`** (CSS selector or ms) for SPAs that lazy-load content. 2000ms is a sensible default; selectors are more reliable than time waits.\n- **`firecrawl_map` before `firecrawl_crawl`** when crawl scope is unclear — discover first, decide what to crawl, then crawl. Saves credits.\n- **`includePaths` / `excludePaths`** dramatically cut crawl cost. Always pass `excludePaths: [\"/admin/*\", \"/api/*\", \"/wp-admin/*\", \"/feed/*\"]` as a default.\n- **Don't request `formats: [\"screenshot\"]`** unless the deliverable visibly uses it. It doubles per-page cost.\n- **Don't use `firecrawl_extract` or `firecrawl_deep_research`.** Both overlap with our own LLM analysis; `firecrawl_extract` has opaque pricing on the free tier; both are explicitly out of scope for `seo-skills`.\n- **Cloudflare / anti-bot:** some sites (especially e-commerce, banking) block Firecrawl's scraper. Surface the error cleanly; defeating WAFs is not a goal of this skill.\n- **Sub-step usage.** When invoked from another skill (`seo-page`, `seo-schema`, etc.), drop the `FIRECRAWL.md` synthesis — the caller wants the raw `META.md` / `RAW.md`. Skip mode-2's `URLS.md` summary too if the caller wants the raw `urls.csv`.\n- **This is the entry point** when you need raw HTML and don't have a more specific skill in mind. If you do — `seo-page` for keyword/traffic verdicts on one URL, `seo-schema` for JSON-LD work, `seo-technical-audit` for crawl-wide issues — use those instead. They orchestrate Firecrawl plus SE Ranking data automatically.\n\n## Works well with\n\n- **Predecessors:** none (entry point) or invoked as a sub-step from another skill.\n- **Successors:**\n - `seo-page` — when a single URL was scraped and now wants keyword/traffic verdicts.\n - `seo-schema` — when JSON-LD audit produced gaps that need generation.\n - `seo-technical-audit` — when a crawl revealed broken pages or noindex issues at scale.\n - `seo-content-audit` — when a crawl produced a corpus to E-E-A-T-audit.\n - `seo-drift baseline` — when the user wants to track this URL or domain over time.","tags":["seo","firecrawl","skills","seranking","agent-skills","ai-search","anthropic","backlinks","claude","claude-code","claude-plugin","claude-skills"],"capabilities":["skill","source-seranking","skill-seo-firecrawl","topic-agent-skills","topic-ai-search","topic-anthropic","topic-backlinks","topic-claude","topic-claude-code","topic-claude-plugin","topic-claude-skills","topic-content-brief","topic-ga4","topic-keyword-research","topic-mcp"],"categories":["seo-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/seranking/seo-skills/seo-firecrawl","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add seranking/seo-skills","source_repo":"https://github.com/seranking/seo-skills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 9 github stars · SKILL.md body (8,393 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:08:36.223Z","embedding":null,"createdAt":"2026-05-18T13:14:09.670Z","updatedAt":"2026-05-18T19:08:36.223Z","lastSeenAt":"2026-05-18T19:08:36.223Z","tsv":"'-1':649,652 '-2':655,1126 '/admin':427,1010 '/api':428,1011 '/article.':706 '/blog':507 '/feed':430,1013 '/meta.md':653 '/products':510 '/raw.md':650,656 '/wp-admin':429,1012 '0':872 '0.5':296,912 '1':227,293,310,316,739,916,920 '100':933 '12':721 '128':508 '2':243,745 '20':463 '200':444 '2000':394 '2000ms':969 '24':512 '3':288 '37':509 '4':355 '404s':871 '5':464,561,746,848 '50':327,438,874 '500':182,422,908 '6':549 '84':511 'abort':164 'ad':5 'ad-hoc':4 'add':381 'ahead':338 'alway':341,445,1007 'analysi':773,1049 'anchor':741 'anoth':1104,1213 'answer':754,883 'anti':1071 'anti-bot':1070 'api':473 'articl':716,861 'ask':826,950 'asynchron':459 'attempt':186 'audit':111,138,781,790,799,806,896,900,1181,1237,1246,1261,1274 'author':719 'automat':1197 'avail':631 'back':701 'bank':1079 'baselin':810,1278 'bash':169 'block':1080 'bodi':481,586 'bot':1072 'breadcrumblist':863 'broken':794,1251 'bullet':562,849 'call':140,340,357 'caller':1118,1133 'came':700 'cannot':39 'canon':29,485,590,867 'cap':424,440,442 'captur':406 'card':852 'case':193 'cheap':270 'check':452 'clean':1087 'cloudflar':1069 'code':644 'command':168,240 'commerc':1078 'complet':822 'concret':743 'confirm':229,282,291 'connect':234 'content':137,798,899,968,1260 'convers':88 'corpus':804,1267 'cost':289,923,1006,1033 'cover':197 'cr':917 'cr/month':909 'cr/page':921 'cr/url':913 'crawl':15,107,211,272,309,314,322,435,453,515,634,689,792,801,841,876,919,985,987,996,998,1005,1184,1249,1264 'crawl-wid':1183 'credit':294,297,311,317,344,557,691,843,934,1000 'credits/month':183 'creditsremain':352 'creditsus':351 'css':397,958 'cut':1004 'data':64,256,744,1196 'data-driven':63 'date':681 'datepublish':720 'dd':685 'decid':993 'deep':1041 'default':257,372,407,420,434,436,439,461,935,973,1016 'defeat':1088 'deliver':387,1024 'depth':629 'didn':881 'direct':52 'discov':277,299,991 'discoveri':628 'distinct':40 'diverg':733 'dom':34,83,404,955 'domain':99,123,204,224,265,274,287,837,1288 'doubl':1029 'dramat':1003 'drift':809,1277 'driven':65 'drop':411,941,1113 'dump':470 'e':1077,1270,1271 'e-commerc':1076 'e-e-a-t-audit':1269 'e.g':506,850 'eat':932 'entri':1141,1203 'error':1086 'especi':1075 'estim':290,301,332 'etc':139,1112 'everi':534,598,625,641 'exampl':702 'example.com':705 'example.com/article.':704 'excerpt':544,667 'excludepath':426,447,1002,1009 'execut':356 'exist':190 'expect':328 'expens':279 'explicit':281,335,417,949,1061 'extensions/firecrawl/install.sh':170 'extract':1038,1051 'fall':60 'fallback':187 'fetch':275 'find':560,738,740,847 'firecrawl':3,17,49,55,151,158,160,231,347,362,364,451,574,581,615,637,662,678,833,981,984,1037,1040,1050,1081,1192 'firecrawl-mcp':150,157,230,361 'firecrawl.dev':177 'firecrawl.md':551,607,632,657,672,673,1115 'first':305,992 'folder':516,571 'follow':674,736,784,866 'follow-up':783 'format':368,382,570,1020 'free':180,906,1057 'front':925 'full':13,255,953 'full-pag':952 'full-sit':12 'gap':1239 'generat':786,1242 'give':47 'go':337 'go-ahead':336 'goal':1093 'group':504,621 'handoff':609,827 'hard':423,441 'hash':495 'header':487,592 'higher':126 'higher-level':125 'hit':543,666 'hoc':6 'html':21,74,370,376,1147 'imag':711,856 'implicit':345 'includepath':1001 'index':735,865 'index.md':532,640 'infer':216 'input':218 'instal':167,239 'instead':1189 'intent':248 'interfac':53 'invok':1102,1206 'issu':1186,1255 'job':457 'js':32,82,402,731 'js-render':31,401,730 'json':27,77,379,490,595,714,779,858,1175,1235 'json-ld':26,76,378,489,594,713,778,857,1174,1234 'key':538,559,645,737,846 'keyword/traffic':1165,1228 'keyword/traffic/ser':48 'known':122 'larg':855 'lazi':966 'lazy-load':965 'ld':28,78,380,491,596,715,780,859,1176,1236 'level':127,531 'limit':421,437,443,462 'line':903 'link':723 'links.csv':496,597 'list':93,266,493,505,623 'llm':1048 'load':967 'loop':565,748,878 'map':10,210,225,263,295,304,325,419,499,612,688,840,910,982 'markdown':87,369,373,480,585,677 'match':359 'matches.md':541,665 'max':563 'mcp':18,56,152,153,156,159,232,360,363 'meta.md':482,526,587,1122 'metadata':23,75,115,350,354 'mind':1157 'mm':684 'mode':208,214,244,477,556,577,611,633,658,686,838,1125 'ms':961 'n':693,845 'nav/footer':412,942 'need':72,91,105,117,782,1145,1241 'next':567,756,885 'noindex':1254 'nois':413,943 'none':1202 'og':24,483,588,707,710 'one':103,250,261,390,696,758,902,1168 'one-lin':901 'one-paragraph':695 'onlymaincont':408,936 'opaqu':1053 'open':564,747,877 'option':206,497,603 'orchestr':50,1191 'organ':862 'outbound':722 'output':467,569 'outsid':61 'overlap':1044 'overrid':414,944 'page':113,131,278,313,518,520,535,601,642,647,724,762,795,875,889,954,1032,1108,1163,1218,1252 'paragraph':697 'pars':22,465,488,593 'pass':367,392,425,446,1008 'pattern':503,620 'pattern-group':502,619 'payload':610,828 'per':298,312,318,476,517,1031 'per-mod':475 'per-pag':1030 'perform':772 'plugin':173 'plus':175,205,513,527,1193 'point':1142,1204 'poll':450 'post':81 'post-j':80 'predecessor':1201 'preflight':228 'prerequisit':147 'present':712,853 'price':1054 'process':226 'produc':802,829,1238,1265 'prose':375 'provid':199 'prune':449 'pull':101 'queri':284 'rank':44,67,546,669,1195 'raw':20,73,472,1121,1136,1146 'raw.md':479,525,584,1123 'read':342 'recommend':566,755,884 'reconnaiss':271 'referenc':870 'relev':548,671 'reliabl':977 'remain':343 'render':33,403,728,732 'repo':174 'request':418,606,1019 'requir':148,280,334 'research':1042 'resolv':246 'respons':349,474 'result':319 'return':19,320,458 'reveal':793,1250 'robot':30,486,591,734,864 'root':554 'run':680,751,930 'sane':433 'save':999 'scale':1257 'schema':134,717,776,892,1111,1172,1232 'scope':307,988,1064 'scrape':8,161,209,222,252,292,366,478,578,687,703,768,839,914,939,1224 'scraper':1083 'screenshot':36,383,1021 'screenshot.png':498,602 'se':43,66,1194 'search':119,212,283,315,460,540,659,690,842 'select':245 'selector':398,959,974 'self':869 'self-referenc':868 'sensibl':972 'seo':2,130,133,136,573,580,614,636,661,761,775,788,797,808,832,888,891,894,898,1067,1107,1110,1162,1171,1179,1217,1231,1244,1259,1276 'seo-content-audit':135,796,897,1258 'seo-drift':807,1275 'seo-firecrawl':1,572,579,613,635,660,831 'seo-pag':129,760,887,1106,1161,1216 'seo-schema':132,774,890,1109,1170,1230 'seo-skil':1066 'seo-technical-audit':787,893,1178,1243 'server':154,727 'server-rend':726 'shape':219,676 'signal':539,646 'signup':178 'singl':220,223,253,264,273,765,929,1221 'site':9,14,109,1074 'skill':45,68,128,189,568,886,1068,1096,1105,1155,1214 'skill-seo-firecrawl' 'skip':1124 'slug':575,582,616,638,648,651,654,663 'slugifi':522 'slugified-url':521 'source-seranking' 'spas':391,963 'specif':1154 'status':454,537,643 'step':146,757,1099,1211 'stop':242 'strip':89 'structur':466 'sub':145,1098,1210 'sub-step':144,1097,1209 'successor':1215 'summari':694,854,1129 'summaris':533 'suppli':260 'surfac':237,330,922,1084 'synthesi':608,1116 'synthesis':550 'target':201,555,679,834 'task':58 'technic':789,895,1180,1245 'tier':181,907,1058 'time':820,979,1290 'tip':905 'titl':708 'tool':365 'top':530 'top-level':529 'topic-agent-skills' 'topic-ai-search' 'topic-anthropic' 'topic-backlinks' 'topic-claude' 'topic-claude-code' 'topic-claude-plugin' 'topic-claude-skills' 'topic-content-brief' 'topic-ga4' 'topic-keyword-research' 'topic-mcp' 'track':816,1284 'true':409,937 'twitter':25,484,589,851 'type':492,860 'unavail':163 'unclear':308,990 'unless':1022 'unspecifi':215 'url':96,179,202,221,254,262,268,300,329,523,536,545,622,626,668,766,818,835,1169,1222,1286 'urls.csv':514,624,1137 'urls.md':500,618,1128 'usag':1100 'use':69,302,389,558,692,844,956,1026,1036,1187 'user':198,247,259,813,824,948,1281 'verdict':1166,1229 'via':16,346 'visibl':388,1025 'waf':1089 'wait':980 'waitfor':393,957 'want':771,814,1119,1134,1227,1282 'warn':926 'web':7 'webfetch':38,85,194 'well':1199 'what-came-back':698 'wide':1185 'within':120,285 'without':100 'work':1177,1198 'yyyi':683 'yyyy-mm-dd':682 'yyyymmdd':576,583,617,639,664","prices":[{"id":"7e40f907-c96a-4d48-b7c8-4fd95044a59c","listingId":"02e57b96-ea7a-4d33-9502-957b44cae386","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"seranking","category":"seo-skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:14:09.670Z"}],"sources":[{"listingId":"02e57b96-ea7a-4d33-9502-957b44cae386","source":"github","sourceId":"seranking/seo-skills/seo-firecrawl","sourceUrl":"https://github.com/seranking/seo-skills/tree/main/skills/seo-firecrawl","isPrimary":false,"firstSeenAt":"2026-05-18T13:14:09.670Z","lastSeenAt":"2026-05-18T19:08:36.223Z"}],"details":{"listingId":"02e57b96-ea7a-4d33-9502-957b44cae386","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"seranking","slug":"seo-firecrawl","github":{"repo":"seranking/seo-skills","stars":9,"topics":["agent-skills","ai-search","answer-engine-optimization","anthropic","backlinks","claude","claude-code","claude-plugin","claude-skills","content-brief","ga4","generative-engine-optimization","keyword-research","mcp","mcp-server","search-console","seo","seo-tools","seranking","site-audit"],"license":"mit","html_url":"https://github.com/seranking/seo-skills","pushed_at":"2026-05-11T20:07:40Z","description":"Claude SEO Skills — production Claude Agent Skills for the SE Ranking MCP server. Content briefs, AI Search share of voice, audits, backlink gaps, keyword clusters, schema, sitemap, GEO, and more.","skill_md_sha":"7c089580a6db5936e7e66d6a07933a21b85976b8","skill_md_path":"skills/seo-firecrawl/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/seranking/seo-skills/tree/main/skills/seo-firecrawl"},"layout":"multi","source":"github","category":"seo-skills","frontmatter":{"name":"seo-firecrawl","description":"Ad-hoc web scraping, site mapping, and full-site crawling via Firecrawl MCP. Returns raw HTML, parsed metadata (og:*, twitter:*, JSON-LD, canonical, robots), JS-rendered DOM, and screenshots that WebFetch cannot. Distinct from the SE Ranking skills (which give keyword/traffic/SERP data) and from WebFetch (which gives markdown prose only). Use when the user says \"scrape this page\", \"crawl this site\", \"map this site\", \"find all pages on\", \"get the OG tags\", \"get the JSON-LD\", \"render this JS-heavy page\", or any task where raw `` metadata, structured-data scripts, or post-JS DOM are the actual deliverable. Also invoked as a sub-step from other skills that need raw HTML."},"skills_sh_url":"https://skills.sh/seranking/seo-skills/seo-firecrawl"},"updatedAt":"2026-05-18T19:08:36.223Z"}}