{"id":"19f2d841-a7e8-4312-9eda-12cb36013400","shortId":"x2urLV","kind":"skill","title":"seo-technical","tagline":"Audit technical SEO across crawlability, indexability, security, URLs, mobile, Core Web Vitals, structured data, JavaScript rendering, and related platform signals like robots.txt and AI crawler access.","description":"# Technical SEO Audit\n\n## When to Use\n- Use when the user wants a technical SEO review focused on crawlability, indexability, performance, or rendering.\n- Use when auditing robots.txt, canonicalization, JavaScript SEO, Core Web Vitals, or AI crawler access.\n- Use when the task is infrastructure- and implementation-oriented rather than content-focused.\n\n## Categories\n\n### 1. Crawlability\n- robots.txt: exists, valid, not blocking important resources\n- XML sitemap: exists, referenced in robots.txt, valid format\n- Noindex tags: intentional vs accidental\n- Crawl depth: important pages within 3 clicks of homepage\n- JavaScript rendering: check if critical content requires JS execution\n- Crawl budget: for large sites (>10k pages), efficiency matters\n\n#### AI Crawler Management\n\nAs of 2025-2026, AI companies actively crawl the web to train models and power AI search. Managing these crawlers via robots.txt is a critical technical SEO consideration.\n\n**Known AI crawlers:**\n\n| Crawler | Company | robots.txt token | Purpose |\n|---------|---------|-----------------|---------|\n| GPTBot | OpenAI | `GPTBot` | Model training |\n| ChatGPT-User | OpenAI | `ChatGPT-User` | Real-time browsing |\n| ClaudeBot | Anthropic | `ClaudeBot` | Model training |\n| PerplexityBot | Perplexity | `PerplexityBot` | Search index + training |\n| Bytespider | ByteDance | `Bytespider` | Model training |\n| Google-Extended | Google | `Google-Extended` | Gemini training (NOT search) |\n| CCBot | Common Crawl | `CCBot` | Open dataset |\n\n**Key distinctions:**\n- Blocking `Google-Extended` prevents Gemini training use but does NOT affect Google Search indexing or AI Overviews (those use `Googlebot`)\n- Blocking `GPTBot` prevents OpenAI training but does NOT prevent ChatGPT from citing your content via browsing (`ChatGPT-User`)\n- ~3-5% of websites now use AI-specific robots.txt rules\n\n**Example, selective AI crawler blocking:**\n```\n# Allow search indexing, block AI training crawlers\nUser-agent: GPTBot\nDisallow: /\n\nUser-agent: Google-Extended\nDisallow: /\n\nUser-agent: Bytespider\nDisallow: /\n\n# Allow all other crawlers (including Googlebot for search)\nUser-agent: *\nAllow: /\n```\n\n**Recommendation:** Consider your AI visibility strategy before blocking. Being cited by AI systems drives brand awareness and referral traffic. Cross-reference the `seo-geo` skill for full AI visibility optimization.\n\n### 2. Indexability\n- Canonical tags: self-referencing, no conflicts with noindex\n- Duplicate content: near-duplicates, parameter URLs, www vs non-www\n- Thin content: pages below minimum word counts per type\n- Pagination: rel=next/prev or load-more pattern\n- Hreflang: correct for multi-language/multi-region sites\n- Index bloat: unnecessary pages consuming crawl budget\n\n### 3. Security\n- HTTPS: enforced, valid SSL certificate, no mixed content\n- Security headers:\n  - Content-Security-Policy (CSP)\n  - Strict-Transport-Security (HSTS)\n  - X-Frame-Options\n  - X-Content-Type-Options\n  - Referrer-Policy\n- HSTS preload: check preload list inclusion for high-security sites\n\n### 4. URL Structure\n- Clean URLs: descriptive, hyphenated, no query parameters for content\n- Hierarchy: logical folder structure reflecting site architecture\n- Redirects: no chains (max 1 hop), 301 for permanent moves\n- URL length: flag >100 characters\n- Trailing slashes: consistent usage\n\n### 5. Mobile Optimization\n- Responsive design: viewport meta tag, responsive CSS\n- Touch targets: minimum 48x48px with 8px spacing\n- Font size: minimum 16px base\n- No horizontal scroll\n- Mobile-first indexing: Google indexes mobile version. **Mobile-first indexing is 100% complete as of July 5, 2024.** Google now crawls and indexes ALL websites exclusively with the mobile Googlebot user-agent.\n\n### 6. Core Web Vitals\n- **LCP** (Largest Contentful Paint): target <2.5s\n- **INP** (Interaction to Next Paint): target <200ms\n  - INP replaced FID on March 12, 2024. FID was fully removed from all Chrome tools (CrUX API, PageSpeed Insights, Lighthouse) on September 9, 2024. Do NOT reference FID anywhere.\n- **CLS** (Cumulative Layout Shift): target <0.1\n- Evaluation uses 75th percentile of real user data\n- Use PageSpeed Insights API or CrUX data if MCP available\n\n### 7. Structured Data\n- Detection: JSON-LD (preferred), Microdata, RDFa\n- Validation against Google's supported types\n- See seo-schema skill for full analysis\n\n### 8. JavaScript Rendering\n- Check if content visible in initial HTML vs requires JS\n- Identify client-side rendered (CSR) vs server-side rendered (SSR)\n- Flag SPA frameworks (React, Vue, Angular) that may cause indexing issues\n- Verify dynamic rendering setup if applicable\n\n#### JavaScript SEO: Canonical & Indexing Guidance (December 2025)\n\nGoogle updated its JavaScript SEO documentation in December 2025 with critical clarifications:\n\n1. **Canonical conflicts:** If a canonical tag in raw HTML differs from one injected by JavaScript, Google may use EITHER one. Ensure canonical tags are identical between server-rendered HTML and JS-rendered output.\n2. **noindex with JavaScript:** If raw HTML contains `<meta name=\"robots\" content=\"noindex\">` but JavaScript removes it, Google MAY still honor the noindex from raw HTML. Serve correct robots directives in the initial HTML response.\n3. **Non-200 status codes:** Google does NOT render JavaScript on pages returning non-200 HTTP status codes. Any content or meta tags injected via JS on error pages will be invisible to Googlebot.\n4. **Structured data in JavaScript:** Product, Article, and other structured data injected via JS may face delayed processing. For time-sensitive structured data (especially e-commerce Product markup), include it in the initial server-rendered HTML.\n\n**Best practice:** Serve critical SEO elements (canonical, meta robots, structured data, title, meta description) in the initial server-rendered HTML rather than relying on JavaScript injection.\n\n### 9. IndexNow Protocol\n- Check if site supports IndexNow for Bing, Yandex, Naver\n- Supported by search engines other than Google\n- Recommend implementation for faster indexing on non-Google engines\n\n## Output\n\n### Technical Score: XX/100\n\n### Category Breakdown\n| Category | Status | Score |\n|----------|--------|-------|\n| Crawlability | pass/warn/fail | XX/100 |\n| Indexability | pass/warn/fail | XX/100 |\n| Security | pass/warn/fail | XX/100 |\n| URL Structure | pass/warn/fail | XX/100 |\n| Mobile | pass/warn/fail | XX/100 |\n| Core Web Vitals | pass/warn/fail | XX/100 |\n| Structured Data | pass/warn/fail | XX/100 |\n| JS Rendering | pass/warn/fail | XX/100 |\n| IndexNow | pass/warn/fail | XX/100 |\n\n### Critical Issues (fix immediately)\n### High Priority (fix within 1 week)\n### Medium Priority (fix within 1 month)\n### Low Priority (backlog)\n\n## DataForSEO Integration (Optional)\n\nIf DataForSEO MCP tools are available, use `on_page_instant_pages` for real page analysis (status codes, page timing, broken links, on-page checks), `on_page_lighthouse` for Lighthouse audits (performance, accessibility, SEO scores), and `domain_analytics_technologies_domain_technologies` for technology stack detection.\n\n## Error Handling\n\n| Scenario | Action |\n|----------|--------|\n| URL unreachable | Report connection error with status code. Suggest verifying URL, checking DNS resolution, and confirming the site is publicly accessible. |\n| robots.txt not found | Note that no robots.txt was detected at the root domain. Recommend creating one with appropriate directives. Continue audit on remaining categories. |\n| HTTPS not configured | Flag as a critical issue. Report whether HTTP is served without redirect, mixed content exists, or SSL certificate is missing/expired. |\n| Core Web Vitals data unavailable | Note that CrUX data is not available (common for low-traffic sites). Suggest using Lighthouse lab data as a proxy and recommend increasing traffic before re-testing. |\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.","tags":["seo","technical","antigravity","awesome","skills","sickn33","agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding","ai-workflows"],"capabilities":["skill","source-sickn33","skill-seo-technical","topic-agent-skills","topic-agentic-skills","topic-ai-agent-skills","topic-ai-agents","topic-ai-coding","topic-ai-workflows","topic-antigravity","topic-antigravity-skills","topic-claude-code","topic-claude-code-skills","topic-codex-cli","topic-codex-skills"],"categories":["antigravity-awesome-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/sickn33/antigravity-awesome-skills/seo-technical","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add sickn33/antigravity-awesome-skills","source_repo":"https://github.com/sickn33/antigravity-awesome-skills","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 34515 github stars · SKILL.md body (8,203 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-04-22T12:51:45.010Z","embedding":null,"createdAt":"2026-04-18T21:44:41.525Z","updatedAt":"2026-04-22T12:51:45.010Z","lastSeenAt":"2026-04-22T12:51:45.010Z","tsv":"'-200':767,779 '-2026':137 '-5':262 '/multi-region':391 '0.1':595 '1':82,468,699,943,949 '100':477,521 '10k':127 '12':566 '16px':503 '2':345,735 '2.5':552 '200ms':560 '2024':527,567,584 '2025':136,686,695 '3':109,261,400,765 '301':470 '4':445,799 '48x48px':496 '5':483,526 '6':543 '7':614 '75th':598 '8':638 '8px':498 '9':583,865 'access':29,65,989,1026 'accident':103 'across':7 'action':1005 'activ':140 'affect':232 'agent':286,291,298,311,542 'ai':27,63,131,138,149,163,237,268,274,281,316,324,342 'ai-specif':267 'allow':277,301,312 'analysi':637,971 'analyt':994 'angular':668 'anthrop':187 'anywher':589 'api':577,607 'applic':679 'appropri':1044 'architectur':463 'articl':805 'ask':1141 'audit':4,32,54,987,1047 'avail':613,962,1085 'awar':328 'backlog':953 'base':504 'best':838 'bing':874 'bloat':394 'block':88,221,242,276,280,320 'boundari':1149 'brand':327 'breakdown':899 'broken':976 'brows':185,257 'budget':123,399 'byted':198 'bytespid':197,199,299 'canon':347,682,700,704,721,844 'canonic':56 'categori':81,898,900,1050 'caus':671 'ccbot':213,216 'certif':406,1071 'chain':466 'charact':478 'chatgpt':176,180,251,259 'chatgpt-us':175,179,258 'check':115,436,641,868,981,1017 'chrome':574 'cite':253,322 'clarif':698,1143 'claudebot':186,188 'clean':448 'clear':1116 'click':110 'client':653 'client-sid':652 'cls':590 'code':769,782,973,1013 'commerc':826 'common':214,1086 'compani':139,166 'complet':522 'configur':1053 'confirm':1021 'conflict':353,701 'connect':1009 'consid':314 'consider':161 'consist':481 'consum':397 'contain':742 'content':79,118,255,357,369,409,413,428,456,549,643,784,1067 'content-focus':78 'content-security-polici':412 'continu':1046 'core':13,59,544,919,1074 'correct':386,757 'count':374 'crawl':104,122,141,215,398,530 'crawlabl':8,47,83,903 'crawler':28,64,132,153,164,165,275,283,304 'creat':1041 'criteria':1152 'critic':117,158,697,841,935,1057 'cross':333 'cross-refer':332 'crux':576,609,1081 'csp':416 'csr':656 'css':492 'cumul':591 'data':17,603,610,616,801,809,822,848,925,1077,1082,1096 'dataforseo':954,958 'dataset':218 'decemb':685,694 'delay':815 'depth':105 'describ':1120 'descript':450,851 'design':487 'detect':617,1001,1035 'differ':709 'direct':759,1045 'disallow':288,295,300 'distinct':220 'dns':1018 'document':692 'domain':993,996,1039 'drive':326 'duplic':356,360 'dynam':675 'e':825 'e-commerc':824 'effici':129 'either':718 'element':843 'enforc':403 'engin':880,893 'ensur':720 'environ':1132 'environment-specif':1131 'error':792,1002,1010 'especi':823 'evalu':596 'exampl':272 'exclus':535 'execut':121 'exist':85,93,1068 'expert':1137 'extend':204,208,224,294 'face':814 'faster':887 'fid':563,568,588 'first':510,518 'fix':937,941,947 'flag':476,663,1054 'focus':45,80 'folder':459 'font':500 'format':98 'found':1029 'frame':424 'framework':665 'full':341,636 'fulli':570 'gemini':209,226 'geo':338 'googl':203,205,207,223,233,293,512,528,626,687,715,747,770,883,892 'google-extend':202,206,222,292 'googlebot':241,306,539,798 'gptbot':170,172,243,287 'guidanc':684 'handl':1003 'header':411 'hierarchi':457 'high':442,939 'high-secur':441 'homepag':112 'honor':750 'hop':469 'horizont':506 'hreflang':385 'hsts':421,434 'html':647,708,729,741,755,763,837,858 'http':780,1061 'https':402,1051 'hyphen':451 'ident':724 'identifi':651 'immedi':938 'implement':74,885 'implementation-ori':73 'import':89,106 'includ':305,829 'inclus':439 'increas':1102 'index':9,48,195,235,279,346,393,511,513,519,532,672,683,888,906 'indexnow':866,872,932 'infrastructur':71 'initi':646,762,833,854 'inject':712,788,810,864 'inp':554,561 'input':1146 'insight':579,606 'instant':966 'integr':955 'intent':101 'interact':555 'invis':796 'issu':673,936,1058 'javascript':18,57,113,639,680,690,714,738,744,774,803,863 'js':120,650,732,790,812,928 'js-render':731 'json':619 'json-ld':618 'juli':525 'key':219 'known':162 'lab':1095 'languag':390 'larg':125 'largest':548 'layout':592 'lcp':547 'ld':620 'length':475 'lighthous':580,984,986,1094 'like':24 'limit':1108 'link':977 'list':438 'load':382 'load-mor':381 'logic':458 'low':951,1089 'low-traff':1088 'manag':133,151 'march':565 'markup':828 'match':1117 'matter':130 'max':467 'may':670,716,748,813 'mcp':612,959 'medium':945 'meta':489,786,845,850 'microdata':622 'minimum':372,495,502 'miss':1154 'missing/expired':1073 'mix':408,1066 'mobil':12,484,509,514,517,538,916 'mobile-first':508,516 'model':146,173,189,200 'month':950 'move':473 'multi':389 'multi-languag':388 'naver':876 'near':359 'near-dupl':358 'next':557 'next/prev':379 'noindex':99,355,736,752 'non':366,766,778,891 'non-googl':890 'non-www':365 'note':1030,1079 'on-pag':978 'one':711,719,1042 'open':217 'openai':171,178,245 'optim':344,485 'option':425,430,956 'orient':75 'output':734,894,1126 'overview':238 'page':107,128,370,396,776,793,965,967,970,974,980,983 'pagespe':578,605 'pagin':377 'paint':550,558 'paramet':361,454 'pass/warn/fail':904,907,910,914,917,922,926,930,933 'pattern':384 'per':375 'percentil':599 'perform':49,988 'perman':472 'permiss':1147 'perplex':192 'perplexitybot':191,193 'platform':22 'polici':415,433 'power':148 'practic':839 'prefer':621 'preload':435,437 'prevent':225,244,250 'prioriti':940,946,952 'process':816 'product':804,827 'protocol':867 'proxi':1099 'public':1025 'purpos':169 'queri':453 'rather':76,859 'raw':707,740,754 'rdfa':623 're':1106 're-test':1105 'react':666 'real':183,601,969 'real-tim':182 'recommend':313,884,1040,1101 'redirect':464,1065 'refer':334,587 'referenc':94,351 'referr':330,432 'referrer-polici':431 'reflect':461 'rel':378 'relat':21 'reli':861 'remain':1049 'remov':571,745 'render':19,51,114,640,655,661,676,728,733,773,836,857,929 'replac':562 'report':1008,1059 'requir':119,649,1145 'resolut':1019 'resourc':90 'respons':486,491,764 'return':777 'review':44,1138 'robot':758,846 'robots.txt':25,55,84,96,155,167,270,1027,1033 'root':1038 'rule':271 'safeti':1148 'scenario':1004 'schema':633 'scope':1119 'score':896,902,991 'scroll':507 'search':150,194,212,234,278,308,879 'secur':10,401,410,414,420,443,909 'see':630 'select':273 'self':350 'self-referenc':349 'sensit':820 'seo':2,6,31,43,58,160,337,632,681,691,842,990 'seo-geo':336 'seo-schema':631 'seo-techn':1 'septemb':582 'serv':756,840,1063 'server':659,727,835,856 'server-rend':726,834,855 'server-sid':658 'setup':677 'shift':593 'side':654,660 'signal':23 'site':126,392,444,462,870,1023,1091 'sitemap':92 'size':501 'skill':339,634,1111 'skill-seo-technical' 'slash':480 'source-sickn33' 'spa':664 'space':499 'specif':269,1133 'ssl':405,1070 'ssr':662 'stack':1000 'status':768,781,901,972,1012 'still':749 'stop':1139 'strategi':318 'strict':418 'strict-transport-secur':417 'structur':16,447,460,615,800,808,821,847,913,924 'substitut':1129 'success':1151 'suggest':1014,1092 'support':628,871,877 'system':325 'tag':100,348,490,705,722,787 'target':494,551,559,594 'task':69,1115 'technic':3,5,30,42,159,895 'technolog':995,997,999 'test':1107,1135 'thin':368 'time':184,819,975 'time-sensit':818 'titl':849 'token':168 'tool':575,960 'topic-agent-skills' 'topic-agentic-skills' 'topic-ai-agent-skills' 'topic-ai-agents' 'topic-ai-coding' 'topic-ai-workflows' 'topic-antigravity' 'topic-antigravity-skills' 'topic-claude-code' 'topic-claude-code-skills' 'topic-codex-cli' 'topic-codex-skills' 'touch':493 'traffic':331,1090,1103 'trail':479 'train':145,174,190,196,201,210,227,246,282 'transport':419 'treat':1124 'type':376,429,629 'unavail':1078 'unnecessari':395 'unreach':1007 'updat':688 'url':11,362,446,449,474,912,1006,1016 'usag':482 'use':35,36,52,66,228,240,266,597,604,717,963,1093,1109 'user':39,177,181,260,285,290,297,310,541,602 'user-ag':284,289,296,309,540 'valid':86,97,404,624,1134 'verifi':674,1015 'version':515 'via':154,256,789,811 'viewport':488 'visibl':317,343,644 'vital':15,61,546,921,1076 'vs':102,364,648,657 'vue':667 'want':40 'web':14,60,143,545,920,1075 'websit':264,534 'week':944 'whether':1060 'within':108,942,948 'without':1064 'word':373 'www':363,367 'x':423,427 'x-content-type-opt':426 'x-frame-opt':422 'xml':91 'xx/100':897,905,908,911,915,918,923,927,931,934 'yandex':875","prices":[{"id":"59a4674c-ecb5-445d-92d8-76fdb05b9a54","listingId":"19f2d841-a7e8-4312-9eda-12cb36013400","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"sickn33","category":"antigravity-awesome-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T21:44:41.525Z"}],"sources":[{"listingId":"19f2d841-a7e8-4312-9eda-12cb36013400","source":"github","sourceId":"sickn33/antigravity-awesome-skills/seo-technical","sourceUrl":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/seo-technical","isPrimary":false,"firstSeenAt":"2026-04-18T21:44:41.525Z","lastSeenAt":"2026-04-22T12:51:45.010Z"}],"details":{"listingId":"19f2d841-a7e8-4312-9eda-12cb36013400","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"sickn33","slug":"seo-technical","github":{"repo":"sickn33/antigravity-awesome-skills","stars":34515,"topics":["agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding","ai-workflows","antigravity","antigravity-skills","claude-code","claude-code-skills","codex-cli","codex-skills","cursor","cursor-skills","developer-tools","gemini-cli","gemini-skills","kiro","mcp","skill-library"],"license":"mit","html_url":"https://github.com/sickn33/antigravity-awesome-skills","pushed_at":"2026-04-22T06:40:00Z","description":"Installable GitHub library of 1,400+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.","skill_md_sha":"9d36fa6307474923a386c7b194458c8c96dfaf34","skill_md_path":"skills/seo-technical/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/seo-technical"},"layout":"multi","source":"github","category":"antigravity-awesome-skills","frontmatter":{"name":"seo-technical","description":"Audit technical SEO across crawlability, indexability, security, URLs, mobile, Core Web Vitals, structured data, JavaScript rendering, and related platform signals like robots.txt and AI crawler access."},"skills_sh_url":"https://skills.sh/sickn33/antigravity-awesome-skills/seo-technical"},"updatedAt":"2026-04-22T12:51:45.010Z"}}