{"id":"5f0170db-302e-44d9-8bf9-7b87aa3bb0ea","shortId":"vXUSmz","kind":"skill","title":"RAG Chunking Strategy Advisor","tagline":"Given a document type and retrieval goal, recommends the optimal chunking strategy for a RAG pipeline to minimize retrieval failures.","description":"# RAG Chunking Strategy Advisor\n\n## What this skill does\n\nThis skill analyzes your document types, content structure, and retrieval goals to recommend the right chunking strategy for your RAG pipeline. Poor chunking is the #1 cause of RAG failures — chunks too large lose precision, chunks too small lose context. This skill picks the right strategy and explains exactly how to implement it.\n\n## How to use\n\n### Claude Code / Cline\n\nCopy this file to `.agents/skills/rag-chunking-advisor/SKILL.md` in your project root.\n\nThen ask:\n- *\"Use the RAG Chunking Strategy Advisor to help me chunk our legal contract PDFs.\"*\n- *\"What chunking strategy should I use for markdown documentation with code blocks?\"*\n\nProvide:\n- Document type (PDFs, markdown, HTML, code, emails, etc.)\n- Typical document length\n- What users will search for (questions, keywords, concepts)\n- Your embedding model if known\n\n### Cursor / Codex\n\nPaste the instructions below along with your document type and retrieval use case.\n\n## The Prompt / Instructions for the Agent\n\nWhen asked to advise on RAG chunking, follow these steps:\n\n### Step 1 — Identify document characteristics\n\nAsk or infer:\n- **Document type**: structured (tables, headers) vs. unstructured (prose) vs. code\n- **Length**: short (< 1 page), medium (1–20 pages), long (20+ pages)\n- **Internal structure**: does it have headers, sections, numbered lists, code blocks?\n- **Query type**: factual lookups, conceptual questions, code search, or multi-hop reasoning?\n\n### Step 2 — Select the primary chunking strategy\n\n| Document Type | Recommended Strategy | Chunk Size |\n|---|---|---|\n| Prose (articles, books) | Sentence-window or recursive character | 512–1024 tokens |\n| Structured docs (markdown, HTML) | Header-based (split on H2/H3) | Full section |\n| PDFs with mixed content | Semantic chunking + page boundary | 512 tokens |\n| Source code | Function/class boundary splitting | Full function |\n| Tables / spreadsheets | Row-level or table-level | 1 row or full table |\n| Emails / short messages | Document-level (no chunking) | Full doc |\n| Legal / contracts | Clause-level splitting | 256–512 tokens |\n\n**Strategies explained:**\n\n- **Fixed-size with overlap** — Split every N tokens with a 10–20% overlap. Fast, simple, works for uniform prose. Use `chunk_size=512, overlap=50`.\n- **Recursive character splitting** — Split on `\\n\\n`, then `\\n`, then `.`, then ` `. Preserves paragraph structure. Best default for general text.\n- **Header-based (Markdown/HTML)** — Split on heading tags (H1/H2/H3). Each section becomes a chunk with its heading prepended. Best for docs sites and wikis.\n- **Semantic chunking** — Use an embedding model to detect topic shifts; split when cosine similarity drops. Best quality, highest cost. Use when documents have abrupt topic changes.\n- **Sentence-window** — Store sentences as chunks but retrieve surrounding N sentences for context. Good for Q&A over dense technical content.\n- **Function-level (code)** — Use AST parsing to split on function/class boundaries. Never split mid-function.\n\n### Step 3 — Configure chunk metadata\n\nEvery chunk must carry metadata for filtering and re-ranking:\n\n```python\n{\n  \"chunk_id\": \"doc_001_chunk_004\",\n  \"source\": \"contracts/nda-2024.pdf\",\n  \"page\": 3,\n  \"section\": \"Confidentiality Obligations\",\n  \"doc_type\": \"legal\",\n  \"created_at\": \"2024-11-01\"\n}\n```\n\nAlways store: `source`, `page_or_line`, `section_heading` (if available), `doc_type`.\n\n### Step 4 — Add a parent-child chunk structure (for complex docs)\n\nFor long structured documents, use a two-level hierarchy:\n- **Parent chunks** (1024–2048 tokens): stored for context retrieval\n- **Child chunks** (128–256 tokens): used for embedding and search\n\nRetrieve by child similarity, return parent for context. This avoids the precision vs. context tradeoff.\n\n```python\n# LlamaIndex example\nfrom llama_index.node_parser import HierarchicalNodeParser\nparser = HierarchicalNodeParser.from_defaults(chunk_sizes=[2048, 512, 128])\n```\n\n### Step 5 — Validate with retrieval testing\n\nBefore deploying, run this checklist:\n- [ ] Sample 20 real user queries against your chunked index\n- [ ] Check if the correct chunk appears in top-3 results\n- [ ] Verify chunk boundaries don't cut mid-sentence or mid-table\n- [ ] Ensure no chunk is under 50 tokens (too small = noise)\n- [ ] Ensure no chunk is over 2048 tokens (too large = diluted embedding)\n\n### Common chunking mistakes to flag\n\n- **Splitting tables row-by-row** when the header row is in a different chunk — always keep headers with data rows\n- **Not adding overlap** on fixed-size splits — 10% overlap prevents losing context at boundaries\n- **Chunking code by line count** instead of by function — always use AST-aware splitting for code\n- **Ignoring document structure** — a 500-token chunk that spans two unrelated sections is worse than two 250-token single-topic chunks\n\n## Example\n\n**Input:**\n> \"Use the RAG Chunking Advisor. I'm building a support chatbot. Documents are markdown product docs, 5–30 pages each, with H2 and H3 headers. Users ask specific how-to questions.\"\n\n**Output:**\n> **Recommended strategy: Header-based chunking**\n>\n> Split on H2 headers as primary boundaries, H3 as secondary. Each chunk = one section, with its full heading path prepended (`Product > Installation > macOS Setup`).\n>\n> Config:\n> ```python\n> from langchain.text_splitter import MarkdownHeaderTextSplitter\n> splitter = MarkdownHeaderTextSplitter(\n>     headers_to_split_on=[(\"##\", \"section\"), (\"###\", \"subsection\")]\n> )\n> ```\n>\n> Add metadata: `{\"source\": filename, \"section\": heading_path, \"doc_type\": \"product_docs\"}`\n>\n> If sections exceed 800 tokens, apply a secondary recursive split with `chunk_size=600, overlap=60`.\n>\n> Validate: run your top 20 support questions against the index and confirm the relevant section appears in top-3.","tags":["rag","chunking","advisor","openagentskills","notysoty","agent-skills","claude","claude-code","claude-skills","cline","cursor","llm"],"capabilities":["skill","source-notysoty","skill-rag-chunking-advisor","topic-agent-skills","topic-claude","topic-claude-code","topic-claude-skills","topic-cline","topic-cursor","topic-llm","topic-llm-skills","topic-skills"],"categories":["openagentskills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/Notysoty/openagentskills/rag-chunking-advisor","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add Notysoty/openagentskills","source_repo":"https://github.com/Notysoty/openagentskills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (5,845 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:13:23.773Z","embedding":null,"createdAt":"2026-05-18T13:20:45.390Z","updatedAt":"2026-05-18T19:13:23.773Z","lastSeenAt":"2026-05-18T19:13:23.773Z","tsv":"'-01':498 '-11':497 '-3':611,852 '001':481 '004':483 '1':58,186,205,208,301 '10':338,681 '1024':261,535 '128':544,582 '2':239 '20':209,212,339,595,838 '2024':496 '2048':536,580,641 '250':721 '256':322,545 '3':462,487 '30':746 '4':512 '5':584,745 '50':352,631 '500':709 '512':260,283,323,350,581 '60':833 '600':831 '800':821 'abrupt':419 'ad':674 'add':513,807 'advis':178 'advisor':4,28,108,733 'agent':174 'agents/skills/rag-chunking-advisor/skill.md':96 'along':160 'alway':499,667,697 'analyz':35 'appear':608,849 'appli':823 'articl':252 'ask':102,176,190,755 'ast':449,700 'ast-awar':699 'avail':508 'avoid':561 'awar':701 'base':269,374,766 'becom':383 'best':367,390,411 'block':128,224 'book':253 'boundari':282,288,455,615,687,774 'build':736 'carri':469 'case':168 'caus':59 'chang':421 'charact':259,354 'characterist':189 'chatbot':739 'check':603 'checklist':593 'child':517,542,554 'chunk':2,15,26,48,55,63,68,106,112,118,181,243,249,280,313,348,385,397,428,464,467,478,482,518,534,543,578,601,607,614,628,638,648,666,688,711,726,732,767,779,829 'claud':89 'claus':319 'clause-level':318 'cline':91 'code':90,127,135,202,223,231,286,447,689,704 'codex':155 'common':647 'complex':521 'concept':148 'conceptu':229 'confidenti':489 'config':792 'configur':463 'confirm':845 'content':39,278,443 'context':72,435,540,559,565,685 'contract':115,317 'contracts/nda-2024.pdf':485 'copi':92 'correct':606 'cosin':408 'cost':414 'count':692 'creat':494 'cursor':154 'cut':618 'data':671 'default':368,577 'dens':441 'deploy':590 'detect':403 'differ':665 'dilut':645 'doc':264,315,392,480,491,509,522,744,814,817 'document':7,37,125,130,139,163,188,193,245,310,417,526,706,740 'document-level':309 'drop':410 'email':136,306 'embed':150,400,549,646 'ensur':626,636 'etc':137 'everi':333,466 'exact':81 'exampl':569,727 'exceed':820 'explain':80,326 'factual':227 'failur':24,62 'fast':341 'file':94 'filenam':810 'filter':472 'fix':328,678 'fixed-s':327,677 'flag':651 'follow':182 'full':273,290,304,314,784 'function':291,445,460,696 'function-level':444 'function/class':287,454 'general':370 'given':5 'goal':11,43 'good':436 'h1/h2/h3':380 'h2':750,770 'h2/h3':272 'h3':752,775 'head':378,388,506,785,812 'header':197,219,268,373,660,669,753,765,771,801 'header-bas':267,372,764 'help':110 'hierarchi':532 'hierarchicalnodepars':574 'hierarchicalnodeparser.from':576 'highest':413 'hop':236 'how-to':757 'html':134,266 'id':479 'identifi':187 'ignor':705 'implement':84 'import':573,797 'index':602,843 'infer':192 'input':728 'instal':789 'instead':693 'instruct':158,171 'intern':214 'keep':668 'keyword':147 'known':153 'langchain.text':795 'larg':65,644 'legal':114,316,493 'length':140,203 'level':296,300,311,320,446,531 'line':504,691 'list':222 'llama_index.node':571 'llamaindex':568 'long':211,524 'lookup':228 'lose':66,71,684 'm':735 'maco':790 'markdown':124,133,265,742 'markdown/html':375 'markdownheadertextsplitt':798,800 'medium':207 'messag':308 'metadata':465,470,808 'mid':459,620,624 'mid-funct':458 'mid-sent':619 'mid-tabl':623 'minim':22 'mistak':649 'mix':277 'model':151,401 'multi':235 'multi-hop':234 'must':468 'n':334,358,359,361,432 'never':456 'nois':635 'number':221 'oblig':490 'one':780 'optim':14 'output':761 'overlap':331,340,351,675,682,832 'page':206,210,213,281,486,502,747 'paragraph':365 'parent':516,533,557 'parent-child':515 'pars':450 'parser':572,575 'past':156 'path':786,813 'pdfs':116,132,275 'pick':75 'pipelin':20,53 'poor':54 'precis':67,563 'prepend':389,787 'preserv':364 'prevent':683 'primari':242,773 'product':743,788,816 'project':99 'prompt':170 'prose':200,251,346 'provid':129 'python':477,567,793 'q':438 'qualiti':412 'queri':225,598 'question':146,230,760,840 'rag':1,19,25,52,61,105,180,731 'rank':476 're':475 're-rank':474 'real':596 'reason':237 'recommend':12,45,247,762 'recurs':258,353,826 'relev':847 'result':612 'retriev':10,23,42,166,430,541,552,587 'return':556 'right':47,77 'root':100 'row':295,302,655,657,661,672 'row-by-row':654 'row-level':294 'run':591,835 'sampl':594 'search':144,232,551 'secondari':777,825 'section':220,274,382,488,505,716,781,805,811,819,848 'select':240 'semant':279,396 'sentenc':255,423,426,433,621 'sentence-window':254,422 'setup':791 'shift':405 'short':204,307 'similar':409,555 'simpl':342 'singl':724 'single-top':723 'site':393 'size':250,329,349,579,679,830 'skill':31,34,74 'skill-rag-chunking-advisor' 'small':70,634 'sourc':285,484,501,809 'source-notysoty' 'span':713 'specif':756 'split':270,289,321,332,355,356,376,406,452,457,652,680,702,768,803,827 'splitter':796,799 'spreadsheet':293 'step':184,185,238,461,511,583 'store':425,500,538 'strategi':3,16,27,49,78,107,119,244,248,325,763 'structur':40,195,215,263,366,519,525,707 'subsect':806 'support':738,839 'surround':431 'tabl':196,292,299,305,625,653 'table-level':298 'tag':379 'technic':442 'test':588 'text':371 'token':262,284,324,335,537,546,632,642,710,722,822 'top':610,837,851 'topic':404,420,725 'topic-agent-skills' 'topic-claude' 'topic-claude-code' 'topic-claude-skills' 'topic-cline' 'topic-cursor' 'topic-llm' 'topic-llm-skills' 'topic-skills' 'tradeoff':566 'two':530,714,720 'two-level':529 'type':8,38,131,164,194,226,246,492,510,815 'typic':138 'uniform':345 'unrel':715 'unstructur':199 'use':88,103,122,167,347,398,415,448,527,547,698,729 'user':142,597,754 'valid':585,834 'verifi':613 'vs':198,201,564 'wiki':395 'window':256,424 'work':343 'wors':718","prices":[{"id":"6edd83f3-2d6f-4d5e-915e-f5c6d3628ae0","listingId":"5f0170db-302e-44d9-8bf9-7b87aa3bb0ea","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"Notysoty","category":"openagentskills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:20:45.390Z"}],"sources":[{"listingId":"5f0170db-302e-44d9-8bf9-7b87aa3bb0ea","source":"github","sourceId":"Notysoty/openagentskills/rag-chunking-advisor","sourceUrl":"https://github.com/Notysoty/openagentskills/tree/main/skills/rag-chunking-advisor","isPrimary":false,"firstSeenAt":"2026-05-18T13:20:45.390Z","lastSeenAt":"2026-05-18T19:13:23.773Z"}],"details":{"listingId":"5f0170db-302e-44d9-8bf9-7b87aa3bb0ea","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"Notysoty","slug":"rag-chunking-advisor","github":{"repo":"Notysoty/openagentskills","stars":8,"topics":["agent-skills","claude","claude-code","claude-skills","cline","cursor","llm","llm-skills","skills"],"license":"mit","html_url":"https://github.com/Notysoty/openagentskills","pushed_at":"2026-03-28T06:50:19Z","description":"A  community-driven library of reusable AI agent skills for Claude Code, Cursor, Codex, Cline, and more.","skill_md_sha":"fc1730d091835dbe222a0ae1630ff1c6d6195eec","skill_md_path":"skills/rag-chunking-advisor/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/Notysoty/openagentskills/tree/main/skills/rag-chunking-advisor"},"layout":"multi","source":"github","category":"openagentskills","frontmatter":{"name":"RAG Chunking Strategy Advisor","description":"Given a document type and retrieval goal, recommends the optimal chunking strategy for a RAG pipeline to minimize retrieval failures."},"skills_sh_url":"https://skills.sh/Notysoty/openagentskills/rag-chunking-advisor"},"updatedAt":"2026-05-18T19:13:23.773Z"}}