{"id":"62ba7f63-5d8c-48cb-97ad-bbe50576b172","shortId":"KtAjAd","kind":"skill","title":"Hybrid Search Architect","tagline":"Designs a hybrid retrieval pipeline combining dense vector search and BM25 sparse search with reciprocal rank fusion, and explains when to use each configuration.","description":"# Hybrid Search Architect\n\n## What this skill does\n\nThis skill designs a hybrid search pipeline that combines dense vector search (semantic similarity) with BM25 sparse search (keyword matching). Hybrid search outperforms either method alone on most retrieval benchmarks because vector search handles semantic meaning while BM25 handles exact keyword matches, product names, codes, and rare terms. This skill picks the right combination and fusion strategy for your use case.\n\n## How to use\n\n### Claude Code / Cline\n\nCopy this file to `.agents/skills/hybrid-search-architect/SKILL.md` in your project root.\n\nThen ask:\n- *\"Use the Hybrid Search Architect to improve our RAG pipeline's retrieval.\"*\n- *\"Design a hybrid search system for our product documentation.\"*\n\nProvide:\n- What you're searching (type of documents)\n- What queries look like (keywords, natural language, codes/IDs)\n- Your current search stack (Pinecone, Weaviate, Elasticsearch, pgvector, etc.)\n- Latency requirements\n\n### Cursor / Codex\n\nDescribe your current retrieval setup and query patterns alongside these instructions.\n\n## The Prompt / Instructions for the Agent\n\n### Step 1 — Determine if hybrid search is needed\n\n| Query pattern | Pure vector | Pure BM25 | Hybrid |\n|---|---|---|---|\n| Natural language questions | ✓ | — | ✓ |\n| Exact product names / SKUs | — | ✓ | ✓ |\n| Technical codes / IDs | — | ✓ | ✓ |\n| Conceptual / semantic | ✓ | — | ✓ |\n| Mixed (most real-world) | — | — | ✓ |\n\n**Use hybrid search when:** queries are mixed (some keyword, some semantic), documents contain both prose and structured data, or pure vector search misses obvious keyword matches.\n\n### Step 2 — Choose a stack\n\n**Option A: Weaviate (easiest hybrid, built-in)**\n```python\n# pip install weaviate-client\nimport weaviate\nfrom weaviate.classes.query import HybridFusion\n\nclient = weaviate.connect_to_local()\ncollection = client.collections.get(\"Documents\")\n\nresults = collection.query.hybrid(\n    query=\"payment processing error\",\n    fusion_type=HybridFusion.RELATIVE_SCORE,  # or RANKED\n    alpha=0.5,   # 0 = pure BM25, 1 = pure vector, 0.5 = balanced\n    limit=10,\n    return_metadata=[\"score\", \"explain_score\"]\n)\n```\n\n**Option B: Elasticsearch / OpenSearch (production-grade)**\n```python\nfrom elasticsearch import Elasticsearch\n\nes = Elasticsearch(\"http://localhost:9200\")\n\nquery = {\n    \"query\": {\n        \"bool\": {\n            \"should\": [\n                # BM25 component\n                {\"match\": {\"content\": {\"query\": user_query, \"boost\": 1.0}}},\n                # Dense vector component (kNN)\n                {\"knn\": {\n                    \"field\": \"embedding\",\n                    \"query_vector\": get_embedding(user_query),\n                    \"num_candidates\": 100,\n                    \"boost\": 1.0\n                }}\n            ]\n        }\n    },\n    \"size\": 10\n}\nresults = es.search(index=\"documents\", body=query)\n```\n\n**Option C: pgvector + custom BM25 (for PostgreSQL users)**\n```python\n# pip install pgvector psycopg2\n# Run both queries, then fuse results\n\nasync def hybrid_search(query: str, k: int = 10) -> list[dict]:\n    embedding = await get_embedding(query)\n\n    # Dense search\n    vector_results = await db.fetch(\"\"\"\n        SELECT id, content, 1 - (embedding <=> $1::vector) as score\n        FROM documents\n        ORDER BY embedding <=> $1::vector\n        LIMIT $2\n    \"\"\", embedding, k * 2)\n\n    # Sparse search (tsvector full-text search)\n    bm25_results = await db.fetch(\"\"\"\n        SELECT id, content, ts_rank(to_tsvector('english', content), plainto_tsquery($1)) as score\n        FROM documents\n        WHERE to_tsvector('english', content) @@ plainto_tsquery($1)\n        ORDER BY score DESC\n        LIMIT $2\n    \"\"\", query, k * 2)\n\n    return reciprocal_rank_fusion(vector_results, bm25_results, k=k)\n```\n\n### Step 3 — Implement Reciprocal Rank Fusion (RRF)\n\nRRF is the standard way to combine results from multiple ranked lists. It's simple, effective, and doesn't require tuning score scales:\n\n```python\ndef reciprocal_rank_fusion(\n    *ranked_lists: list[dict],\n    k: int = 60,\n    top_n: int = 10\n) -> list[dict]:\n    \"\"\"\n    Combine multiple ranked result lists using Reciprocal Rank Fusion.\n    k=60 is the standard constant (from the original RRF paper).\n    \"\"\"\n    scores: dict[str, float] = {}\n    docs: dict[str, dict] = {}\n\n    for ranked_list in ranked_lists:\n        for rank, doc in enumerate(ranked_list, start=1):\n            doc_id = doc[\"id\"]\n            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)\n            docs[doc_id] = doc\n\n    sorted_ids = sorted(scores, key=lambda x: scores[x], reverse=True)\n    return [docs[doc_id] for doc_id in sorted_ids[:top_n]]\n```\n\n### Step 4 — Tune the alpha parameter\n\nIf your backend supports an alpha/weight parameter:\n\n| Use case | Alpha (vector weight) |\n|---|---|\n| Technical docs with many exact terms | 0.3 |\n| General knowledge / FAQ | 0.5 |\n| Semantic / conceptual search | 0.7 |\n| Code search | 0.4 |\n| Mixed content (default) | 0.5 |\n\nTest with your actual query distribution — sample 50 real queries and compare precision at k=5 across alpha values.\n\n### Step 5 — Add a reranker (optional but high-impact)\n\nAfter hybrid retrieval, a cross-encoder reranker re-scores the top results with full query-document attention. This is the single highest-impact quality improvement after hybrid search:\n\n```python\n# pip install sentence-transformers\nfrom sentence_transformers import CrossEncoder\n\nreranker = CrossEncoder(\"cross-encoder/ms-marco-MiniLM-L-6-v2\")\n\ndef rerank(query: str, results: list[dict], top_n: int = 5) -> list[dict]:\n    \"\"\"Re-rank results using a cross-encoder. Retrieve wide, rerank narrow.\"\"\"\n    pairs = [(query, r[\"content\"]) for r in results]\n    scores = reranker.predict(pairs)\n\n    ranked = sorted(zip(results, scores), key=lambda x: x[1], reverse=True)\n    return [doc for doc, _ in ranked[:top_n]]\n\n# Usage: retrieve top 20 via hybrid, rerank to top 5\ncandidates = hybrid_search(query, k=20)\nfinal_results = rerank(query, candidates, top_n=5)\n```\n\n### Step 6 — Evaluation\n\nMeasure retrieval quality with Recall@K and Mean Reciprocal Rank:\n\n```python\ndef recall_at_k(relevant_ids: set, retrieved_ids: list[str], k: int) -> float:\n    return len(relevant_ids & set(retrieved_ids[:k])) / len(relevant_ids)\n\ndef mrr(relevant_ids: set, retrieved_ids: list[str]) -> float:\n    for rank, doc_id in enumerate(retrieved_ids, 1):\n        if doc_id in relevant_ids:\n            return 1 / rank\n    return 0.0\n```\n\nBenchmark: run 50 labeled queries, compare Recall@5 for pure vector vs. pure BM25 vs. hybrid. Hybrid should outperform both by 10–25% on mixed query sets.","tags":["hybrid","search","architect","openagentskills","notysoty","agent-skills","claude","claude-code","claude-skills","cline","cursor","llm"],"capabilities":["skill","source-notysoty","skill-hybrid-search-architect","topic-agent-skills","topic-claude","topic-claude-code","topic-claude-skills","topic-cline","topic-cursor","topic-llm","topic-llm-skills","topic-skills"],"categories":["openagentskills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/Notysoty/openagentskills/hybrid-search-architect","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add Notysoty/openagentskills","source_repo":"https://github.com/Notysoty/openagentskills","install_from":"skills.sh"}},"qualityScore":"0.454","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 8 github stars · SKILL.md body (6,651 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:13:22.106Z","embedding":null,"createdAt":"2026-05-18T13:20:43.121Z","updatedAt":"2026-05-18T19:13:22.106Z","lastSeenAt":"2026-05-18T19:13:22.106Z","tsv":"'/ms-marco-minilm-l-6-v2':719 '0':284,571 '0.0':869 '0.3':626 '0.4':637 '0.5':283,290,630,641 '0.7':634 '1':181,287,398,400,409,438,450,560,572,766,858,866 '1.0':327,345 '10':293,347,381,515,891 '100':343 '2':239,412,415,456,459 '20':780,792 '25':892 '3':471 '4':603 '5':657,662,730,786,800,877 '50':649,872 '6':802 '60':511,528 '9200':314 'across':658 'actual':645 'add':663 'agent':179 'agents/skills/hybrid-search-architect/skill.md':106 'alon':60 'alongsid':171 'alpha':282,606,617,659 'alpha/weight':613 'architect':3,30,117 'ask':112 'async':373 'attent':690 'await':385,393,425 'b':300 'backend':610 'balanc':291 'benchmark':64,870 'bm25':14,50,72,193,286,319,358,423,466,883 'bodi':352 'bool':317 'boost':326,344 'built':249 'built-in':248 'c':355 'candid':342,787,797 'case':95,616 'choos':240 'claud':99 'client':256,263 'client.collections.get':268 'cline':101 'code':79,100,203,635 'codes/ids':149 'codex':162 'collect':267 'collection.query.hybrid':271 'combin':9,43,88,483,518 'compar':653,875 'compon':320,330 'conceptu':205,632 'configur':27 'constant':532 'contain':224 'content':322,397,429,435,447,639,749 'copi':102 'cross':676,717,740 'cross-encod':675,716,739 'crossencod':713,715 'current':151,165 'cursor':161 'custom':357 'data':229 'db.fetch':394,426 'def':374,501,720,815,840 'default':640 'dens':10,44,328,389 'desc':454 'describ':163 'design':4,37,125 'determin':182 'dict':383,508,517,539,543,545,726,732 'distribut':647 'doc':542,554,561,563,566,569,575,576,578,591,592,595,621,770,772,852,860 'document':133,141,223,269,351,405,442,689 'doesn':494 'easiest':246 'effect':492 'either':58 'elasticsearch':156,301,308,310,312 'embed':334,338,384,387,399,408,413 'encod':677,718,741 'english':434,446 'enumer':556,855 'error':275 'es':311 'es.search':349 'etc':158 'evalu':803 'exact':74,198,624 'explain':22,297 'faq':629 'field':333 'file':104 'final':793 'float':541,828,849 'full':420,686 'full-text':419 'fuse':371 'fusion':20,90,276,463,475,504,526 'general':627 'get':337,386 'grade':305 'handl':68,73 'high':669 'high-impact':668 'highest':696 'highest-impact':695 'hybrid':1,6,28,39,55,115,127,184,194,213,247,375,672,701,782,788,885,886 'hybridfus':262 'hybridfusion.relative':278 'id':204,396,428,562,564,567,570,577,580,593,596,599,820,823,832,835,839,843,846,853,857,861,864 'impact':670,697 'implement':472 'import':257,261,309,712 'improv':119,699 'index':350 'instal':253,364,705 'instruct':173,176 'int':380,510,514,729,827 'k':379,414,458,468,469,509,527,573,656,791,809,818,826,836 'key':583,762 'keyword':53,75,146,220,236 'knn':331,332 'knowledg':628 'label':873 'lambda':584,763 'languag':148,196 'latenc':159 'len':830,837 'like':145 'limit':292,411,455 'list':382,488,506,507,516,522,548,551,558,725,731,824,847 'local':266 'localhost':313 'look':144 'mani':623 'match':54,76,237,321 'mean':70,811 'measur':804 'metadata':295 'method':59 'miss':234 'mix':207,218,638,894 'mrr':841 'multipl':486,519 'n':513,601,728,776,799 'name':78,200 'narrow':745 'natur':147,195 'need':187 'num':341 'obvious':235 'opensearch':302 'option':243,299,354,666 'order':406,451 'origin':535 'outperform':57,888 'pair':746,756 'paper':537 'paramet':607,614 'pattern':170,189 'payment':273 'pgvector':157,356,365 'pick':85 'pinecon':154 'pip':252,363,704 'pipelin':8,41,122 'plainto':436,448 'postgresql':360 'precis':654 'process':274 'product':77,132,199,304 'production-grad':303 'project':109 'prompt':175 'prose':226 'provid':134 'psycopg2':366 'pure':190,192,231,285,288,879,882 'python':251,306,362,500,703,814 'qualiti':698,806 'queri':143,169,188,216,272,315,316,323,325,335,340,353,369,377,388,457,646,651,688,722,747,790,796,874,895 'query-docu':687 'question':197 'r':748,751 'rag':121 'rank':19,281,431,462,474,487,503,505,520,525,547,550,553,557,574,735,757,774,813,851,867 'rare':81 're':137,680,734 're-rank':733 're-scor':679 'real':210,650 'real-world':209 'recal':808,816,876 'reciproc':18,461,473,502,524,812 'relev':819,831,838,842,863 'requir':160,496 'rerank':665,678,714,721,744,783,795 'reranker.predict':755 'result':270,348,372,392,424,465,467,484,521,684,724,736,753,760,794 'retriev':7,63,124,166,673,742,778,805,822,834,845,856 'return':294,460,590,769,829,865,868 'revers':588,767 'right':87 'root':110 'rrf':476,477,536 'run':367,871 'sampl':648 'scale':499 'score':279,296,298,403,440,453,498,538,565,582,586,681,754,761 'scores.get':568 'search':2,12,16,29,40,46,52,56,67,116,128,138,152,185,214,233,376,390,417,422,633,636,702,789 'select':395,427 'semant':47,69,206,222,631 'sentenc':707,710 'sentence-transform':706 'set':821,833,844,896 'setup':167 'similar':48 'simpl':491 'singl':694 'size':346 'skill':33,36,84 'skill-hybrid-search-architect' 'skus':201 'sort':579,581,598,758 'source-notysoty' 'spars':15,51,416 'stack':153,242 'standard':480,531 'start':559 'step':180,238,470,602,661,801 'str':378,540,544,723,825,848 'strategi':91 'structur':228 'support':611 'system':129 'technic':202,620 'term':82,625 'test':642 'text':421 'top':512,600,683,727,775,779,785,798 'topic-agent-skills' 'topic-claude' 'topic-claude-code' 'topic-claude-skills' 'topic-cline' 'topic-cursor' 'topic-llm' 'topic-llm-skills' 'topic-skills' 'transform':708,711 'true':589,768 'ts':430 'tsqueri':437,449 'tsvector':418,433,445 'tune':497,604 'type':139,277 'usag':777 'use':25,94,98,113,212,523,615,737 'user':324,339,361 'valu':660 'vector':11,45,66,191,232,289,329,336,391,401,410,464,618,880 'via':781 'vs':881,884 'way':481 'weaviat':155,245,255,258 'weaviate-cli':254 'weaviate.classes.query':260 'weaviate.connect':264 'weight':619 'wide':743 'world':211 'x':585,587,764,765 'zip':759","prices":[{"id":"d12caffd-b998-4d00-a9ae-fbd341bf130e","listingId":"62ba7f63-5d8c-48cb-97ad-bbe50576b172","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"Notysoty","category":"openagentskills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:20:43.121Z"}],"sources":[{"listingId":"62ba7f63-5d8c-48cb-97ad-bbe50576b172","source":"github","sourceId":"Notysoty/openagentskills/hybrid-search-architect","sourceUrl":"https://github.com/Notysoty/openagentskills/tree/main/skills/hybrid-search-architect","isPrimary":false,"firstSeenAt":"2026-05-18T13:20:43.121Z","lastSeenAt":"2026-05-18T19:13:22.106Z"}],"details":{"listingId":"62ba7f63-5d8c-48cb-97ad-bbe50576b172","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"Notysoty","slug":"hybrid-search-architect","github":{"repo":"Notysoty/openagentskills","stars":8,"topics":["agent-skills","claude","claude-code","claude-skills","cline","cursor","llm","llm-skills","skills"],"license":"mit","html_url":"https://github.com/Notysoty/openagentskills","pushed_at":"2026-03-28T06:50:19Z","description":"A  community-driven library of reusable AI agent skills for Claude Code, Cursor, Codex, Cline, and more.","skill_md_sha":"0afd7243ed955ba5b50c70c59f73c7eded490c6e","skill_md_path":"skills/hybrid-search-architect/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/Notysoty/openagentskills/tree/main/skills/hybrid-search-architect"},"layout":"multi","source":"github","category":"openagentskills","frontmatter":{"name":"Hybrid Search Architect","description":"Designs a hybrid retrieval pipeline combining dense vector search and BM25 sparse search with reciprocal rank fusion, and explains when to use each configuration."},"skills_sh_url":"https://skills.sh/Notysoty/openagentskills/hybrid-search-architect"},"updatedAt":"2026-05-18T19:13:22.106Z"}}