{"id":"2db23d60-5025-44bb-8e4b-60ef52fcc622","shortId":"aae63K","kind":"skill","title":"dspy-haystack-integration","tagline":"This skill should be used when the user asks to \"integrate DSPy with Haystack\", \"optimize Haystack prompts using DSPy\", \"use DSPy to improve Haystack pipeline\", mentions \"Haystack pipeline optimization\", \"combining DSPy and Haystack\", \"extract DSPy prompt for Haystack\", or wants ","description":"# DSPy + Haystack Integration\n\n## Goal\n\nUse DSPy's optimization capabilities to automatically improve prompts in Haystack pipelines.\n\n## When to Use\n\n- You have existing Haystack pipelines\n- Manual prompt tuning is tedious\n- Need data-driven prompt optimization\n- Want to combine Haystack components with DSPy optimization\n\n## Inputs\n\n| Input | Type | Description |\n|-------|------|-------------|\n| `haystack_pipeline` | `Pipeline` | Existing Haystack pipeline |\n| `trainset` | `list[dspy.Example]` | Training examples |\n| `metric` | `callable` | Evaluation function |\n\n## Outputs\n\n| Output | Type | Description |\n|--------|------|-------------|\n| `optimized_prompt` | `str` | DSPy-optimized prompt |\n| `optimized_pipeline` | `Pipeline` | Updated Haystack pipeline |\n\n## Workflow\n\n### Phase 1: Build Initial Haystack Pipeline\n\n```python\nfrom haystack import Pipeline\nfrom haystack.components.generators import OpenAIGenerator\nfrom haystack.components.builders import PromptBuilder\nfrom haystack.components.retrievers.in_memory import InMemoryBM25Retriever\nfrom haystack.document_stores.in_memory import InMemoryDocumentStore\n\n# Setup document store\ndoc_store = InMemoryDocumentStore()\ndoc_store.write_documents(documents)\n\n# Initial generic prompt\ninitial_prompt = \"\"\"\nContext: {{context}}\nQuestion: {{question}}\nAnswer:\n\"\"\"\n\n# Build pipeline\npipeline = Pipeline()\npipeline.add_component(\"retriever\", InMemoryBM25Retriever(document_store=doc_store))\npipeline.add_component(\"prompt_builder\", PromptBuilder(template=initial_prompt))\npipeline.add_component(\"generator\", OpenAIGenerator(model=\"gpt-4o-mini\"))\n\npipeline.connect(\"retriever\", \"prompt_builder.context\")\npipeline.connect(\"prompt_builder\", \"generator\")\n```\n\n### Phase 2: Create DSPy RAG Module\n\n```python\nimport dspy\n\nclass HaystackRAG(dspy.Module):\n    \"\"\"DSPy module wrapping Haystack retriever.\"\"\"\n    \n    def __init__(self, retriever, k=3):\n        super().__init__()\n        self.retriever = retriever\n        self.k = k\n        self.generate = dspy.ChainOfThought(\"context, question -> answer\")\n    \n    def forward(self, question):\n        # Use Haystack retriever\n        results = self.retriever.run(query=question, top_k=self.k)\n        context = [doc.content for doc in results['documents']]\n        \n        # Use DSPy for generation\n        pred = self.generate(context=context, question=question)\n        return dspy.Prediction(context=context, answer=pred.answer)\n```\n\n### Phase 3: Define Custom Metric\n\n```python\nfrom haystack.components.evaluators import SASEvaluator\n\n# Haystack semantic evaluator\nsas_evaluator = SASEvaluator(model=\"sentence-transformers/all-MiniLM-L6-v2\")\n\ndef mixed_metric(example, pred, trace=None):\n    \"\"\"Combine semantic accuracy with conciseness.\"\"\"\n    \n    # Semantic similarity (Haystack SAS)\n    sas_result = sas_evaluator.run(\n        ground_truth_answers=[example.answer],\n        predicted_answers=[pred.answer]\n    )\n    semantic_score = sas_result['score']\n    \n    # Conciseness penalty\n    word_count = len(pred.answer.split())\n    conciseness = 1.0 if word_count <= 20 else max(0, 1 - (word_count - 20) / 50)\n    \n    return 0.7 * semantic_score + 0.3 * conciseness\n```\n\n### Phase 4: Optimize with DSPy\n\n```python\nfrom dspy.teleprompt import BootstrapFewShot\n\nlm = dspy.LM(\"openai/gpt-4o-mini\")\ndspy.configure(lm=lm)\n\n# Create DSPy module with Haystack retriever\nrag_module = HaystackRAG(retriever=pipeline.get_component(\"retriever\"))\n\n# Optimize\noptimizer = BootstrapFewShot(\n    metric=mixed_metric,\n    max_bootstrapped_demos=4,\n    max_labeled_demos=4\n)\n\ncompiled = optimizer.compile(rag_module, trainset=trainset)\n```\n\n### Phase 5: Extract and Apply Optimized Prompt\n\nAfter optimization, extract the optimized prompt and apply it to your Haystack pipeline.\n\nSee [Prompt Extraction Guide](references/prompt-extraction.md) for detailed steps on:\n- Extracting prompts from compiled DSPy modules\n- Mapping DSPy demos to Haystack templates\n- Building optimized Haystack pipelines\n\n## Production Example\n\nFor a complete production-ready implementation, see [HaystackDSPyOptimizer](examples/haystack-dspy-optimizer.py).\n\nThis class provides:\n- Wrapper for Haystack retrievers in DSPy modules\n- Automatic optimization with BootstrapFewShot\n- Prompt extraction and Haystack pipeline rebuilding\n- Complete usage example with document store setup\n\n## Best Practices\n\n1. **Match retrievers** - Use same retriever in DSPy module as Haystack pipeline\n2. **Custom metrics** - Combine Haystack evaluators with DSPy optimization\n3. **Prompt extraction** - Carefully map DSPy demos to Haystack template format\n4. **Test both** - Validate DSPy module AND final Haystack pipeline\n\n## Limitations\n\n- Prompt template conversion can be tricky\n- Some Haystack features don't map directly to DSPy\n- Requires maintaining two codebases initially\n- Complex pipelines may need custom integration\n\n## Official Documentation\n\n- **DSPy Documentation**: https://dspy.ai/\n- **DSPy GitHub**: https://github.com/stanfordnlp/dspy\n- **Haystack Documentation**: https://docs.haystack.deepset.ai/","tags":["dspy","haystack","integration","skills","omidzamani","agent-skills","claude-code","claude-skills","llm","prompt-optimization","rag"],"capabilities":["skill","source-omidzamani","skill-dspy-haystack-integration","topic-agent-skills","topic-claude-code","topic-claude-skills","topic-dspy","topic-llm","topic-prompt-optimization","topic-rag"],"categories":["dspy-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/OmidZamani/dspy-skills/dspy-haystack-integration","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add OmidZamani/dspy-skills","source_repo":"https://github.com/OmidZamani/dspy-skills","install_from":"skills.sh"}},"qualityScore":"0.487","qualityRationale":"deterministic score 0.49 from registry signals: · indexed on github topic:agent-skills · 74 github stars · SKILL.md body (5,074 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-02T06:55:44.422Z","embedding":null,"createdAt":"2026-04-18T22:14:13.391Z","updatedAt":"2026-05-02T06:55:44.422Z","lastSeenAt":"2026-05-02T06:55:44.422Z","tsv":"'/all-minilm-l6-v2':300 '/stanfordnlp/dspy':571 '0':346 '0.3':356 '0.7':353 '1':126,347,493 '1.0':339 '2':210,505 '20':343,350 '3':231,281,514 '4':359,396,400,525 '4o':200 '5':408 '50':351 'accuraci':310 'answer':172,242,278,322,325 'appli':411,421 'ask':13 'automat':55,474 'best':491 'bootstrap':394 'bootstrapfewshot':367,389,477 'build':127,173,448 'builder':188,207 'callabl':104 'capabl':53 'care':517 'class':218,465 'codebas':554 'combin':34,82,308,508 'compil':401,439 'complet':456,484 'complex':556 'compon':84,178,186,194,385 'concis':312,332,338,357 'context':168,169,240,257,270,271,276,277 'convers':538 'count':335,342,349 'creat':211,374 'custom':283,506,560 'data':76 'data-driven':75 'def':226,243,301 'defin':282 'demo':395,399,444,520 'descript':91,110 'detail':433 'direct':548 'doc':157,183,260 'doc.content':258 'doc_store.write':160 'docs.haystack.deepset.ai':574 'document':155,161,162,181,263,488,563,565,573 'driven':77 'dspi':2,16,23,25,35,39,45,50,86,115,212,217,221,265,362,375,440,443,472,500,512,519,529,550,564,567 'dspy-haystack-integr':1 'dspy-optim':114 'dspy.ai':566 'dspy.chainofthought':239 'dspy.configure':371 'dspy.example':100 'dspy.lm':369 'dspy.module':220 'dspy.prediction':275 'dspy.teleprompt':365 'els':344 'evalu':105,292,294,510 'exampl':102,304,453,486 'example.answer':323 'examples/haystack-dspy-optimizer.py':463 'exist':66,95 'extract':38,409,416,429,436,479,516 'featur':544 'final':532 'format':524 'forward':244 'function':106 'generat':195,208,267 'generic':164 'github':568 'github.com':570 'github.com/stanfordnlp/dspy':569 'goal':48 'gpt':199 'gpt-4o-mini':198 'ground':320 'guid':430 'haystack':3,18,20,28,31,37,42,46,59,67,83,92,96,122,129,133,224,248,290,315,378,425,446,450,469,481,503,509,522,533,543,572 'haystack.components.builders':141 'haystack.components.evaluators':287 'haystack.components.generators':137 'haystack.components.retrievers.in':145 'haystack.document_stores.in':150 'haystackdspyoptim':462 'haystackrag':219,382 'implement':460 'import':134,138,142,147,152,216,288,366 'improv':27,56 'init':227,233 'initi':128,163,166,191,555 'inmemorybm25retriever':148,180 'inmemorydocumentstor':153,159 'input':88,89 'integr':4,15,47,561 'k':230,237,255 'label':398 'len':336 'limit':535 'list':99 'lm':368,372,373 'maintain':552 'manual':69 'map':442,518,547 'match':494 'max':345,393,397 'may':558 'memori':146,151 'mention':30 'metric':103,284,303,390,392,507 'mini':201 'mix':302,391 'model':197,296 'modul':214,222,376,381,404,441,473,501,530 'need':74,559 'none':307 'offici':562 'openai/gpt-4o-mini':370 'openaigener':139,196 'optim':19,33,52,79,87,111,116,118,360,387,388,412,415,418,449,475,513 'optimizer.compile':402 'output':107,108 'penalti':333 'phase':125,209,280,358,407 'pipelin':29,32,60,68,93,94,97,119,120,123,130,135,174,175,176,426,451,482,504,534,557 'pipeline.add':177,185,193 'pipeline.connect':202,205 'pipeline.get':384 'practic':492 'pred':268,305 'pred.answer':279,326 'pred.answer.split':337 'predict':324 'product':452,458 'production-readi':457 'prompt':21,40,57,70,78,112,117,165,167,187,192,206,413,419,428,437,478,515,536 'prompt_builder.context':204 'promptbuild':143,189 'provid':466 'python':131,215,285,363 'queri':252 'question':170,171,241,246,253,272,273 'rag':213,380,403 'readi':459 'rebuild':483 'references/prompt-extraction.md':431 'requir':551 'result':250,262,318,330 'retriev':179,203,225,229,235,249,379,383,386,470,495,498 'return':274,352 'sas':293,316,317,329 'sas_evaluator.run':319 'sasevalu':289,295 'score':328,331,355 'see':427,461 'self':228,245 'self.generate':238,269 'self.k':236,256 'self.retriever':234 'self.retriever.run':251 'semant':291,309,313,327,354 'sentenc':298 'sentence-transform':297 'setup':154,490 'similar':314 'skill':6 'skill-dspy-haystack-integration' 'source-omidzamani' 'step':434 'store':156,158,182,184,489 'str':113 'super':232 'tedious':73 'templat':190,447,523,537 'test':526 'top':254 'topic-agent-skills' 'topic-claude-code' 'topic-claude-skills' 'topic-dspy' 'topic-llm' 'topic-prompt-optimization' 'topic-rag' 'trace':306 'train':101 'trainset':98,405,406 'transform':299 'tricki':541 'truth':321 'tune':71 'two':553 'type':90,109 'updat':121 'usag':485 'use':9,22,24,49,63,247,264,496 'user':12 'valid':528 'want':44,80 'word':334,341,348 'workflow':124 'wrap':223 'wrapper':467","prices":[{"id":"7e9bb6a7-168c-407a-8b50-af374e11d4a6","listingId":"2db23d60-5025-44bb-8e4b-60ef52fcc622","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"OmidZamani","category":"dspy-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T22:14:13.391Z"}],"sources":[{"listingId":"2db23d60-5025-44bb-8e4b-60ef52fcc622","source":"github","sourceId":"OmidZamani/dspy-skills/dspy-haystack-integration","sourceUrl":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-haystack-integration","isPrimary":false,"firstSeenAt":"2026-04-18T22:14:13.391Z","lastSeenAt":"2026-05-02T06:55:44.422Z"}],"details":{"listingId":"2db23d60-5025-44bb-8e4b-60ef52fcc622","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"OmidZamani","slug":"dspy-haystack-integration","github":{"repo":"OmidZamani/dspy-skills","stars":74,"topics":["agent-skills","claude-code","claude-skills","dspy","llm","prompt-optimization","rag"],"license":"mit","html_url":"https://github.com/OmidZamani/dspy-skills","pushed_at":"2026-02-21T12:49:43Z","description":"Collection of Claude Skills for DSPy framework - program language models, optimize prompts, and build RAG pipelines systematically","skill_md_sha":"be729e3832290afffe9010145efca8d0a0dcdcf3","skill_md_path":"skills/dspy-haystack-integration/SKILL.md","default_branch":"master","skill_tree_url":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-haystack-integration"},"layout":"multi","source":"github","category":"dspy-skills","frontmatter":{"name":"dspy-haystack-integration","description":"This skill should be used when the user asks to \"integrate DSPy with Haystack\", \"optimize Haystack prompts using DSPy\", \"use DSPy to improve Haystack pipeline\", mentions \"Haystack pipeline optimization\", \"combining DSPy and Haystack\", \"extract DSPy prompt for Haystack\", or wants to use DSPy's optimization capabilities to automatically improve prompts in existing Haystack pipelines."},"skills_sh_url":"https://skills.sh/OmidZamani/dspy-skills/dspy-haystack-integration"},"updatedAt":"2026-05-02T06:55:44.422Z"}}