{"id":"ce62eba7-8b3c-4b7f-b2f9-9fe2180d5bed","shortId":"9KJNB9","kind":"skill","title":"dspy-custom-module-design","tagline":"This skill should be used when the user asks to \"create custom DSPy module\", \"design a DSPy module\", \"extend dspy.Module\", \"build reusable DSPy component\", mentions \"custom module patterns\", \"module serialization\", \"stateful modules\", \"module testing\", or needs to design producti","description":"# DSPy Custom Module Design\n\n## Goal\n\nDesign production-quality custom DSPy modules with proper architecture, state management, serialization, and testing patterns.\n\n## When to Use\n\n- Building reusable DSPy components\n- Complex logic beyond built-in modules\n- Need custom state management\n- Sharing modules across projects\n- Production deployment requirements\n\n## Related Skills\n\n- Module composition: [dspy-advanced-module-composition](../dspy-advanced-module-composition/SKILL.md)\n- Signature design: [dspy-signature-designer](../dspy-signature-designer/SKILL.md)\n- Optimization: [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md)\n\n## Inputs\n\n| Input | Type | Description |\n|-------|------|-------------|\n| `task_description` | `str` | What the module should do |\n| `components` | `list` | Sub-modules or predictors |\n| `state` | `dict` | Stateful attributes |\n\n## Outputs\n\n| Output | Type | Description |\n|--------|------|-------------|\n| `custom_module` | `dspy.Module` | Production-ready module |\n\n## Workflow\n\n### Phase 1: Basic Module Structure\n\nAll custom modules inherit from `dspy.Module`:\n\n```python\nimport dspy\n\nclass BasicQA(dspy.Module):\n    \"\"\"Simple question answering module.\"\"\"\n\n    def __init__(self):\n        super().__init__()\n        self.predictor = dspy.Predict(\"question -> answer\")\n\n    def forward(self, question):\n        \"\"\"Entry point for module execution.\"\"\"\n        return self.predictor(question=question)\n\n# Usage\ndspy.configure(lm=dspy.LM(\"openai/gpt-4o-mini\"))\nqa = BasicQA()\nresult = qa(question=\"What is Python?\")\nprint(result.answer)\n```\n\n### Phase 2: Stateful Modules\n\nModules can maintain state across calls:\n\n```python\nimport dspy\nimport logging\n\nlogger = logging.getLogger(__name__)\n\nclass StatefulRAG(dspy.Module):\n    \"\"\"RAG with query caching.\"\"\"\n\n    def __init__(self, cache_size=100):\n        super().__init__()\n        self.retrieve = dspy.Retrieve(k=3)\n        self.generate = dspy.ChainOfThought(\"context, question -> answer\")\n        self.cache = {}\n        self.cache_size = cache_size\n\n    def forward(self, question):\n        # Check cache\n        if question in self.cache:\n            return self.cache[question]\n\n        # Retrieve and generate\n        passages = self.retrieve(question).passages\n        result = self.generate(context=passages, question=question)\n\n        # Update cache with size limit\n        if len(self.cache) >= self.cache_size:\n            self.cache.pop(next(iter(self.cache)))\n        self.cache[question] = result\n\n        return result\n```\n\n### Phase 3: Error Handling and Validation\n\nProduction modules need robust error handling:\n\n```python\nimport dspy\nfrom typing import Optional\nimport logging\n\nlogger = logging.getLogger(__name__)\n\nclass RobustClassifier(dspy.Module):\n    \"\"\"Classifier with validation.\"\"\"\n\n    def __init__(self, valid_labels: list[str]):\n        super().__init__()\n        self.valid_labels = set(valid_labels)\n        self.classify = dspy.Predict(\"text -> label: str, confidence: float\")\n\n    def forward(self, text: str) -> dspy.Prediction:\n        if not text or not text.strip():\n            return dspy.Prediction(label=\"unknown\", confidence=0.0, error=\"Empty input\")\n\n        try:\n            result = self.classify(text=text)\n\n            # Validate label\n            if result.label not in self.valid_labels:\n                result.label = \"unknown\"\n                result.confidence = 0.0\n\n            return result\n\n        except Exception as e:\n            logger.error(f\"Classification failed: {e}\")\n            return dspy.Prediction(label=\"unknown\", confidence=0.0, error=str(e))\n```\n\n### Phase 4: Serialization\n\nModules support save/load:\n\n```python\nimport dspy\n\n# Save module state\nmodule = MyCustomModule()\nmodule.save(\"my_module.json\")\n\n# Load requires creating instance first, then loading state\nloaded = MyCustomModule()\nloaded.load(\"my_module.json\")\n\n# For loading entire programs (dspy>=2.6.0)\nmodule.save(\"./my_module/\", save_program=True)\nloaded = dspy.load(\"./my_module/\")\n```\n\n## Production Example\n\n```python\nimport dspy\nfrom typing import List, Optional\nimport logging\n\nlogger = logging.getLogger(__name__)\n\nclass ProductionRAG(dspy.Module):\n    \"\"\"Production-ready RAG with all best practices.\"\"\"\n\n    def __init__(\n        self,\n        retriever_k: int = 5,\n        cache_enabled: bool = True,\n        cache_size: int = 1000\n    ):\n        super().__init__()\n\n        # Configuration\n        self.retriever_k = retriever_k\n        self.cache_enabled = cache_enabled\n        self.cache_size = cache_size\n\n        # Components\n        self.retrieve = dspy.Retrieve(k=retriever_k)\n        self.generate = dspy.ChainOfThought(\"context, question -> answer\")\n\n        # State\n        self.cache = {} if cache_enabled else None\n        self.call_count = 0\n\n    def forward(self, question: str) -> dspy.Prediction:\n        \"\"\"Execute RAG pipeline with caching.\"\"\"\n        self.call_count += 1\n\n        # Validation\n        if not question or not question.strip():\n            return dspy.Prediction(\n                answer=\"Please provide a valid question.\",\n                error=\"Invalid input\"\n            )\n\n        # Cache check\n        if self.cache_enabled and question in self.cache:\n            logger.info(f\"Cache hit (call #{self.call_count})\")\n            return self.cache[question]\n\n        # Execute pipeline\n        try:\n            passages = self.retrieve(question).passages\n\n            if not passages:\n                logger.warning(\"No passages retrieved\")\n                return dspy.Prediction(\n                    answer=\"No relevant information found.\",\n                    passages=[]\n                )\n\n            result = self.generate(context=passages, question=question)\n            result.passages = passages\n\n            # Update cache\n            if self.cache_enabled:\n                self._update_cache(question, result)\n\n            return result\n\n        except Exception as e:\n            logger.error(f\"RAG execution failed: {e}\")\n            return dspy.Prediction(\n                answer=\"An error occurred while processing your question.\",\n                error=str(e)\n            )\n\n    def _update_cache(self, key: str, value: dspy.Prediction):\n        \"\"\"Manage cache with size limit.\"\"\"\n        if len(self.cache) >= self.cache_size:\n            self.cache.pop(next(iter(self.cache)))\n        self.cache[key] = value\n\n    def clear_cache(self):\n        \"\"\"Clear cache.\"\"\"\n        if self.cache_enabled:\n            self.cache.clear()\n```\n\n## Best Practices\n\n1. **Single responsibility** - Each module does one thing well\n2. **Validate inputs** - Check for None, empty strings, invalid types\n3. **Handle errors** - Return Predictions with error fields, never raise\n4. **Log important events** - Cache hits, errors, validation failures\n5. **Test independently** - Unit test modules before composition\n\n## Limitations\n\n- State increases memory usage (careful with large caches)\n- Serialization doesn't automatically save custom state\n- Module testing requires mocking LM calls\n- Deep module hierarchies can be hard to debug\n- Performance overhead from validation in hot paths\n\n## Official Documentation\n\n- **DSPy Documentation**: https://dspy.ai/\n- **DSPy GitHub**: https://github.com/stanfordnlp/dspy\n- **Custom Modules Guide**: https://dspy.ai/tutorials/custom_module/\n- **Module API**: https://dspy.ai/api/modules/","tags":["dspy","custom","module","design","skills","omidzamani","agent-skills","claude-code","claude-skills","llm","prompt-optimization","rag"],"capabilities":["skill","source-omidzamani","skill-dspy-custom-module-design","topic-agent-skills","topic-claude-code","topic-claude-skills","topic-dspy","topic-llm","topic-prompt-optimization","topic-rag"],"categories":["dspy-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/OmidZamani/dspy-skills/dspy-custom-module-design","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add OmidZamani/dspy-skills","source_repo":"https://github.com/OmidZamani/dspy-skills","install_from":"skills.sh"}},"qualityScore":"0.487","qualityRationale":"deterministic score 0.49 from registry signals: · indexed on github topic:agent-skills · 74 github stars · SKILL.md body (7,407 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-02T06:55:44.067Z","embedding":null,"createdAt":"2026-04-18T22:14:09.503Z","updatedAt":"2026-05-02T06:55:44.067Z","lastSeenAt":"2026-05-02T06:55:44.067Z","tsv":"'/api/modules/':781 '/dspy-advanced-module-composition/skill.md':100 '/dspy-miprov2-optimizer/skill.md':113 '/dspy-signature-designer/skill.md':107 '/my_module':443,449 '/stanfordnlp/dspy':770 '/tutorials/custom_module/':776 '0':526 '0.0':367,387,404 '1':150,540,678 '100':237 '1000':490 '2':208,687 '2.6.0':441 '3':243,300,697 '4':409,707 '5':482,716 'across':86,215 'advanc':97 'answer':168,178,248,516,550,594,630 'api':778 'architectur':59 'ask':14 'attribut':136 'automat':736 'basic':151 'basicqa':164,198 'best':474,676 'beyond':75 'bool':485 'build':26,69 'built':77 'built-in':76 'cach':231,235,252,259,281,483,487,500,504,520,537,559,570,609,643,650,668,671,711,732 'call':216,572,745 'care':729 'check':258,560,690 'class':163,225,323,465 'classif':396 'classifi':326 'clear':667,670 'complex':73 'compon':29,72,126,506 'composit':94,99,723 'confid':348,366,403 'configur':493 'context':246,276,514,602 'count':525,539,574 'creat':16,426 'custom':3,17,31,46,54,81,141,155,738,771 'debug':753 'deep':746 'def':170,179,232,254,329,350,476,527,641,666 'deploy':89 'descript':117,119,140 'design':5,20,43,48,50,102,106 'dict':134 'document':762,764 'doesn':734 'dspi':2,18,22,28,45,55,71,96,104,110,162,219,313,416,440,454,763,766 'dspy-advanced-module-composit':95 'dspy-custom-module-design':1 'dspy-miprov2-optimizer':109 'dspy-signature-design':103 'dspy.ai':765,775,780 'dspy.ai/api/modules/':779 'dspy.ai/tutorials/custom_module/':774 'dspy.chainofthought':245,513 'dspy.configure':193 'dspy.lm':195 'dspy.load':448 'dspy.module':25,143,159,165,227,325,467 'dspy.predict':176,344 'dspy.prediction':355,363,400,532,549,593,629,648 'dspy.retrieve':241,508 'e':393,398,407,621,627,640 'els':522 'empti':369,693 'enabl':484,499,501,521,563,612,674 'entir':438 'entri':183 'error':301,309,368,405,556,632,638,699,703,713 'event':710 'exampl':451 'except':390,391,618,619 'execut':187,533,578,625 'extend':24 'f':395,569,623 'fail':397,626 'failur':715 'field':704 'first':428 'float':349 'forward':180,255,351,528 'found':598 'generat':269 'github':767 'github.com':769 'github.com/stanfordnlp/dspy':768 'goal':49 'guid':773 'handl':302,310,698 'hard':751 'hierarchi':748 'hit':571,712 'hot':759 'import':161,218,220,312,316,318,415,453,457,460,709 'increas':726 'independ':718 'inform':597 'inherit':157 'init':171,174,233,239,330,337,477,492 'input':114,115,370,558,689 'instanc':427 'int':481,489 'invalid':557,695 'iter':292,661 'k':242,480,495,497,509,511 'key':645,664 'label':333,339,342,346,364,377,383,401 'larg':731 'len':286,655 'limit':284,653,724 'list':127,334,458 'lm':194,744 'load':424,430,432,437,447 'loaded.load':434 'log':221,319,461,708 'logger':222,320,462 'logger.error':394,622 'logger.info':568 'logger.warning':588 'logging.getlogger':223,321,463 'logic':74 'maintain':213 'manag':61,83,649 'memori':727 'mention':30 'miprov2':111 'mock':743 'modul':4,19,23,32,34,37,38,47,56,79,85,93,98,123,130,142,147,152,156,169,186,210,211,306,411,418,420,682,721,740,747,772,777 'module.save':422,442 'my_module.json':423,435 'mycustommodul':421,433 'name':224,322,464 'need':41,80,307 'never':705 'next':291,660 'none':523,692 'occur':633 'offici':761 'one':684 'openai/gpt-4o-mini':196 'optim':108,112 'option':317,459 'output':137,138 'overhead':755 'passag':270,273,277,581,584,587,590,599,603,607 'path':760 'pattern':33,65 'perform':754 'phase':149,207,299,408 'pipelin':535,579 'pleas':551 'point':184 'practic':475,677 'predict':701 'predictor':132 'print':205 'process':635 'product':52,88,145,305,450,469 'producti':44 'production-qu':51 'production-readi':144,468 'productionrag':466 'program':439,445 'project':87 'proper':58 'provid':552 'python':160,204,217,311,414,452 'qa':197,200 'qualiti':53 'queri':230 'question':167,177,182,190,191,201,247,257,261,266,272,278,279,295,515,530,544,555,565,577,583,604,605,614,637 'question.strip':547 'rag':228,471,534,624 'rais':706 'readi':146,470 'relat':91 'relev':596 'requir':90,425,742 'respons':680 'result':199,274,296,298,372,389,600,615,617 'result.answer':206 'result.confidence':386 'result.label':379,384 'result.passages':606 'retriev':267,479,496,510,591 'return':188,264,297,362,388,399,548,575,592,616,628,700 'reusabl':27,70 'robust':308 'robustclassifi':324 'save':417,444,737 'save/load':413 'self':172,181,234,256,331,352,478,529,644,669 'self._update_cache':613 'self.cache':249,250,263,265,287,288,293,294,498,502,518,562,567,576,611,656,657,662,663,673 'self.cache.clear':675 'self.cache.pop':290,659 'self.call':524,538,573 'self.classify':343,373 'self.generate':244,275,512,601 'self.predictor':175,189 'self.retrieve':240,271,507,582 'self.retriever':494 'self.valid':338,382 'serial':35,62,410,733 'set':340 'share':84 'signatur':101,105 'simpl':166 'singl':679 'size':236,251,253,283,289,488,503,505,652,658 'skill':7,92 'skill-dspy-custom-module-design' 'source-omidzamani' 'state':36,60,82,133,135,209,214,419,431,517,725,739 'statefulrag':226 'str':120,335,347,354,406,531,639,646 'string':694 'structur':153 'sub':129 'sub-modul':128 'super':173,238,336,491 'support':412 'task':118 'test':39,64,717,720,741 'text':345,353,358,374,375 'text.strip':361 'thing':685 'topic-agent-skills' 'topic-claude-code' 'topic-claude-skills' 'topic-dspy' 'topic-llm' 'topic-prompt-optimization' 'topic-rag' 'tri':371,580 'true':446,486 'type':116,139,315,456,696 'unit':719 'unknown':365,385,402 'updat':280,608,642 'usag':192,728 'use':10,68 'user':13 'valid':304,328,332,341,376,541,554,688,714,757 'valu':647,665 'well':686 'workflow':148","prices":[{"id":"ddbb4817-3549-4ed9-970e-66502c495a74","listingId":"ce62eba7-8b3c-4b7f-b2f9-9fe2180d5bed","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"OmidZamani","category":"dspy-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T22:14:09.503Z"}],"sources":[{"listingId":"ce62eba7-8b3c-4b7f-b2f9-9fe2180d5bed","source":"github","sourceId":"OmidZamani/dspy-skills/dspy-custom-module-design","sourceUrl":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-custom-module-design","isPrimary":false,"firstSeenAt":"2026-04-18T22:14:09.503Z","lastSeenAt":"2026-05-02T06:55:44.067Z"}],"details":{"listingId":"ce62eba7-8b3c-4b7f-b2f9-9fe2180d5bed","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"OmidZamani","slug":"dspy-custom-module-design","github":{"repo":"OmidZamani/dspy-skills","stars":74,"topics":["agent-skills","claude-code","claude-skills","dspy","llm","prompt-optimization","rag"],"license":"mit","html_url":"https://github.com/OmidZamani/dspy-skills","pushed_at":"2026-02-21T12:49:43Z","description":"Collection of Claude Skills for DSPy framework - program language models, optimize prompts, and build RAG pipelines systematically","skill_md_sha":"b418e67fd94f7e0435eb783565f4298e1af29443","skill_md_path":"skills/dspy-custom-module-design/SKILL.md","default_branch":"master","skill_tree_url":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-custom-module-design"},"layout":"multi","source":"github","category":"dspy-skills","frontmatter":{"name":"dspy-custom-module-design","description":"This skill should be used when the user asks to \"create custom DSPy module\", \"design a DSPy module\", \"extend dspy.Module\", \"build reusable DSPy component\", mentions \"custom module patterns\", \"module serialization\", \"stateful modules\", \"module testing\", or needs to design production-quality custom DSPy modules with proper architecture, state management, and testing."},"skills_sh_url":"https://skills.sh/OmidZamani/dspy-skills/dspy-custom-module-design"},"updatedAt":"2026-05-02T06:55:44.067Z"}}