{"id":"a3692430-0658-46c1-aa74-0560cec5e423","shortId":"dEBsfD","kind":"skill","title":"dspy-bootstrap-fewshot","tagline":"This skill should be used when the user asks to \"bootstrap few-shot examples\", \"generate demonstrations\", \"use BootstrapFewShot\", \"optimize with limited data\", \"create training demos automatically\", mentions \"teacher model for few-shot\", \"10-50 training examples\", or wants automa","description":"# DSPy Bootstrap Few-Shot Optimizer\n\n## Goal\n\nAutomatically generate and select optimal few-shot demonstrations for your DSPy program using a teacher model.\n\n## When to Use\n\n- You have **10-50 labeled examples**\n- Manual example selection is tedious or suboptimal\n- You want demonstrations with reasoning traces\n- Quick optimization without extensive compute\n\n## Related Skills\n\n- For more data (200+ examples): [dspy-miprov2-optimizer](../dspy-miprov2-optimizer/SKILL.md)\n- For agentic systems: [dspy-gepa-reflective](../dspy-gepa-reflective/SKILL.md)\n- Measure improvements: [dspy-evaluation-suite](../dspy-evaluation-suite/SKILL.md)\n\n## Inputs\n\n| Input | Type | Description |\n|-------|------|-------------|\n| `program` | `dspy.Module` | Your DSPy program to optimize |\n| `trainset` | `list[dspy.Example]` | Training examples |\n| `metric` | `callable` | Evaluation function |\n| `metric_threshold` | `float` | Numerical threshold for accepting demos (optional) |\n| `max_bootstrapped_demos` | `int` | Max teacher-generated demos (default: 4) |\n| `max_labeled_demos` | `int` | Max direct labeled demos (default: 16) |\n| `max_rounds` | `int` | Max bootstrapping attempts per example (default: 1) |\n| `teacher_settings` | `dict` | Configuration for teacher model (optional) |\n\n## Outputs\n\n| Output | Type | Description |\n|--------|------|-------------|\n| `compiled_program` | `dspy.Module` | Optimized program with demos |\n\n## Workflow\n\n### Phase 1: Setup\n\n```python\nimport dspy\nfrom dspy.teleprompt import BootstrapFewShot\n\n# Configure LMs\ndspy.configure(lm=dspy.LM(\"openai/gpt-4o-mini\"))\n```\n\n### Phase 2: Define Program and Metric\n\n```python\nclass QA(dspy.Module):\n    def __init__(self):\n        self.generate = dspy.ChainOfThought(\"question -> answer\")\n    \n    def forward(self, question):\n        return self.generate(question=question)\n\ndef validate_answer(example, pred, trace=None):\n    return example.answer.lower() in pred.answer.lower()\n```\n\n### Phase 3: Compile\n\n```python\noptimizer = BootstrapFewShot(\n    metric=validate_answer,\n    max_bootstrapped_demos=4,\n    max_labeled_demos=4,\n    teacher_settings={'lm': dspy.LM(\"openai/gpt-4o\")}\n)\n\ncompiled_qa = optimizer.compile(QA(), trainset=trainset)\n```\n\n### Phase 4: Use and Save\n\n```python\n# Use optimized program\nresult = compiled_qa(question=\"What is photosynthesis?\")\n\n# Save for production (state-only, recommended)\ncompiled_qa.save(\"qa_optimized.json\", save_program=False)\n```\n\n## Production Example\n\n```python\nimport dspy\nfrom dspy.teleprompt import BootstrapFewShot\nfrom dspy.evaluate import Evaluate\nimport logging\n\nlogging.basicConfig(level=logging.INFO)\nlogger = logging.getLogger(__name__)\n\nclass ProductionQA(dspy.Module):\n    def __init__(self):\n        self.cot = dspy.ChainOfThought(\"question -> answer\")\n    \n    def forward(self, question: str):\n        try:\n            return self.cot(question=question)\n        except Exception as e:\n            logger.error(f\"Generation failed: {e}\")\n            return dspy.Prediction(answer=\"Unable to answer\")\n\ndef robust_metric(example, pred, trace=None):\n    if not pred.answer or pred.answer == \"Unable to answer\":\n        return 0.0\n    return float(example.answer.lower() in pred.answer.lower())\n\ndef optimize_with_bootstrap(trainset, devset):\n    \"\"\"Full optimization pipeline with validation.\"\"\"\n    \n    # Baseline\n    baseline = ProductionQA()\n    evaluator = Evaluate(devset=devset, metric=robust_metric, num_threads=4)\n    baseline_score = evaluator(baseline)\n    logger.info(f\"Baseline: {baseline_score:.2%}\")\n    \n    # Optimize\n    optimizer = BootstrapFewShot(\n        metric=robust_metric,\n        max_bootstrapped_demos=4,\n        max_labeled_demos=4\n    )\n    \n    compiled = optimizer.compile(baseline, trainset=trainset)\n    optimized_score = evaluator(compiled)\n    logger.info(f\"Optimized: {optimized_score:.2%}\")\n    \n    if optimized_score > baseline_score:\n        compiled.save(\"production_qa.json\", save_program=False)\n        return compiled\n    \n    logger.warning(\"Optimization didn't improve; keeping baseline\")\n    return baseline\n```\n\n## Best Practices\n\n1. **Quality over quantity** - 10 excellent examples beat 100 noisy ones\n2. **Use stronger teacher** - GPT-4 as teacher for GPT-3.5 student\n3. **Validate with held-out set** - Always test on unseen data\n4. **Start with 4 demos** - More isn't always better\n\n## Limitations\n\n- Requires labeled training data\n- Teacher model costs can add up\n- May not generalize to very different inputs\n- Limited exploration compared to MIPROv2\n\n## Official Documentation\n\n- **DSPy Documentation**: https://dspy.ai/\n- **DSPy GitHub**: https://github.com/stanfordnlp/dspy\n- **BootstrapFewShot API**: https://dspy.ai/api/optimizers/BootstrapFewShot/\n- **Optimization Guide**: https://dspy.ai/learn/optimization/optimizers/","tags":["dspy","bootstrap","fewshot","skills","omidzamani","agent-skills","claude-code","claude-skills","llm","prompt-optimization","rag"],"capabilities":["skill","source-omidzamani","skill-dspy-bootstrap-fewshot","topic-agent-skills","topic-claude-code","topic-claude-skills","topic-dspy","topic-llm","topic-prompt-optimization","topic-rag"],"categories":["dspy-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/OmidZamani/dspy-skills/dspy-bootstrap-fewshot","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add OmidZamani/dspy-skills","source_repo":"https://github.com/OmidZamani/dspy-skills","install_from":"skills.sh"}},"qualityScore":"0.487","qualityRationale":"deterministic score 0.49 from registry signals: · indexed on github topic:agent-skills · 74 github stars · SKILL.md body (4,810 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-02T06:55:43.987Z","embedding":null,"createdAt":"2026-04-18T22:14:08.738Z","updatedAt":"2026-05-02T06:55:43.987Z","lastSeenAt":"2026-05-02T06:55:43.987Z","tsv":"'-3.5':497 '-4':492 '-50':40,76 '/api/optimizers/bootstrapfewshot/':558 '/dspy-evaluation-suite/skill.md':123 '/dspy-gepa-reflective/skill.md':116 '/dspy-miprov2-optimizer/skill.md':108 '/learn/optimization/optimizers/':563 '/stanfordnlp/dspy':553 '0.0':384 '1':183,205,476 '10':39,75,480 '100':484 '16':173 '2':221,423,452,487 '200':102 '3':257,499 '4':163,268,272,285,413,433,437,511,514 'accept':150 'add':530 'agent':110 'alway':506,519 'answer':236,247,264,342,364,367,382 'api':555 'ask':13 'attempt':179 'automa':45 'automat':31,53 'baselin':401,402,414,417,420,421,440,456,471,473 'beat':483 'best':474 'better':520 'bootstrap':3,15,47,154,178,266,393,431 'bootstrapfewshot':23,213,261,320,426,554 'callabl':141 'class':227,333 'compar':541 'compil':196,258,278,294,438,446,464 'compiled.save':458 'compiled_qa.save':307 'comput':96 'configur':187,214 'cost':528 'creat':28 'data':27,101,510,525 'def':230,237,245,336,343,368,390 'default':162,172,182 'defin':222 'demo':30,151,155,161,166,171,202,267,271,432,436,515 'demonstr':21,61,88 'descript':127,195 'devset':395,406,407 'dict':186 'didn':467 'differ':537 'direct':169 'document':545,547 'dspi':2,46,64,105,113,120,131,209,316,546,549 'dspy-bootstrap-fewshot':1 'dspy-evaluation-suit':119 'dspy-gepa-reflect':112 'dspy-miprov2-optimizer':104 'dspy.ai':548,557,562 'dspy.ai/api/optimizers/bootstrapfewshot/':556 'dspy.ai/learn/optimization/optimizers/':561 'dspy.chainofthought':234,340 'dspy.configure':216 'dspy.evaluate':322 'dspy.example':137 'dspy.lm':218,276 'dspy.module':129,198,229,335 'dspy.prediction':363 'dspy.teleprompt':211,318 'e':356,361 'evalu':121,142,324,404,405,416,445 'exampl':19,42,78,80,103,139,181,248,313,371,482 'example.answer.lower':253,387 'excel':481 'except':353,354 'explor':540 'extens':95 'f':358,419,448 'fail':360 'fals':311,462 'few-shot':16,36,48,58 'fewshot':4 'float':146,386 'forward':238,344 'full':396 'function':143 'general':534 'generat':20,54,160,359 'gepa':114 'github':550 'github.com':552 'github.com/stanfordnlp/dspy':551 'goal':52 'gpt':491,496 'guid':560 'held':503 'held-out':502 'import':208,212,315,319,323,325 'improv':118,469 'init':231,337 'input':124,125,538 'int':156,167,176 'isn':517 'keep':470 'label':77,165,170,270,435,523 'level':328 'limit':26,521,539 'list':136 'lm':217,275 'lms':215 'log':326 'logger':330 'logger.error':357 'logger.info':418,447 'logger.warning':465 'logging.basicconfig':327 'logging.getlogger':331 'logging.info':329 'manual':79 'max':153,157,164,168,174,177,265,269,430,434 'may':532 'measur':117 'mention':32 'metric':140,144,225,262,370,408,410,427,429 'miprov2':106,543 'model':34,69,190,527 'name':332 'noisi':485 'none':251,374 'num':411 'numer':147 'offici':544 'one':486 'openai/gpt-4o':277 'openai/gpt-4o-mini':219 'optim':24,51,57,93,107,134,199,260,291,391,397,424,425,443,449,450,454,466,559 'optimizer.compile':280,439 'option':152,191 'output':192,193 'per':180 'phase':204,220,256,284 'photosynthesi':299 'pipelin':398 'practic':475 'pred':249,372 'pred.answer':377,379 'pred.answer.lower':255,389 'product':302,312 'production_qa.json':459 'productionqa':334,403 'program':65,128,132,197,200,223,292,310,461 'python':207,226,259,289,314 'qa':228,279,281,295 'qa_optimized.json':308 'qualiti':477 'quantiti':479 'question':235,240,243,244,296,341,346,351,352 'quick':92 'reason':90 'recommend':306 'reflect':115 'relat':97 'requir':522 'result':293 'return':241,252,349,362,383,385,463,472 'robust':369,409,428 'round':175 'save':288,300,309,460 'score':415,422,444,451,455,457 'select':56,81 'self':232,239,338,345 'self.cot':339,350 'self.generate':233,242 'set':185,274,505 'setup':206 'shot':18,38,50,60 'skill':6,98 'skill-dspy-bootstrap-fewshot' 'source-omidzamani' 'start':512 'state':304 'state-on':303 'str':347 'stronger':489 'student':498 'suboptim':85 'suit':122 'system':111 'teacher':33,68,159,184,189,273,490,494,526 'teacher-gener':158 'tedious':83 'test':507 'thread':412 'threshold':145,148 'topic-agent-skills' 'topic-claude-code' 'topic-claude-skills' 'topic-dspy' 'topic-llm' 'topic-prompt-optimization' 'topic-rag' 'trace':91,250,373 'train':29,41,138,524 'trainset':135,282,283,394,441,442 'tri':348 'type':126,194 'unabl':365,380 'unseen':509 'use':9,22,66,72,286,290,488 'user':12 'valid':246,263,400,500 'want':44,87 'without':94 'workflow':203","prices":[{"id":"7a691b8a-76dc-4f74-b93a-7dcd80d0aa44","listingId":"a3692430-0658-46c1-aa74-0560cec5e423","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"OmidZamani","category":"dspy-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T22:14:08.738Z"}],"sources":[{"listingId":"a3692430-0658-46c1-aa74-0560cec5e423","source":"github","sourceId":"OmidZamani/dspy-skills/dspy-bootstrap-fewshot","sourceUrl":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-bootstrap-fewshot","isPrimary":false,"firstSeenAt":"2026-04-18T22:14:08.738Z","lastSeenAt":"2026-05-02T06:55:43.987Z"}],"details":{"listingId":"a3692430-0658-46c1-aa74-0560cec5e423","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"OmidZamani","slug":"dspy-bootstrap-fewshot","github":{"repo":"OmidZamani/dspy-skills","stars":74,"topics":["agent-skills","claude-code","claude-skills","dspy","llm","prompt-optimization","rag"],"license":"mit","html_url":"https://github.com/OmidZamani/dspy-skills","pushed_at":"2026-02-21T12:49:43Z","description":"Collection of Claude Skills for DSPy framework - program language models, optimize prompts, and build RAG pipelines systematically","skill_md_sha":"c11cd2cde837239d95c4e1203ebf807ca8f37fbe","skill_md_path":"skills/dspy-bootstrap-fewshot/SKILL.md","default_branch":"master","skill_tree_url":"https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-bootstrap-fewshot"},"layout":"multi","source":"github","category":"dspy-skills","frontmatter":{"name":"dspy-bootstrap-fewshot","description":"This skill should be used when the user asks to \"bootstrap few-shot examples\", \"generate demonstrations\", \"use BootstrapFewShot\", \"optimize with limited data\", \"create training demos automatically\", mentions \"teacher model for few-shot\", \"10-50 training examples\", or wants automatic demonstration generation for a DSPy program without extensive compute."},"skills_sh_url":"https://skills.sh/OmidZamani/dspy-skills/dspy-bootstrap-fewshot"},"updatedAt":"2026-05-02T06:55:43.987Z"}}