{"id":"faeaabfe-2765-4938-a77b-e09a485f2e19","shortId":"3s7pgQ","kind":"skill","title":"pilot-map-reduce","tagline":"Distributed map-reduce over agent swarms for parallel data processing.  Use this skill when: 1. You need to process large datasets across multiple workers 2. You want parallel map phase followed by aggregating reduce phase 3. You have embarrassingly parallel tasks with combine st","description":"# pilot-map-reduce\n\nImplement distributed map-reduce patterns for parallel data processing across agent swarms.\n\n## Commands\n\n### Submit map tasks to workers\n```bash\nTOTAL_WORKERS=$(pilotctl --json peers --search \"role:mapper\" | jq 'length')\n\nfor i in $(seq 0 $((TOTAL_WORKERS - 1))); do\n  WORKER=$(pilotctl --json peers --search \"role:mapper\" | jq -r \".[$i].address\")\n\n  pilotctl --json send-message \"$WORKER\" \\\n    --data \"{\\\"type\\\":\\\"map_task\\\",\\\"job_id\\\":\\\"$JOB_ID\\\",\\\"chunk_start\\\":$((i * 1000)),\\\"chunk_end\\\":$(((i + 1) * 1000))}\"\ndone\n```\n\n### Collect map results\n```bash\nEXPECTED_RESULTS=$TOTAL_WORKERS\nRECEIVED=0\n\nwhile [ $RECEIVED -lt $EXPECTED_RESULTS ]; do\n  RESULTS=$(pilotctl --json received \\\n    | jq '[.messages[] | select(.payload.type == \"map_result\" and .payload.job_id == \"'$JOB_ID'\")] | length')\n  RECEIVED=$RESULTS\n  sleep 1\ndone\n```\n\n### Shuffle and reduce\n```bash\nMAP_RESULTS=$(cat /tmp/map-results-$JOB_ID.json)\nKEYS=$(echo \"$MAP_RESULTS\" | jq -r '.[].payload.results | to_entries | .[].key' | sort -u)\n\nfor key in $KEYS; do\n  VALUES=$(echo \"$MAP_RESULTS\" | jq -r '[.[].payload.results.\"'$key'\" // empty] | flatten')\n\n  pilotctl --json send-message \"$REDUCER\" \\\n    --data \"{\\\"type\\\":\\\"reduce_task\\\",\\\"job_id\\\":\\\"$JOB_ID\\\",\\\"key\\\":\\\"$key\\\",\\\"values\\\":$VALUES}\"\ndone\n```\n\n## Workflow Example\n\nWord count across distributed text corpus:\n\n```bash\n#!/bin/bash\nJOB_ID=\"wordcount-$(date +%s)\"\n\n# MAP phase\nMAPPERS=$(pilotctl --json peers --search \"role:mapper\" | jq -r '.[].address')\nfor i in $(seq 0 9); do\n  pilotctl --json send-message \"${MAPPERS[$i]}\" \\\n    --data \"{\\\"type\\\":\\\"map_task\\\",\\\"job_id\\\":\\\"$JOB_ID\\\",\\\"chunk\\\":$i}\" &\ndone\nwait\n\n# REDUCE phase\nsleep 5\nMAP_RESULTS=$(pilotctl --json received \\\n  | jq '[.messages[] | select(.payload.type == \"map_result\")]')\n\nFINAL=$(echo \"$MAP_RESULTS\" | jq 'map({(.payload.word): .payload.count}) | add')\necho \"$FINAL\"\n```\n\n## Dependencies\n\nRequires pilot-protocol skill, jq, and sort.","tags":["pilot","map","reduce","skills","teoslayer","agent-skills","ai-agents","clawhub","networking","openclaw","overlay-network","p2p"],"capabilities":["skill","source-teoslayer","skill-pilot-map-reduce","topic-agent-skills","topic-ai-agents","topic-clawhub","topic-networking","topic-openclaw","topic-overlay-network","topic-p2p","topic-pilot-protocol"],"categories":["pilot-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/TeoSlayer/pilot-skills/pilot-map-reduce","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add TeoSlayer/pilot-skills","source_repo":"https://github.com/TeoSlayer/pilot-skills","install_from":"skills.sh"}},"qualityScore":"0.453","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 6 github stars · SKILL.md body (1,954 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:14:57.735Z","embedding":null,"createdAt":"2026-05-18T13:22:43.258Z","updatedAt":"2026-05-18T19:14:57.735Z","lastSeenAt":"2026-05-18T19:14:57.735Z","tsv":"'/bin/bash':229 '/tmp/map-results-':172 '0':88,137,251 '1':20,91,125,163 '1000':121,126 '2':30 '3':41 '5':276 '9':252 'across':27,64,224 'add':296 'address':103,246 'agent':10,65 'aggreg':38 'bash':73,131,168,228 'cat':171 'chunk':118,122,269 'collect':128 'combin':48 'command':67 'corpus':227 'count':223 'data':14,62,110,207,261 'dataset':26 'date':233 'depend':299 'distribut':5,55,225 'done':127,164,219,271 'echo':175,192,289,297 'embarrass':44 'empti':199 'end':123 'entri':182 'exampl':221 'expect':132,141 'final':288,298 'flatten':200 'follow':36 'id':115,117,156,158,212,214,231,266,268 'implement':54 'job':114,116,157,211,213,230,265,267 'job_id.json':173 'jq':82,100,148,178,195,244,282,292,305 'json':77,95,105,146,202,239,255,280 'key':174,183,187,189,198,215,216 'larg':25 'length':83,159 'lt':140 'map':3,7,34,52,57,69,112,129,152,169,176,193,235,263,277,286,290,293 'map-reduc':6,56 'mapper':81,99,237,243,259 'messag':108,149,205,258,283 'multipl':28 'need':22 'parallel':13,33,45,61 'pattern':59 'payload.count':295 'payload.job':155 'payload.results':180,197 'payload.type':151,285 'payload.word':294 'peer':78,96,240 'phase':35,40,236,274 'pilot':2,51,302 'pilot-map-reduc':1,50 'pilot-protocol':301 'pilotctl':76,94,104,145,201,238,254,279 'process':15,24,63 'protocol':303 'r':101,179,196,245 'receiv':136,139,147,160,281 'reduc':4,8,39,53,58,167,206,209,273 'requir':300 'result':130,133,142,144,153,161,170,177,194,278,287,291 'role':80,98,242 'search':79,97,241 'select':150,284 'send':107,204,257 'send-messag':106,203,256 'seq':87,250 'shuffl':165 'skill':18,304 'skill-pilot-map-reduce' 'sleep':162,275 'sort':184,307 'source-teoslayer' 'st':49 'start':119 'submit':68 'swarm':11,66 'task':46,70,113,210,264 'text':226 'topic-agent-skills' 'topic-ai-agents' 'topic-clawhub' 'topic-networking' 'topic-openclaw' 'topic-overlay-network' 'topic-p2p' 'topic-pilot-protocol' 'total':74,89,134 'type':111,208,262 'u':185 'use':16 'valu':191,217,218 'wait':272 'want':32 'word':222 'wordcount':232 'worker':29,72,75,90,93,109,135 'workflow':220","prices":[{"id":"3a6793d3-d16d-4e26-ae30-f0f1cd6046e5","listingId":"faeaabfe-2765-4938-a77b-e09a485f2e19","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"TeoSlayer","category":"pilot-skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:22:43.258Z"}],"sources":[{"listingId":"faeaabfe-2765-4938-a77b-e09a485f2e19","source":"github","sourceId":"TeoSlayer/pilot-skills/pilot-map-reduce","sourceUrl":"https://github.com/TeoSlayer/pilot-skills/tree/main/skills/pilot-map-reduce","isPrimary":false,"firstSeenAt":"2026-05-18T13:22:43.258Z","lastSeenAt":"2026-05-18T19:14:57.735Z"}],"details":{"listingId":"faeaabfe-2765-4938-a77b-e09a485f2e19","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"TeoSlayer","slug":"pilot-map-reduce","github":{"repo":"TeoSlayer/pilot-skills","stars":6,"topics":["agent-skills","ai-agents","clawhub","networking","openclaw","overlay-network","p2p","pilot-protocol"],"license":"agpl-3.0","html_url":"https://github.com/TeoSlayer/pilot-skills","pushed_at":"2026-05-13T06:08:49Z","description":"80+ agent skills for Pilot Protocol — communication, file transfer, trust, task routing, swarm coordination, and more","skill_md_sha":"35255600257f6e9f8acc8108d96dd2b83c6a5630","skill_md_path":"skills/pilot-map-reduce/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/TeoSlayer/pilot-skills/tree/main/skills/pilot-map-reduce"},"layout":"multi","source":"github","category":"pilot-skills","frontmatter":{"name":"pilot-map-reduce","license":"AGPL-3.0","description":"Distributed map-reduce over agent swarms for parallel data processing.  Use this skill when: 1. You need to process large datasets across multiple workers 2. You want parallel map phase followed by aggregating reduce phase 3. You have embarrassingly parallel tasks with combine step  Do NOT use this skill when: - Tasks are not parallelizable (use single worker) - You need streaming results (use pilot-load-balancer)","compatibility":"Requires pilot-protocol skill and pilotctl binary on PATH. The daemon must be running (pilotctl daemon start)."},"skills_sh_url":"https://skills.sh/TeoSlayer/pilot-skills/pilot-map-reduce"},"updatedAt":"2026-05-18T19:14:57.735Z"}}