{"id":"9ea93634-8394-427f-8b80-e30931626a8c","shortId":"SKYqqk","kind":"skill","title":"recsys-pipeline-architect","tagline":"Designs composable recommendation, ranking, and feed pipelines using the six-stage Source→Hydrator→Filter→Scorer→Selector→SideEffect framework","description":"# recsys-pipeline-architect\n\n## Overview\n\nA spec-and-scaffold skill for building composable recommendation, ranking, and feed pipelines. It encodes the six-stage **Source → Hydrator → Filter → Scorer → Selector → SideEffect** framework popularized by xAI's open-sourced [For You algorithm](https://github.com/xai-org/x-algorithm) (Apache 2.0). This skill is an independent reimplementation of the *pattern* — no code is copied from the original — licensed MIT. Use it whenever you need \"the top K items for a (user, context)\": social feeds, content CMSs, RAG rerankers, task prioritizers, notification triage, search reranking, ad ranking.\n\n## When to Use This Skill\n\n- Use when the user wants to build any system that picks \"the top K items for a user/context\"\n- Use when the user asks \"how should I rank X\" or describes a feed/personalization problem\n- Use when the user has a scoring function and needs the pipeline plumbing around it\n- Use when the user wants to migrate from a single relevance score to multi-action prediction with tunable weights\n- Use when the user is wrapping an LLM/ML scorer and needs filters, hydrators, side-effects, and a runnable scaffold in their stack (TypeScript / Go / Python)\n\n## How It Works\n\n### Step 1: Clarify the use case\n\nAsk the user three questions (only what is missing):\n\n1. What are the items being ranked? (posts, products, tasks, alerts, documents...)\n2. What is the input context? (user ID, search query, current document, time window...)\n3. What language / runtime? (TypeScript/Node, Go, Python, Rust...)\n\n### Step 2: Walk the eight steps of the spec\n\nThe full SKILL walks through: clarify use case → identify candidate sources → list required hydrations → list filters → design scorer chain → selector → side effects → generate scaffold. Each step surfaces the architectural trade-offs (multi-action vs single-score, candidate isolation vs joint scoring, online vs offline batch) so the user makes them explicitly rather than defaulting silently.\n\n### Step 3: Emit a runnable scaffold\n\nThe upstream repository ships three runnable example scaffolds — every one green on its test suite:\n\n- **Strapi v5 plugin** (TypeScript, Jest, 3/3 pass) — adds `GET /api/feed/for-you` with multi-action scoring and author diversity\n- **Zentra-compatible pipeline** (Go with generics, 3/3 pass) — engine.Module-compatible, standalone-usable\n- **PMAI task prioritizer** (Python / FastAPI / pytest, 3/3 pass) — `GET /tasks/next?user_id=42&limit=10`\n\nWhen the user's stack doesn't match, the skill generates from scratch following the interface definitions in `references/interfaces.md` (TypeScript, Go, Python, Rust).\n\n## Examples\n\n### Example 1: Strapi content feed\n\nUser: \"I'm running a Strapi v5 instance with 50k articles. I want a 'for you' feed personalized to each logged-in user based on their reading history.\"\n\nSkill walks through the 8 steps, generates a Strapi plugin scaffold using the Strapi example as the template.\n\n### Example 2: RAG retrieval reranker\n\nUser: \"My RAG returns top-50 chunks from a vector DB. I want to rerank them with a more expensive scorer and return top-5.\"\n\nSkill recognizes this as a single-source pipeline with a scorer chain (cheap retrieval + expensive rerank). Generates a Python async pipeline.\n\n### Example 3: Notification triage\n\nUser: \"We send too many notifications. I want a daily digest that picks the top 10 from the last 24h queue.\"\n\nSkill identifies this as an offline-batch pipeline. Generates a scheduled job scaffold.\n\n## Best Practices\n\n- ✅ Surface the multi-action vs single-score trade-off explicitly — don't default silently\n- ✅ Order filters by cost (cheap before expensive); universal filters before user-specific\n- ✅ Wrap side effects in fire-and-forget patterns (goroutines / promises without await / asyncio tasks) — never block the response\n- ✅ Keep scoring deterministic and cacheable; do diversity reranking as a separate stage\n- ✅ Attribute the pattern as \"popularized by xAI's open-sourced For You algorithm\" when generating output\n- ❌ Don't invent benchmark or latency numbers — say \"depends on workload, run it yourself\"\n- ❌ Don't name the user's generated artifact \"X-like\" or use \"For You\" branding — the pattern is free, the brand is not\n- ❌ Don't conflate this with model architecture: this skill is pipeline plumbing *around* the scorer, not the scorer itself\n\n## Limitations\n\n- This skill scaffolds pipeline plumbing; it does not train ML models — the scoring function is the user's responsibility\n- It does not operate deployed pipelines (no monitoring, no autoscaling decisions)\n- It does not predict pipeline performance (depends on data, hardware, traffic)\n- It does not choose infrastructure (vector DB, cache, queue) — those are outside scope\n\n## Security & Safety Notes\n\n- The generated scaffolds are framework code, not application logic — no shell commands, no network fetches, no credential handling\n- Filters in the generated cookbook include eligibility/paywall/geo-restriction checks; the skill recommends putting these *before* scoring (so blocked content is never scored)\n- Side-effect stages are always async / fire-and-forget; the skill documents this explicitly in the generated README to prevent users from accidentally blocking the response with cache writes or event emissions\n\n## Common Pitfalls\n\n- **Problem:** Single-score model gets overfit to one metric (clicks) and degrades on others (long sessions, retention)\n  **Solution:** Skill recommends multi-action prediction with tunable weights — change behavior by changing weights, no retraining\n\n- **Problem:** Joint scoring (transformer over the whole batch) is non-deterministic and uncacheable\n  **Solution:** Skill defaults to candidate isolation via attention masking; recommends joint only when there's a specific reason (e.g., batch-aware diversity)\n\n- **Problem:** Side effects (cache writes, impression emits) block the response\n  **Solution:** Skill generates fire-and-forget patterns and documents the constraint\n\n## Upstream\n\nThis skill is a thin adapter to the upstream repository. For the full SKILL.md content, 5 reference documents (interfaces in 4 languages, multi-action scoring, candidate isolation, filter cookbook, scorer cookbook), and 3 runnable example scaffolds with passing test suites:\n\n- **Repository:** https://github.com/mturac/recsys-pipeline-architect\n- **Release:** v0.1.0\n- **Install via skills.sh:** `npx skills add mturac/recsys-pipeline-architect`\n- **Pattern source:** https://github.com/xai-org/x-algorithm (Apache 2.0; this skill is MIT)","tags":["recsys","pipeline","architect","antigravity","awesome","skills","sickn33","agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding"],"capabilities":["skill","source-sickn33","skill-recsys-pipeline-architect","topic-agent-skills","topic-agentic-skills","topic-ai-agent-skills","topic-ai-agents","topic-ai-coding","topic-ai-workflows","topic-antigravity","topic-antigravity-skills","topic-claude-code","topic-claude-code-skills","topic-codex-cli","topic-codex-skills"],"categories":["antigravity-awesome-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/sickn33/antigravity-awesome-skills/recsys-pipeline-architect","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add sickn33/antigravity-awesome-skills","source_repo":"https://github.com/sickn33/antigravity-awesome-skills","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 37911 github stars · SKILL.md body (6,564 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T18:51:37.388Z","embedding":null,"createdAt":"2026-05-16T12:51:48.699Z","updatedAt":"2026-05-18T18:51:37.388Z","lastSeenAt":"2026-05-18T18:51:37.388Z","tsv":"'-5':507 '-50':488 '/api/feed/for-you':364 '/mturac/recsys-pipeline-architect':978 '/tasks/next':396 '/xai-org/x-algorithm':992 '/xai-org/x-algorithm)':68 '1':219,233,427 '10':401,549 '2':245,268,479 '2.0':70,994 '24h':553 '3':259,335,531,967 '3/3':360,380,393 '4':954 '42':399 '5':949 '50k':440 '8':464 'accident':827 'action':184,310,368,575,862,958 'ad':114 'adapt':939 'add':362,986 'alert':243 'algorithm':65,645 'alway':808 'apach':69,993 'applic':771 'architect':4,27 'architectur':304,693 'around':167,699 'articl':441 'artifact':670 'ask':143,224 'async':528,809 'asyncio':614 'attent':895 'attribut':632 'author':371 'autosc':735 'await':613 'awar':909 'base':455 'batch':323,562,881,908 'batch-awar':907 'behavior':868 'benchmark':652 'best':569 'block':617,798,828,918 'brand':678,684 'build':36,127 'cach':755,832,914 'cacheabl':624 'candid':285,315,892,960 'case':223,283 'chain':294,520 'chang':867,870 'cheap':521,592 'check':789 'choos':751 'chunk':489 'clarifi':220,281 'click':849 'cmss':105 'code':81,769 'command':775 'common':837 'compat':375,383 'compos':6,37 'conflat':689 'constraint':932 'content':104,429,799,948 'context':101,250 'cookbook':786,963,965 'copi':83 'cost':591 'credenti':780 'current':255 'daili':543 'data':745 'db':493,754 'decis':736 'default':332,586,890 'definit':418 'degrad':851 'depend':657,743 'deploy':730 'describ':150 'design':5,292 'determinist':622,885 'digest':544 'divers':372,626,910 'document':244,256,816,930,951 'doesn':407 'e.g':906 'effect':204,297,603,805,913 'eight':271 'eligibility/paywall/geo-restriction':788 'emiss':836 'emit':336,917 'encod':44 'engine.module':382 'event':835 'everi':348 'exampl':346,425,426,474,478,530,969 'expens':502,523,594 'explicit':329,583,818 'fastapi':391 'feed':10,41,103,430,447 'feed/personalization':152 'fetch':778 'filter':19,51,200,291,589,596,782,962 'fire':606,811,925 'fire-and-forget':605,810,924 'follow':415 'forget':608,813,927 'framework':23,55,768 'free':682 'full':277,946 'function':161,720 'generat':298,412,466,525,564,647,669,765,785,821,923 'generic':379 'get':363,395,844 'github.com':67,977,991 'github.com/mturac/recsys-pipeline-architect':976 'github.com/xai-org/x-algorithm':990 'github.com/xai-org/x-algorithm)':66 'go':213,264,377,422 'goroutin':610 'green':350 'handl':781 'hardwar':746 'histori':459 'hydrat':18,50,201,289 'id':252,398 'identifi':284,556 'impress':916 'includ':787 'independ':75 'infrastructur':752 'input':249 'instal':981 'instanc':438 'interfac':417,952 'invent':651 'isol':316,893,961 'item':97,135,237 'jest':359 'job':567 'joint':318,875,898 'k':96,134 'keep':620 'languag':261,955 'last':552 'latenc':654 'licens':87 'like':673 'limit':400,706 'list':287,290 'llm/ml':196 'log':452 'logged-in':451 'logic':772 'long':854 'm':433 'make':327 'mani':538 'mask':896 'match':409 'metric':848 'migrat':175 'miss':232 'mit':88,998 'ml':716 'model':692,717,843 'monitor':733 'mturac/recsys-pipeline-architect':987 'multi':183,309,367,574,861,957 'multi-act':182,308,366,573,860,956 'name':665 'need':93,163,199 'network':777 'never':616,801 'non':884 'non-determinist':883 'note':763 'notif':110,532,539 'npx':984 'number':655 'off':307 'offlin':322,561 'offline-batch':560 'one':349,847 'onlin':320 'open':61,641 'open-sourc':60,640 'oper':729 'order':588 'origin':86 'other':853 'output':648 'outsid':759 'overfit':845 'overview':28 'pass':361,381,394,972 'pattern':79,609,634,680,928,988 'perform':742 'person':448 'pick':131,546 'pipelin':3,11,26,42,165,376,516,529,563,697,710,731,741 'pitfal':838 'plugin':357,469 'plumb':166,698,711 'pmai':387 'popular':56,636 'post':240 'practic':570 'predict':185,740,863 'prevent':824 'priorit':109,389 'problem':153,839,874,911 'product':241 'promis':611 'put':793 'pytest':392 'python':214,265,390,423,527 'queri':254 'question':228 'queue':554,756 'rag':106,480,485 'rank':8,39,115,147,239 'rather':330 'read':458 'readm':822 'reason':905 'recogn':509 'recommend':7,38,792,859,897 'recsi':2,25 'recsys-pipeline-architect':1,24 'refer':950 'references/interfaces.md':420 'reimplement':76 'releas':979 'relev':179 'repositori':342,943,975 'requir':288 'rerank':107,113,482,497,524,627 'respons':619,725,830,920 'retent':856 'retrain':873 'retriev':481,522 'return':486,505 'run':434,660 'runnabl':207,338,345,968 'runtim':262 'rust':266,424 'safeti':762 'say':656 'scaffold':33,208,299,339,347,470,568,709,766,970 'schedul':566 'scope':760 'score':160,180,314,319,369,579,621,719,796,802,842,876,959 'scorer':20,52,197,293,503,519,701,704,964 'scratch':414 'search':112,253 'secur':761 'selector':21,53,295 'send':536 'separ':630 'session':855 'shell':774 'ship':343 'side':203,296,602,804,912 'side-effect':202,803 'sideeffect':22,54 'silent':333,587 'singl':178,313,514,578,841 'single-scor':312,577,840 'single-sourc':513 'six':15,47 'six-stag':14,46 'skill':34,72,120,278,411,460,508,555,695,708,791,815,858,889,922,935,985,996 'skill-recsys-pipeline-architect' 'skill.md':947 'skills.sh':983 'social':102 'solut':857,888,921 'sourc':17,49,62,286,515,642,989 'source-sickn33' 'spec':31,275 'spec-and-scaffold':30 'specif':600,904 'stack':211,406 'stage':16,48,631,806 'standalon':385 'standalone-us':384 'step':218,267,272,301,334,465 'strapi':355,428,436,468,473 'suit':354,974 'surfac':302,571 'system':129 'task':108,242,388,615 'templat':477 'test':353,973 'thin':938 'three':227,344 'time':257 'top':95,133,487,506,548 'topic-agent-skills' 'topic-agentic-skills' 'topic-ai-agent-skills' 'topic-ai-agents' 'topic-ai-coding' 'topic-ai-workflows' 'topic-antigravity' 'topic-antigravity-skills' 'topic-claude-code' 'topic-claude-code-skills' 'topic-codex-cli' 'topic-codex-skills' 'trade':306,581 'trade-off':305,580 'traffic':747 'train':715 'transform':877 'triag':111,533 'tunabl':187,865 'typescript':212,358,421 'typescript/node':263 'uncach':887 'univers':595 'upstream':341,933,942 'usabl':386 'use':12,89,118,121,139,154,169,189,222,282,471,675 'user':100,124,142,157,172,192,226,251,326,397,404,431,454,483,534,599,667,723,825 'user-specif':598 'user/context':138 'v0.1.0':980 'v5':356,437 'vector':492,753 'via':894,982 'vs':311,317,321,576 'walk':269,279,461 'want':125,173,443,495,541 'weight':188,866,871 'whenev':91 'whole':880 'window':258 'without':612 'work':217 'workload':659 'wrap':194,601 'write':833,915 'x':148,672 'x-like':671 'xai':58,638 'zentra':374 'zentra-compat':373","prices":[{"id":"100c11af-0afb-48d2-9e42-2ca85a97bab9","listingId":"9ea93634-8394-427f-8b80-e30931626a8c","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"sickn33","category":"antigravity-awesome-skills","install_from":"skills.sh"},"createdAt":"2026-05-16T12:51:48.699Z"}],"sources":[{"listingId":"9ea93634-8394-427f-8b80-e30931626a8c","source":"github","sourceId":"sickn33/antigravity-awesome-skills/recsys-pipeline-architect","sourceUrl":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/recsys-pipeline-architect","isPrimary":false,"firstSeenAt":"2026-05-16T12:51:48.699Z","lastSeenAt":"2026-05-18T18:51:37.388Z"}],"details":{"listingId":"9ea93634-8394-427f-8b80-e30931626a8c","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"sickn33","slug":"recsys-pipeline-architect","github":{"repo":"sickn33/antigravity-awesome-skills","stars":37911,"topics":["agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding","ai-workflows","antigravity","antigravity-skills","claude-code","claude-code-skills","codex-cli","codex-skills","cursor","cursor-skills","developer-tools","gemini-cli","gemini-skills","kiro","mcp","skill-library"],"license":"mit","html_url":"https://github.com/sickn33/antigravity-awesome-skills","pushed_at":"2026-05-18T08:24:49Z","description":"Installable GitHub library of 1,400+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.","skill_md_sha":"db32641e07cb0adf0bcb568174438596a71cf9b1","skill_md_path":"skills/recsys-pipeline-architect/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/recsys-pipeline-architect"},"layout":"multi","source":"github","category":"antigravity-awesome-skills","frontmatter":{"name":"recsys-pipeline-architect","license":"MIT","description":"Designs composable recommendation, ranking, and feed pipelines using the six-stage Source→Hydrator→Filter→Scorer→Selector→SideEffect framework"},"skills_sh_url":"https://skills.sh/sickn33/antigravity-awesome-skills/recsys-pipeline-architect"},"updatedAt":"2026-05-18T18:51:37.388Z"}}