{"id":"4ef742a6-7c01-49e9-9722-0a9250207229","shortId":"eX444D","kind":"skill","title":"analytical-pm","tagline":"Structured analytical and metrics framework for AI product roles. Covers: metrics, goal-setting, root-cause analysis, trade-offs, A/B tests.","description":"# Analytical PM Skill\n\nApply a structured framework to PM analytical, metrics, root-cause, and trade-off questions targeting AI product roles.\n\n## When to Use\n- User asks \"What metrics would you use for X\"\n- User asks \"How would you measure success for X\"\n- User asks \"Metric X dropped 20%, diagnose it\"\n- User asks about trade-offs between two product decisions\n- User asks \"Define a North Star metric for X\"\n- User says `/analytical-pm` followed by a question\n- Any question about metrics, goals, root-cause analysis, A/B tests, or trade-offs\n\n## Context\n- **Tuned for**: AI product roles at frontier AI companies\n- **What matters**: Translating product intuition into measurable outcomes and debugging complex systems with data\n- **Common pitfall**: Picking vanity metrics or being too qualitative. Be rigorous and quantitative.\n\n## Three Question Types\n\n### TYPE A: Metrics / Goal-Setting Questions\n\"Define success metrics for X\" / \"What would you measure for X\" / \"Set goals for X\"\n\n**Framework: Analytical (6 Steps)**\n\n#### Step 1: Clarify the Product\n- What is the product? Who uses it? What value does it deliver?\n- What stage is it in? (launch, growth, mature, declining)\n- What's the business model? (subscription, API usage, freemium, enterprise)\n\n#### Step 2: Define the North Star Metric (NSM)\nThe NSM must capture the **core value exchange** between product and user.\n- Formula: NSM = [engagement unit] per [user segment] per [time period]\n- Example (ChatGPT): # of successful conversations per weekly active user\n- Example (LLM API platform): # of API calls generating production value per monthly active developer\n- Example (Claude): # of tasks completed per weekly active user\n\n**Decompose the NSM** into a metric tree:\n```\nNSM = Factor A x Factor B x Factor C\n```\n\n#### Step 3: Supporting Metrics (3-5)\nLeading indicators that the NSM will grow. Organized by AARRR:\n- **Acquisition**: New users/developers, sign-up conversion\n- **Activation**: First successful use, time-to-value\n- **Retention**: D7/D30 retention, usage frequency\n- **Revenue**: ARPU, conversion to paid, API spend\n- **Referral**: Organic invites, word-of-mouth, virality coefficient\n\n#### Step 4: Counter / Guardrail Metrics (2-3)\nWhat we must NOT break while optimizing the NSM:\n- **Quality**: Response accuracy, hallucination rate, harmful content rate\n- **Safety**: Content policy violations, user reports, model refusals (false positive rate)\n- **Trust**: User satisfaction (CSAT/NPS), enterprise churn, data privacy incidents\n- **System**: Latency (TTFT, TPS), error rate, uptime\n\n#### Step 5: Ecosystem Metrics\nFor platform companies, measure ecosystem health:\n- Developer ecosystem: # of apps built, API integrations, plugin adoption\n- Partner ecosystem: Revenue through partners, integration depth\n- Content ecosystem: User-generated content, model fine-tunes, custom GPTs\n\n#### Step 6: Trade-offs Between Metrics\nIdentify 2-3 key tensions:\n- Growth vs. Safety (more users vs. more moderation needed)\n- Speed vs. Quality (faster responses vs. more accurate responses)\n- Revenue vs. Access (monetization vs. mission of broad access)\n\nState how you'd resolve each (e.g., set guardrail thresholds, A/B test, phased rollout).\n\n---\n\n### TYPE B: Root-Cause / Diagnostic Questions\n\"Metric X dropped 20% this week. Diagnose it.\"\n\n**Framework: MECE (Mutually Exclusive, Collectively Exhaustive)**\n\n#### Step 1: Clarify\n- Which metric exactly? Over what timeframe? What's the baseline?\n- Is this relative or absolute? Sudden or gradual?\n- Any known events (launches, incidents, seasonality)?\n\n#### Step 2: Segment to Isolate\nBreak the metric down systematically:\n- **By user segment**: New vs. existing, free vs. paid, geography, platform (web/mobile/API)\n- **By product surface**: Which feature/page/endpoint is affected?\n- **By time**: When exactly did the drop start? Correlated with any deploy/event?\n- **By funnel stage**: Where in the funnel is the drop?\n\n#### Step 3: Hypothesize (MECE)\nGenerate hypotheses that are mutually exclusive and collectively exhaustive:\n\n**Internal factors:**\n- Product change (new deploy, A/B test, feature removal)\n- Technical issue (latency increase, outage, bug, model regression)\n- Data/instrumentation issue (logging break, tracking change, attribution error)\n\n**External factors:**\n- Seasonality (holiday, weekend, school schedule)\n- Competitor action (new feature launch, pricing change)\n- Market event (news cycle, regulatory change, viral moment)\n- Platform change (app store policy, browser update, API deprecation)\n\n#### Step 4: Validate\nFor each hypothesis, state:\n- What data would confirm/deny it\n- What team/tool you'd use to investigate\n- Priority order for investigation\n\n#### Step 5: Recommend Action\n- Short-term: Immediate mitigation\n- Medium-term: Root cause fix\n- Long-term: Monitoring/alerting to catch this earlier\n\n---\n\n### TYPE C: Trade-off Questions\n\"Feature A would increase engagement but decrease revenue. Ship or not?\"\n\n**Framework: 3 Trade-off Types**\n\n#### Type 1: Similar Product Cannibalization\nProduct A vs. Product B serving overlapping users.\n- Quantify cannibalization risk (user overlap, usage substitution)\n- Measure incremental value (does total pie grow?)\n- Run holdout experiment\n\n#### Type 2: Same Product, Different Variations\nVersion A vs. Version B of the same feature.\n- Define ship/no-ship criteria upfront\n- A/B test with clear primary metric and guardrails\n- Set duration and statistical significance threshold\n- Consider long-term effects (novelty bias, learning curves)\n\n#### Type 3: Different Products, Same Surface\nFeature X vs. Feature Y competing for the same real estate.\n- Score each on: impact to NSM, strategic value, user demand, effort\n- Consider: Can they coexist? Is this a false dichotomy?\n- Propose: Experiment design, phased rollout, or user segmentation\n\n**For all trade-offs:**\n- State the decision framework explicitly\n- Quantify where possible (even rough estimates)\n- Identify the reversibility of each option\n- Recommend with conviction, then acknowledge what you'd monitor\n\n## AI-Specific Analytical Considerations\n- **Model metrics**: Perplexity, BLEU/ROUGE, human eval scores, Elo ratings\n- **Safety metrics**: Harmful content rate, jailbreak success rate, refusal accuracy\n- **Cost metrics**: Cost per query, GPU utilization, inference cost per token\n- **Latency metrics**: Time to first token (TTFT), tokens per second (TPS), end-to-end response time\n- **Quality metrics**: Hallucination rate, factual accuracy, instruction-following score\n\n## Output Format\nStructure as a rigorous analytical walkthrough. Be quantitative where possible. For metrics questions, draw the metric tree. For root-cause, walk through the diagnostic systematically. Aim for ~2000 words.\n\n## Research-First Workflow\nBefore generating the answer:\n1. **Research** — Use web search to find latest benchmarks, industry metrics, and analytical frameworks relevant to the question. Do 5-10 searches.\n2. **Cite sources** — Include `[linked source](url)` inline for data points and benchmarks.\n3. **Display** the complete structured answer.\n\n## What Good Looks Like\n- Starts with clarifying the metric/situation (don't assume)\n- NSM captures core user value (not vanity metrics)\n- Metric tree is decomposable and actionable\n- Counter metrics show product maturity (especially safety for AI)\n- Root-cause analysis is structured and exhaustive (MECE)\n- Trade-off analysis is quantitative, not just qualitative\n- Shows awareness of AI-specific measurement challenges","tags":["analytical","skills","aroyburman-codes","agent-skills","claude-code","claude-skills","frameworks","metrics","pm-tools","product-management","product-strategy"],"capabilities":["skill","source-aroyburman-codes","skill-analytical-pm","topic-agent-skills","topic-claude-code","topic-claude-skills","topic-frameworks","topic-metrics","topic-pm-tools","topic-product-management","topic-product-strategy"],"categories":["pm-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/aroyburman-codes/pm-skills/analytical-pm","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add aroyburman-codes/pm-skills","source_repo":"https://github.com/aroyburman-codes/pm-skills","install_from":"skills.sh"}},"qualityScore":"0.453","qualityRationale":"deterministic score 0.45 from registry signals: · indexed on github topic:agent-skills · 6 github stars · SKILL.md body (7,530 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:14:47.096Z","embedding":null,"createdAt":"2026-05-18T13:22:16.120Z","updatedAt":"2026-05-18T19:14:47.096Z","lastSeenAt":"2026-05-18T19:14:47.096Z","tsv":"'-10':1002 '-3':358,450 '-5':305 '/analytical-pm':100 '1':187,516,733,982 '2':223,357,449,543,763,1004 '20':76,504 '2000':972 '3':301,304,594,727,805,1017 '4':353,664 '5':404,687,1001 '6':184,442 'a/b':25,114,490,612,781 'aarrr':315 'absolut':532 'access':473,479 'accur':469 'accuraci':370,903,937 'acknowledg':875 'acquisit':316 'action':640,689,1048 'activ':259,273,282,323 'adopt':421 'affect':570 'ai':10,47,123,128,881,1057,1080 'ai-specif':880,1079 'aim':970 'analysi':21,113,1061,1070 'analyt':2,5,27,36,183,883,948,994 'analytical-pm':1 'answer':981,1022 'api':218,263,266,341,418,661 'app':416,656 'appli':30 'arpu':337 'ask':54,63,72,80,90 'assum':1034 'attribut':630 'awar':1077 'b':296,495,741,772 'baselin':527 'benchmark':990,1016 'bias':801 'bleu/rouge':888 'break':363,547,627 'broad':478 'browser':659 'bug':621 'built':417 'busi':215 'c':299,710 'call':267 'cannib':736,746 'captur':233,1036 'catch':706 'caus':20,40,112,498,699,964,1060 'challeng':1083 'chang':609,629,645,651,655 'chatgpt':253 'churn':392 'cite':1005 'clarifi':188,517,1029 'claud':276 'clear':784 'coeffici':351 'coexist':835 'collect':513,604 'common':144 'compani':129,409 'compet':815 'competitor':639 'complet':279,1020 'complex':140 'confirm/deny':673 'consid':795,832 'consider':884 'content':374,377,429,434,897 'context':120 'convers':256,322,338 'convict':873 'core':235,1037 'correl':579 'cost':904,906,912 'counter':354,1049 'cover':13 'criteria':779 'csat/nps':390 'curv':803 'custom':439 'cycl':649 'd':483,678,878 'd7/d30':332 'data':143,393,671,1013 'data/instrumentation':624 'debug':139 'decis':88,856 'declin':211 'decompos':284,1046 'decreas':721 'defin':91,167,224,777 'deliv':202 'demand':830 'deploy':611 'deploy/event':582 'deprec':662 'depth':428 'design':843 'develop':274,413 'diagnos':77,507 'diagnost':499,968 'dichotomi':840 'differ':766,806 'display':1018 'draw':957 'drop':75,503,577,592 'durat':790 'e.g':486 'earlier':708 'ecosystem':405,411,414,423,430 'effect':799 'effort':831 'elo':892 'end':927,929 'end-to-end':926 'engag':244,719 'enterpris':221,391 'error':400,631 'especi':1054 'estat':820 'estim':864 'eval':890 'even':862 'event':538,647 'exact':520,574 'exampl':252,261,275 'exchang':237 'exclus':512,602 'exhaust':514,605,1065 'exist':557 'experi':761,842 'explicit':858 'extern':632 'factor':292,295,298,607,633 'factual':936 'fals':384,839 'faster':465 'featur':614,642,715,776,810,813 'feature/page/endpoint':568 'find':988 'fine':437 'fine-tun':436 'first':324,919,976 'fix':700 'follow':101,940 'format':943 'formula':242 'framework':8,33,182,509,726,857,995 'free':558 'freemium':220 'frequenc':335 'frontier':127 'funnel':584,589 'generat':268,433,597,979 'geographi':561 'goal':16,109,164,179 'goal-set':15,163 'good':1024 'gpts':440 'gpu':909 'gradual':535 'grow':312,758 'growth':209,453 'guardrail':355,488,788 'hallucin':371,934 'harm':373,896 'health':412 'holdout':760 'holiday':635 'human':889 'hypothes':595,598 'hypothesi':668 'identifi':448,865 'immedi':693 'impact':824 'incid':395,540 'includ':1007 'increas':619,718 'increment':753 'indic':307 'industri':991 'infer':911 'inlin':1011 'instruct':939 'instruction-follow':938 'integr':419,427 'intern':606 'intuit':134 'investig':681,685 'invit':345 'isol':546 'issu':617,625 'jailbreak':899 'key':451 'known':537 'latenc':397,618,915 'latest':989 'launch':208,539,643 'lead':306 'learn':802 'like':1026 'link':1008 'llm':262 'log':626 'long':702,797 'long-term':701,796 'look':1025 'market':646 'matter':131 'matur':210,1053 'measur':67,136,175,410,752,1082 'mece':510,596,1066 'medium':696 'medium-term':695 'metric':7,14,37,56,73,95,108,148,162,169,228,289,303,356,406,447,501,519,549,786,886,895,905,916,933,955,959,992,1042,1043,1050 'metric/situation':1031 'mission':476 'mitig':694 'model':216,382,435,622,885 'moder':460 'moment':653 'monet':474 'monitor':879 'monitoring/alerting':704 'month':272 'mouth':349 'must':232,361 'mutual':511,601 'need':461 'new':317,555,610,641 'news':648 'north':93,226 'novelti':800 'nsm':229,231,243,286,291,310,367,826,1035 'off':24,84,119,445,853 'optim':365 'option':870 'order':683 'organ':313,344 'outag':620 'outcom':137 'output':942 'overlap':743,749 'paid':340,560 'partner':422,426 'per':246,249,257,271,280,907,913,923 'period':251 'perplex':887 'phase':492,844 'pick':146 'pie':757 'pitfal':145 'platform':264,408,562,654 'plugin':420 'pm':3,28,35 'point':1014 'polici':378,658 'posit':385 'possibl':861,953 'price':644 'primari':785 'prioriti':682 'privaci':394 'product':11,48,87,124,133,190,194,239,269,565,608,735,737,740,765,807,1052 'propos':841 'qualit':152,1075 'qualiti':368,464,932 'quantifi':745,859 'quantit':156,951,1072 'queri':908 'question':45,104,106,158,166,500,714,956,999 'rate':372,375,386,401,893,898,901,935 'real':819 'recommend':688,871 'referr':343 'refus':383,902 'regress':623 'regulatori':650 'relat':530 'relev':996 'remov':615 'report':381 'research':975,983 'research-first':974 'resolv':484 'respons':369,466,470,930 'retent':331,333 'revenu':336,424,471,722 'revers':867 'rigor':154,947 'risk':747 'role':12,49,125 'rollout':493,845 'root':19,39,111,497,698,963,1059 'root-caus':18,38,110,496,962,1058 'rough':863 'run':759 'safeti':376,455,894,1055 'satisfact':389 'say':99 'schedul':638 'school':637 'score':821,891,941 'search':986,1003 'season':541,634 'second':924 'segment':248,544,554,848 'serv':742 'set':17,165,178,487,789 'ship':723 'ship/no-ship':778 'short':691 'short-term':690 'show':1051,1076 'sign':320 'sign-up':319 'signific':793 'similar':734 'skill':29 'skill-analytical-pm' 'sourc':1006,1009 'source-aroyburman-codes' 'specif':882,1081 'speed':462 'spend':342 'stage':204,585 'star':94,227 'start':578,1027 'state':480,669,854 'statist':792 'step':185,186,222,300,352,403,441,515,542,593,663,686 'store':657 'strateg':827 'structur':4,32,944,1021,1063 'subscript':217 'substitut':751 'success':68,168,255,325,900 'sudden':533 'support':302 'surfac':566,809 'system':141,396 'systemat':551,969 'target':46 'task':278 'team/tool':676 'technic':616 'tension':452 'term':692,697,703,798 'test':26,115,491,613,782 'three':157 'threshold':489,794 'time':250,328,572,917,931 'time-to-valu':327 'timefram':523 'token':914,920,922 'topic-agent-skills' 'topic-claude-code' 'topic-claude-skills' 'topic-frameworks' 'topic-metrics' 'topic-pm-tools' 'topic-product-management' 'topic-product-strategy' 'total':756 'tps':399,925 'track':628 'trade':23,43,83,118,444,712,729,852,1068 'trade-off':22,42,82,117,443,711,728,851,1067 'translat':132 'tree':290,960,1044 'trust':387 'ttft':398,921 'tune':121,438 'two':86 'type':159,160,494,709,731,732,762,804 'unit':245 'updat':660 'upfront':780 'uptim':402 'url':1010 'usag':219,334,750 'use':52,59,196,326,679,984 'user':53,62,71,79,89,98,241,247,260,283,380,388,432,457,553,744,748,829,847,1038 'user-gener':431 'users/developers':318 'util':910 'valid':665 'valu':199,236,270,330,754,828,1039 'vaniti':147,1041 'variat':767 'version':768,771 'violat':379 'viral':350,652 'vs':454,458,463,467,472,475,556,559,739,770,812 'walk':965 'walkthrough':949 'web':985 'web/mobile/api':563 'week':258,281,506 'weekend':636 'word':347,973 'word-of-mouth':346 'workflow':977 'would':57,65,173,672,717 'x':61,70,74,97,171,177,181,294,297,502,811 'y':814","prices":[{"id":"1806bc7f-2566-4899-9373-7f22a810fd77","listingId":"4ef742a6-7c01-49e9-9722-0a9250207229","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"aroyburman-codes","category":"pm-skills","install_from":"skills.sh"},"createdAt":"2026-05-18T13:22:16.120Z"}],"sources":[{"listingId":"4ef742a6-7c01-49e9-9722-0a9250207229","source":"github","sourceId":"aroyburman-codes/pm-skills/analytical-pm","sourceUrl":"https://github.com/aroyburman-codes/pm-skills/tree/main/skills/analytical-pm","isPrimary":false,"firstSeenAt":"2026-05-18T13:22:16.120Z","lastSeenAt":"2026-05-18T19:14:47.096Z"}],"details":{"listingId":"4ef742a6-7c01-49e9-9722-0a9250207229","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"aroyburman-codes","slug":"analytical-pm","github":{"repo":"aroyburman-codes/pm-skills","stars":6,"topics":["agent-skills","ai","claude-code","claude-skills","frameworks","metrics","pm-tools","product-management","product-strategy"],"license":"mit","html_url":"https://github.com/aroyburman-codes/pm-skills","pushed_at":"2026-02-17T06:52:03Z","description":"PM workflow and product thinking skills for AI product managers. 17 structured frameworks for PRDs, metrics, strategy, writing, prioritization, and more.","skill_md_sha":"d7d2e0fc6d6cc7a20a4f363d95df8db9afec6269","skill_md_path":"skills/analytical-pm/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/aroyburman-codes/pm-skills/tree/main/skills/analytical-pm"},"layout":"multi","source":"github","category":"pm-skills","frontmatter":{"name":"analytical-pm","description":"Structured analytical and metrics framework for AI product roles. Covers: metrics, goal-setting, root-cause analysis, trade-offs, A/B tests."},"skills_sh_url":"https://skills.sh/aroyburman-codes/pm-skills/analytical-pm"},"updatedAt":"2026-05-18T19:14:47.096Z"}}