{"id":"6cc2648d-2142-4cf1-a117-1a5bbc8808c2","shortId":"guMsYE","kind":"skill","title":"kafka-topic-audit","tagline":"Audit all Kafka topic configurations against production best practices using the Lenses MCP server. Checks replication factor, retention, partitions, compaction, naming conventions, orphaned topics and missing metadata. Use when user says \"audit my topics\", \"check topic configs\",","description":"# Kafka Topic Configuration Audit\n\nAudits all topic configurations against production best practices. Misconfigured topics are the #1 cause of Kafka data loss - engineers create topics and forget to tune them.\n\nTarget environment: $ARGUMENTS\n\n## Workflow\n\nCopy this checklist and track your progress:\n\n```\nAudit Progress:\n- [ ] Step 1: Check environment health\n- [ ] Step 2: Fetch all topics\n- [ ] Step 3: Audit configurations against best practices\n- [ ] Step 4: Check metadata completeness\n- [ ] Step 5: Detect orphaned topics\n- [ ] Step 6: Run consistency checks\n- [ ] Step 7: Generate report\n```\n\n1. **Check environment health** for a high-level summary\n2. **Fetch all topics** and their configurations\n3. **Audit each topic** against best practices (see `references/audit-rules.md`)\n4. **Cross-reference metadata** for completeness\n5. **Detect orphaned topics** with no consumers\n6. **Report findings** with prioritised recommendations\n\n## Step 1: Environment Overview\n\nUse the Lenses MCP `check_environment_health` tool to get a quick summary:\n- Broker count, topic count, consumer count, connector count\n- Any existing issues flagged by Lenses\n\nExpected output: Environment health summary with broker, topic and consumer counts.\n\n**Validation**: If the environment is unhealthy or unreachable, stop and report the connection issue before proceeding.\n\n## Step 2: Fetch All Topics\n\nUse the Lenses MCP `list_topics` tool to retrieve all topics with their configurations in one call.\n\nFor topics that need deeper inspection, use:\n- `get_topic` for detailed config including partitions and consumers\n- `get_topic_broker_configs` for broker-level config overrides\n- `get_topic_partitions` for partition-level message counts and bytes\n\nExpected output: Full list of topics with their configurations. If zero topics are returned, report this and stop.\n\n## Step 3: Audit Configurations\n\nFor each topic, check against the thresholds in `references/audit-rules.md`:\n- **Replication factor** - RF=1 is critical, RF=2 is a warning in production\n- **Retention policies** - unbounded growth, too short or excessively long\n- **Partition count** - single-partition bottlenecks or excessive partitions\n- **Compaction settings** - compact without keys, delete for state topics\n- **Naming conventions** - must follow `{domain}.{entity}.{event}` pattern\n\n## Step 4: Metadata Completeness\n\nUse the Lenses MCP `list_topic_metadata` tool to check:\n- Topics missing descriptions\n- Topics missing tags\n- Topics without registered schemas (key or value)\n\nUse `list_datasets` with filters (`is_compacted`, `has_records`) to find anomalies.\n\n## Step 5: Orphan Detection\n\nFor each topic, use `list_consumer_groups_by_topic` to check for active consumers.\n- **Warning**: Topics with zero consumer groups (may be orphaned/dead)\n- **Suggestion**: Topics with only inactive/empty consumer groups\n\n## Step 6: Consistency Checks\n\n- Flag topics in the same domain with different retention policies\n- Flag topics in the same domain with different replication factors\n- Flag topics with inconsistent serialisation formats within a domain\n\n## Success Criteria\n\n### Quantitative\n- Triggers on 90% of topic-related queries (test with 10-20 varied phrasings)\n- Completes full audit in under 15 MCP tool calls\n- 0 failed MCP calls per run\n\n### Qualitative\n- Report is actionable without follow-up questions from the user\n- Consistent severity categorisation (critical/warning/suggestion) across runs\n- Every finding includes a concrete remediation step\n\n## Examples\n\n### Example 1: Routine weekly audit\n\nUser says: \"Run a topic audit on the staging environment\"\n\nActions:\n1. Check staging environment health via Lenses MCP\n2. Fetch all topics and configs\n3. Audit each topic against rules in `references/audit-rules.md`\n4. Check metadata completeness and orphaned topics\nResult: Full audit report with prioritised findings\n\n### Example 2: Pre-deployment check\n\nUser says: \"Check if my topic configs are production-ready\"\n\nActions:\n1. Audit all topics for RF < 3, unbounded retention, single partitions\n2. Flag any critical issues that would block a production deployment\nResult: Report highlighting critical issues that must be fixed before go-live\n\n### Example 3: Investigate a specific topic\n\nUser says: \"Is the orders.payment.completed topic configured correctly?\"\n\nActions:\n1. Fetch detailed config for the specific topic using `get_topic`\n2. Check broker-level overrides with `get_topic_broker_configs`\n3. Verify metadata and consumer groups\nResult: Focused report on a single topic with all findings\n\n## Troubleshooting\n\n### Lenses MCP connection failed\nCause: Environment name is incorrect or Lenses agent is offline.\nSolution: Run `check_environment_health` first. Verify the environment name matches what `list_environments` returns.\n\n### No topics returned\nCause: Environment exists but has no topics or permissions are restricted.\nSolution: Confirm the cluster has topics via the Lenses UI. Check that the Lenses agent has read access.\n\n### Metadata endpoint returns empty\nCause: Schema Registry is not configured or topics have no registered schemas.\nSolution: This is a valid finding - report it as missing metadata rather than treating it as an error.\n\n## Output Format\n\n```\n## Topic Audit Report\n\n### Environment: {name}\n- Brokers: X | Topics: Y | Consumer groups: Z\n\n### Critical (must fix)\n- [topic-name] Description of the issue\n  Current: {current value} | Recommended: {recommended value}\n\n### Warning (should fix)\n- [topic-name] Description of the issue\n  Current: {current value} | Recommended: {recommended value}\n\n### Suggestion (consider improving)\n- [topic-name] Description of the issue\n  Recommendation: How to fix it\n\n### Summary\n- X critical issues found\n- Y warnings found\n- Z suggestions found\n- Topics audited: N\n- Orphaned topics: M\n```","tags":["kafka","topic","audit","agentic","engineering","for","apache","lensesio","agent-skills","agentic-engineering","apache-kafka","claude-code"],"capabilities":["skill","source-lensesio","skill-kafka-topic-audit","topic-agent-skills","topic-agentic-engineering","topic-apache-kafka","topic-claude-code","topic-context-engineering","topic-cursor","topic-data-engineering","topic-devops","topic-kafka","topic-kafka-connect","topic-lenses","topic-mcp"],"categories":["agentic-engineering-for-apache-kafka"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/lensesio/agentic-engineering-for-apache-kafka/kafka-topic-audit","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add lensesio/agentic-engineering-for-apache-kafka","source_repo":"https://github.com/lensesio/agentic-engineering-for-apache-kafka","install_from":"skills.sh"}},"qualityScore":"0.463","qualityRationale":"deterministic score 0.46 from registry signals: · indexed on github topic:agent-skills · 26 github stars · SKILL.md body (5,833 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:05:01.044Z","embedding":null,"createdAt":"2026-05-15T19:05:07.361Z","updatedAt":"2026-05-18T19:05:01.044Z","lastSeenAt":"2026-05-18T19:05:01.044Z","tsv":"'-20':483 '0':495 '1':58,86,121,168,318,528,543,597,647 '10':482 '15':491 '2':91,131,226,322,551,580,608,658 '3':96,138,303,557,603,633,669 '4':103,147,364,565 '5':108,154,403 '6':113,161,437 '7':118 '90':474 'access':746 'across':517 'action':504,542,596,646 'activ':418 'agent':697,743 'anomali':401 'argument':74 'audit':4,5,36,45,46,83,97,139,304,488,531,537,558,574,598,784,854 'best':12,52,100,143 'block':615 'bottleneck':342 'broker':184,204,265,269,661,667,788 'broker-level':268,660 'byte':283 'call':246,494,498 'categoris':515 'caus':59,690,718,751 'check':19,39,87,104,116,122,175,309,376,416,439,544,566,584,587,659,702,739 'checklist':78 'cluster':732 'compact':24,346,348,396 'complet':106,153,366,486,568 'concret':523 'config':41,258,266,271,556,591,650,668 'configur':9,44,49,98,137,243,292,305,644,756 'confirm':730 'connect':221,688 'connector':190 'consid':828 'consist':115,438,513 'consum':160,188,207,262,411,419,424,434,673,792 'convent':26,356 'copi':76 'correct':645 'count':185,187,189,191,208,281,338 'creat':65 'criteria':470 'critic':320,611,622,795,844 'critical/warning/suggestion':516 'cross':149 'cross-refer':148 'current':805,806,821,822 'data':62 'dataset':392 'deeper':251 'delet':351 'deploy':583,618 'descript':379,801,817,833 'detail':257,649 'detect':109,155,405 'differ':447,457 'domain':359,445,455,468 'empti':750 'endpoint':748 'engin':64 'entiti':360 'environ':73,88,123,169,176,200,212,541,546,691,703,708,713,719,786 'error':780 'event':361 'everi':519 'exampl':526,527,579,632 'excess':335,344 'exist':193,720 'expect':198,284 'factor':21,316,459 'fail':496,689 'fetch':92,132,227,552,648 'filter':394 'find':163,400,520,578,684,768 'first':705 'fix':627,797,813,840 'flag':195,440,450,460,609 'focus':676 'follow':358,507 'follow-up':506 'forget':68 'format':465,782 'found':846,849,852 'full':286,487,573 'generat':119 'get':180,254,263,273,656,665 'go':630 'go-liv':629 'group':412,425,435,674,793 'growth':331 'health':89,124,177,201,547,704 'high':128 'high-level':127 'highlight':621 'improv':829 'inactive/empty':433 'includ':259,521 'inconsist':463 'incorrect':694 'inspect':252 'investig':634 'issu':194,222,612,623,804,820,836,845 'kafka':2,7,42,61 'kafka-topic-audit':1 'key':350,387 'lens':16,173,197,232,369,549,686,696,737,742 'level':129,270,279,662 'list':234,287,371,391,410,712 'live':631 'long':336 'loss':63 'm':858 'match':710 'may':426 'mcp':17,174,233,370,492,497,550,687 'messag':280 'metadata':31,105,151,365,373,567,671,747,773 'misconfigur':54 'miss':30,378,381,772 'must':357,625,796 'n':855 'name':25,355,692,709,787,800,816,832 'need':250 'offlin':699 'one':245 'orders.payment.completed':642 'orphan':27,110,156,404,570,856 'orphaned/dead':428 'output':199,285,781 'overrid':272,663 'overview':170 'partit':23,260,275,278,337,341,345,607 'partition-level':277 'pattern':362 'per':499 'permiss':726 'phrase':485 'polici':329,449 'practic':13,53,101,144 'pre':582 'pre-deploy':581 'prioritis':165,577 'proceed':224 'product':11,51,327,594,617 'production-readi':593 'progress':82,84 'qualit':501 'quantit':471 'queri':479 'question':509 'quick':182 'rather':774 'read':745 'readi':595 'recommend':166,808,809,824,825,837 'record':398 'refer':150 'references/audit-rules.md':146,314,564 'regist':385,761 'registri':753 'relat':478 'remedi':524 'replic':20,315,458 'report':120,162,219,298,502,575,620,677,769,785 'restrict':728 'result':572,619,675 'retent':22,328,448,605 'retriev':238 'return':297,714,717,749 'rf':317,321,602 'routin':529 'rule':562 'run':114,500,518,534,701 'say':35,533,586,639 'schema':386,752,762 'see':145 'serialis':464 'server':18 'set':347 'sever':514 'short':333 'singl':340,606,680 'single-partit':339 'skill' 'skill-kafka-topic-audit' 'solut':700,729,763 'source-lensesio' 'specif':636,653 'stage':540,545 'state':353 'step':85,90,95,102,107,112,117,167,225,302,363,402,436,525 'stop':217,301 'success':469 'suggest':429,827,851 'summari':130,183,202,842 'tag':382 'target':72 'test':480 'threshold':312 'tool':178,236,374,493 'topic':3,8,28,38,40,43,48,55,66,94,111,134,141,157,186,205,229,235,240,248,255,264,274,289,295,308,354,372,377,380,383,408,414,421,430,441,451,461,477,536,554,560,571,590,600,637,643,654,657,666,681,716,724,734,758,783,790,799,815,831,853,857 'topic-agent-skills' 'topic-agentic-engineering' 'topic-apache-kafka' 'topic-claude-code' 'topic-context-engineering' 'topic-cursor' 'topic-data-engineering' 'topic-devops' 'topic-kafka' 'topic-kafka-connect' 'topic-lenses' 'topic-mcp' 'topic-nam':798,814,830 'topic-rel':476 'track':80 'treat':776 'trigger':472 'troubleshoot':685 'tune':70 'ui':738 'unbound':330,604 'unhealthi':214 'unreach':216 'use':14,32,171,230,253,367,390,409,655 'user':34,512,532,585,638 'valid':209,767 'valu':389,807,810,823,826 'vari':484 'verifi':670,706 'via':548,735 'warn':325,420,811,848 'week':530 'within':466 'without':349,384,505 'workflow':75 'would':614 'x':789,843 'y':791,847 'z':794,850 'zero':294,423","prices":[{"id":"bf1ba4a3-6243-44d6-9019-2f2145ca6aef","listingId":"6cc2648d-2142-4cf1-a117-1a5bbc8808c2","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"lensesio","category":"agentic-engineering-for-apache-kafka","install_from":"skills.sh"},"createdAt":"2026-05-15T19:05:07.361Z"}],"sources":[{"listingId":"6cc2648d-2142-4cf1-a117-1a5bbc8808c2","source":"github","sourceId":"lensesio/agentic-engineering-for-apache-kafka/kafka-topic-audit","sourceUrl":"https://github.com/lensesio/agentic-engineering-for-apache-kafka/tree/main/skills/kafka-topic-audit","isPrimary":false,"firstSeenAt":"2026-05-15T19:05:07.361Z","lastSeenAt":"2026-05-18T19:05:01.044Z"}],"details":{"listingId":"6cc2648d-2142-4cf1-a117-1a5bbc8808c2","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"lensesio","slug":"kafka-topic-audit","github":{"repo":"lensesio/agentic-engineering-for-apache-kafka","stars":26,"topics":["agent-skills","agentic-engineering","apache-kafka","claude-code","context-engineering","cursor","data-engineering","devops","kafka","kafka-connect","lenses","mcp","model-context-protocol","platform-engineering","real-time-data","schema-registry","skills","streaming","streaming-data"],"license":"mit","html_url":"https://github.com/lensesio/agentic-engineering-for-apache-kafka","pushed_at":"2026-05-15T11:34:19Z","description":"AI agent skills for building, operating and troubleshooting Apache Kafka applications. Topic audit, consumer lag, schema review, security, connectors and DLQ","skill_md_sha":"701bcccfd1050d5d025a806a4bb3b3aa9eb80095","skill_md_path":"skills/kafka-topic-audit/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/lensesio/agentic-engineering-for-apache-kafka/tree/main/skills/kafka-topic-audit"},"layout":"multi","source":"github","category":"agentic-engineering-for-apache-kafka","frontmatter":{"name":"kafka-topic-audit","license":"MIT","description":"Audit all Kafka topic configurations against production best practices using the Lenses MCP server. Checks replication factor, retention, partitions, compaction, naming conventions, orphaned topics and missing metadata. Use when user says \"audit my topics\", \"check topic configs\", \"topic health check\" or asks about retention, replication or partition settings. Do NOT use for creating, deleting or modifying topics.","compatibility":"Recommended - the Lenses MCP server (lenses-mcp) connected and configured with a valid environment. Any Kafka MCP that exposes an equivalent tool surface (list_topics, get_topic, etc.) will also work; without an MCP server, the skill falls back to codebase-only inspection and skips live-cluster checks."},"skills_sh_url":"https://skills.sh/lensesio/agentic-engineering-for-apache-kafka/kafka-topic-audit"},"updatedAt":"2026-05-18T19:05:01.044Z"}}