{"id":"da81a91c-917a-478a-aa18-9637cdf247fe","shortId":"jtLJXq","kind":"skill","title":"kafka-connector-review","tagline":"Review Kafka Connect connector configurations for common misconfigurations using the Lenses MCP server. Checks error handling, DLQ setup, converters, transforms, task count and task health. Use when user says \"review connectors\", \"check connector configs\", \"why is my connector fa","description":"# Kafka Connect Configuration Review\n\nReviews Kafka Connect connector configurations for common misconfigurations. Connectors are defined as JSON/YAML and are entirely language-agnostic.\n\nTarget environment: $ARGUMENTS\n\n## Workflow\n\nCopy this checklist and track your progress:\n\n```\nConnector Review Progress:\n- [ ] Step 1: List all connectors\n- [ ] Step 2: Inspect each connector's configuration\n- [ ] Step 3: Validate configurations against plugin schemas\n- [ ] Step 4: Audit for common misconfigurations\n- [ ] Step 5: Generate report\n```\n\n1. **List all connectors** with status and task states\n2. **Inspect each connector's configuration** in detail\n3. **Validate configurations** against plugin schemas\n4. **Audit for common misconfigurations**\n5. **Report findings** with current and recommended configs\n\n## Step 1: List All Connectors\n\nUse the Lenses MCP `list_kafka_connectors` tool to get all connectors with:\n- Connector name, class and type (source/sink)\n- Status (RUNNING, PAUSED, FAILED, UNASSIGNED)\n- Task states and count\n- Cluster info\n\nFlag immediately:\n- **Critical**: Connectors in FAILED state\n- **Critical**: Tasks in FAILED state\n- **Warning**: Connectors in PAUSED state with no obvious reason\n\nExpected output: List of all connectors with name, class, status and task states.\n\n**Validation**: If no connectors are returned, report that no Connect cluster is configured and stop.\n\n## Step 2: Inspect Configurations\n\nUse the Lenses MCP `get_kafka_connector_target_definition` tool to get the full configuration YAML for each connector.\n\n## Step 3: Validate Configurations\n\nUse the Lenses MCP `validate_connector_configuration` tool to validate each connector's config against its plugin's schema. This catches:\n- Missing required configuration fields\n- Invalid configuration values\n- Type mismatches\n\n## Step 4: Audit Common Misconfigurations\n\n### Error Handling\n- **Critical**: `errors.tolerance=all` without `errors.deadletterqueue.topic.name` (silently drops messages)\n- **Warning**: Missing `errors.tolerance` (defaults to `none`, connector stops on any error)\n- **Warning**: Missing `errors.log.enable=true` (errors not logged)\n- **Suggestion**: Add `errors.deadletterqueue.context.headers.enable=true` for richer DLQ metadata\n\n### Converters\n- **Warning**: `key.converter` or `value.converter` mismatch with topic serialisation format\n- **Warning**: Using `org.apache.kafka.connect.storage.StringConverter` for structured data (should use Avro/JSON/Protobuf converter)\n- **Warning**: Missing `schemas.enable` setting where schemas are expected\n\n### Transforms (SMTs)\n- **Warning**: Complex transform chains (> 3 SMTs) that may be hard to debug\n- **Suggestion**: Consider Kafka Streams or ksqlDB for complex transformations\n- Verify transform class names are valid and in the correct order\n\n### Task Count\n- **Warning**: `tasks.max=1` on high-throughput connectors (limits parallelism)\n- **Suggestion**: Source connectors - `tasks.max` should align with source partitioning\n- **Suggestion**: Sink connectors - `tasks.max` should align with topic partition count\n\n### Connection and Retry\n- **Warning**: Missing retry configuration for connectors that interact with external systems\n- **Warning**: Missing connection timeout settings\n- **Suggestion**: Set explicit `consumer.override.*` or `producer.override.*` for performance tuning\n\n### Naming Conventions\n- **Suggestion**: Connector names should follow a consistent pattern (e.g., `{source/sink}-{system}-{entity}`)\n\n## Success Criteria\n\n### Quantitative\n- Triggers on 90% of connector-related queries (test with 10-20 varied phrasings)\n- Completes review in under 10 MCP tool calls\n- Catches 100% of validation errors reported by the plugin schema\n\n### Qualitative\n- Failed connectors are flagged immediately in the status overview\n- Error handling gaps (missing DLQ, silent drops) are always identified\n- Report is useful for both connector operators and developers\n\n## Examples\n\n### Example 1: Routine connector review\n\nUser says: \"Review all connectors in staging\"\n\nActions:\n1. List all connectors with status\n2. Inspect and validate each connector's configuration\n3. Audit for common misconfigurations\nResult: Full report covering all connectors with prioritised findings\n\n### Example 2: Investigating a failed connector\n\nUser says: \"My sink connector is failing, can you check why?\"\n\nActions:\n1. List connectors and identify those in FAILED state\n2. Get the full configuration for the failed connector\n3. Validate config against plugin schema\n4. Check for common issues (DLQ, converters, error handling)\nResult: Diagnosis of the specific failure with remediation steps\n\n### Example 3: Single connector deep dive\n\nUser says: \"Review the config for the elasticsearch-sink connector\"\n\nActions:\n1. Fetch the target definition for the named connector\n2. Validate against its plugin schema\n3. Audit error handling, converters, task count\nResult: Focused report on a single connector\n\n## Troubleshooting\n\n### No connectors returned\nCause: No Kafka Connect cluster is configured in the environment or no connectors are deployed.\nSolution: Verify that a Connect cluster exists via Lenses UI. Check `get_deployment_targets` for available Connect clusters.\n\n### Validation fails with unknown plugin\nCause: The connector plugin class is not installed on the Connect cluster.\nSolution: Report the missing plugin. This is a deployment issue, not a configuration issue.\n\n### Connector shows RUNNING but tasks are FAILED\nCause: The connector framework is running but individual tasks have encountered errors.\nSolution: Check each task's status. Common causes include authentication failures, network issues or schema mismatches with the external system.\n\n## Output Format\n\n```\n## Connector Review Report\n\n### Environment: {name}\n\n### Connector Status Overview\n| Connector | Class | Status | Tasks | Failed Tasks |\n|-----------|-------|--------|-------|-------------|\n| name | class | RUNNING | 3/3 | 0 |\n\n### Critical (must fix)\n- [connector-name] Description of the issue\n  Current: {current config} | Recommended: {recommended config}\n\n### Warning (should fix)\n- [connector-name] Description of the issue\n  Current: {current config} | Recommended: {recommended config}\n\n### Validation Errors\n- [connector-name] {field}: {validation error message}\n\n### Suggestion (consider improving)\n- [connector-name] Description of the suggestion\n  Recommendation: {how to improve}\n\n### Summary\n- X connectors reviewed\n- Y critical issues found\n- Z warnings found\n- Failed connectors: N\n- Failed tasks: M\n```","tags":["kafka","connector","review","agentic","engineering","for","apache","lensesio","agent-skills","agentic-engineering","apache-kafka","claude-code"],"capabilities":["skill","source-lensesio","skill-kafka-connector-review","topic-agent-skills","topic-agentic-engineering","topic-apache-kafka","topic-claude-code","topic-context-engineering","topic-cursor","topic-data-engineering","topic-devops","topic-kafka","topic-kafka-connect","topic-lenses","topic-mcp"],"categories":["agentic-engineering-for-apache-kafka"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/lensesio/agentic-engineering-for-apache-kafka/kafka-connector-review","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add lensesio/agentic-engineering-for-apache-kafka","source_repo":"https://github.com/lensesio/agentic-engineering-for-apache-kafka","install_from":"skills.sh"}},"qualityScore":"0.463","qualityRationale":"deterministic score 0.46 from registry signals: · indexed on github topic:agent-skills · 26 github stars · SKILL.md body (6,565 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T19:05:00.344Z","embedding":null,"createdAt":"2026-05-15T19:05:06.497Z","updatedAt":"2026-05-18T19:05:00.344Z","lastSeenAt":"2026-05-18T19:05:00.344Z","tsv":"'-20':477 '0':803 '1':82,110,147,394,529,541,587,647 '10':476,484 '100':489 '2':87,119,231,547,570,596,656 '3':94,127,254,362,555,605,630,662 '3/3':802 '4':101,133,288,611 '5':107,138 '90':468 'action':540,586,646 'add':321 'agnost':66 'align':407,416 'alway':516 'argument':69 'audit':102,134,289,556,663 'authent':772 'avail':710 'avro/json/protobuf':346 'call':487 'catch':277,488 'caus':680,718,751,770 'chain':361 'check':18,36,584,612,705,764 'checklist':73 'class':166,210,381,722,794,800 'cluster':179,225,684,700,712,729 'common':11,54,104,136,290,558,614,769 'complet':480 'complex':359,377 'config':38,145,270,607,639,816,819,832,835 'configur':9,46,52,92,96,124,129,227,233,248,256,263,280,283,427,554,600,686,742 'connect':7,45,50,224,421,437,683,699,711,728 'connector':3,8,35,37,42,51,56,78,85,90,113,122,150,157,162,164,184,194,207,218,240,252,262,268,308,399,404,413,429,452,471,500,523,531,537,544,552,565,574,579,589,604,632,645,655,675,678,692,720,744,753,785,790,793,808,824,839,849,861,871 'connector-nam':807,823,838,848 'connector-rel':470 'consid':371,846 'consist':457 'consumer.override':443 'convent':450 'convert':23,328,347,617,666 'copi':71 'correct':388 'count':26,178,391,420,668 'cover':563 'criteria':464 'critic':183,188,294,804,864 'current':142,814,815,830,831 'data':343 'debug':369 'deep':633 'default':305 'defin':58 'definit':242,651 'deploy':694,707,738 'descript':810,826,851 'detail':126 'develop':526 'diagnosi':621 'dive':634 'dlq':21,326,512,616 'drop':300,514 'e.g':459 'elasticsearch':643 'elasticsearch-sink':642 'encount':761 'entir':63 'entiti':462 'environ':68,689,788 'error':19,292,312,317,492,508,618,664,762,837,843 'errors.deadletterqueue.context.headers.enable':322 'errors.deadletterqueue.topic.name':298 'errors.log.enable':315 'errors.tolerance':295,304 'exampl':527,528,569,629 'exist':701 'expect':202,355 'explicit':442 'extern':433,781 'fa':43 'fail':173,186,191,499,573,581,594,603,714,750,797,870,873 'failur':625,773 'fetch':648 'field':281,841 'find':140,568 'fix':806,822 'flag':181,502 'focus':670 'follow':455 'format':337,784 'found':866,869 'framework':754 'full':247,561,599 'gap':510 'generat':108 'get':160,238,245,597,706 'handl':20,293,509,619,665 'hard':367 'health':29 'high':397 'high-throughput':396 'identifi':517,591 'immedi':182,503 'improv':847,858 'includ':771 'individu':758 'info':180 'inspect':88,120,232,548 'instal':725 'interact':431 'invalid':282 'investig':571 'issu':615,739,743,775,813,829,865 'json/yaml':60 'kafka':2,6,44,49,156,239,372,682 'kafka-connector-review':1 'key.converter':330 'ksqldb':375 'languag':65 'language-agnost':64 'lens':15,153,236,259,703 'limit':400 'list':83,111,148,155,204,542,588 'log':319 'm':875 'may':365 'mcp':16,154,237,260,485 'messag':301,844 'metadata':327 'misconfigur':12,55,105,137,291,559 'mismatch':286,333,778 'miss':278,303,314,349,425,436,511,733 'must':805 'n':872 'name':165,209,382,449,453,654,789,799,809,825,840,850 'network':774 'none':307 'obvious':200 'oper':524 'order':389 'org.apache.kafka.connect.storage.stringconverter':340 'output':203,783 'overview':507,792 'parallel':401 'partit':410,419 'pattern':458 'paus':172,196 'perform':447 'phrase':479 'plugin':98,131,273,496,609,660,717,721,734 'prioritis':567 'producer.override':445 'progress':77,80 'qualit':498 'quantit':465 'queri':473 'reason':201 'recommend':144,817,818,833,834,855 'relat':472 'remedi':627 'report':109,139,221,493,518,562,671,731,787 'requir':279 'result':560,620,669 'retri':423,426 'return':220,679 'review':4,5,34,47,48,79,481,532,535,637,786,862 'richer':325 'routin':530 'run':171,746,756,801 'say':33,534,576,636 'schema':99,132,275,353,497,610,661,777 'schemas.enable':350 'serialis':336 'server':17 'set':351,439,441 'setup':22 'show':745 'silent':299,513 'singl':631,674 'sink':412,578,644 'skill' 'skill-kafka-connector-review' 'smts':357,363 'solut':695,730,763 'sourc':403,409 'source-lensesio' 'source/sink':169,460 'specif':624 'stage':539 'state':118,176,187,192,197,214,595 'status':115,170,211,506,546,768,791,795 'step':81,86,93,100,106,146,230,253,287,628 'stop':229,309 'stream':373 'structur':342 'success':463 'suggest':320,370,402,411,440,451,845,854 'summari':859 'system':434,461,782 'target':67,241,650,708 'task':25,28,117,175,189,213,390,667,748,759,766,796,798,874 'tasks.max':393,405,414 'test':474 'throughput':398 'timeout':438 'tool':158,243,264,486 'topic':335,418 'topic-agent-skills' 'topic-agentic-engineering' 'topic-apache-kafka' 'topic-claude-code' 'topic-context-engineering' 'topic-cursor' 'topic-data-engineering' 'topic-devops' 'topic-kafka' 'topic-kafka-connect' 'topic-lenses' 'topic-mcp' 'track':75 'transform':24,356,360,378,380 'trigger':466 'troubleshoot':676 'true':316,323 'tune':448 'type':168,285 'ui':704 'unassign':174 'unknown':716 'use':13,30,151,234,257,339,345,520 'user':32,533,575,635 'valid':95,128,215,255,261,266,384,491,550,606,657,713,836,842 'valu':284 'value.converter':332 'vari':478 'verifi':379,696 'via':702 'warn':193,302,313,329,338,348,358,392,424,435,820,868 'without':297 'workflow':70 'x':860 'y':863 'yaml':249 'z':867","prices":[{"id":"e100e9f4-6c48-4736-8ccd-550647a384b3","listingId":"da81a91c-917a-478a-aa18-9637cdf247fe","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"lensesio","category":"agentic-engineering-for-apache-kafka","install_from":"skills.sh"},"createdAt":"2026-05-15T19:05:06.497Z"}],"sources":[{"listingId":"da81a91c-917a-478a-aa18-9637cdf247fe","source":"github","sourceId":"lensesio/agentic-engineering-for-apache-kafka/kafka-connector-review","sourceUrl":"https://github.com/lensesio/agentic-engineering-for-apache-kafka/tree/main/skills/kafka-connector-review","isPrimary":false,"firstSeenAt":"2026-05-15T19:05:06.497Z","lastSeenAt":"2026-05-18T19:05:00.344Z"}],"details":{"listingId":"da81a91c-917a-478a-aa18-9637cdf247fe","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"lensesio","slug":"kafka-connector-review","github":{"repo":"lensesio/agentic-engineering-for-apache-kafka","stars":26,"topics":["agent-skills","agentic-engineering","apache-kafka","claude-code","context-engineering","cursor","data-engineering","devops","kafka","kafka-connect","lenses","mcp","model-context-protocol","platform-engineering","real-time-data","schema-registry","skills","streaming","streaming-data"],"license":"mit","html_url":"https://github.com/lensesio/agentic-engineering-for-apache-kafka","pushed_at":"2026-05-15T11:34:19Z","description":"AI agent skills for building, operating and troubleshooting Apache Kafka applications. Topic audit, consumer lag, schema review, security, connectors and DLQ","skill_md_sha":"b9d1466890c0298b2332d3160739c3ebbe1ad130","skill_md_path":"skills/kafka-connector-review/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/lensesio/agentic-engineering-for-apache-kafka/tree/main/skills/kafka-connector-review"},"layout":"multi","source":"github","category":"agentic-engineering-for-apache-kafka","frontmatter":{"name":"kafka-connector-review","license":"MIT","description":"Review Kafka Connect connector configurations for common misconfigurations using the Lenses MCP server. Checks error handling, DLQ setup, converters, transforms, task count and task health. Use when user says \"review connectors\", \"check connector configs\", \"why is my connector failing\" or asks about Kafka Connect configuration. Do NOT use for creating, deploying or controlling connectors.","compatibility":"Recommended - the Lenses MCP server (lenses-mcp) connected and configured with a valid environment. Any Kafka MCP that exposes an equivalent Kafka Connect tool surface (list_connectors, get_connector, etc.) will also work; without an MCP server, the skill falls back to codebase-only inspection and skips live-cluster checks."},"skills_sh_url":"https://skills.sh/lensesio/agentic-engineering-for-apache-kafka/kafka-connector-review"},"updatedAt":"2026-05-18T19:05:00.344Z"}}