{"id":"64ae7c9e-6e1c-48a9-be20-7914924916f8","shortId":"hhCnvY","kind":"skill","title":"wiztelemetry-tracing","tagline":"Use when working with WizTelemetry Tracing extension for KubeSphere, including installation, configuration, and tracing query API","description":"# WizTelemetry Tracing\n\n## Overview\n\nWizTelemetry Tracing is an extension component in the KubeSphere Observability Platform that provides distributed tracing functionality based on the [OpenTelemetry](https://opentelemetry.io/docs/specs/otel/trace/) standard.\n\n## When to Use\n\n- Installing or configuring the WizTelemetry Tracing extension\n- Understanding tracing architecture (Generator + Operator + Collector + Agent)\n- Using the tracing query API to query traces, spans, service graphs\n- Configuring OpenTelemetry auto-instrumentation for applications\n\n## Components\n\n| Component | Description | Default Enabled |\n|-----------|-------------|-----------------|\n| generator | WizTelemetry Tracing Generator: generates service graphs from tracing data (StatefulSet) | true |\n| operator | OpenTelemetry Operator: manages OpenTelemetry Collector and auto-instrumentation | true |\n| collector | OpenTelemetry Collector for WizTelemetry: receives traces, exports to Vector and OpenSearch | true |\n| agent | WizTelemetry Tracing Agent: collects tracing data from local log files (DaemonSet) | false |\n| demo | OpenTelemetry Demo: generates sample tracing data for demonstration | false |\n\n## Dependencies\n\n- **WizTelemetry Platform Service** (whizard-telemetry): Required\n- **OpenSearch** (opensearch): Required\n\n## Installation\n\n### Prerequisites\n\n**REQUIRED: Complete all steps in order before generating InstallPlan.**\n\n#### Step 1: Get Available Clusters and Confirm Target\n\n**CRITICAL: DO NOT proceed until target clusters are determined.**\n\n**Step 1.1: Get available clusters**\n\n```bash\nkubectl get clusters -o jsonpath='{.items[*].metadata.name}'\n```\n\n**Step 1.2: Determine target clusters**\n\n- If user **explicitly specified** target clusters in the request -> Use those clusters directly, proceed to Step 2\n- If user **did NOT specify** target clusters -> Ask user to confirm which clusters to deploy to, then proceed to Step 2\n\n**Ask user (if not specified):**\n```\nAvailable clusters: host, dev\nWhich clusters do you want to deploy WizTelemetry Tracing to?\n```\n\n#### Step 2: Get Latest Version (if not provided by user)\n\n**MUST do this to get the latest version:**\n\n```bash\nkubectl get extensionversions -n kubesphere-system -l kubesphere.io/extension-ref=wiztelemetry-tracing -o jsonpath='{range .items[*]}{.spec.version}{\"\\n\"}{end}' | sort -V | tail -1\n```\n\nThis outputs the latest version (e.g., `1.0.6`). Note this down - you'll use it in the InstallPlan.\n\n#### Step 3: Get Generator Endpoint\n\n**Get a node IP from the target cluster:**\n\n```bash\nkubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type==\"ExternalIP\")].address}' 2>/dev/null || \\\nkubectl get nodes -o jsonpath='{.items[0].status.addresses[?(@.type==\"InternalIP\")].address}'\n```\n\nThis outputs a node IP (e.g., `192.168.1.100`). Use this as the generator endpoint.\n\n**If multiple shards (e.g., `generator.shardCount: 3`), generate all endpoints:**\n\n```bash\nNODE_IP=\"<NODE_IP>\"\nfor i in 0 1 2; do\n  PORT=$((32318 + i))\n  echo \"  - http://${NODE_IP}:${PORT}\"\ndone\n```\n\nReplace `<NODE_IP>` with the actual node IP from the previous command.\n\n#### Step 4: Confirm Configuration with User\n\nBefore creating the InstallPlan, confirm the following with the user:\n\n```\nI'll install WizTelemetry Tracing with the following configuration:\n- Version: <VERSION>\n- Target clusters: <TARGET_CLUSTERS>\n- Generator endpoint: http://<NODE_IP>:32318\n- OpenSearch endpoint: <OPENSEARCH_ENDPOINT>\n\nDo you want to proceed? (yes/no)\n```\n\nIf user confirms, proceed to create the InstallPlan. If not, adjust the configuration based on user feedback.\n\n#### Step 5: Create InstallPlan\n\n**CRITICAL: InstallPlan `metadata.name` MUST be `wiztelemetry-tracing`. DO NOT use any other name.**\n\n**CRITICAL: `config` field is YAML format. You MUST:**\n- Use the config structure exactly as shown in the template\n- **DO NOT** add configuration fields that are not shown in the template\n- **DO NOT** modify the structure or hierarchy\n\n**CRITICAL: All placeholders MUST be replaced with actual values. DO NOT leave them as placeholders.**\n\n**Based on your selections:**\n- **Target clusters**: User-confirmed cluster names\n- **OpenSearch endpoint**: User-provided (default: `https://opensearch-cluster-data.kubesphere-logging-system.svc:9200`)\n- **OpenSearch credentials**: User-provided (default user: `admin`)\n\n#### Template\n\n```yaml\napiVersion: kubesphere.io/v1alpha1\nkind: InstallPlan\nmetadata:\n  name: wiztelemetry-tracing\n  namespace: kubesphere-system\nspec:\n  extension:\n    name: wiztelemetry-tracing\n    version: <VERSION>  # From Step 2\n  enabled: true\n  upgradeStrategy: Manual\n  config: |\n    global:\n      generator:\n        endpoints:\n          - http://<HOST_NODE_IP>:32318\n      storage:\n        opensearch:\n          auth:\n            strategy: basic\n            user: <OPENSEARCH_USER>\n            password: <OPENSEARCH_PASSWORD>\n          endpoints:\n            - <OPENSEARCH_ENDPOINT>\n    generator:\n      shardCount: 1\n      service:\n        nodePort: 32318\n  clusterScheduling:\n    placement:\n      clusters:\n        - <TARGET_CLUSTERS>\n```\n\n**Replace placeholders:**\n- `<VERSION>`: From Step 2 (e.g., `1.0.6`)\n- `<HOST_NODE_IP>`: Node IP from Step 3 (e.g., `192.168.1.100`)\n- `<OPENSEARCH_USER>`: OpenSearch username\n- `<OPENSEARCH_PASSWORD>`: OpenSearch password\n- `<OPENSEARCH_ENDPOINT>`: OpenSearch endpoint (e.g., `https://opensearch-cluster-data.kubesphere-logging-system.svc:9200`)\n- `<TARGET_CLUSTERS>`: User-confirmed cluster names\n\n### Enable Agent (Log File Collection)\n\nTo enable tracing data collection from local log files, set `agent.enabled: true` in the config. This deploys a DaemonSet that scans log files and forwards traces to the generator.\n\n### Enable Demo\n\nTo enable the OpenTelemetry Demo (generates sample trace data), set `demo.enabled: true` in the config.\n\n### Send Traces to Tempo\n\nTo forward traces to Tempo instead of OpenSearch, modify the `collector.collector.config`:\n\n```yaml\ncollector:\n  enabled: true\n  collector:\n    config: |\n      exporters:\n        otlp:\n          endpoint: <TEMPO_DISTRIBUTOR_GRPC_ENDPOINT>\n          headers:\n            x-scope-orgid: wiztelemetry-tracing-ks\n          tls:\n            insecure: true\n      service:\n        pipelines:\n          traces:\n            exporters:\n              - otlp\n              - otlphttp\n```\n\n### Multiple Generator Instances\n\nTo scale the generator for higher throughput, increase `generator.shardCount` and update `global.generator.endpoints`:\n\n```yaml\nglobal:\n  generator:\n    endpoints:\n      - http://<HOST_NODE_IP>:32318\n      - http://<HOST_NODE_IP>:32319\n      - http://<HOST_NODE_IP>:32320\n\ngenerator:\n  shardCount: 3\n  service:\n    nodePort: 32318\n```\n\n> `shardCount` must match the number of endpoints in `global.generator.endpoints`.\n> The nodePort of each shard increases in sequence starting from the configured `service.nodePort`.\n\n## Configuration Parameters\n\n### Global Parameters\n\n#### DNS Service\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `global.dnsService` | string | `coredns` | DNS service name |\n\n#### Generator Global Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `global.generator.endpoints` | list | `[\"http://<ip>:32318\"]` | Endpoints of all generator shards. Tracing data is routed to different shards by `traceId`. |\n\n#### OpenSearch Storage Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `global.storage.opensearch.auth.strategy` | string | `basic` | Auth strategy |\n| `global.storage.opensearch.auth.user` | string | `admin` | OpenSearch username |\n| `global.storage.opensearch.auth.password` | string | `admin` | OpenSearch password |\n| `global.storage.opensearch.endpoints` | list | | OpenSearch endpoint URLs |\n| `global.storage.opensearch.index.prefix` | string | `wiz-tracing-span` | Index prefix |\n| `global.storage.opensearch.index.timestring` | string | `%Y.%m.%d` | Index time format (strftime pattern) |\n\n### Generator Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `generator.shardCount` | int | `1` | Number of generator shards. Must equal the number of entries in `global.generator.endpoints`. |\n| `generator.image.tag` | string | `v1.0.2` | Generator image tag |\n| `generator.image.imagePullPolicy` | string | `IfNotPresent` | Image pull policy |\n| `generator.service.nodePort` | int | `32318` | NodePort of the first shard. Subsequent shards use consecutive ports. |\n| `generator.resources.limits.cpu` | string | `2` | CPU limit |\n| `generator.resources.limits.memory` | string | `2000Mi` | Memory limit |\n| `generator.resources.requests.cpu` | string | `100m` | CPU request |\n| `generator.resources.requests.memory` | string | `100Mi` | Memory request |\n\n### Operator Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `operator.enabled` | bool | `true` | Enable OpenTelemetry Operator |\n| `operator.fullnameOverride` | string | `wiztelemetry-tracing-operator` | Override name for the operator deployment |\n\n### Collector Parameters\n\n#### ISM Policy Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `collector.ism_policy.enable` | bool | `true` | Enable OpenSearch Index State Management policy |\n| `collector.ism_policy.span_index_pattern` | string | `*wiz-tracing-span*` | Index pattern for span indices |\n| `collector.ism_policy.service_index_pattern` | string | `*wiz-tracing-service*` | Index pattern for service indices |\n| `collector.ism_policy.min_index_age` | string | `7d` | Minimum index retention period |\n| `collector.ism_policy.span_index_priority` | int | `9950` | Index priority for span indices |\n\n#### Collector Deployment Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `collector.enabled` | bool | `true` | Enable OpenTelemetry Collector |\n| `collector.collector.mode` | string | `deployment` | Deployment mode (`deployment` or `daemonset`) |\n| `collector.collector.replicas` | int | `2` | Number of collector replicas |\n| `collector.collector.upgradeStrategy` | string | `automatic` | Upgrade strategy |\n\n### Agent Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `agent.enabled` | bool | `false` | Enable Tracing Agent (DaemonSet) |\n| `agent.dockerRootDir` | string | `/var/lib/docker` | Docker root directory |\n| `agent.config.scanInterval` | string | `1m` | Log file scan interval |\n| `agent.config.deletionDelay` | string | `5m` | Delay before deleting processed files |\n| `agent.config.logPath` | string | `/app/logs/*/*trace*.log` | Path pattern for trace log files |\n| `agent.config.podLabelSelector` | map | `{}` | Filter pods by label selector |\n| `agent.config.namespaceLabelSelector` | map | `{}` | Filter namespaces by label selector |\n| `agent.config.includeNamespaces` | list | `[]` | List of namespaces to include |\n| `agent.config.excludeNamespaces` | list | `[]` | List of namespaces to exclude |\n| `agent.resources.limits.cpu` | string | `500m` | CPU limit |\n| `agent.resources.limits.memory` | string | `500Mi` | Memory limit |\n| `agent.resources.requests.cpu` | string | `10m` | CPU request |\n| `agent.resources.requests.memory` | string | `10Mi` | Memory request |\n| `agent.vector.resources.limits.cpu` | string | `2` | Vector sidecar CPU limit |\n| `agent.vector.resources.limits.memory` | string | `2000Mi` | Vector sidecar memory limit |\n| `agent.vector.resources.requests.cpu` | string | `100m` | Vector sidecar CPU request |\n| `agent.vector.resources.requests.memory` | string | `100Mi` | Vector sidecar memory request |\n\n### Demo Parameters\n\n| Parameter | Type | Default | Description |\n|-----------|------|---------|-------------|\n| `demo.enabled` | bool | `false` | Enable OpenTelemetry Demo |\n\n## Tracing Query API\n\n### Query Traces\n\n```bash\ncurl -X POST \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/traces\" \\\n  -H \"X-Remote-User: admin\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"condition\": {}, \"limit\": 20}'\n```\n\n### Query Spans\n\n```bash\ncurl -X POST \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/spans\" \\\n  -H \"X-Remote-User: admin\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"condition\": {}, \"limit\": 20}'\n```\n\n### Get Service Graph\n\n```bash\ncurl -X POST \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/servicegraphs\" \\\n  -H \"X-Remote-User: admin\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"condition\": {\"start\": \"<START_TIME>\", \"end\": \"<END_TIME>\"}}'\n```\n\n### Get Services\n\n```bash\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/services\" \\\n  -H \"X-Remote-User: admin\"\n```\n\n### Get Tags\n\n```bash\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/tags\" \\\n  -H \"X-Remote-User: admin\"\n```\n\n### Get Values by Tag\n\n```bash\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/values?tags=span.kind&limit=100\" \\\n  -H \"X-Remote-User: admin\"\n```\n\n### Get Histogram\n\n```bash\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/histogram?key=<KEY>&kind=1&interval=15m&startTime=<START>&endTime=<END>\" \\\n  -H \"X-Remote-User: admin\"\n```\n\n**Query Parameters:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `key` | string | The key of node or edge |\n| `kind` | int | Histogram kind: 1=RequestTotal, 2=RequestTimeAverage, 3=FailedRequestTotal, 4=ResponseTotal, 5=ResponseTimeAverage, 6=FailedResponseTotal |\n| `interval` | string | Time interval (e.g., `15m`, `1h`, `1d`) |\n| `startTime` | string | Start time in seconds since epoch |\n| `endTime` | string | End time in seconds since epoch |\n\n### Get Associated Workloads\n\n```bash\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/workloads?keys=<KEYS>&startTime=<START>&endTime=<END>\" \\\n  -H \"X-Remote-User: admin\"\n```\n\n**Query Parameters:**\n\n| Parameter | Type | Description |\n|-----------|------|-------------|\n| `keys` | string | Comma-separated list of node keys |\n| `startTime` | string | Start time in seconds since epoch |\n| `endTime` | string | End time in seconds since epoch |\n\n### Get/Set Apdex Thresholds\n\n```bash\n# Get apdex thresholds\ncurl -X GET \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds?keys=<KEYS>\" \\\n  -H \"X-Remote-User: admin\"\n\n# Set apdex thresholds\ncurl -X PUT \"http://whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds\" \\\n  -H \"X-Remote-User: admin\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"serviceA:serviceB\": 0.5}'\n```\n\n## OpenTelemetry Auto-Instrumentation\n\n### Create Instrumentation Resource\n\n```bash\nkubectl apply -f - <<EOF\napiVersion: opentelemetry.io/v1alpha1\nkind: Instrumentation\nmetadata:\n  name: my-instrumentation\nspec:\n  exporter:\n    endpoint: http://wiztelemetry-tracing-collector.wiz-telemetry-tracing:4317\n  propagators:\n    - tracecontext\n    - baggage\n    - b3\n  sampler:\n    type: parentbased_traceidratio\n    argument: \"0.25\"\n  python:\n    env:\n      - name: OTEL_EXPORTER_OTLP_ENDPOINT\n        value: http://wiztelemetry-tracing-collector.wiz-telemetry-tracing:4318\n  dotnet:\n    env:\n      - name: OTEL_EXPORTER_OTLP_ENDPOINT\n        value: http://wiztelemetry-tracing-collector.wiz-telemetry-tracing:4318\n  go:\n    env:\n      - name: OTEL_EXPORTER_OTLP_ENDPOINT\n        value: http://wiztelemetry-tracing-collector.wiz-telemetry-tracing:4318\nEOF\n```\n\n### Enable Instrumentation via Annotations\n\nAdd the following annotation to a pod or namespace:\n\n| Language | Annotation |\n|----------|-----------|\n| Java | `instrumentation.opentelemetry.io/inject-java: \"true\"` |\n| NodeJS | `instrumentation.opentelemetry.io/inject-nodejs: \"true\"` |\n| Python | `instrumentation.opentelemetry.io/inject-python: \"true\"` |\n| .NET | `instrumentation.opentelemetry.io/inject-dotnet: \"true\"` |\n| Go | `instrumentation.opentelemetry.io/inject-go: \"true\"` |\n| Apache HTTPD | `instrumentation.opentelemetry.io/inject-apache-httpd: \"true\"` |\n| Nginx | `instrumentation.opentelemetry.io/inject-nginx: \"true\"` |\n| SDK only | `instrumentation.opentelemetry.io/inject-sdk: \"true\"` |\n\nAnnotation values:\n- `\"true\"` - inject from the namespace's Instrumentation resource\n- `\"my-instrumentation\"` - use a specific Instrumentation CR in the current namespace\n- `\"my-ns/my-instrumentation\"` - use an Instrumentation CR in a different namespace\n- `\"false\"` - do not inject\n\n## Extension Operations\n\n### Check Extension Status\n\n```bash\nkubectl get installplan wiztelemetry-tracing\nkubectl get extensions wiztelemetry-tracing\n```\n\n### Check Component Status\n\n```bash\nkubectl get statefulset -n wiz-telemetry-tracing\nkubectl get deployment -n wiz-telemetry-tracing\nkubectl get daemonset -n wiz-telemetry-tracing\nkubectl get pods -n wiz-telemetry-tracing\n```\n\n### Uninstall Extension\n\n**Uninstall from all clusters:**\n\n```bash\nkubectl delete installplan wiztelemetry-tracing\n```\n\n**Uninstall from specific cluster:**\n\nTo remove WizTelemetry Tracing from a specific cluster, update the InstallPlan by removing that cluster from `clusterScheduling.placement.clusters`:\n\n```yaml\napiVersion: kubesphere.io/v1alpha1\nkind: InstallPlan\nmetadata:\n  name: wiztelemetry-tracing\nspec:\n  extension:\n    name: wiztelemetry-tracing\n    version: <VERSION>\n  enabled: true\n  upgradeStrategy: Manual\n  clusterScheduling:\n    placement:\n      clusters:\n        - <REMAINING_CLUSTERS>  # Remove the cluster you want to uninstall from\n```","tags":["wiztelemetry","tracing","kubesphere","agent-skills","cloud-native","cncf","devops","ebpf","hacktoberfest","kubernetes","llm","multi-cluster"],"capabilities":["skill","source-kubesphere","skill-wiztelemetry-tracing","topic-agent-skills","topic-cloud-native","topic-cncf","topic-devops","topic-ebpf","topic-hacktoberfest","topic-kubernetes","topic-kubesphere","topic-llm","topic-multi-cluster","topic-multi-tenancy","topic-observability"],"categories":["kubesphere"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/kubesphere/kubesphere/wiztelemetry-tracing","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add kubesphere/kubesphere","source_repo":"https://github.com/kubesphere/kubesphere","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 16934 github stars · SKILL.md body (18,131 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T18:52:41.036Z","embedding":null,"createdAt":"2026-04-27T06:52:20.525Z","updatedAt":"2026-05-18T18:52:41.036Z","lastSeenAt":"2026-05-18T18:52:41.036Z","tsv":"'-1':300 '/app/logs':1088 '/dev/null':344 '/docs/specs/otel/trace/)':45 '/extension-ref=wiztelemetry-tracing':289 '/inject-apache-httpd:':1598 '/inject-dotnet:':1587 '/inject-go:':1592 '/inject-java:':1572 '/inject-nginx:':1603 '/inject-nodejs:':1577 '/inject-python:':1582 '/inject-sdk:':1609 '/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds':1457 '/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds?keys=':1442 '/kapis/tracing.wiztelemetry.io/v1alpha1/histogram?key=':1316 '/kapis/tracing.wiztelemetry.io/v1alpha1/servicegraphs':1245 '/kapis/tracing.wiztelemetry.io/v1alpha1/services':1269 '/kapis/tracing.wiztelemetry.io/v1alpha1/spans':1220 '/kapis/tracing.wiztelemetry.io/v1alpha1/tags':1284 '/kapis/tracing.wiztelemetry.io/v1alpha1/traces':1196 '/kapis/tracing.wiztelemetry.io/v1alpha1/values?tags=span.kind&limit=100':1301 '/kapis/tracing.wiztelemetry.io/v1alpha1/workloads?keys=':1391 '/my-instrumentation':1636 '/v1alpha1':563,1488,1741 '/var/lib/docker':1067 '0':338,351,384 '0.25':1513 '0.5':1472 '1':169,385,604,874,1318,1346 '1.0.6':307,617 '1.1':186 '1.2':199 '100m':924,1161 '100mi':929,1168 '10m':1137 '10mi':1142 '15m':1320,1363 '192.168.1.100':362,624 '1d':1365 '1h':1364 '1m':1073 '2':219,240,261,343,386,584,615,914,1042,1147,1348 '20':1211,1235 '2000mi':919,1154 '3':319,374,622,755,1350 '32318':389,436,593,607,750,758,806,901 '32319':751 '32320':752 '4':407,1352 '4317':1503 '4318':1526,1539,1552 '5':463,1354 '500m':1127 '500mi':1132 '5m':1080 '6':1356 '7d':1004 '9950':1013 'actual':399,524 'add':500,1558 'address':342,355 'adjust':455 'admin':557,835,840,1202,1226,1251,1275,1290,1307,1328,1399,1448,1463 'age':1002 'agent':63,123,126,639,1052,1063 'agent.config.deletiondelay':1078 'agent.config.excludenamespaces':1118 'agent.config.includenamespaces':1111 'agent.config.logpath':1086 'agent.config.namespacelabelselector':1104 'agent.config.podlabelselector':1097 'agent.config.scaninterval':1071 'agent.dockerrootdir':1065 'agent.enabled':653,1058 'agent.resources.limits.cpu':1125 'agent.resources.limits.memory':1130 'agent.resources.requests.cpu':1135 'agent.resources.requests.memory':1140 'agent.vector.resources.limits.cpu':1145 'agent.vector.resources.limits.memory':1152 'agent.vector.resources.requests.cpu':1159 'agent.vector.resources.requests.memory':1166 'annot':1557,1561,1568,1611 'apach':1594 'apdex':1431,1435,1450 'api':19,68,1187 'apivers':560,1485,1738 'appli':1482 'applic':81 'application/json':1207,1231,1256,1468 'architectur':59 'argument':1512 'ask':227,241 'associ':1383 'auth':596,831 'auto':78,107,1475 'auto-instrument':77,106,1474 'automat':1049 'avail':171,188,246 'b3':1507 'baggag':1506 'base':39,458,532 'bash':190,278,331,378,1190,1214,1239,1263,1278,1295,1310,1385,1433,1480,1654,1670,1709 'basic':598,830 'bool':939,966,1027,1059,1180 'check':1651,1667 'cluster':172,182,189,193,202,208,214,226,232,247,251,330,433,537,541,610,636,1708,1719,1727,1734,1762,1765 'clusterschedul':608,1760 'clusterscheduling.placement.clusters':1736 'collect':127,642,647 'collector':62,104,110,112,705,708,956,1019,1031,1045 'collector.collector.config':703 'collector.collector.mode':1032 'collector.collector.replicas':1040 'collector.collector.upgradestrategy':1047 'collector.enabled':1026 'collector.ism_policy.enable':965 'collector.ism_policy.min':1000 'collector.ism_policy.service':987 'collector.ism_policy.span':974,1009 'comma':1408 'comma-separ':1407 'command':405 'complet':160 'compon':28,82,83,1668 'condit':1209,1233,1258 'config':481,490,589,657,688,709 'configur':15,52,75,409,430,457,501,779,781 'confirm':174,230,408,416,447,540,635 'consecut':910 'content':1205,1229,1254,1466 'content-typ':1204,1228,1253,1465 'coredn':793 'cpu':915,925,1128,1138,1150,1164 'cr':1628,1640 'creat':413,450,464,1477 'credenti':551 'critic':176,466,480,517 'curl':1191,1215,1240,1264,1279,1296,1311,1386,1437,1452 'current':1631 'd':860,1208,1232,1257,1469 'daemonset':134,661,1039,1064,1689 'data':96,129,142,646,682,813 'default':85,548,555,789,802,826,870,936,963,1024,1056,1177 'delay':1081 'delet':1083,1711 'demo':136,138,673,678,1173,1184 'demo.enabled':684,1179 'demonstr':144 'depend':146 'deploy':234,256,659,955,1020,1034,1035,1037,1681 'descript':84,790,803,827,871,937,964,1025,1057,1178,1333,1404 'determin':184,200 'dev':249 'differ':817,1643 'direct':215 'directori':1070 'distribut':36 'dns':785,794 'docker':1068 'done':395 'dotnet':1527 'e.g':306,361,372,616,623,631,1362 'echo':391 'edg':1341 'enabl':86,585,638,644,672,675,706,941,968,1029,1061,1182,1554,1756 'end':296,1260,1376,1424 'endpoint':322,368,377,435,438,544,592,601,630,712,749,765,807,846,1498,1520,1533,1546 'endtim':1322,1374,1393,1422 'entri':884 'env':1515,1528,1541 'eof':1484,1553 'epoch':1373,1381,1421,1429 'equal':880 'exact':492 'exclud':1124 'explicit':205 'export':117,710,728,1497,1518,1531,1544 'extens':10,27,56,576,1649,1652,1663,1704,1750 'extensionvers':281 'externalip':341 'f':1483 'failedrequesttot':1351 'failedresponsetot':1357 'fals':135,145,1060,1181,1645 'feedback':461 'field':482,502 'file':133,641,651,665,1075,1085,1096 'filter':1099,1106 'first':905 'follow':418,429,1560 'format':485,863 'forward':667,694 'function':38 'generat':60,87,90,91,139,166,321,367,375,434,591,602,671,679,732,737,748,753,797,810,866,877,890 'generator.image.imagepullpolicy':893 'generator.image.tag':887 'generator.resources.limits.cpu':912 'generator.resources.limits.memory':917 'generator.resources.requests.cpu':922 'generator.resources.requests.memory':927 'generator.service.nodeport':899 'generator.shardcount':373,742,872 'get':170,187,192,262,274,280,320,323,333,346,1236,1261,1266,1276,1281,1291,1298,1308,1313,1382,1388,1434,1439,1656,1662,1672,1680,1688,1696 'get/set':1430 'global':590,747,783,798 'global.dnsservice':791 'global.generator.endpoints':745,767,804,886 'global.storage.opensearch.auth.password':838 'global.storage.opensearch.auth.strategy':828 'global.storage.opensearch.auth.user':833 'global.storage.opensearch.endpoints':843 'global.storage.opensearch.index.prefix':848 'global.storage.opensearch.index.timestring':856 'go':1540,1589 'graph':74,93,1238 'h':1197,1203,1221,1227,1246,1252,1270,1285,1302,1323,1394,1443,1458,1464 'header':713 'hierarchi':516 'higher':739 'histogram':1309,1344 'host':248 'httpd':1595 'ifnotpres':895 'imag':891,896 'includ':13,1117 'increas':741,773 'index':854,861,970,975,982,988,995,1001,1006,1010,1014 'indic':986,999,1018 'inject':1614,1648 'insecur':723 'instal':14,50,157,424 'installplan':167,317,415,452,465,467,565,1657,1712,1730,1743 'instanc':733 'instead':698 'instrument':79,108,1476,1478,1490,1495,1555,1619,1623,1627,1639 'instrumentation.opentelemetry.io':1571,1576,1581,1586,1591,1597,1602,1608 'instrumentation.opentelemetry.io/inject-apache-httpd:':1596 'instrumentation.opentelemetry.io/inject-dotnet:':1585 'instrumentation.opentelemetry.io/inject-go:':1590 'instrumentation.opentelemetry.io/inject-java:':1570 'instrumentation.opentelemetry.io/inject-nginx:':1601 'instrumentation.opentelemetry.io/inject-nodejs:':1575 'instrumentation.opentelemetry.io/inject-python:':1580 'instrumentation.opentelemetry.io/inject-sdk:':1607 'int':873,900,1012,1041,1343 'internalip':354 'interv':1077,1319,1358,1361 'ip':326,360,380,393,401,619 'ism':958 'item':196,293,337,350 'java':1569 'jsonpath':195,291,336,349 'key':1334,1337,1405,1413 'kind':564,1317,1342,1345,1489,1742 'ks':721 'kubectl':191,279,332,345,1481,1655,1661,1671,1679,1687,1695,1710 'kubespher':12,31,284,573 'kubesphere-system':283,572 'kubesphere.io':288,562,1740 'kubesphere.io/extension-ref=wiztelemetry-tracing':287 'kubesphere.io/v1alpha1':561,1739 'l':286 'label':1102,1109 'languag':1567 'latest':263,276,304 'leav':528 'limit':916,921,1129,1134,1151,1158,1210,1234 'list':805,844,1112,1113,1119,1120,1410 'll':312,423 'local':131,649 'log':132,640,650,664,1074,1090,1095 'm':859 'manag':102,972 'manual':588,1759 'map':1098,1105 'match':761 'memori':920,930,1133,1143,1157,1171 'metadata':566,1491,1744 'metadata.name':197,468 'minimum':1005 'mode':1036 'modifi':512,701 'multipl':370,731 'must':270,469,487,520,760,879 'my-instrument':1493,1621 'my-n':1633 'n':282,295,1674,1682,1690,1698 'name':479,542,567,577,637,796,951,1492,1516,1529,1542,1745,1751 'namespac':571,1107,1115,1122,1566,1617,1632,1644 'net':1584 'nginx':1600 'node':325,334,347,359,379,392,400,618,1339,1412 'nodej':1574 'nodeport':606,757,769,902 'note':308 'ns':1635 'number':763,875,882,1043 'o':194,290,335,348 'observ':32 'opensearch':121,154,155,437,543,550,595,625,627,629,700,821,836,841,845,969 'opensearch-cluster-data.kubesphere-logging-system.svc:9200':549,632 'opentelemetri':42,76,100,103,111,137,677,942,1030,1183,1473 'opentelemetry.io':44,1487 'opentelemetry.io/docs/specs/otel/trace/)':43 'opentelemetry.io/v1alpha1':1486 'oper':61,99,101,932,943,949,954,1650 'operator.enabled':938 'operator.fullnameoverride':944 'order':164 'orgid':717 'otel':1517,1530,1543 'otlp':711,729,1519,1532,1545 'otlphttp':730 'output':302,357 'overrid':950 'overview':22 'paramet':782,784,787,799,800,823,824,867,868,933,934,957,960,961,1021,1022,1053,1054,1174,1175,1330,1331,1401,1402 'parentbas':1510 'password':600,628,842 'path':1091 'pattern':865,976,983,989,996,1092 'period':1008 'pipelin':726 'placehold':519,531,612 'placement':609,1761 'platform':33,148 'pod':1100,1564,1697 'polici':898,959,973 'port':388,394,911 'post':1193,1217,1242 'prefix':855 'prerequisit':158 'previous':404 'prioriti':1011,1015 'proceed':179,216,237,443,448 'process':1084 'propag':1504 'provid':35,267,547,554 'pull':897 'put':1454 'python':1514,1579 'queri':18,67,70,1186,1188,1212,1329,1400 'rang':292 'receiv':115 'remot':1200,1224,1249,1273,1288,1305,1326,1397,1446,1461 'remov':1721,1732,1763 'replac':396,522,611 'replica':1046 'request':211,926,931,1139,1144,1165,1172 'requesttimeaverag':1349 'requesttot':1347 'requir':153,156,159 'resourc':1479,1620 'responsetimeaverag':1355 'responsetot':1353 'retent':1007 'root':1069 'rout':815 'sampl':140,680 'sampler':1508 'scale':735 'scan':663,1076 'scope':716 'sdk':1605 'second':1371,1379,1419,1427 'select':535 'selector':1103,1110 'send':689 'separ':1409 'sequenc':775 'servic':73,92,149,605,725,756,786,795,994,998,1237,1262 'service.nodeport':780 'servicea':1470 'serviceb':1471 'set':652,683,1449 'shard':371,772,811,818,878,906,908 'shardcount':603,754,759 'shown':494,506 'sidecar':1149,1156,1163,1170 'sinc':1372,1380,1420,1428 'skill' 'skill-wiztelemetry-tracing' 'sort':297 'source-kubesphere' 'span':72,853,981,985,1017,1213 'spec':575,1496,1749 'spec.version':294 'specif':1626,1718,1726 'specifi':206,224,245 'standard':46 'start':776,1259,1368,1416 'starttim':1321,1366,1392,1414 'state':971 'statefulset':97,1673 'status':1653,1669 'status.addresses':339,352 'step':162,168,185,198,218,239,260,318,406,462,583,614,621 'storag':594,822 'strategi':597,832,1051 'strftime':864 'string':792,829,834,839,849,857,888,894,913,918,923,928,945,977,990,1003,1033,1048,1066,1072,1079,1087,1126,1131,1136,1141,1146,1153,1160,1167,1335,1359,1367,1375,1406,1415,1423 'structur':491,514 'subsequ':907 'system':285,574 'tag':892,1277,1294 'tail':299 'target':175,181,201,207,225,329,432,536 'telemetri':152,1501,1524,1537,1550,1677,1685,1693,1701 'telemetry-trac':1500,1523,1536,1549 'templat':497,509,558 'tempo':692,697 'threshold':1432,1436,1451 'throughput':740 'time':862,1360,1369,1377,1417,1425 'tls':722 'topic-agent-skills' 'topic-cloud-native' 'topic-cncf' 'topic-devops' 'topic-ebpf' 'topic-hacktoberfest' 'topic-kubernetes' 'topic-kubesphere' 'topic-llm' 'topic-multi-cluster' 'topic-multi-tenancy' 'topic-observability' 'trace':3,9,17,21,24,37,55,58,66,71,89,95,116,125,128,141,258,426,473,570,580,645,668,681,690,695,720,727,812,852,948,980,993,1062,1089,1094,1185,1189,1502,1525,1538,1551,1660,1666,1678,1686,1694,1702,1715,1723,1748,1754 'tracecontext':1505 'traceid':820 'traceidratio':1511 'true':98,109,122,586,654,685,707,724,940,967,1028,1573,1578,1583,1588,1593,1599,1604,1610,1613,1757 'type':340,353,788,801,825,869,935,962,1023,1055,1176,1206,1230,1255,1332,1403,1467,1509 'understand':57 'uninstal':1703,1705,1716,1769 'updat':744,1728 'upgrad':1050 'upgradestrategi':587,1758 'url':847 'use':4,49,64,212,313,363,476,488,909,1624,1637 'user':204,221,228,242,269,411,421,446,460,539,546,553,556,599,634,1201,1225,1250,1274,1289,1306,1327,1398,1447,1462 'user-confirm':538,633 'user-provid':545,552 'usernam':626,837 'v':298 'v1.0.2':889 'valu':525,1292,1521,1534,1547,1612 'vector':119,1148,1155,1162,1169 'version':264,277,305,431,581,1755 'via':1556 'want':254,441,1767 'whizard':151 'whizard-telemetri':150 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80':1195,1219,1244,1268,1283,1300,1315,1390,1441,1456 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds':1455 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/apdex/thresholds?keys=':1440 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/histogram?key=':1314 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/servicegraphs':1243 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/services':1267 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/spans':1218 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/tags':1282 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/traces':1194 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/values?tags=span.kind&limit=100':1299 'whizard-telemetry-apiserver.extension-whizard-telemetry.svc:80/kapis/tracing.wiztelemetry.io/v1alpha1/workloads?keys=':1389 'wiz':851,979,992,1676,1684,1692,1700 'wiz-telemetry-trac':1675,1683,1691,1699 'wiz-tracing-servic':991 'wiz-tracing-span':850,978 'wiztelemetri':2,8,20,23,54,88,114,124,147,257,425,472,569,579,719,947,1659,1665,1714,1722,1747,1753 'wiztelemetry-trac':1,471,568,578,1658,1664,1713,1746,1752 'wiztelemetry-tracing-collector.wiz':1499,1522,1535,1548 'wiztelemetry-tracing-k':718 'wiztelemetry-tracing-oper':946 'work':6 'workload':1384 'x':715,1192,1199,1216,1223,1241,1248,1265,1272,1280,1287,1297,1304,1312,1325,1387,1396,1438,1445,1453,1460 'x-remote-us':1198,1222,1247,1271,1286,1303,1324,1395,1444,1459 'x-scope-orgid':714 'y':858 'yaml':484,559,704,746,1737 'yes/no':444","prices":[{"id":"3a27f17d-410f-40aa-a3b7-f7f2c3381da1","listingId":"64ae7c9e-6e1c-48a9-be20-7914924916f8","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"kubesphere","category":"kubesphere","install_from":"skills.sh"},"createdAt":"2026-04-27T06:52:20.525Z"}],"sources":[{"listingId":"64ae7c9e-6e1c-48a9-be20-7914924916f8","source":"github","sourceId":"kubesphere/kubesphere/wiztelemetry-tracing","sourceUrl":"https://github.com/kubesphere/kubesphere/tree/master/skills/wiztelemetry-tracing","isPrimary":false,"firstSeenAt":"2026-04-27T06:52:20.525Z","lastSeenAt":"2026-05-18T18:52:41.036Z"}],"details":{"listingId":"64ae7c9e-6e1c-48a9-be20-7914924916f8","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"kubesphere","slug":"wiztelemetry-tracing","github":{"repo":"kubesphere/kubesphere","stars":16934,"topics":["agent-skills","ai","cloud-native","cncf","devops","ebpf","hacktoberfest","kubernetes","kubesphere","llm","multi-cluster","multi-tenancy","observability","servicemesh","skills","skills-sh","skillsmp"],"license":"other","html_url":"https://github.com/kubesphere/kubesphere","pushed_at":"2026-05-06T07:16:53Z","description":"The container platform tailored for Kubernetes multi-cloud, datacenter, and edge management ⎈ 🖥 ☁️","skill_md_sha":"e0cf3ae5b28e4afcefb5243782216b0aa85ea086","skill_md_path":"skills/wiztelemetry-tracing/SKILL.md","default_branch":"master","skill_tree_url":"https://github.com/kubesphere/kubesphere/tree/master/skills/wiztelemetry-tracing"},"layout":"multi","source":"github","category":"kubesphere","frontmatter":{"name":"wiztelemetry-tracing","description":"Use when working with WizTelemetry Tracing extension for KubeSphere, including installation, configuration, and tracing query API"},"skills_sh_url":"https://skills.sh/kubesphere/kubesphere/wiztelemetry-tracing"},"updatedAt":"2026-05-18T18:52:41.036Z"}}