{"id":"a4e75515-6f51-40b3-98b8-00fd6bfb08a4","shortId":"eC9LGA","kind":"skill","title":"sn-da-image-caption","tagline":"图片理解与数据提取 skill。当图片文件（.png/.jpg/.jpeg/.gif/.webp/.bmp）是主要输入且用户需要理解、提取数据或分析图片内容时使用。提供预配置的 caption 脚本（scripts/caption.py），通过 vision 模型将图片转为文本描述，无需额外配置 API Key。覆盖：(1) 通过 scripts/caption.py 对图表/表格/截图/流程图进行 caption，(2) 将 caption 文本解析为结构化 DataFrame，(3) 基于提取数据重新生成可视化图表，(4) 导出为 Excel/CSV","description":"# Image Caption Analysis — 图片描述与数据提取\n\n## Overview\n\nAnalyze, extract data from, or understand image files (.png, .jpg, .jpeg, .gif, .webp, .bmp). The core workflow:\n\n1. Run `scripts/caption.py` to get a text description of the image\n2. Parse the description into structured data (DataFrame, etc.)\n3. Analyze, visualize, or export\n\n## scripts/caption.py — Image Caption\n\nThe script converts images to text descriptions via a vision model. Configure via `SN_API_KEY` (minimum required), or use `SN_VISION_API_KEY` / `SN_VISION_BASE_URL` / `SN_VISION_MODEL` for fine-grained control. See the project environment variable spec for the full fallback chain.\n\n### Usage\n\n```bash\n# Basic — get text description\npython3 scripts/caption.py /mnt/data/image.png\n\n# Custom prompt — guide what to extract\npython3 scripts/caption.py /mnt/data/chart.png --prompt \"提取所有数值，Markdown 表格格式\"\n\n# JSON output — includes detected type, usage stats, cache info\npython3 scripts/caption.py /mnt/data/image.png --json\n\n# Batch — process all images in a directory\npython3 scripts/caption.py /mnt/data/images/ --batch --output /mnt/data/captions.json\n\n# Override model (optional)\npython3 scripts/caption.py /mnt/data/image.png --model gemini-3.1-flash-lite-preview\n```\n\n### Options\n\n| Option | Description |\n|--------|------------|\n| `--prompt, -p` | Custom prompt (overrides auto-detection) |\n| `--model, -m` | Vision model (default: sensenova-6.7-flash-lite) |\n| `--json` | Output structured JSON instead of plain text |\n| `--batch` | Process all images in a directory |\n| `--output, -o` | Output file for batch results |\n| `--no-cache` | Skip MD5 cache |\n\n### What it does automatically\n\n- **Type detection**: Detects image type from filename (chart/table/UI/diagram/general) and picks the best prompt\n- **Compression**: Images >5MB or >2048px are compressed before sending\n- **Caching**: Same image + same prompt → instant cached result, no API cost\n- **Error handling**: Retries on failure, returns error message on permanent failure\n\n### JSON output format\n\n```json\n{\n  \"file\": \"/mnt/data/image.png\",\n  \"type\": \"chart\",\n  \"description\": \"这是一张柱状图...\",\n  \"usage\": {\"prompt_tokens\": 1100, \"completion_tokens\": 400},\n  \"cached\": false\n}\n```\n\n## Calling from Python\n\n```python\nimport subprocess, json\n\nCAPTION = \"/path/to/skills/sn-da-image-caption/scripts/caption.py\"\n\n# Single image\nresult = subprocess.run(\n    [\"python3\", CAPTION, \"/mnt/data/chart.png\", \"--json\",\n     \"--prompt\", \"提取图表数据，Markdown 表格输出\"],\n    capture_output=True, text=True, timeout=60\n)\ndata = json.loads(result.stdout)\ndescription = data[\"description\"]\n\n# Batch\nresult = subprocess.run(\n    [\"python3\", CAPTION, \"/mnt/data/images/\", \"--batch\",\n     \"--output\", \"/mnt/data/captions.json\"],\n    capture_output=True, text=True, timeout=300\n)\nwith open(\"/mnt/data/captions.json\") as f:\n    all_captions = json.load(f)\n```\n\n## Prompt Strategy\n\nDifferent image types need different prompts. The script auto-detects, but specifying `--prompt` gives better results.\n\n| Image Type | When | Recommended --prompt |\n|-----------|------|---------------------|\n| Data chart | 柱状图/折线图/饼图 | `\"提取图表标题、坐标轴、每个数据点数值、图例。Markdown 表格输出。\"` |\n| Table screenshot | 表格截图 | `\"提取表格所有内容，Markdown 表格格式，保持行列结构，数值不四舍五入。\"` |\n| UI screenshot | 界面截图 | `\"以前端开发者视角描述：布局、组件、文字、颜色。\"` |\n| Diagram | 流程图/架构图 | `\"描述所有节点、连接关系（A→B）、分支条件。\"` |\n| General | 照片、其他 | 不传 --prompt，用默认 |\n\n## Parsing Caption Results\n\nCaption 通常返回 Markdown 表格，解析为 DataFrame：\n\n```python\nimport pandas as pd\n\ndef parse_markdown_table(text):\n    lines = text.strip().split('\\n')\n    table_lines = []\n    in_table = False\n    for line in lines:\n        stripped = line.strip()\n        if '|' in stripped:\n            in_table = True\n            table_lines.append(stripped)\n        elif in_table:\n            break\n\n    data_lines = []\n    for l in table_lines:\n        cells = [c.strip() for c in l.split('|') if c.strip()]\n        if cells and not all(set(c) <= set('-: ') for c in cells):\n            data_lines.append(cells)\n\n    if len(data_lines) < 2:\n        return None\n\n    header = data_lines[0]\n    rows = [r for r in data_lines[1:] if len(r) == len(header)]\n    df = pd.DataFrame(rows, columns=header)\n\n    # Auto numeric conversion\n    for col in df.columns:\n        try:\n            cleaned = df[col].str.replace(',', '').str.strip()\n            if cleaned.str.endswith('%').any():\n                df[col] = pd.to_numeric(cleaned.str.rstrip('%'), errors='coerce')\n            else:\n                converted = pd.to_numeric(cleaned, errors='coerce')\n                if converted.notna().sum() > len(df) * 0.5:\n                    df[col] = converted\n        except Exception:\n            pass\n    return df\n```\n\n## Visualization\n\n### Chinese Font Setup (MANDATORY)\n\n```python\nimport matplotlib.pyplot as plt\nimport matplotlib\nimport os\n\nfont_path = '/usr/share/fonts/truetype/wqy/wqy-zenhei.ttc'\nif os.path.exists(font_path):\n    matplotlib.rcParams['font.family'] = 'WenQuanYi Zen Hei'\nmatplotlib.rcParams['axes.unicode_minus'] = False\n```\n\n### Color Palette\n\n```python\nCOLORS = ['#4C72B0', '#55A868', '#C44E52', '#8172B2', '#CCB974', '#64B5CD']\n```\n\n### Save & Display\n\n```python\nplt.savefig('/mnt/data/chart.png', dpi=150, bbox_inches='tight')\nplt.show()\nprint(\"![图表](sandbox:/mnt/data/chart.png)\")\n```\n\n## Export to Excel\n\n```python\nfrom openpyxl.styles import Font, PatternFill, Alignment\n\noutput_path = \"/mnt/data/result.xlsx\"\nwith pd.ExcelWriter(output_path, engine='openpyxl') as writer:\n    df.to_excel(writer, index=False, sheet_name='提取数据')\n    ws = writer.sheets['提取数据']\n    fill = PatternFill(start_color='4472C4', end_color='4472C4', fill_type='solid')\n    for cell in ws[1]:\n        cell.font = Font(bold=True, color='FFFFFF')\n        cell.fill = fill\n        cell.alignment = Alignment(horizontal='center')\n    for i, col in enumerate(df.columns, 1):\n        w = max(df[col].astype(str).str.len().max(), len(str(col))) + 2\n        ws.column_dimensions[chr(64 + i)].width = min(w * 1.2, 40)\n\nprint(f\"[下载](sandbox:{output_path})\")\n```\n\n## Multi-Image Processing\n\n```python\nimport glob\n\nimage_files = sorted(glob.glob(\"/mnt/data/*.png\"))\nall_dfs = []\n\nfor img in image_files:\n    r = subprocess.run(\n        [\"python3\", CAPTION, img, \"--json\", \"--prompt\", \"提取数据，Markdown 表格\"],\n        capture_output=True, text=True, timeout=60\n    )\n    desc = json.loads(r.stdout)[\"description\"]\n    df = parse_markdown_table(desc)\n    if df is not None:\n        all_dfs.append(df)\n\ncombined = pd.concat(all_dfs, ignore_index=True) if all_dfs else None\n```\n\nOr batch mode:\n\n```bash\npython3 scripts/caption.py /mnt/data/images/ --batch --output /mnt/data/captions.json\n```\n\n## Common Pitfalls\n\n- **Always caption first** — don't guess image content from filenames\n- **Use --prompt for precision** — auto-detect is OK, explicit prompt is better\n- **Verify extracted data** — check sums, percentages, row counts after parsing\n- **Large tables truncate** — caption in two passes: `\"提取前半部分\"` + `\"提取后半部分\"`\n- **Chinese font** — must set before any matplotlib call, or output is garbled\n- **Timeout** — single image ~10-30s, batch set timeout accordingly","tags":["image","caption","sensenova","skills","opensensenova","agent","agent-skills","ai-agents","ai-assistant","data-analysis","document-processing","office-automation"],"capabilities":["skill","source-opensensenova","skill-sn-da-image-caption","topic-agent","topic-agent-skills","topic-ai-agents","topic-ai-assistant","topic-data-analysis","topic-document-processing","topic-office-automation","topic-presentation-slides"],"categories":["SenseNova-Skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/OpenSenseNova/SenseNova-Skills/sn-da-image-caption","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add OpenSenseNova/SenseNova-Skills","source_repo":"https://github.com/OpenSenseNova/SenseNova-Skills","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 1627 github stars · SKILL.md body (7,125 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-05-18T18:53:04.472Z","embedding":null,"createdAt":"2026-05-15T06:53:09.636Z","updatedAt":"2026-05-18T18:53:04.472Z","lastSeenAt":"2026-05-18T18:53:04.472Z","tsv":"'-3.1':194 '-30':872 '-6.7':216 '/mnt/data':748 '/mnt/data/captions.json':185,357,367,811 '/mnt/data/chart.png':155,330,631,641 '/mnt/data/image.png':146,171,191,301 '/mnt/data/images':182,354,808 '/mnt/data/result.xlsx':654 '/path/to/skills/sn-da-image-caption/scripts/caption.py':323 '/usr/share/fonts/truetype/wqy/wqy-zenhei.ttc':603 '0':524 '0.5':578 '1':23,63,532,689,708 '1.2':729 '10':871 '1100':309 '150':633 '2':31,74,518,720 '2048px':269 '3':36,83 '300':364 '4':38 '40':730 '400':312 '4472c4':678,681 '4c72b0':621 '55a868':622 '5mb':267 '60':342,773 '64':724 '64b5cd':626 '8172b2':624 'accord':877 'align':651,699 'all_dfs.append':788 'alway':814 'analysi':43 'analyz':46,84 'api':20,105,113,283 'astyp':713 'auto':208,385,543,829 'auto-detect':207,384,828 'automat':251 'axes.unicode':614 'b':431 'base':117 'bash':139,805 'basic':140 'batch':173,183,228,240,349,355,803,809,874 'bbox':634 'best':263 'better':391,836 'bmp':59 'bold':692 'break':484 'c':495,506,509 'c.strip':493,499 'c44e52':623 'cach':167,244,247,274,280,313 'call':315,863 'caption':5,13,30,33,42,90,322,329,353,371,440,442,760,815,850 'captur':336,358,767 'ccb974':625 'cell':492,501,511,513,686 'cell.alignment':698 'cell.fill':696 'cell.font':690 'center':701 'chain':137 'chart':303,399 'chart/table/ui/diagram/general':259 'check':840 'chines':588,856 'chr':723 'clean':551,570 'cleaned.str.endswith':557 'cleaned.str.rstrip':563 'coerc':565,572 'col':547,553,560,580,704,712,719 'color':617,620,677,680,694 'column':541 'combin':790 'common':812 'complet':310 'compress':265,271 'configur':102 'content':821 'control':126 'convers':545 'convert':93,567,581 'converted.notna':574 'core':61 'cost':284 'count':844 'custom':147,204 'da':3 'data':48,80,343,347,398,485,516,522,530,839 'data_lines.append':512 'datafram':35,81,447 'def':453 'default':214 'desc':774,782 'descript':70,77,97,143,201,304,346,348,777 'detect':163,209,253,254,386,830 'df':538,552,559,577,579,586,711,778,784,789 'df.columns':549,707 'df.to':663 'dfs':751,793,799 'diagram':425 'differ':376,380 'dimens':722 'directori':179,234 'display':628 'dpi':632 'elif':481 'els':566,800 'end':679 'engin':659 'enumer':706 'environ':130 'error':285,291,564,571 'etc':82 'excel':644,664 'excel/csv':40 'except':582,583 'explicit':833 'export':87,642 'extract':47,152,838 'f':369,373,732 'failur':289,295 'fallback':136 'fals':314,466,616,667 'ffffff':695 'file':53,238,300,745,756 'filenam':258,823 'fill':674,682,697 'fine':124 'fine-grain':123 'first':816 'flash':196,218 'flash-lit':217 'flash-lite-preview':195 'font':589,601,606,649,691,857 'font.family':609 'format':298 'full':135 'garbl':867 'gemini':193 'general':433 'get':67,141 'gif':57 'give':390 'glob':743 'glob.glob':747 'grain':125 'guess':819 'guid':149 'handl':286 'header':521,537,542 'hei':612 'horizont':700 'ignor':794 'imag':4,41,52,73,89,94,176,231,255,266,276,325,377,393,739,744,755,820,870 'img':753,761 'import':319,449,593,597,599,648,742 'inch':635 'includ':162 'index':666,795 'info':168 'instant':279 'instead':224 'jpeg':56 'jpg':55 'json':160,172,220,223,296,299,321,331,762 'json.load':372 'json.loads':344,775 'key':21,106,114 'l':488 'l.split':497 'larg':847 'len':515,534,536,576,717 'line':458,463,468,470,486,491,517,523,531 'line.strip':472 'lite':197,219 'm':211 'mandatori':591 'markdown':158,334,407,413,444,455,765,780 'matplotlib':598,862 'matplotlib.pyplot':594 'matplotlib.rcparams':608,613 'max':710,716 'md5':246 'messag':292 'min':727 'minimum':107 'minus':615 'mode':804 'model':101,121,187,192,210,213 'multi':738 'multi-imag':737 'must':858 'n':461 'name':669 'need':379 'no-cach':242 'none':520,787,801 'numer':544,562,569 'o':236 'ok':832 'open':366 'openpyxl':660 'openpyxl.styles':647 'option':188,199,200 'os':600 'os.path.exists':605 'output':161,184,221,235,237,297,337,356,359,652,657,735,768,810,865 'overrid':186,206 'overview':45 'p':203 'palett':618 'panda':450 'pars':75,439,454,779,846 'pass':584,853 'path':602,607,653,658,736 'patternfil':650,675 'pd':452 'pd.concat':791 'pd.dataframe':539 'pd.excelwriter':656 'pd.to':561,568 'percentag':842 'perman':294 'pick':261 'pitfal':813 'plain':226 'plt':596 'plt.savefig':630 'plt.show':637 'png':54,749 'png/.jpg/.jpeg/.gif/.webp/.bmp':9 'precis':827 'preview':198 'print':638,731 'process':174,229,740 'project':129 'prompt':148,156,202,205,264,278,307,332,374,381,389,397,437,763,825,834 'python':317,318,448,592,619,629,645,741 'python3':144,153,169,180,189,328,352,759,806 'r':526,528,535,757 'r.stdout':776 'recommend':396 'requir':108 'result':241,281,326,350,392,441 'result.stdout':345 'retri':287 'return':290,519,585 'row':525,540,843 'run':64 'sandbox':640,734 'save':627 'screenshot':410,418 'script':92,383 'scripts/caption.py':15,25,65,88,145,154,170,181,190,807 'see':127 'send':273 'sensenova':215 'set':505,507,859,875 'setup':590 'sheet':668 'singl':324,869 'skill':7 'skill-sn-da-image-caption' 'skip':245 'sn':2,104,111,115,119 'sn-da-image-capt':1 'solid':684 'sort':746 'source-opensensenova' 'spec':132 'specifi':388 'split':460 'start':676 'stat':166 'str':714,718 'str.len':715 'str.replace':554 'str.strip':555 'strategi':375 'strip':471,475,480 'structur':79,222 'subprocess':320 'subprocess.run':327,351,758 'sum':575,841 'tabl':409,456,462,465,477,483,490,781,848 'table_lines.append':479 'text':69,96,142,227,339,361,457,770 'text.strip':459 'tight':636 'timeout':341,363,772,868,876 'token':308,311 'topic-agent' 'topic-agent-skills' 'topic-ai-agents' 'topic-ai-assistant' 'topic-data-analysis' 'topic-document-processing' 'topic-office-automation' 'topic-presentation-slides' 'tri':550 'true':338,340,360,362,478,693,769,771,796 'truncat':849 'two':852 'type':164,252,256,302,378,394,683 'ui':417 'understand':51 'url':118 'usag':138,165,306 'use':110,824 'variabl':131 'verifi':837 'via':98,103 'vision':17,100,112,116,120,212 'visual':85,587 'w':709,728 'webp':58 'wenquanyi':610 'width':726 'workflow':62 'writer':662,665 'writer.sheets':672 'ws':671,688 'ws.column':721 'zen':611 '下载':733 '不传':436 '以前端开发者视角描述':420 '保持行列结构':415 '其他':435 '分支条件':432 '图例':406 '图片描述与数据提取':44 '图片理解与数据提取':6 '图表':639 '坐标轴':404 '基于提取数据重新生成可视化图表':37 '对图表':26 '导出为':39 '将':32 '布局':421 '当图片文件':8 '截图':28 '折线图':401 '描述所有节点':428 '提供预配置的':12 '提取前半部分':854 '提取后半部分':855 '提取图表数据':333 '提取图表标题':403 '提取所有数值':157 '提取数据':670,673,764 '提取数据或分析图片内容时使用':11 '提取表格所有内容':412 '数值不四舍五入':416 '文字':423 '文本解析为结构化':34 '无需额外配置':19 '是主要输入且用户需要理解':10 '架构图':427 '柱状图':400 '模型将图片转为文本描述':18 '每个数据点数值':405 '流程图':426 '流程图进行':29 '照片':434 '用默认':438 '界面截图':419 '组件':422 '脚本':14 '表格':27,445,766 '表格截图':411 '表格格式':159,414 '表格输出':335,408 '覆盖':22 '解析为':446 '这是一张柱状图':305 '连接关系':429 '通常返回':443 '通过':16,24 '颜色':424 '饼图':402","prices":[{"id":"50166ec8-024b-41b2-9e1e-67035b7a3ca5","listingId":"a4e75515-6f51-40b3-98b8-00fd6bfb08a4","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"OpenSenseNova","category":"SenseNova-Skills","install_from":"skills.sh"},"createdAt":"2026-05-15T06:53:09.636Z"}],"sources":[{"listingId":"a4e75515-6f51-40b3-98b8-00fd6bfb08a4","source":"github","sourceId":"OpenSenseNova/SenseNova-Skills/sn-da-image-caption","sourceUrl":"https://github.com/OpenSenseNova/SenseNova-Skills/tree/main/skills/sn-da-image-caption","isPrimary":false,"firstSeenAt":"2026-05-15T06:53:09.636Z","lastSeenAt":"2026-05-18T18:53:04.472Z"}],"details":{"listingId":"a4e75515-6f51-40b3-98b8-00fd6bfb08a4","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"OpenSenseNova","slug":"sn-da-image-caption","github":{"repo":"OpenSenseNova/SenseNova-Skills","stars":1627,"topics":["agent","agent-skills","ai-agents","ai-assistant","data-analysis","document-processing","office-automation","presentation-slides"],"license":"mit","html_url":"https://github.com/OpenSenseNova/SenseNova-Skills","pushed_at":"2026-05-15T04:43:37Z","description":"Modular SenseNova skills for building AI-powered office assistants and productivity workflows","skill_md_sha":"93190fb4a22cf09d1b2dd20df5cff63d684f7cae","skill_md_path":"skills/sn-da-image-caption/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/OpenSenseNova/SenseNova-Skills/tree/main/skills/sn-da-image-caption"},"layout":"multi","source":"github","category":"SenseNova-Skills","frontmatter":{"name":"sn-da-image-caption","description":"图片理解与数据提取 skill。当图片文件（.png/.jpg/.jpeg/.gif/.webp/.bmp）是主要输入且用户需要理解、提取数据或分析图片内容时使用。提供预配置的 caption 脚本（scripts/caption.py），通过 vision 模型将图片转为文本描述，无需额外配置 API Key。覆盖：(1) 通过 scripts/caption.py 对图表/表格/截图/流程图进行 caption，(2) 将 caption 文本解析为结构化 DataFrame，(3) 基于提取数据重新生成可视化图表，(4) 导出为 Excel/CSV。**遇到以下任一情况就主动使用本 skill，不要自行猜测图片内容**：①用户出现触发词：图片分析 / 图表提取 / 表格识别 / OCR / 图片描述 / 截图分析 / 图表数据 / 提取图片中的数据 / 图片转表格 / 识别图片 / image caption / extract data from image / chart analysis / table OCR；②用户上传或指定了图片文件（.png / .jpg / .jpeg / .gif / .webp / .bmp）并要求理解、提取数据或分析内容；③任务需要从图表截图、表格截图、UI 截图、流程图中提取结构化信息；④用户要求将图片中的数据转为 Excel/CSV 或重新生成可视化图表。仅不用于：图片编辑（裁剪、滤镜、缩放）、图片生成、不含数据的风景/人物照片描述。"},"skills_sh_url":"https://skills.sh/OpenSenseNova/SenseNova-Skills/sn-da-image-caption"},"updatedAt":"2026-05-18T18:53:04.472Z"}}