{"id":"40c1f5a0-abe3-4a75-8a32-6934047cb7c0","shortId":"rZU53y","kind":"skill","title":"review-and-promote-traces","tagline":"Judge flagged trace outputs and promote judged items back to datasets. Use when an evaluation run requires human judgment or when review queue items need to be judged for promotion into the dataset.","description":"# Review and Promote Traces\n\nUse this skill for the judgment and promotion loop after trace evaluation.\n\n## Interactive Q&A protocol (mandatory)\n\n\nBEFORE the first scoping question, search for a structured question tool (e.g., `AskUserQuestion` or similar interactive widget) and load it. Use that tool for EVERY scoping question. Fall back to plain-text lettered options ONLY if no such tool exists in the environment.\n\n\nIf context is unclear, ask one question at a time using the structured question tool (loaded per the HARD-GATE above).\n\nExample question structure:\n\n```\nWhat do you want to review first?\nA) One specific run\nB) Pending queue triage across runs\n```\n\nRules:\n- Ask one question per message.\n- Use the structured question tool for every question. Structure each with a short header, 2-4 options with labels and descriptions, and place the recommended option first. Do not add \"(Recommended)\" or similar annotations to option labels.\n- Ask one follow-up when needed, then continue.\n\n## Workflow\n\n1. If needed, flag evaluated run:\n - Use `flag_review_item` with `run_id`.\n2. Retrieve review items:\n - Use `list_review_items` with pagination.\n3. Collect human judgments:\n - Present review items to the user and ask for judgment one item at a time when needed.\n - Capture `judgment_value` and optional `notes` for each item.\n - Use the structured question tool (loaded per the HARD-GATE above) to present options that match the live evaluation setup:\n - Binary example:\n - A) Pass\n - B) Fail\n - Categorical example:\n - A)