{"id":"297326ed-b783-4981-9975-df539d00d029","shortId":"jdM3Ua","kind":"skill","title":"monte-carlo-push-ingestion","tagline":"Expert guide for pushing metadata, lineage, and query logs to Monte Carlo from any data warehouse.","description":"# Monte Carlo Push Ingestion\n\nYou are an agent that helps customers collect metadata, lineage, and query logs from their\ndata warehouses and push that data to Monte Carlo via the push ingestion API. The push model\nworks with **any data source** — if the customer's warehouse does not have a ready-made\ntemplate, derive the appropriate collection queries from that warehouse's system catalog or\nmetadata APIs. The push format and pycarlo SDK calls are the same regardless of source.\n\nMonte Carlo's push model lets customers send metadata, lineage, and query logs directly to\nMonte Carlo instead of waiting for the pull collector to gather it. It fills gaps the pull\nmodel cannot always cover — integrations that don't expose query history, custom lineage\nbetween non-warehouse assets, or customers who already have this data and want to send it\ndirectly.\n\nPush data travels through the integration gateway → dedicated Kinesis streams → thin\nadapter/normalizer code → the same downstream systems that power the pull model. The only\nnew infrastructure is the ingress layer; everything after it is shared.\n\n## MANDATORY — Always start from templates\n\nWhen generating any push-ingestion script, you MUST:\n\n1. **Read the corresponding template** before writing any code. Templates live in this skill's\n   directory under `scripts/templates/<warehouse>/`. To find them, glob for\n   `**/push-ingestion/scripts/templates/<warehouse>/*.py` — this works regardless of where the\n   skill is installed. Do NOT search from the current working directory alone.\n2. **Adapt the template** to the customer's needs — do not write pycarlo imports, model constructors,\n   or SDK method calls from memory.\n3. If no template exists for the target warehouse, read the **Snowflake template** as the canonical\n   reference and adapt only the warehouse-specific collection queries.\n\nTemplate files follow this naming pattern:\n- `collect_<flow>.py` — collection only (queries the warehouse, writes a JSON manifest)\n- `push_<flow>.py` — push only (reads the manifest, sends to Monte Carlo)\n- `collect_and_push_<flow>.py` — combined (imports from both, runs in sequence)\n\n**After running any push script**, you MUST surface the `invocation_id`(s) returned by the API\nto the user. The invocation ID is the only way to trace pushed data through downstream systems\nand is required for validation. Never let a push complete without showing the user the\ninvocation IDs — they need them for `/mc-validate-metadata`, `/mc-validate-lineage`, and\ndebugging.\n\n## Canonical pycarlo API — authoritative reference\n\nThe following imports, classes, and method signatures are the **ONLY** correct pycarlo API for\npush ingestion. If your training data suggests different names, **it is wrong**. Use exactly\nwhat is listed here.\n\n### Imports and client setup\n\n```python\nfrom pycarlo.core import Client, Session\nfrom pycarlo.features.ingestion import IngestionService\nfrom pycarlo.features.ingestion.models import (\n    # Metadata\n    RelationalAsset, AssetMetadata, AssetField, AssetVolume, AssetFreshness, Tag,\n    # Lineage\n    LineageEvent, LineageAssetRef, ColumnLineageField, ColumnLineageSourceField,\n    # Query logs\n    QueryLogEntry,\n)\n\nclient = Client(session=Session(mcd_id=key_id, mcd_token=key_token, scope=\"Ingestion\"))\nservice = IngestionService(mc_client=client)\n```\n\n### Method signatures\n\n```python\n# Metadata\nservice.send_metadata(resource_uuid=..., resource_type=..., events=[RelationalAsset(...)])\n\n# Lineage (table or column)\nservice.send_lineage(resource_uuid=..., resource_type=..., events=[LineageEvent(...)])\n\n# Query logs — note: log_type, NOT resource_type\nservice.send_query_logs(resource_uuid=..., log_type=..., events=[QueryLogEntry(...)])\n\n# Extract invocation ID from any response\nservice.extract_invocation_id(result)\n```\n\n### RelationalAsset structure (nested, NOT flat)\n\n```python\nRelationalAsset(\n    type=\"TABLE\",  # ONLY \"TABLE\" or \"VIEW\" (uppercase) — normalize warehouse-native values\n    metadata=AssetMetadata(\n        name=\"my_table\",\n        database=\"analytics\",\n        schema=\"public\",\n        description=\"optional description\",\n    ),\n    fields=[\n        AssetField(name=\"id\", type=\"INTEGER\", description=None),\n        AssetField(name=\"amount\", type=\"DECIMAL(10,2)\"),\n    ],\n    volume=AssetVolume(row_count=1000000, byte_count=111111111),  # optional\n    freshness=AssetFreshness(last_update_time=\"2026-03-12T14:30:00Z\"),  # optional\n)\n```\n\n## Environment variable conventions\n\nAll generated scripts MUST use these exact variable names. Do NOT invent alternatives like\n`MCD_KEY_ID`, `MC_TOKEN`, `MONTE_CARLO_KEY`, etc.\n\n| Variable | Purpose | Used by |\n|---|---|---|\n| `MCD_INGEST_ID` | Ingestion key ID (scope=Ingestion) | push scripts |\n| `MCD_INGEST_TOKEN` | Ingestion key secret | push scripts |\n| `MCD_ID` | GraphQL API key ID | verification scripts |\n| `MCD_TOKEN` | GraphQL API key secret | verification scripts |\n| `MCD_RESOURCE_UUID` | Warehouse resource UUID | all scripts |\n\n## What this skill can build for you\n\nTell Claude your warehouse or data platform and Monte Carlo resource UUID and this skill will\ngenerate a ready-to-run Python script that:\n- Connects to your warehouse using the idiomatic driver for that platform\n- Discovers databases, schemas, and tables\n- Extracts the right columns — names, types, row counts, byte counts, last modified time, descriptions\n- Builds the correct pycarlo `RelationalAsset`, `LineageEvent`, or `QueryLogEntry` objects\n- Pushes to Monte Carlo and saves an output manifest with the `invocation_id` for tracing\n\nTemplates are available for common warehouses (Snowflake, BigQuery, BigQuery Iceberg,\nDatabricks, Redshift, Hive). For any other platform, Claude will derive the appropriate\ncollection queries from the warehouse's system catalog or metadata APIs and generate an\nequivalent script.\n\n### Ready-to-run examples\n\nProduction-ready example scripts built from these templates are published in the\n[mcd-public-resources](https://github.com/monte-carlo-data/mcd-public-resources) repo:\n\n- **[BigQuery Iceberg (BigLake) tables](https://github.com/monte-carlo-data/mcd-public-resources/tree/main/examples/push-ingestion/bigquery/push-iceberg-tables)** —\n  metadata and query log collection for BigQuery Iceberg tables that are invisible to Monte\n  Carlo's standard pull collector (which uses `__TABLES__`). Includes a `--only-freshness-and-volume`\n  flag for fast periodic pushes that skip the schema/fields query — useful for hourly cron jobs\n  after the initial full metadata push.\n\n## Reference docs — when to load\n\n| Reference file | Load when… |\n|---|---|\n| `references/prerequisites.md` | Customer is setting up for the first time, has auth errors, or needs help creating API keys |\n| `references/push-metadata.md` | Building or debugging a metadata collection script |\n| `references/push-lineage.md` | Building or debugging a lineage collection script |\n| `references/push-query-logs.md` | Building or debugging a query log collection script |\n| `references/custom-lineage.md` | Customer needs custom lineage nodes or edges via GraphQL |\n| `references/validation.md` | Verifying pushed data, running GraphQL checks, or deleting push-ingested tables |\n| `references/direct-http-api.md` | Customer wants to call push APIs directly via curl/HTTP without pycarlo |\n| `references/anomaly-detection.md` | Customer asks why freshness or volume detectors aren't firing |\n\n## Prerequisites — read this first\n\n→ Load `references/prerequisites.md`\n\nTwo separate API keys are required. This is the most common setup stumbling block:\n- **Ingestion key** (scope=Ingestion) — for pushing data\n- **GraphQL API key** — for verification queries\n\nBoth use the same `x-mcd-id` / `x-mcd-token` headers but point to different endpoints.\n\n## What you can push\n\n| Flow | pycarlo method | Push endpoint | Type field | Expiration |\n|---|---|---|---|---|\n| Table metadata | `send_metadata()` | `/ingest/v1/metadata` | `resource_type` (e.g. `\"data-lake\"`) | **Never expires** |\n| Table lineage | `send_lineage()` | `/ingest/v1/lineage` | `resource_type` (same as metadata) | **Never expires** |\n| Column lineage | `send_lineage()` (events include `fields`) | `/ingest/v1/lineage` | `resource_type` (same as metadata) | **Expires after 10 days** |\n| Query logs | `send_query_logs()` | `/ingest/v1/querylogs` | **`log_type`** (not `resource_type`!) | Same as pulled |\n| Custom lineage | GraphQL mutations | `api.getmontecarlo.com/graphql` | N/A — uses GraphQL API key | 7 days default; set `expireAt: \"9999-12-31\"` for permanent |\n\n**Important**: Query logs use `log_type` instead of `resource_type`. This is the only push\nendpoint where the field name differs. See `references/push-query-logs.md` for the full list\nof supported `log_type` values.\n\nThe pycarlo SDK is optional — you can also call the push APIs directly via HTTP/curl. See\n`references/direct-http-api.md` for examples.\n\nEvery push returns an `invocation_id` — save it. It is your primary debugging handle across\nall downstream systems.\n\n## Step 1 — Generate your collection scripts\n\nAsk Claude to build the script for your warehouse:\n\n> \"Build me a metadata collection script for Snowflake. My MC resource UUID is `abc-123`.\"\n\nThe script templates in `**/push-ingestion/scripts/templates/` (Snowflake, BigQuery, BigQuery Iceberg, Databricks, Redshift, Hive)\nare the **mandatory starting point** for script generation — they contain the correct pycarlo\nimports, model constructors, and SDK calls. **They are not an exhaustive list.** If the\ncustomer's warehouse is not listed, use the templates as a guide and determine the appropriate\nqueries or file-collection approach for their platform. For file-based sources (like Hive\nMetastore logs), provide the command to retrieve the file, parse it, and transform it into the\nformat required by the push APIs. The push format and SDK calls are identical regardless of\nsource; only the collection queries change.\n\n**Batching**: For large payloads, split events into batches. Use a batch size of **50 assets**\nper push call. The pycarlo HTTP client has a hardcoded 10-second read timeout that cannot be\noverridden (`Session` and `Client` do not accept a `timeout` parameter) — larger batches (200+)\nwill timeout on warehouses with thousands of tables. The compressed request body must also not\nexceed **1MB** (Kinesis limit). All push endpoints support batching.\n\n**Push frequency**: Push at most **once per hour**. Sub-hourly pushes produce unpredictable\nanomaly detector behavior because the training pipeline aggregates into hourly buckets.\n\n**Per flow, see:**\n- Metadata (schema + volume + freshness): `references/push-metadata.md`\n- Table and column lineage: `references/push-lineage.md`\n- Query logs: `references/push-query-logs.md`\n\n## Step 2 — Validate pushed data\n\nAfter pushing, verify data is visible in Monte Carlo using the GraphQL API (GraphQL API key).\n\n→ `references/validation.md` — all verification queries (getTable, getMetricsV4,\ngetTableLineage, getDerivedTablesPartialLineage, getAggregatedQueries)\n\nTiming expectations:\n- **Metadata**: visible within a few minutes\n- **Table lineage**: visible within seconds to a few minutes (fast direct path to Neo4j)\n- **Column lineage**: a few minutes\n- **Query logs**: at least **15-20 minutes** (async processing pipeline)\n\n## Step 3 — Anomaly detection (optional)\n\nIf you want Monte Carlo's freshness and volume detectors to fire on pushed data, you need to\npush consistently over time — detectors require historical data to train.\n\n→ `references/anomaly-detection.md` — recommended push frequency, minimum samples,\ntraining windows, and what to tell customers who ask why detectors aren't activating\n\n## Custom lineage nodes and edges\n\nFor non-warehouse assets (dbt models, Airflow DAGs, custom ETL pipelines) or cross-resource\nlineage, use the GraphQL mutations directly:\n\n→ `references/custom-lineage.md` — `createOrUpdateLineageNode`, `createOrUpdateLineageEdge`,\n`deleteLineageNode`, and the critical `expireAt: \"9999-12-31\"` rule\n\n## Deleting push-ingested tables\n\nPush tables are excluded from the normal pull-based deletion flow (intentionally). To delete\nthem explicitly, use `deletePushIngestedTables` — covered in `references/validation.md`\nunder \"Table management operations\".\n\n## Available slash commands\n\nCustomers can invoke these explicitly instead of describing their intent in prose:\n\n| Command | Purpose |\n|---|---|\n| `/mc-build-metadata-collector` | Generate a metadata collection script |\n| `/mc-build-lineage-collector` | Generate a lineage collection script |\n| `/mc-build-query-log-collector` | Generate a query log collection script |\n| `/mc-validate-metadata` | Verify pushed metadata via the GraphQL API |\n| `/mc-validate-lineage` | Verify pushed lineage via the GraphQL API |\n| `/mc-validate-query-logs` | Verify pushed query logs via the GraphQL API |\n| `/mc-create-lineage-node` | Create a custom lineage node |\n| `/mc-create-lineage-edge` | Create a custom lineage edge |\n| `/mc-delete-lineage-node` | Delete a custom lineage node |\n| `/mc-delete-push-tables` | Delete push-ingested tables |\n\n## Debugging checkpoints\n\nWhen pushed data isn't appearing, work through these five checkpoints in order:\n\n1. **Did the SDK return a `202` and an `invocation_id`?**\n   If not, the gateway rejected the request — check auth headers and `resource.uuid`.\n\n2. **Is the integration key the right type?**\n   Must be scope `Ingestion`, created via `montecarlo integrations create-key --scope Ingestion`.\n   A standard GraphQL API key will not work for push.\n\n3. **Is `resource.uuid` correct and authorized?**\n   The key can be scoped to specific warehouse UUIDs. If the UUID doesn't match, you get `403`.\n\n4. **Did the normalizer process it?**\n   Use the `invocation_id` to search CloudWatch logs for the relevant Lambda. For query logs,\n   check the `log_type` — Hive requires `\"hive-s3\"`, not `\"hive\"`.\n\n5. **Did the downstream system pick it up?**\n   - Metadata: query `getTable` in GraphQL\n   - Table lineage: check Neo4j within seconds–minutes (fast path via PushLineageProcessor)\n   - Query logs: wait at least 15-20 minutes; check `getAggregatedQueries`\n\n## Known gotchas\n\n- **`log_type` vs `resource_type`**: metadata and lineage use `resource_type` (e.g. `\"data-lake\"`);\n  query logs use **`log_type`** — the only endpoint where the field name differs. Wrong value →\n  `Unsupported ingest query-log log_type` error.\n- **`invocation_id` must be saved**: every output manifest should include it — it's your\n  only tracing handle once the request leaves the SDK.\n- **Query log async delay**: at least 15-20 minutes. `getAggregatedQueries` will return 0 until\n  processing completes — this is expected, not a bug.\n- **Custom lineage `expireAt` defaults to 7 days**: nodes vanish silently unless you set\n  `expireAt: \"9999-12-31\"` for permanent nodes.\n- **Push tables are never auto-deleted**: the periodic cleanup job excludes them by default\n  (`exclude_push_tables=True`). Delete them explicitly via `deletePushIngestedTables` (max\n  1,000 MCONs per call; also deletes lineage nodes and all edges touching those nodes).\n- **Anomaly detectors need history**: pushing once is not enough. Freshness needs 7+ pushes\n  over ~2 weeks; volume needs 10–48 samples over ~42 days. Push at most once per hour.\n- **Batching required for large payloads**: the compressed request body must not exceed 1MB.\n  Split large event lists into batches.\n- **Column lineage expires after 10 days**: unlike table metadata and table lineage (which\n  never expire), column lineage has a 10-day TTL, same as pulled column lineage.\n- **Quote SQL identifiers in warehouse queries**: database, schema, and table names must be\n  quoted to handle mixed-case or special characters. The quoting syntax varies by warehouse —\n  Snowflake and Redshift use double quotes (`\"{db}\"`), BigQuery/Databricks/Hive use backticks\n  (`` `db` ``). The templates already handle this correctly for each warehouse — follow the\n  same quoting pattern when adapting.\n\n## Memory safety\n\nGenerated scripts must include a startup memory check. The collection phase loads query history\nrows into memory for parsing — on large warehouses with long lookback windows, this can exhaust\navailable RAM and cause the process to be silently killed (SIGKILL / exit 137) with no traceback.\n\nAdd this pattern near the top of every generated script, after imports:\n\n```python\nimport os\n\ndef _check_available_memory(min_gb: float = 2.0) -> None:\n    \"\"\"Warn if available memory is below the threshold.\"\"\"\n    try:\n        if hasattr(os, \"sysconf\"):  # Linux / macOS\n            page_size = os.sysconf(\"SC_PAGE_SIZE\")\n            avail_pages = os.sysconf(\"SC_AVPHYS_PAGES\")\n            avail_gb = (page_size * avail_pages) / (1024 ** 3)\n        else:\n            return  # Windows — skip check\n    except (ValueError, OSError):\n        return\n    if avail_gb < min_gb:\n        print(\n            f\"WARNING: Only {avail_gb:.1f} GB of memory available \"\n            f\"(minimum recommended: {min_gb:.1f} GB). \"\n            f\"Consider reducing the lookback window or increasing available memory.\"\n        )\n```\n\nCall `_check_available_memory()` before connecting to the warehouse.\n\nAdditionally, when fetching query history:\n- Use `cursor.fetchmany(batch_size)` in a loop instead of `cursor.fetchall()` when possible\n- For very large result sets, consider adding a LIMIT clause and processing in windows\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.","tags":["monte","carlo","push","ingestion","antigravity","awesome","skills","sickn33","agent-skills","agentic-skills","ai-agent-skills","ai-agents"],"capabilities":["skill","source-sickn33","skill-monte-carlo-push-ingestion","topic-agent-skills","topic-agentic-skills","topic-ai-agent-skills","topic-ai-agents","topic-ai-coding","topic-ai-workflows","topic-antigravity","topic-antigravity-skills","topic-claude-code","topic-claude-code-skills","topic-codex-cli","topic-codex-skills"],"categories":["antigravity-awesome-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/sickn33/antigravity-awesome-skills/monte-carlo-push-ingestion","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add sickn33/antigravity-awesome-skills","source_repo":"https://github.com/sickn33/antigravity-awesome-skills","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 34666 github stars · SKILL.md body (17,777 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-04-23T06:51:35.550Z","embedding":null,"createdAt":"2026-04-18T21:41:02.226Z","updatedAt":"2026-04-23T06:51:35.550Z","lastSeenAt":"2026-04-23T06:51:35.550Z","tsv":"'-03':603 '-12':604,1123,1601,1979 '-123':1225 '-20':1507,1875,1949 '-31':1124,1602,1980 '/graphql':1111 '/ingest/v1/lineage':1066,1081 '/ingest/v1/metadata':1053 '/ingest/v1/querylogs':1096 '/mc-build-lineage-collector':1658 '/mc-build-metadata-collector':1652 '/mc-build-query-log-collector':1664 '/mc-create-lineage-edge':1702 '/mc-create-lineage-node':1696 '/mc-delete-lineage-node':1708 '/mc-delete-push-tables':1714 '/mc-validate-lineage':400,1679 '/mc-validate-metadata':399,1671 '/mc-validate-query-logs':1687 '/monte-carlo-data/mcd-public-resources)':829 '/monte-carlo-data/mcd-public-resources/tree/main/examples/push-ingestion/bigquery/push-iceberg-tables)**':837 '/push-ingestion/scripts/templates':238,1230 '0':1954 '000':2010 '00z':607 '1':215,1197,1735,2009 '10':586,1089,1360,2042,2077,2092 '1000000':592 '1024':2259 '111111111':595 '137':2198 '15':1506,1874,1948 '1f':2281,2291 '1mb':1396,2066 '2':258,587,1446,1758,2038 '2.0':2224 '200':1379 '202':1741 '2026':602 '3':280,1513,1789,2260 '30':606 '4':1813 '403':1812 '42':2046 '48':2043 '5':1845 '50':1348 '7':1117,1969,2035 '9999':1122,1600,1978 'abc':1224 'accept':1373 'across':1192 'activ':1564 'ad':2335 'adapt':259,298,2154 'adapter/normalizer':177 'add':2202 'addit':2312 'agent':29 'aggreg':1425 'airflow':1577 'alon':257 'alreadi':156,2141 'also':1166,1393,2014 'altern':624 'alway':137,202 'amount':583 'analyt':567 'anomali':1418,1514,2024 'api':54,89,360,405,420,660,668,799,913,969,994,1014,1115,1170,1318,1462,1464,1678,1686,1695,1782 'api.getmontecarlo.com':1110 'api.getmontecarlo.com/graphql':1109 'appear':1727 'approach':1286 'appropri':78,788,1280 'aren':983,1562 'ask':977,1202,1559,2376 'asset':152,1349,1574 'assetfield':460,574,581 'assetfresh':462,598 'assetmetadata':459,562 'assetvolum':461,589 'async':1509,1944 'auth':907,1754 'author':1794 'authorit':406 'auto':1989 'auto-delet':1988 'avail':769,1635,2186,2219,2228,2247,2253,2257,2271,2279,2285,2301,2305 'avphi':2251 'backtick':2137 'base':1293,1618 'batch':1335,1342,1345,1378,1403,2054,2072,2319 'behavior':1420 'biglak':833 'bigqueri':774,775,831,844,1232,1233 'bigquery/databricks/hive':2135 'block':1005 'bodi':1391,2062 'boundari':2384 'bucket':1428 'bug':1963 'build':685,743,916,924,932,1205,1211 'built':815 'byte':593,737 'call':96,277,967,1167,1256,1324,1352,2013,2303 'cannot':136,1365 'canon':295,403 'carlo':3,17,23,49,104,119,333,632,697,755,852,1458,1521 'case':2118 'catalog':86,796 'caus':2189 'chang':1334 'charact':2121 'check':956,1753,1834,1860,1877,2164,2218,2265,2304 'checkpoint':1721,1732 'clarif':2378 'class':411 'claud':689,784,1203 'claus':2338 'cleanup':1993 'clear':2351 'client':442,448,472,473,489,490,1356,1370 'cloudwatch':1825 'code':178,223 'collect':33,79,304,312,314,334,789,842,921,929,938,1200,1215,1285,1332,1656,1662,1669,2166 'collector':126,856 'column':506,732,1074,1439,1497,2073,2088,2098 'columnlineagefield':467 'columnlineagesourcefield':468 'combin':338 'command':1301,1637,1650 'common':771,1002 'complet':387,1957 'compress':1389,2060 'connect':713,2308 'consid':2294,2334 'consist':1536 'constructor':273,1253 'contain':1247 'convent':611 'correct':418,745,1249,1792,2144 'correspond':218 'count':591,594,736,738 'cover':138,1628 'creat':912,1697,1703,1770,1775 'create-key':1774 'createorupdatelineageedg':1594 'createorupdatelineagenod':1593 'criteria':2387 'critic':1598 'cron':880 'cross':1584 'cross-resourc':1583 'curl/http':972 'current':254 'cursor.fetchall':2326 'cursor.fetchmany':2318 'custom':32,65,109,146,154,264,898,941,943,964,976,1105,1265,1557,1565,1579,1638,1699,1705,1711,1964 'dag':1578 'data':20,41,46,61,159,167,374,427,693,953,1012,1058,1449,1453,1531,1542,1724,1894 'data-lak':1057,1893 'databas':566,725,2106 'databrick':777,1235 'day':1090,1118,1970,2047,2078,2093 'db':2134,2138 'dbt':1575 'debug':402,918,926,934,1190,1720 'decim':585 'dedic':173 'def':2217 'default':1119,1967,1998 'delay':1945 'delet':958,1604,1619,1623,1709,1715,1990,2003,2015 'deletelineagenod':1595 'deletepushingestedt':1627,2007 'deriv':76,786 'describ':1645,2355 'descript':570,572,579,742 'detect':1515 'detector':982,1419,1526,1539,1561,2025 'determin':1278 'differ':429,1035,1147,1908 'direct':116,165,970,1171,1493,1591 'directori':230,256 'discov':724 'doc':889 'doesn':1807 'doubl':2132 'downstream':181,376,1194,1848 'driver':720 'e.g':1056,1892 'edg':947,1569,1707,2020 'els':2261 'endpoint':1036,1045,1142,1401,1903 'enough':2032 'environ':609,2367 'environment-specif':2366 'equival':803 'error':908,1918 'etc':634 'etl':1580 'event':501,513,530,1078,1340,2069 'everi':1178,1924,2209 'everyth':196 'exact':435,618 'exampl':809,813,1177 'exceed':1395,2065 'except':2266 'exclud':1612,1995,1999 'exhaust':1261,2185 'exist':284 'exit':2197 'expect':1476,1960 'expert':6,2372 'expir':1048,1061,1073,1087,2075,2087 'expireat':1121,1599,1966,1977 'explicit':1625,1642,2005 'expos':143 'extract':532,729 'f':2276,2286,2293 'fast':869,1492,1865 'fetch':2314 'field':573,1047,1080,1145,1906 'file':307,894,1284,1292,1305 'file-bas':1291 'file-collect':1283 'fill':131 'find':234 'fire':985,1528 'first':904,989 'five':1731 'flag':867 'flat':546 'float':2223 'flow':1041,1430,1620 'follow':308,409,2148 'format':92,1313,1321 'frequenc':1405,1548 'fresh':597,864,979,1435,1523,2033 'full':885,1152 'gap':132 'gateway':172,1749 'gather':128 'gb':2222,2254,2272,2274,2280,2282,2290,2292 'generat':207,613,704,801,1198,1245,1653,1659,1665,2157,2210 'get':1811 'getaggregatedqueri':1474,1878,1951 'getderivedtablespartiallineag':1473 'getmetricsv4':1471 'gettabl':1470,1855 'gettablelineag':1472 'github.com':828,836 'github.com/monte-carlo-data/mcd-public-resources)':827 'github.com/monte-carlo-data/mcd-public-resources/tree/main/examples/push-ingestion/bigquery/push-iceberg-tables)**':835 'glob':236 'gotcha':1880 'graphql':659,667,949,955,1013,1107,1114,1461,1463,1589,1677,1685,1694,1781,1857 'guid':7,1276 'handl':1191,1935,2115,2142 'hardcod':1359 'hasattr':2236 'header':1031,1755 'help':31,911 'histor':1541 'histori':145,2027,2170,2316 'hive':779,1237,1296,1838,1841,1844 'hive-s3':1840 'hour':879,1411,1414,1427,2053 'http':1355 'http/curl':1173 'iceberg':776,832,845,1234 'id':355,366,394,477,479,534,540,576,628,641,644,658,662,764,1026,1183,1745,1822,1920 'ident':1326 'identifi':2102 'idiomat':719 'import':271,339,410,440,447,452,456,1127,1251,2213,2215 'includ':860,1079,1928,2160 'increas':2300 'infrastructur':191 'ingest':5,25,53,211,423,485,640,642,646,650,652,961,1006,1009,1607,1718,1769,1778,1912 'ingestionservic':453,487 'ingress':194 'initi':884 'input':2381 'instal':248 'instead':120,1133,1643,2324 'integ':578 'integr':139,171,1761,1773 'intent':1621,1647 'invent':623 'invis':849 'invoc':354,365,393,533,539,763,1182,1744,1821,1919 'invok':1640 'isn':1725 'job':881,1994 'json':321 'key':478,482,627,633,643,653,661,669,914,995,1007,1015,1116,1465,1762,1776,1783,1796 'kill':2195 'kinesi':174,1397 'known':1879 'lake':1059,1895 'lambda':1830 'larg':1337,2057,2068,2177,2331 'larger':1377 'last':599,739 'layer':195 'least':1505,1873,1947 'leav':1939 'let':108,384 'like':625,1295 'limit':1398,2337,2343 'lineag':11,35,112,147,464,503,508,928,944,1063,1065,1075,1077,1106,1440,1484,1498,1566,1586,1661,1682,1700,1706,1712,1859,1888,1965,2016,2074,2084,2089,2099 'lineageassetref':466 'lineageev':465,514,748 'linux':2239 'list':438,1153,1262,1270,2070 'live':225 'load':892,895,990,2168 'log':14,38,115,470,516,518,525,528,841,937,1092,1095,1097,1129,1131,1156,1298,1443,1503,1668,1691,1826,1833,1836,1870,1881,1897,1899,1915,1916,1943 'long':2180 'lookback':2181,2297 'loop':2323 'maco':2240 'made':74 'manag':1633 'mandatori':201,1240 'manifest':322,329,760,1926 'match':1809,2352 'max':2008 'mc':488,629,1220 'mcd':476,480,626,639,649,657,665,673,824,1025,1029 'mcd-public-resourc':823 'mcon':2011 'memori':279,2155,2163,2173,2220,2229,2284,2302,2306 'metadata':10,34,88,111,457,494,496,561,798,838,886,920,1050,1052,1071,1086,1214,1432,1477,1655,1674,1853,1886,2081 'metastor':1297 'method':276,413,491,1043 'min':2221,2273,2289 'minimum':1549,2287 'minut':1482,1491,1501,1508,1864,1876,1950 'miss':2389 'mix':2117 'mixed-cas':2116 'model':57,107,135,187,272,1252,1576 'modifi':740 'mont':2,16,22,48,103,118,332,631,696,754,851,1457,1520 'monte-carlo-push-ingest':1 'montecarlo':1772 'must':214,351,615,1392,1766,1921,2063,2111,2159 'mutat':1108,1590 'n/a':1112 'name':310,430,563,575,582,620,733,1146,1907,2110 'nativ':559 'near':2205 'need':266,396,910,942,1533,2026,2034,2041 'neo4j':1496,1861 'nest':544 'never':383,1060,1072,1987,2086 'new':190 'node':945,1567,1701,1713,1971,1983,2017,2023 'non':150,1572 'non-warehous':149,1571 'none':580,2225 'normal':556,1615,1816 'note':517 'object':751 'only-freshness-and-volum':862 'oper':1634 'option':571,596,608,1163,1516 'order':1734 'os':2216,2237 'os.sysconf':2243,2249 'oserror':2268 'output':759,1925,2361 'overridden':1367 'page':2241,2245,2248,2252,2255,2258 'paramet':1376 'pars':1306,2175 'path':1494,1866 'pattern':311,2152,2204 'payload':1338,2058 'per':1350,1410,1429,2012,2052 'period':870,1992 'perman':1126,1982 'permiss':2382 'phase':2167 'pick':1850 'pipelin':1424,1511,1581 'platform':694,723,783,1289 'point':1033,1242 'possibl':2328 'power':184 'prerequisit':986 'primari':1189 'print':2275 'process':1510,1817,1956,2191,2340 'produc':1416 'product':811 'production-readi':810 'prose':1649 'provid':1299 'public':569,825 'publish':820 'pull':125,134,186,855,1104,1617,2097 'pull-bas':1616 'purpos':636,1651 'push':4,9,24,44,52,56,91,106,166,210,323,325,336,348,373,386,422,647,655,752,871,887,952,960,968,1011,1040,1044,1141,1169,1179,1317,1320,1351,1400,1404,1406,1415,1448,1451,1530,1535,1547,1606,1609,1673,1681,1689,1717,1723,1788,1984,2000,2028,2036,2048 'push-ingest':209,959,1605,1716 'pushlineageprocessor':1868 'py':239,313,324,337 'pycarlo':94,270,404,419,746,974,1042,1160,1250,1354 'pycarlo.core':446 'pycarlo.features.ingestion':451 'pycarlo.features.ingestion.models':455 'python':444,493,547,710,2214 'queri':13,37,80,114,144,305,316,469,515,524,790,840,876,936,1018,1091,1094,1128,1281,1333,1442,1469,1502,1667,1690,1832,1854,1869,1896,1914,1942,2105,2169,2315 'query-log':1913 'querylogentri':471,531,750 'quot':2100,2113,2123,2133,2151 'ram':2187 'read':216,289,327,987,1362 'readi':73,707,806,812 'ready-mad':72 'ready-to-run':706,805 'recommend':1546,2288 'redshift':778,1236,2130 'reduc':2295 'refer':296,407,888,893 'references/anomaly-detection.md':975,1545 'references/custom-lineage.md':940,1592 'references/direct-http-api.md':963,1175 'references/prerequisites.md':897,991 'references/push-lineage.md':923,1441 'references/push-metadata.md':915,1436 'references/push-query-logs.md':931,1149,1444 'references/validation.md':950,1466,1630 'regardless':100,242,1327 'reject':1750 'relationalasset':458,502,542,548,747 'relev':1829 'repo':830 'request':1390,1752,1938,2061 'requir':380,997,1314,1540,1839,2055,2380 'resourc':497,499,509,511,521,526,674,677,698,826,1054,1067,1082,1100,1135,1221,1585,1884,1890 'resource.uuid':1757,1791 'respons':537 'result':541,2332 'retriev':1303 'return':357,1180,1739,1953,2262,2269 'review':2373 'right':731,1764 'row':590,735,2171 'rule':1603 'run':342,346,709,808,954 's3':1842 'safeti':2156,2383 'sampl':1550,2044 'save':757,1184,1923 'sc':2244,2250 'schema':568,726,1433,2107 'schema/fields':875 'scope':484,645,1008,1768,1777,1799,2354 'script':212,349,614,648,656,664,672,680,711,804,814,922,930,939,1201,1207,1216,1227,1244,1657,1663,1670,2158,2211 'scripts/templates':232 'sdk':95,275,1161,1255,1323,1738,1941 'search':251,1824 'second':1361,1487,1863 'secret':654,670 'see':1148,1174,1431 'send':110,163,330,1051,1064,1076,1093 'separ':993 'sequenc':344 'servic':486 'service.extract':538 'service.send':495,507,523 'session':449,474,475,1368 'set':900,1120,1976,2333 'setup':443,1003 'share':200 'show':389 'sigkil':2196 'signatur':414,492 'silent':1973,2194 'size':1346,2242,2246,2256,2320 'skill':228,246,683,702,2346 'skill-monte-carlo-push-ingestion' 'skip':873,2264 'slash':1636 'snowflak':291,773,1218,1231,2128 'sourc':62,102,1294,1329 'source-sickn33' 'special':2120 'specif':303,1801,2368 'split':1339,2067 'sql':2101 'standard':854,1780 'start':203,1241 'startup':2162 'step':1196,1445,1512 'stop':2374 'stream':175 'structur':543 'stumbl':1004 'sub':1413 'sub-hour':1412 'substitut':2364 'success':2386 'suggest':428 'support':1155,1402 'surfac':352 'syntax':2124 'sysconf':2238 'system':85,182,377,795,1195,1849 't14':605 'tabl':504,550,552,565,728,834,846,859,962,1049,1062,1387,1437,1483,1608,1610,1632,1719,1858,1985,2001,2080,2083,2109 'tag':463 'target':287 'task':2350 'tell':688,1556 'templat':75,205,219,224,261,283,292,306,767,818,1228,1273,2140 'test':2370 'thin':176 'thousand':1385 'threshold':2233 'time':601,741,905,1475,1538 'timeout':1363,1375,1381 'token':481,483,630,651,666,1030 'top':2207 'topic-agent-skills' 'topic-agentic-skills' 'topic-ai-agent-skills' 'topic-ai-agents' 'topic-ai-coding' 'topic-ai-workflows' 'topic-antigravity' 'topic-antigravity-skills' 'topic-claude-code' 'topic-claude-code-skills' 'topic-codex-cli' 'topic-codex-skills' 'touch':2021 'trace':372,766,1934 'traceback':2201 'train':426,1423,1544,1551 'transform':1309 'travel':168 'treat':2359 'tri':2234 'true':2002 'ttl':2094 'two':992 'type':500,512,519,522,529,549,577,584,734,1046,1055,1068,1083,1098,1101,1132,1136,1157,1765,1837,1882,1885,1891,1900,1917 'unless':1974 'unlik':2079 'unpredict':1417 'unsupport':1911 'updat':600 'uppercas':555 'use':434,616,637,717,858,877,1020,1113,1130,1271,1343,1459,1587,1626,1819,1889,1898,2131,2136,2317,2344 'user':363,391 'uuid':498,510,527,675,678,699,1222,1803,1806 'valid':382,1447,2369 'valu':560,1158,1910 'valueerror':2267 'vanish':1972 'vari':2125 'variabl':610,619,635 'verif':663,671,1017,1468 'verifi':951,1452,1672,1680,1688 'via':50,948,971,1172,1675,1683,1692,1771,1867,2006 'view':554 'visibl':1455,1478,1485 'volum':588,866,981,1434,1525,2040 'vs':1883 'wait':122,1871 'want':161,965,1519 'warehous':21,42,67,83,151,288,302,318,558,676,691,716,772,793,1210,1267,1383,1573,1802,2104,2127,2147,2178,2311 'warehouse-n':557 'warehouse-specif':301 'warn':2226,2277 'way':370 'week':2039 'window':1552,2182,2263,2298,2342 'within':1479,1486,1862 'without':388,973 'work':58,241,255,1728,1786 'write':221,269,319 'wrong':433,1909 'x':1024,1028 'x-mcd-id':1023 'x-mcd-token':1027","prices":[{"id":"cf9b46d4-9863-48e8-a51c-9d8a0faa6cfc","listingId":"297326ed-b783-4981-9975-df539d00d029","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"sickn33","category":"antigravity-awesome-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T21:41:02.226Z"}],"sources":[{"listingId":"297326ed-b783-4981-9975-df539d00d029","source":"github","sourceId":"sickn33/antigravity-awesome-skills/monte-carlo-push-ingestion","sourceUrl":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/monte-carlo-push-ingestion","isPrimary":false,"firstSeenAt":"2026-04-18T21:41:02.226Z","lastSeenAt":"2026-04-23T06:51:35.550Z"}],"details":{"listingId":"297326ed-b783-4981-9975-df539d00d029","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"sickn33","slug":"monte-carlo-push-ingestion","github":{"repo":"sickn33/antigravity-awesome-skills","stars":34666,"topics":["agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding","ai-workflows","antigravity","antigravity-skills","claude-code","claude-code-skills","codex-cli","codex-skills","cursor","cursor-skills","developer-tools","gemini-cli","gemini-skills","kiro","mcp","skill-library"],"license":"mit","html_url":"https://github.com/sickn33/antigravity-awesome-skills","pushed_at":"2026-04-23T06:41:03Z","description":"Installable GitHub library of 1,400+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.","skill_md_sha":"5f9090a9a5fc80139447ca2b50a232c5ddba15d0","skill_md_path":"skills/monte-carlo-push-ingestion/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/monte-carlo-push-ingestion"},"layout":"multi","source":"github","category":"antigravity-awesome-skills","frontmatter":{"name":"monte-carlo-push-ingestion","description":"Expert guide for pushing metadata, lineage, and query logs to Monte Carlo from any data warehouse."},"skills_sh_url":"https://skills.sh/sickn33/antigravity-awesome-skills/monte-carlo-push-ingestion"},"updatedAt":"2026-04-23T06:51:35.550Z"}}