{"id":"ba21640f-a958-42be-92da-ef06fdefb540","shortId":"n6sRRm","kind":"skill","title":"azure-storage-file-datalake-py","tagline":"Azure Data Lake Storage Gen2 SDK for Python. Use for hierarchical file systems, big data analytics, and file/directory operations.","description":"# Azure Data Lake Storage Gen2 SDK for Python\n\nHierarchical file system for big data analytics workloads.\n\n## Installation\n\n```bash\npip install azure-storage-file-datalake azure-identity\n```\n\n## Environment Variables\n\n```bash\nAZURE_STORAGE_ACCOUNT_URL=https://<account>.dfs.core.windows.net\n```\n\n## Authentication\n\n```python\nfrom azure.identity import DefaultAzureCredential\nfrom azure.storage.filedatalake import DataLakeServiceClient\n\ncredential = DefaultAzureCredential()\naccount_url = \"https://<account>.dfs.core.windows.net\"\n\nservice_client = DataLakeServiceClient(account_url=account_url, credential=credential)\n```\n\n## Client Hierarchy\n\n| Client | Purpose |\n|--------|---------|\n| `DataLakeServiceClient` | Account-level operations |\n| `FileSystemClient` | Container (file system) operations |\n| `DataLakeDirectoryClient` | Directory operations |\n| `DataLakeFileClient` | File operations |\n\n## File System Operations\n\n```python\n# Create file system (container)\nfile_system_client = service_client.create_file_system(\"myfilesystem\")\n\n# Get existing\nfile_system_client = service_client.get_file_system_client(\"myfilesystem\")\n\n# Delete\nservice_client.delete_file_system(\"myfilesystem\")\n\n# List file systems\nfor fs in service_client.list_file_systems():\n    print(fs.name)\n```\n\n## Directory Operations\n\n```python\nfile_system_client = service_client.get_file_system_client(\"myfilesystem\")\n\n# Create directory\ndirectory_client = file_system_client.create_directory(\"mydir\")\n\n# Create nested directories\ndirectory_client = file_system_client.create_directory(\"path/to/nested/dir\")\n\n# Get directory client\ndirectory_client = file_system_client.get_directory_client(\"mydir\")\n\n# Delete directory\ndirectory_client.delete_directory()\n\n# Rename/move directory\ndirectory_client.rename_directory(new_name=\"myfilesystem/newname\")\n```\n\n## File Operations\n\n### Upload File\n\n```python\n# Get file client\nfile_client = file_system_client.get_file_client(\"path/to/file.txt\")\n\n# Upload from local file\nwith open(\"local-file.txt\", \"rb\") as data:\n    file_client.upload_data(data, overwrite=True)\n\n# Upload bytes\nfile_client.upload_data(b\"Hello, Data Lake!\", overwrite=True)\n\n# Append data (for large files)\nfile_client.append_data(data=b\"chunk1\", offset=0, length=6)\nfile_client.append_data(data=b\"chunk2\", offset=6, length=6)\nfile_client.flush_data(12)  # Commit the data\n```\n\n### Download File\n\n```python\nfile_client = file_system_client.get_file_client(\"path/to/file.txt\")\n\n# Download all content\ndownload = file_client.download_file()\ncontent = download.readall()\n\n# Download to file\nwith open(\"downloaded.txt\", \"wb\") as f:\n    download = file_client.download_file()\n    download.readinto(f)\n\n# Download range\ndownload = file_client.download_file(offset=0, length=100)\n```\n\n### Delete File\n\n```python\nfile_client.delete_file()\n```\n\n## List Contents\n\n```python\n# List paths (files and directories)\nfor path in file_system_client.get_paths():\n    print(f\"{'DIR' if path.is_directory else 'FILE'}: {path.name}\")\n\n# List paths in directory\nfor path in file_system_client.get_paths(path=\"mydir\"):\n    print(path.name)\n\n# Recursive listing\nfor path in file_system_client.get_paths(path=\"mydir\", recursive=True):\n    print(path.name)\n```\n\n## File/Directory Properties\n\n```python\n# Get properties\nproperties = file_client.get_file_properties()\nprint(f\"Size: {properties.size}\")\nprint(f\"Last modified: {properties.last_modified}\")\n\n# Set metadata\nfile_client.set_metadata(metadata={\"processed\": \"true\"})\n```\n\n## Access Control (ACL)\n\n```python\n# Get ACL\nacl = directory_client.get_access_control()\nprint(f\"Owner: {acl['owner']}\")\nprint(f\"Permissions: {acl['permissions']}\")\n\n# Set ACL\ndirectory_client.set_access_control(\n    owner=\"user-id\",\n    permissions=\"rwxr-x---\"\n)\n\n# Update ACL entries\nfrom azure.storage.filedatalake import AccessControlChangeResult\ndirectory_client.update_access_control_recursive(\n    acl=\"user:user-id:rwx\"\n)\n```\n\n## Async Client\n\n```python\nfrom azure.storage.filedatalake.aio import DataLakeServiceClient\nfrom azure.identity.aio import DefaultAzureCredential\n\nasync def datalake_operations():\n    credential = DefaultAzureCredential()\n    \n    async with DataLakeServiceClient(\n        account_url=\"https://<account>.dfs.core.windows.net\",\n        credential=credential\n    ) as service_client:\n        file_system_client = service_client.get_file_system_client(\"myfilesystem\")\n        file_client = file_system_client.get_file_client(\"test.txt\")\n        \n        await file_client.upload_data(b\"async content\", overwrite=True)\n        \n        download = await file_client.download_file()\n        content = await download.readall()\n\nimport asyncio\nasyncio.run(datalake_operations())\n```\n\n## Best Practices\n\n1. **Use hierarchical namespace** for file system semantics\n2. **Use `append_data` + `flush_data`** for large file uploads\n3. **Set ACLs at directory level** and inherit to children\n4. **Use async client** for high-throughput scenarios\n5. **Use `get_paths` with `recursive=True`** for full directory listing\n6. **Set metadata** for custom file attributes\n7. **Consider Blob API** for simple object storage use cases\n\n## When to Use\nThis skill is applicable to execute the workflow or actions described in the overview.\n\n## Limitations\n- Use this skill only when the task clearly matches the scope described above.\n- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.\n- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.","tags":["azure","storage","file","datalake","antigravity","awesome","skills","sickn33","agent-skills","agentic-skills","ai-agent-skills","ai-agents"],"capabilities":["skill","source-sickn33","skill-azure-storage-file-datalake-py","topic-agent-skills","topic-agentic-skills","topic-ai-agent-skills","topic-ai-agents","topic-ai-coding","topic-ai-workflows","topic-antigravity","topic-antigravity-skills","topic-claude-code","topic-claude-code-skills","topic-codex-cli","topic-codex-skills"],"categories":["antigravity-awesome-skills"],"synonyms":[],"warnings":[],"endpointUrl":"https://skills.sh/sickn33/antigravity-awesome-skills/azure-storage-file-datalake-py","protocol":"skill","transport":"skills-sh","auth":{"type":"none","details":{"cli":"npx skills add sickn33/antigravity-awesome-skills","source_repo":"https://github.com/sickn33/antigravity-awesome-skills","install_from":"skills.sh"}},"qualityScore":"0.700","qualityRationale":"deterministic score 0.70 from registry signals: · indexed on github topic:agent-skills · 34928 github stars · SKILL.md body (5,554 chars)","verified":false,"liveness":"unknown","lastLivenessCheck":null,"agentReviews":{"count":0,"score_avg":null,"cost_usd_avg":null,"success_rate":null,"latency_p50_ms":null,"narrative_summary":null,"summary_updated_at":null},"enrichmentModel":"deterministic:skill-github:v1","enrichmentVersion":1,"enrichedAt":"2026-04-24T18:50:34.458Z","embedding":null,"createdAt":"2026-04-18T21:33:13.459Z","updatedAt":"2026-04-24T18:50:34.458Z","lastSeenAt":"2026-04-24T18:50:34.458Z","tsv":"'0':243,298 '1':494 '100':300 '12':257 '2':502 '3':512 '4':522 '5':531 '6':245,252,254,542 '7':549 'access':380,388,403,421 'accesscontrolchangeresult':419 'account':59,74,80,82,92,450 'account-level':91 'acl':382,385,386,393,398,401,414,424,514 'action':571 'analyt':22,40 'api':552 'append':232,504 'applic':565 'ask':609 'async':430,441,447,476,524 'asyncio':488 'asyncio.run':489 'attribut':548 'authent':62 'await':472,481,485 'azur':2,7,26,47,52,57 'azure-ident':51 'azure-storage-file-datalak':46 'azure-storage-file-datalake-pi':1 'azure.identity':65 'azure.identity.aio':438 'azure.storage.filedatalake':69,417 'azure.storage.filedatalake.aio':434 'b':226,240,249,475 'bash':43,56 'best':492 'big':20,38 'blob':551 'boundari':617 'byte':223 'case':558 'children':521 'chunk1':241 'chunk2':250 'clarif':611 'clear':584 'client':78,86,88,116,125,129,152,156,161,169,175,177,180,200,202,205,265,268,431,457,460,464,467,470,525 'commit':258 'consid':550 'contain':96,113 'content':272,276,307,477,484 'control':381,389,404,422 'creat':110,158,165 'credenti':72,84,85,445,453,454 'criteria':620 'custom':546 'data':8,21,27,39,216,218,219,225,228,233,238,239,247,248,256,260,474,505,507 'datalak':5,50,443,490 'datalakedirectorycli':100 'datalakefilecli':103 'datalakeservicecli':71,79,90,436,449 'def':442 'defaultazurecredenti':67,73,440,446 'delet':131,182,301 'describ':572,588 'dfs.core.windows.net':61,76,452 'dir':321 'directori':101,147,159,160,163,167,168,171,174,176,179,183,185,187,189,313,324,331,516,540 'directory_client.delete':184 'directory_client.get':387 'directory_client.rename':188 'directory_client.set':402 'directory_client.update':420 'download':261,270,273,278,287,292,294,480 'download.readall':277,486 'download.readinto':290 'downloaded.txt':283 'els':325 'entri':415 'environ':54,600 'environment-specif':599 'execut':567 'exist':122 'expert':605 'f':286,291,320,364,368,391,396 'file':4,18,35,49,97,104,106,111,114,118,123,127,133,137,143,150,154,193,196,199,201,204,210,236,262,264,267,275,280,289,296,302,305,311,326,361,458,462,466,469,483,499,510,547 'file/directory':24,354 'file_client.append':237,246 'file_client.delete':304 'file_client.download':274,288,295,482 'file_client.flush':255 'file_client.get':360 'file_client.set':375 'file_client.upload':217,224,473 'file_system_client.create':162,170 'file_system_client.get':178,203,266,317,335,346,468 'filesystemcli':95 'flush':506 'fs':140 'fs.name':146 'full':539 'gen2':11,30 'get':121,173,198,357,384,533 'hello':227 'hierarch':17,34,496 'hierarchi':87 'high':528 'high-throughput':527 'id':408,428 'ident':53 'import':66,70,418,435,439,487 'inherit':519 'input':614 'instal':42,45 'lake':9,28,229 'larg':235,509 'last':369 'length':244,253,299 'level':93,517 'limit':576 'list':136,306,309,328,342,541 'local':209 'local-file.txt':213 'match':585 'metadata':374,376,377,544 'miss':622 'modifi':370,372 'mydir':164,181,338,349 'myfilesystem':120,130,135,157,465 'myfilesystem/newname':192 'name':191 'namespac':497 'nest':166 'new':190 'object':555 'offset':242,251,297 'open':212,282 'oper':25,94,99,102,105,108,148,194,444,491 'output':594 'overview':575 'overwrit':220,230,478 'owner':392,394,405 'path':310,315,318,329,333,336,337,344,347,348,534 'path.is':323 'path.name':327,340,353 'path/to/file.txt':206,269 'path/to/nested/dir':172 'permiss':397,399,409,615 'pip':44 'practic':493 'print':145,319,339,352,363,367,390,395 'process':378 'properti':355,358,359,362 'properties.last':371 'properties.size':366 'purpos':89 'py':6 'python':14,33,63,109,149,197,263,303,308,356,383,432 'rang':293 'rb':214 'recurs':341,350,423,536 'rename/move':186 'requir':613 'review':606 'rwx':429 'rwxr':411 'rwxr-x':410 'safeti':616 'scenario':530 'scope':587 'sdk':12,31 'semant':501 'servic':77,456 'service_client.create':117 'service_client.delete':132 'service_client.get':126,153,461 'service_client.list':142 'set':373,400,513,543 'simpl':554 'size':365 'skill':563,579 'skill-azure-storage-file-datalake-py' 'source-sickn33' 'specif':601 'stop':607 'storag':3,10,29,48,58,556 'substitut':597 'success':619 'system':19,36,98,107,112,115,119,124,128,134,138,144,151,155,459,463,500 'task':583 'test':603 'test.txt':471 'throughput':529 'topic-agent-skills' 'topic-agentic-skills' 'topic-ai-agent-skills' 'topic-ai-agents' 'topic-ai-coding' 'topic-ai-workflows' 'topic-antigravity' 'topic-antigravity-skills' 'topic-claude-code' 'topic-claude-code-skills' 'topic-codex-cli' 'topic-codex-skills' 'treat':592 'true':221,231,351,379,479,537 'updat':413 'upload':195,207,222,511 'url':60,75,81,83,451 'use':15,495,503,523,532,557,561,577 'user':407,425,427 'user-id':406,426 'valid':602 'variabl':55 'wb':284 'workflow':569 'workload':41 'x':412","prices":[{"id":"4899ef3d-381e-448e-817d-4f81fae5308b","listingId":"ba21640f-a958-42be-92da-ef06fdefb540","amountUsd":"0","unit":"free","nativeCurrency":null,"nativeAmount":null,"chain":null,"payTo":null,"paymentMethod":"skill-free","isPrimary":true,"details":{"org":"sickn33","category":"antigravity-awesome-skills","install_from":"skills.sh"},"createdAt":"2026-04-18T21:33:13.459Z"}],"sources":[{"listingId":"ba21640f-a958-42be-92da-ef06fdefb540","source":"github","sourceId":"sickn33/antigravity-awesome-skills/azure-storage-file-datalake-py","sourceUrl":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/azure-storage-file-datalake-py","isPrimary":false,"firstSeenAt":"2026-04-18T21:33:13.459Z","lastSeenAt":"2026-04-24T18:50:34.458Z"}],"details":{"listingId":"ba21640f-a958-42be-92da-ef06fdefb540","quickStartSnippet":null,"exampleRequest":null,"exampleResponse":null,"schema":null,"openapiUrl":null,"agentsTxtUrl":null,"citations":[],"useCases":[],"bestFor":[],"notFor":[],"kindDetails":{"org":"sickn33","slug":"azure-storage-file-datalake-py","github":{"repo":"sickn33/antigravity-awesome-skills","stars":34928,"topics":["agent-skills","agentic-skills","ai-agent-skills","ai-agents","ai-coding","ai-workflows","antigravity","antigravity-skills","claude-code","claude-code-skills","codex-cli","codex-skills","cursor","cursor-skills","developer-tools","gemini-cli","gemini-skills","kiro","mcp","skill-library"],"license":"mit","html_url":"https://github.com/sickn33/antigravity-awesome-skills","pushed_at":"2026-04-24T06:41:17Z","description":"Installable GitHub library of 1,400+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.","skill_md_sha":"f0cf30c09d4e59c9e0e9ae4ca3f9431309f51bd8","skill_md_path":"skills/azure-storage-file-datalake-py/SKILL.md","default_branch":"main","skill_tree_url":"https://github.com/sickn33/antigravity-awesome-skills/tree/main/skills/azure-storage-file-datalake-py"},"layout":"multi","source":"github","category":"antigravity-awesome-skills","frontmatter":{"name":"azure-storage-file-datalake-py","description":"Azure Data Lake Storage Gen2 SDK for Python. Use for hierarchical file systems, big data analytics, and file/directory operations."},"skills_sh_url":"https://skills.sh/sickn33/antigravity-awesome-skills/azure-storage-file-datalake-py"},"updatedAt":"2026-04-24T18:50:34.458Z"}}