Extract schema.org, Open Graph, and JSON-LD metadata from web pages for indexing

Uses extruct to pull machine-readable metadata from raw HTML so an agent can classify, deduplicate, or enrich pages without brittle full-page parsing. It is best for metadata harvesting workflows, not for crawling an entire site or rendering JavaScript-heavy pages.

Prerequisites

Python 3 environment

Installation

Use the upstream install or setup path that matches your environment:

• pip install extruct
• pip install 'extruct[cli]'
• pip install -r requirements-dev.txt

Requirements and caveats from upstream:

• :target: https://pypi.python.org/pypi/extruct
• .. _rdflib: https://pypi.python.org/pypi/rdflib/
• First fetch the HTML using python-requests and then feed the response body to extruct::

Basic usage or getting-started notes:

---

• ::

---

• Source: https://github.com/scrapinghub/extruct
• Extracted from upstream docs: https://raw.githubusercontent.com/scrapinghub/extruct/HEAD/README.rst

Documentation

• https://github.com/scrapinghub/extruct#readme

Source

• Agent Skill Exchange