Crawl4AI RAG

Combines web crawling with vector search to enable content extraction, semantic indexing, and retrieval-augmented generation over web-based documentation and code repositories.

MCP server that integrates web crawling capabilities with RAG (Retrieval-Augmented Generation) functionality, built by Jason Guo to enable AI agents and coding assistants to crawl websites, extract content, and perform semantic search over the collected data. The implementation combines Crawl4AI for web scraping with Supabase for vector storage, supporting advanced features like contextual embeddings, hybrid search, code example extraction, and AI hallucination detection through Neo4j knowledge graphs. Designed for AI coding workflows where assistants need to gather, index, and query web-based documentation or code repositories, with Docker deployment and SearXNG integration for enhanced search capabilities.