unclecode/crawl4ai

What it does

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

Summary based on the GitHub description (opens in a new tab)

What the README covers

• 🚀🤖 Crawl4AI: the open-source web crawler for LLMs and AI agents
• Two ways to use Crawl4AI
• 🐍 Run it yourself: open source, forever
• ☁️ Or use the cloud: no browsers, no proxies
• Which one?
• ✨ Features
• 🛠️ Installation
• 🔬 Advanced usage examples
• 📖 Documentation
• 🤝 Contributing
• 📄 License & Attribution
• Attribution Requirements

Getting started

Excerpt from the README · Checked 15 Oct 2026

pip install -U crawl4ai
crawl4ai-setup        # installs the browser, once

Shown exactly as published. Top 200 Repos has not run or tested it. View in README (opens in a new tab)

Ranking trend

Global rank in each recorded snapshot. A lower number is a higher rank.

From #186 to #189 across 2 snapshots since 15 Oct 2026.

What people build with it

Related repositories

• firecrawl/firecrawl#32

  Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

  Shared topics: scraper, web-crawler, crawler, data-extraction

• D4Vinci/Scrapling#138

  An adaptive Python web scraping framework for individual requests and larger crawls.

  Shared topics: crawler, data-extraction, playwright, web-scraping

• browser-use/browser-use#100

  Open-source Python library for AI browser agents, plus Browser Use Cloud: stealth browsers and a hosted agent API.

  Shared topics: playwright, web-scraping, ai-agents, mcp

• langchain-ai/langchain#64

  The agent engineering platform.

  Shared topics: open-source, rag, ai-agents, llm

• Graphify-Labs/graphify#69

  Turn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.

  Shared topics: rag, ai-agents, mcp, llm

• ollama/ollama#45

  Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

  Used together in Turn public competitor pages into a structured market scan with Crawl4AI