MCP Web Tools

Self-hosted Model Context Protocol (MCP) server giving AI agents free web_search and web_extractor tools — search via SearXNG, extract via Crawl4AI, no paid API keys.

Previously known as Agent Web Tool MCP / agent-web-tool-mcp. GitHub redirects old URLs — existing clones and links keep working.

MCP Web Tools is a self-hosted Model Context Protocol server that lets developers serve web_search and web_extractor to their AI agents locally — instead of paying per-query for Brave Search API, Tavily, or Serper. Search is backed by a self-hosted SearXNG metasearch instance aggregating ~70 engines; extraction is backed by Crawl4AI, which fetches pages with a headless Chromium and returns clean Markdown.

The whole stack is a 4-service Docker Compose deployment: FastMCP server + SearXNG + Crawl4AI + Valkey cache. Apache-2.0 licensed.

Why self-host MCP Web Tools?

• Free per-query: replace Brave Search API / Tavily API / Serper / Bing keys (each paid per query) with a stack you control.
• Private by default: agent queries never leave your network — only the underlying SearXNG / Crawl4AI fetches do.
• One docker compose up: runs on a dev laptop, a shared internal box, or a VM. No SaaS account, no rotating API keys.
• Reproducible: pin every upstream image for byte-identical redeploys.
• Standard MCP surface: works unchanged with Claude Code, Cursor, Hermes, or any MCP-compatible client.

Use cases

• Agent web access without paying per-query for SaaS search APIs.
• Markdown extraction from arbitrary URLs (for RAG ingestion, summarization, agentic browsing, web scraping).
• A self-hostable, auditable web-tool layer that does not leak agent prompts to a third-party SaaS.
• A single docker compose up to add web capability to Claude Code, Cursor, or any MCP-compatible client.

Quick links

• Install guide — one-liner installer, manual clone, make targets, smoke tests, agent connect snippets.
• Configuration reference — every .env variable + full web_search / web_extractor parameter tables + dev playground.
• Architecture — service topology, request flow, Mermaid diagrams, gotchas.
• Changelog — release notes.
• Source on GitHub — Apache-2.0.

At a glance

Hosting	Self-hosted (Docker Compose)
API key required	None
Tools	web_search, web_extractor
Search backend	SearXNG (~70 engines)
Extraction backend	Crawl4AI (headless Chromium)
Transport	Streamable HTTP (default) or stdio
Cache	Valkey + in-process TTL cache
License	Apache-2.0

One-liner install

curl -fsSL https://github.com/datvietvac-techhub/mcp-web-tools/releases/latest/download/install.sh | bash
cd ~/.local/share/mcp-web-tool
make up && make smoke
claude mcp add --transport http web-tool http://localhost:8000/mcp

See the install guide for details.