| --- |
| title: Web MCP |
| emoji: 🔎 |
| colorFrom: blue |
| colorTo: green |
| sdk: gradio |
| sdk_version: 5.36.2 |
| app_file: app.py |
| pinned: false |
| short_description: Search & fetch the web with per-tool analytics |
| thumbnail: >- |
| https://cdn-uploads.huggingface.co/production/uploads/5f17f0a0925b9863e28ad517/tfYtTMw9FgiWdyyIYz6A6.png |
| --- |
| |
| # Web MCP Server |
|
|
| A Model Context Protocol (MCP) server that exposes two composable tools—`search` (Serper metadata) and `fetch` (single-page extraction)—alongside a live analytics dashboard that tracks daily usage for each tool. The UI runs on Gradio and can be reached directly or via MCP-compatible clients like Claude Desktop and Cursor. |
|
|
| ## Highlights |
| - Dual MCP tools with shared rate limiting (`360 requests/hour`) and structured JSON responses. |
| - Daily analytics split by tool: the **Analytics** tab renders "Daily Search" (left) and "Daily Fetch" (right) bar charts covering the last 14 days. |
| - Persistent request counters keyed by UTC date and tool: `{"YYYY-MM-DD": {"search": n, "fetch": m}}`, with automatic migration from legacy totals. |
| - Pluggable storage: respects `ANALYTICS_DATA_DIR`, otherwise falls back to `/data` (if writable) or `./data` for local development. |
| - Ready-to-serve Gradio app with MCP endpoints exposed via `gr.api` for direct client consumption. |
|
|
| ## Requirements |
| - Python 3.8 or newer. |
| - Serper API key (`SERPER_API_KEY`) with access to the Search and News endpoints. |
| - Dependencies listed in `requirements.txt`, including `filelock` and `pandas` for analytics storage. |
|
|
| Install everything with: |
| ```bash |
| pip install -r requirements.txt |
| ``` |
|
|
| ## Configuration |
| 1. Export your Serper API key: |
| ```bash |
| export SERPER_API_KEY="your-api-key" |
| ``` |
| 2. (Optional) Override the analytics storage path: |
| ```bash |
| export ANALYTICS_DATA_DIR="/path/to/persistent/storage" |
| ``` |
| If unset, the app automatically prefers `/data` when available, otherwise `./data`. |
|
|
| 3. (Optional) Control private/local address policy for `fetch`: |
| - `FETCH_ALLOW_PRIVATE` — set to `1`/`true` to disable the SSRF guard entirely (not recommended except for trusted, local testing). |
| - `FETCH_PRIVATE_ALLOWLIST` — comma/space separated host patterns allowed even if they resolve to private/local IPs, e.g.: |
| ```bash |
| export FETCH_PRIVATE_ALLOWLIST="*.corp.local, my-proxy.internal" |
| ``` |
| If neither is set, the fetcher refuses URLs whose host resolves to private, loopback, link‑local, multicast, reserved, or unspecified addresses. It also re-checks the final redirect target. |
| |
| The request counters live in `<DATA_DIR>/request_counts.json`, guarded by a file lock to support concurrent MCP calls. |
|
|
| ## Running Locally |
| Launch the Gradio server (with MCP support enabled) via: |
| ```bash |
| python app.py |
| ``` |
| This starts a local UI at `http://localhost:7860` and exposes the MCP SSE endpoint at `http://localhost:7860/gradio_api/mcp/sse`. |
|
|
| ### Connecting From MCP Clients |
| - **Claude Desktop** – update `claude_desktop_config.json`: |
| ```json |
| { |
| "mcpServers": { |
| "web-search": { |
| "command": "python", |
| "args": ["/absolute/path/to/app.py"], |
| "env": { |
| "SERPER_API_KEY": "your-api-key" |
| } |
| } |
| } |
| } |
| ``` |
| - **URL-based MCP clients** – run `python app.py`, then point the client to `http://localhost:7860/gradio_api/mcp/sse`. |
|
|
| ## Tool Reference |
| ### `search` |
| - **Purpose**: Retrieve metadata-only results from Serper (general web or news). |
| - **Inputs**: |
| - `query` *(str, required)* – search terms. |
| - `search_type` *("search" | "news", default "search")* – switch to `news` for recency-focused results. |
| - `num_results` *(int, default 4, range 1–20)* – number of hits to return. |
| - **Output**: JSON containing the query echo, result count, timing, and an array of entries with `position`, `title`, `link`, `domain`, and optional `source`/`date` for news. |
|
|
| ### `fetch` |
| - **Purpose**: Download a single URL and extract the readable article text via Trafilatura. |
| - **Inputs**: |
| - `url` *(str, required)* – must start with `http://` or `https://`. |
| - `timeout` *(int, default 20 seconds)* – client timeout for the HTTP request. |
| - **Output**: JSON with the original and final URL, domain, HTTP status, title, ISO timestamp of the fetch, word count, cleaned `content`, and duration. |
|
|
| Both tools increment their respective analytics buckets on every invocation, including validation failures and rate-limit denials, ensuring the dashboard mirrors real traffic. |
|
|
| ## Analytics Dashboard |
| Open the **Analytics** tab in the Gradio UI to inspect daily activity: |
| - **Daily Search Count** (left column) – bar chart for the past 14 days of `search` tool requests. |
| - **Daily Fetch Count** (right column) – bar chart for the past 14 days of `fetch` tool requests. |
| - Tooltips reveal the display label (e.g., `Sep 17`), raw count, and ISO date key. |
|
|
| Data is stored in JSON and can be safely externalized for long-term tracking. Existing totals in the legacy integer-only format are automatically migrated during the first write. |
|
|
| ## Rate Limiting & Error Handling |
| - Global moving-window limit of `360` requests per hour shared across both tools (powered by `limits`). |
| - Standardized error payloads for missing parameters, invalid URLs, Serper issues, HTTP failures, and rate-limit hits, each preserving analytics increments. |
|
|
| ## Troubleshooting |
| - **`SERPER_API_KEY is not set`** – export the key in the environment where the server runs. |
| - **`Rate limit exceeded`** – pause requests or reduce client concurrency. |
| - **Empty extraction** – some sites block bots; try another URL. |
| - **Storage permissions** – ensure the chosen data directory is writable; adjust `ANALYTICS_DATA_DIR` if necessary. |
|
|
| ## Licensing & Contributions |
| Feel free to fork and adapt for your MCP workflows. Contributions are welcome—open a PR or issue with proposed analytics enhancements, additional tooling, or documentation tweaks. |
|
|
