Spaces:
Sleeping
Download README.md from bugrahans/google-search-scraper: direct link, hf CLI and curl.
- Browser
- Download file 3.58 kB
-
https://huggingface.co/spaces/bugrahans/google-search-scraper/resolve/main/README.md
- Command line
-
hf download hf://spaces/bugrahans/google-search-scraper/README.md
-
curl -L -o README.md https://huggingface.co/spaces/bugrahans/google-search-scraper/resolve/main/README.md
A newer version of the Gradio SDK is available: 6.32.0
title: Google Search Scraper
short_description: Any Google results page as JSON, plus a rank checker
emoji: 🔎
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 6.0.2
app_file: app.py
pinned: false
tags:
- google
- google-search
- serp
- serp-api
- google-scraper
- rank-tracker
- seo
- ai-overview
- people-also-ask
- web-scraping
- scraper
- data-extraction
- smolagents
- tool
- scrape.do
Google Search Scraper — Powered by Scrape.do
Run a Google search from any country and get the whole results page back as structured data: organic rankings, ads, People Also Ask, related searches, the AI Overview, knowledge panel, local pack, top stories and videos.
🔑 Get your free API token
👉 Sign up free on Scrape.do — 1,000 free credits per month, no credit card required. One results page costs 10 credits, so that's about 100 searches.
Two tabs
| Tab | Input | Output |
|---|---|---|
| Search results | Any query | A live preview of the results page, a table of organic results, CSV download and the raw JSON |
| Rank checker | Keyword + your domain | Your position, the page it's on, and every site ranking above you — highlighted on the results page |
Both tabs share the same settings: 14 countries (each with its own Google domain and language), desktop or mobile results, 1–5 pages deep, a time filter, and optional city-level location.
Why an API instead of parsing Google's HTML
Google renames the code behind its results page without warning, so a scraper built on page selectors breaks silently. This Space uses the Scrape.do Google Search API, which returns the same field names whatever Google changes, and handles proxies and CAPTCHAs.
Two details are handled for you:
- Multi-page searches stay consistent. Every page request carries a start offset from the first page on, so all pages come from the same browsing session instead of reshuffling or repeating results.
- Late AI Overviews are collected. When Google is still writing its AI Overview as the page returns, the tool fetches the finished version with one follow-up call (5 credits), immediately — the key for it expires in 90 seconds.
Use as a smolagents Tool
from smolagents import CodeAgent, InferenceClientModel
from tool import ScrapeDoGoogleSearchTool
agent = CodeAgent(
tools=[ScrapeDoGoogleSearchTool(api_token="YOUR_SCRAPEDO_TOKEN")],
model=InferenceClientModel(),
)
agent.run("Who ranks in the top 5 for 'best web scraping api' in Germany, and do any run ads?")
You can also set SCRAPE_DO_API_TOKEN in your environment and instantiate
ScrapeDoGoogleSearchTool() with no arguments.
Inputs
query(required) — The search, as you'd type it into Google.country(optional) — Two-letter code (us,gb,de,tr…). Sets ranking country, language and Google domain. Defaults tous.pages(optional) — 1–5 result pages. 10 credits each. Defaults to1.device(optional) —desktopormobile.time_period(optional) —last_hour,last_day,last_week,last_month,last_year.location(optional) — City in Google's format, e.g.Austin,Texas,United States.
Learn more at scrape.do · Documentation · Google search scraping guide