Spaces:
Sleeping
Sleeping
Commit History
-
Detect Google Scholar's block page, and unwrap Bing's redirect links
Claude +1: Claude Opus 5Unverified signatureBoth found by running the live smoke test against the deployed MCP server, not by the suite - the division of labour CONTRIBUTING.md already describes. Google Scholar serves an anti-bot interstitial to the Space's IP instead of results. That page has no `div.gs_ri`, so the scraper sat in `wait_for_selector` for the full 30 seconds and then reported a bare "Timeout 30000ms exceeded" - slow for every caller, and worded like a broken selector rather than a blocked deployment. It now waits for the results selector and Google's challenge elements together, so a block is recognised as soon as the page loads and raises GoogleScholarBlockedException with a message that says what actually happened. A challenge page carrying none of those elements falls back to matching Google's wording, but only after the selector wait has failed: sniffing the text up front would be faster and wrong, since a legitimate Scholar search for "unusual traffic" returns papers whose snippets contain that phrase. A p…f493ed5