Transient Errors & Retry Guidance

Why 502 no-results and empty-array responses happen, when to retry, and the retry-once pattern

Google's data surfaces occasionally return an empty response shell even when the underlying data exists. Every Google Scraper sub-API surfaces this as either a 502 no results error or, less commonly, a 200 response with an empty results array. A single retry recovers the data ~99% of the time. This page documents the pattern so your crawlers handle it gracefully.

When You'll See It

Symptom	Plugins where it appears	Typical cause
`502 {"error":"no results"}`	Search · Maps · Maps Reviews · Flights · Hotels · News (with stale token)	Google returned an empty data shell; plugin parses zero results
`200 { "shopping_results": [] }`	Shopping (deeper paginated pages)	Same upstream cause; Shopping plugin returns 200 since the response was structurally valid
`502 {"error":"unexpected response"}`	News only	Stale `*_token`; fetch a fresh one from a recent search response
`502 {"error":"request failed"}`	All	Generic upstream failure; safe to retry

Empirical Picture

Based on 260 stress requests across the Google Scraper sub-APIs (excluding outlier query patterns):

Plugin family	First-try success	After one retry
Most queries	99%	~100%
Edge cases (deep pagination on dense queries)	60–80%	96–99%

The takeaway: build retry-once into your client by default. A single retry takes the worst-case cell from 60% to 96% effective success.

Retry-Once Pattern (Python)

import time
import requests

def fetch_with_retry(url, max_attempts=2):
    for attempt in range(1, max_attempts + 1):
        r = requests.get(url, timeout=120)
        try:
            data = r.json()
        except Exception:
            data = None

        # Success: 200 with non-empty primary results
        if r.status_code == 200 and data and any(
            (data.get(k) or []) for k in
            ("local_results", "shopping_results", "news_results",
             "properties", "best_flights", "reviews")
        ):
            return data

        # Transient empty/502 — retry
        if attempt < max_attempts:
            time.sleep(0.5)
            continue

        # Final attempt failed — return whatever we got
        r.raise_for_status()
        return data

# Usage
data = fetch_with_retry(
    "https://api.scrape.do/plugin/google/maps/search?q=pizza+new+york&token=YOUR_TOKEN"
)

What NOT to Treat as Transient

Symptom	Treat as
`400` with a clear validation message	Permanent. Fix the request
`200` with `error` field present (validation that returned 200 in some legacy paths)	Permanent. Fix the request
`502 no results` after 2–3 retries with the same parameters	Likely a real empty result (e.g., `start=200` past the end of the result set)
`500` with a parser error message	Possibly a new upstream HTML/JSON shape. Log it and report to support

News-Specific: Stale Tokens

The Google News API uses opaque tokens (topic_token, section_token, story_token, publication_token) that rotate occasionally. When a stored token stops working, the API returns:

{
  "error": "unexpected response",
  "message": "Unexpected response. Please try again, and if the issue continues, contact support."
}

Recovery: fetch a fresh token from a recent q=<keyword> response and retry. Don't store tokens long-term; treat them as ephemeral cursors. The two stable identifiers are q (keyword strings) and kgmid (Knowledge Graph entity IDs).

See News navigation for the full chaining playbook.