Transient Errors & Retry Guidance

Google's data surfaces occasionally return an empty response shell even when the underlying data exists. Every Google Scraper sub-API surfaces this as either a 502 no results error or, less commonly, a 200 response with an empty results array. A single retry recovers the data ~99% of the time. This page documents the pattern so your crawlers handle it gracefully.

When You'll See It

Empirical Picture

Based on 260 stress requests across the Google Scraper sub-APIs (excluding outlier query patterns):

The takeaway: build retry-once into your client by default. A single retry takes the worst-case cell from 60% to 96% effective success.

Retry-Once Pattern (Python)

import time
import requests

def fetch_with_retry(url, max_attempts=2):
    for attempt in range(1, max_attempts + 1):
        r = requests.get(url, timeout=120)
        try:
            data = r.json()
        except Exception:
            data = None

        # Success: 200 with non-empty primary results
        if r.status_code == 200 and data and any(
            (data.get(k) or []) for k in
            ("local_results", "shopping_results", "news_results",
             "properties", "best_flights", "reviews")
        ):
            return data

        # Transient empty/502 — retry
        if attempt < max_attempts:
            time.sleep(0.5)
            continue

        # Final attempt failed — return whatever we got
        r.raise_for_status()
        return data

# Usage
data = fetch_with_retry(
    "https://api.scrape.do/plugin/google/maps/search?q=pizza+new+york&token=YOUR_TOKEN"
)

What NOT to Treat as Transient

News-Specific: Stale Tokens

The Google News API uses opaque tokens (topic_token, section_token, story_token, publication_token) that rotate occasionally. When a stored token stops working, the API returns:

{
  "error": "unexpected response",
  "message": "Unexpected response. Please try again, and if the issue continues, contact support."
}

Recovery: fetch a fresh token from a recent q=<keyword> response and retry. Don't store tokens long-term; treat them as ephemeral cursors. The two stable identifiers are q (keyword strings) and kgmid (Knowledge Graph entity IDs).

See News navigation for the full chaining playbook.

Session Expiry on Immersive Product Tokens

The Google Shopping Product API uses an opaque immersive_product_page_token minted by the search endpoint. Each token is tied to an upstream Google product-viewer session that ages out after several hours. When a stored token is past its lifetime, /plugin/google/shopping/product returns HTTP 410 Gone:

{
  "error": "session expired",
  "message": "The immersive_product_page_token is older than 6h0m0s and is past Google's product-viewer session lifetime. Fetch /plugin/google/shopping again to mint a fresh token, then retry."
}

(The exact lifetime in the message may vary; the recovery is always the same.)

Recovery: re-fetch /plugin/google/shopping with the same query, grab a fresh immersive_product_page_token from a shopping_results[] entry, then retry the product call with the new token. Don't store tokens long-term; treat them as ephemeral cursors. The stable identifier across re-mints is the search query (q), which always resolves to the same product cluster (gpcid) in the result set.

See Google Shopping Product for the per-endpoint details.