# Transient Errors & Retry Guidance

> Source: https://scrape.do/documentation/google-scraper-api/transient-errors/

Why 502 no-results and empty-array responses happen, when to retry, and the retry-once pattern

Google's data surfaces occasionally return an empty response shell even when the underlying data exists. Every Google Scraper sub-API surfaces this as either a `502 no results` error or, less commonly, a `200` response with an empty results array. **A single retry recovers the data ~99% of the time.** This page documents the pattern so your crawlers handle it gracefully.

---

## When You'll See It

| Symptom | Plugins where it appears | Typical cause |
|---|---|---|
| `502 {"error":"no results"}` | Search · Maps · Maps Reviews · Flights · Hotels · News (with stale token) | Google returned an empty data shell; plugin parses zero results |
| `200 { "shopping_results": [] }` | Shopping (deeper paginated pages) | Same upstream cause; Shopping plugin returns 200 since the response was structurally valid |
| `502 {"error":"unexpected response"}` | News only | Stale `*_token`; fetch a fresh one from a recent search response |
| `410 {"error":"session expired"}` | Shopping Product | `immersive_product_page_token` aged out (typically several hours after mint). Re-mint via `/plugin/google/shopping` |
| `502 {"error":"request failed"}` | All | Generic upstream failure; safe to retry |

---

## Empirical Picture

Based on 260 stress requests across the Google Scraper sub-APIs (excluding outlier query patterns):

| Plugin family | First-try success | After one retry |
|---|---:|---:|
| Most queries | **99%** | **~100%** |
| Edge cases (deep pagination on dense queries) | 60–80% | 96–99% |

The takeaway: **build retry-once into your client by default.** A single retry takes the worst-case cell from 60% to 96% effective success.

---

## Retry-Once Pattern (Python)

```python
import time
import requests

def fetch_with_retry(url, max_attempts=2):
    for attempt in range(1, max_attempts + 1):
        r = requests.get(url, timeout=120)
        try:
            data = r.json()
        except Exception:
            data = None

        # Success: 200 with non-empty primary results
        if r.status_code == 200 and data and any(
            (data.get(k) or []) for k in
            ("local_results", "shopping_results", "news_results",
             "properties", "best_flights", "reviews")
        ):
            return data

        # Transient empty/502 — retry
        if attempt < max_attempts:
            time.sleep(0.5)
            continue

        # Final attempt failed — return whatever we got
        r.raise_for_status()
        return data

# Usage
data = fetch_with_retry(
    "https://api.scrape.do/plugin/google/maps/search?q=pizza+new+york&token=YOUR_TOKEN"
)
```

---

## What NOT to Treat as Transient

| Symptom | Treat as |
|---|---|
| `400` with a clear validation message | Permanent. Fix the request |
| `200` with `error` field present (validation that returned 200 in some legacy paths) | Permanent. Fix the request |
| `502 no results` after 2–3 retries with the same parameters | Likely a real empty result (e.g., `start=200` past the end of the result set) |
| `500` with a parser error message | Possibly a new upstream HTML/JSON shape. Log it and report to support |

---

## News-Specific: Stale Tokens

The Google News API uses opaque tokens (`topic_token`, `section_token`, `story_token`, `publication_token`) that **rotate occasionally**. When a stored token stops working, the API returns:

```json
{
  "error": "unexpected response",
  "message": "Unexpected response. Please try again, and if the issue continues, contact support."
}
```

Recovery: fetch a fresh token from a recent `q=<keyword>` response and retry. Don't store tokens long-term; treat them as ephemeral cursors. The two stable identifiers are `q` (keyword strings) and `kgmid` (Knowledge Graph entity IDs).

See [News navigation](/documentation/google-scraper-api/news) for the full chaining playbook.

---

## Session Expiry on Immersive Product Tokens

The Google Shopping Product API uses an opaque `immersive_product_page_token` minted by the search endpoint. Each token is tied to an upstream Google product-viewer session that **ages out after several hours**. When a stored token is past its lifetime, `/plugin/google/shopping/product` returns HTTP `410 Gone`:

```json
{
  "error": "session expired",
  "message": "The immersive_product_page_token is older than 6h0m0s and is past Google's product-viewer session lifetime. Fetch /plugin/google/shopping again to mint a fresh token, then retry."
}
```

(The exact lifetime in the message may vary; the recovery is always the same.)

Recovery: re-fetch `/plugin/google/shopping` with the same query, grab a fresh `immersive_product_page_token` from a `shopping_results[]` entry, then retry the product call with the new token. Don't store tokens long-term; treat them as ephemeral cursors. The stable identifier across re-mints is the search query (`q`), which always resolves to the same product cluster (`gpcid`) in the result set.

See [Google Shopping Product](/documentation/google-scraper-api/shopping/product#handling-session-expiry) for the per-endpoint details.

---