# Tray.ai - Scrape.do Integration

> Source: https://scrape.do/documentation/integrations/tray-ai/

Step-by-step guide to integrate Scrape.do with Tray.ai for bulk web scraping workflows from Google Sheets

Scrape.do works in Tray.ai through the **HTTP Client** connector. Your workflow reads a list of URLs from Google Sheets, sends each one to Scrape.do, parses the page and writes one row per URL to a results sheet.

## 1. Create a Project and Add Config Data

Your token lives in the project's config data, so no step carries it.

- From your Tray dashboard, click **Create new** > **Project** and give it a name.
- Open the **Config data** tab, click **Add** and create these keys:
    - **scrape_do_token** - Your API token from your [dashboard](https://dashboard.scrape.do)
    - **source_sheet_id** - The Google Sheets ID holding your URL list (one header: `url`)
    - **results_sheet_id** - The Google Sheets ID where results are written (headers: `URL`, `ASIN`, `Name`, `Price`, `Rating`, `Status`, `Scraped`)
- Click **Save**. Any step can now reference a key, for example `$.config.scrape_do_token`.

Tray's project export carries config values, so clear `scrape_do_token` before you share an exported project.

## 2. Read the URL List and Loop

- Inside your project, click **Create workflow** and select the **Manual** trigger.
- Add a **Google Sheets** connector (`google-sheets-1`):
    - **Operation** - **Get rows**
    - **Spreadsheet** - `$.config.source_sheet_id`
    - **Worksheet** - `Sheet1`
    - **Has headers** - ON
- Add a **Loop** connector (`loop-1`):
    - **Operation** - **Loop list**
    - **List** - `$.steps.google-sheets-1.rows`
    - **Parallel processing** - ON
    - **Concurrency** - Your Scrape.do plan concurrency

The URL for each iteration is `$.steps.loop-1.value.url`.

## 3. Send Each URL to Scrape.do

Inside the loop, add an **HTTP Client** connector (`http-client-1`):

- **Operation** - **GET**
- **URL** - `https://api.scrape.do/`
- **Query parameters**:

| Parameter | Value |
|---|---|
| `token` | `$.config.scrape_do_token` |
| `url` | `$.steps.loop-1.value.url` |
| `output` | `raw` to parse HTML with a script, or `markdown` for an AI step |
| `render` | `false`, or `true` for JavaScript-heavy pages |
| `super` | `false`, or `true` for residential and mobile proxies |
| `geoCode` | A country code such as `us`. View available locations [here](/documentation/proxies/geo-code) |

- Set **Parse response** to OFF.
- Under **Error handling**, set **On error** to **Continue** with 2 retries.
- Map the target URL into the `url` parameter. Don't add it to the URL field yourself: Tray encodes query parameters for you, and a hand-built query string breaks on targets with `?` or `&`.
- Click **Run step** and check for a 200 with page content in `$.steps.http-client-1.response.body`.

> If you're unable to get a response, visit the [playground](https://dashboard.scrape.do/playground) and experiment with parameters like **Super**, **Render JavaScript**, and **Block Resources** until you get a successful response.

## 4. Extract the Fields

Add a **Script** connector (`script-1`) inside the loop with these **Variables**:

- **source_url** - `$.steps.loop-1.value.url`
- **response** - `$.steps.http-client-1.response.body`
- **status** - `$.steps.http-client-1.response.status_code`

The script returns `url`, `asin`, `name`, `price`, `rating`, `status` and `scraped_at`. It matches HTML tags, so pair it with `output=raw`. For sites with different layouts, put an **Anthropic** or **OpenAI** step before the script and use `output=markdown`.

## 5. Write Results to Google Sheets

Add a second **Google Sheets** connector (`google-sheets-2`) inside the loop:

- **Operation** - **Add row**
- **Spreadsheet** - `$.config.results_sheet_id`
- **Worksheet** - `Sheet1`
- Map each column to `$.steps.script-1.result.<field>`, for example **URL** to `$.steps.script-1.result.url`.

## 6. Test and Deploy

- Click **Run workflow** and open the **Logs** tab. Check that the loop ran once per row and the results sheet has one row per URL.
- Switch the trigger from **Manual** to **Scheduled** when the run is clean.

**<a href="/uploads/blog/scrapedo-tray-workflow.json" download="scrapedo-tray-workflow.json">Download this Tray.ai workflow</a>** to skip steps 2 to 5. Importing doesn't bring authentications with it: connect Google Sheets and fill in the three config data keys before the first run.

Full tutorial with every step: [How to Build a Bulk Web Scraper in Tray.ai with Scrape.do (Low Code)](/blog/tray-ai-web-scraper/)