# Getting Started > Source: https://scrape.do/documentation/ Scrape.do documentation. Scrape.do provides _**unblocked access to public web data at scale**_ by: - Avoiding all anti-bot, WAF, and CAPTCHA through custom bypass solutions. - Rotating 150M+ datacenter, residential, and mobile proxies in 150 countries. - Seamlessly rendering and interacting web pages with its managed headless browser. **After signing up for Scrape.do and generating your API key, test your setup with a basic request:** **cURL (API mode)** ```bash curl --location --request GET 'https://api.scrape.do/?token=YOUR_TOKEN&url=https://httpbin.co/anything' ``` **Python (API mode)** ```python import requests import urllib.parse token = "YOUR_TOKEN" targetUrl = urllib.parse.quote("https://httpbin.co/anything") url = "http://api.scrape.do/?token={}&url={}".format(token, targetUrl) response = requests.request("GET", url) print(response.text) ``` **Node.js (API mode)** ```javascript const axios = require('axios'); const token = "YOUR_TOKEN"; const targetUrl = encodeURIComponent("https://httpbin.co/anything"); const config = { 'method': 'GET', 'url': `https://api.scrape.do/?token=${token}&url=${targetUrl}`, 'headers': {} }; axios(config) .then(function (response) { console.log(response.data); }) .catch(function (error) { console.log(error); }); ``` **cURL (Proxy mode)** ```bash curl -k -x "http://YOUR_TOKEN:@proxy.scrape.do:8080" 'https://httpbin.co/anything' -v ``` **Python (Proxy mode)** ```python import requests import urllib3 urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) url = "https://httpbin.co/anything" token = "YOUR_TOKEN" proxyModeUrl = "http://{}:@proxy.scrape.do:8080".format(token) proxies = { "http": proxyModeUrl, "https": proxyModeUrl, } response = requests.request("GET", url, proxies=proxies, verify=False) print(response.text) ``` **Node.js (Proxy mode)** ```javascript const axios = require('axios'); const token = "YOUR_TOKEN"; const targetUrl = "https://httpbin.co/anything"; axios({ method:"GET", url:targetUrl, proxy: { protocol:'http', host: 'proxy.scrape.do', port: 8080, auth: { username: token, password: '' } } }) .then(response => { console.log(response.data); }) .catch(error => { console.error(error.message); }); ``` *(Go, Ruby, Java, C#, PHP examples are also available on the HTML version.)* --- ## Generate API Token All your requests are authorized with your account token, which is **automatically generated when you first sign up**. Your token is located in your dashboard here: ![Token Location](/images/docs/token.gif) If you haven't signed up yet, [create a free-forever account here](https://dashboard.scrape.do/sign-up). --- ## API Playground The [API Playground](https://dashboard.scrape.do/playground) is your interactive testing environment where you can experiment with Scrape.do parameters without writing code. Use it to: - Test different parameter combinations in real-time - Generate code snippets in multiple programming languages (Python, Node.js, PHP, cURL, etc.) - Preview API responses before implementing in your application - Learn how parameters affect request behavior ![API Playground Dashboard](/images/docs/dashboard.png) Access the [playground](https://dashboard.scrape.do/playground) from your dashboard to build and refine your scraping requests with instant feedback. --- ## Encode Your Target URL Pass the target website URL you want to scrape using the `url` parameter. When using API mode, **you must URL-encode the parameter** to prevent it from being misinterpreted as multiple query parameters (supported protocols: `HTTP` and `HTTPS`). **cURL (API mode)** ```bash sudo apt-get install gridsite-clients urlencode "YOUR_URL" ``` **Python (API mode)** ```python import urllib.parse encoded_url = urllib.parse.quote("YOUR_URL") ``` **Node.js (API mode)** ```javascript let encoded_url = encodeURIComponent("YOUR_URL") ``` *(Go, Ruby, Java, C#, PHP examples are also available on the HTML version.)* > [!NOTE] > No need to encode the URL if you use **Proxy Mode**. --- ## API Parameters Overview You can view all the parameters of Scrape.do from the table below and have an overview of all of them quickly. | Parameter | Type | Default | Description | Details | | --------------------- | ------ | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | | **token\*** | string | | The token to use for authentication. | [more](/documentation/#generate-api-token) | | **url\*** | string | | Target web page URL. | [more](/documentation/#encode-your-target-url) | | super | bool | false | Use Residential & Mobile Proxy Networks | [more](/documentation/proxies/super) | | geoCode | string | | Choose the right country for your target web page | [more](/documentation/proxies/geo-code) | | regionalGeoCode | string | | Choose continent for your target web page | [more](/documentation/proxies/regional-geo-code) | | sessionId | int | | Use the same IP address continuously with a session | [more](/documentation/api-response/session-id) | | customHeaders | bool | false | Handle all request headers for the target web page | [more](/documentation/headers-cookies/custom-headers) | | extraHeaders | bool | false | Use it to change header values or add new headers over the ones we have added | [more](/documentation/headers-cookies/extra-headers) | | forwardHeaders | bool | false | Forward your own headers to the target website | [more](/documentation/headers-cookies/forward-headers) | | setCookies | string | | Set cookies for the target web page | [more](/documentation/headers-cookies/set-cookies) | | disableRedirection | bool | false | Disable request redirection for your use-case | [more](/documentation/api-response/disable-redirection) | | callback | string | | **Deprecated.** Use [Async API webhooks](/documentation/async-api/) instead | [more](/documentation/async-api/) | | timeout | int | 60000 | Set maximum timeout for your requests | [more](/documentation/api-response/timeout) | | retryTimeout | int | 15000 | Set maximum timeout for retry mechanism | [more](/documentation/api-response/retry-settings#retry-timeout) | | disableRetry | bool | false | Disable retry mechanism for your use-case | [more](/documentation/api-response/retry-settings#disable-retry) | | device | string | desktop | Specify the device type (desktop, mobile, tablet) | [more](/documentation/api-response/device) | | render | bool | false | Use a headless browser to render JavaScript and wait for content to load | [more](/documentation/headless-browser#js-render) | | waitUntil | string | domcontentloaded | Control when the browser considers the page loaded | [more](/documentation/headless-browser/wait#wait-until) | | customWait | int | 0 | Set the browser wait time on the target web page after content loaded | [more](/documentation/headless-browser/wait#custom-wait) | | waitSelector | string | | CSS selector to wait for in the target web page | [more](/documentation/headless-browser/wait#wait-css-selector) | | width | int | 1920 | Browser viewport width in pixels | [more](/documentation/headless-browser/viewport#width) | | height | int | 1080 | Browser viewport height in pixels | [more](/documentation/headless-browser/viewport#height) | | blockResources | bool | true | Block CSS, images, and fonts on your target web page | [more](/documentation/headless-browser/block-resources) | | screenShot | bool | false | Return a screenshot from your target web page | [more](/documentation/headless-browser/screenshot#normal-screenshot) | | fullScreenShot | bool | false | Return a full page screenshot from your target web page | [more](/documentation/headless-browser/screenshot#full-page-screenshot) | | particularScreenShot | string | | Return a screenshot of a particular area from your target web page | [more](/documentation/headless-browser/screenshot#partial-screenshot) | | playWithBrowser | string | | Simulate browser actions like click, scroll, execute js, etc. | [more](/documentation/headless-browser/browser-interactions) | | output | string | raw | Get the output in raw or markdown format | [more](/documentation/api-response/response-output#output-format) | | transparentResponse | bool | false | Return pure response from target web page without Scrape.do processing | [more](/documentation/api-response/response-output#transparent-response) | | returnJSON | bool | false | Returns network requests with content as a property string | [more](/documentation/headless-browser/returnjson) | | showFrames | bool | false | Returns all iframe content from the target webpage (requires render=true and returnJSON=true) | [more](/documentation/headless-browser/returnjson#show-frames) | | showWebsocketRequests | bool | false | Display websocket requests (requires render=true and returnJSON=true) | [more](/documentation/headless-browser/returnjson#show-websocket-requests) | | pureCookies | bool | false | Returns the original Set-Cookie headers from the target website | [more](/documentation/api-response/response-output#pure-cookies) | --- ## How Does Scrape.do Work? Modern websites use sophisticated anti-bot systems (Cloudflare, PerimeterX, DataDome, Akamai) that fingerprint TLS handshakes, validate HTTP headers, and blacklist datacenter IPs to block automated traffic. Scrape.do solves this by handling all anti-bot evasion on your behalf. Technically, **we act as an intelligent proxy layer between your application and target websites**. Every request is intercepted, upgraded to mimic legitimate browser behavior at multiple layers (TLS, HTTP, JavaScript), routed through our residential/mobile proxy network, and delivered as if from a real user. Your request goes through our infrastructure which: 1. **Routes through rotating proxies** — Your request is forwarded through our pool of 150M+ datacenter, residential, and mobile IPs across 150 countries, automatically rotating to avoid rate limits and IP bans. 2. **Mimics real browser behavior** — We manipulate TLS fingerprints and HTTP headers to match legitimate browser traffic, making your requests indistinguishable from organic users to bypass WAFs and anti-bot systems. 3. **Renders JavaScript if needed** — Setting `render=true` spins up a headless browser (Chromium) to execute JavaScript and load dynamic content, essential for modern SPAs built with React, Vue, or Angular. 4. **Handles CAPTCHAs automatically** — When target sites present CAPTCHA challenges (reCAPTCHA, hCaptcha, etc.), our system detects and solves them transparently without your intervention. 5. **Retries intelligently** — If a request fails due to temporary issues (502/503, timeouts, rate limits), we automatically retry with a different IP until success or timeout. 6. **Returns clean data** — You receive the raw HTML, JSON, or any content type the target website returns. API credits are only consumed on successful requests (2xx status codes). For asynchronous processing of long-running jobs, use the [Async API](/documentation/async-api/) to submit jobs and receive results via webhook instead of keeping connections open. ### Credits Credits are the unit of billing in Scrape.do. You are only charged for successful requests (status codes `2xx`, `400`, `404`, `410`). Failed requests are free. Not all requests cost the same. Some websites require advanced bypass methods (residential proxies, headless browser rendering, or custom anti-detection solutions), which consume more credits per request. For example, a basic datacenter request costs 1 credit, while a residential proxy request with JS rendering costs 25. See the full [Request Costs](/documentation/request-costs) table for credit pricing by request type and domain-specific pricing. ### Concurrency Concurrency is the number of requests your account can process simultaneously. Each plan has a concurrency limit that determines how many requests can be in-flight at the same time. If you hit your concurrency limit, additional requests will be queued or rejected until a slot opens up. The [Async API](/documentation/async-api/) has a separate concurrency pool (30% of your plan limit) that runs independently from your main API concurrency, so you can use both without interference.