Category:Scraping Use Cases

How to Scrape Any Website in KNIME (Free Workflow Template)

2 Mins Read

Created Date: April 16, 2026

Updated Date: April 16, 2026

Growth

Building scrapers that get past anti-bot walls, CAPTCHAs, and proxy bans usually means Python scripts, constant maintenance, and a lot of broken pipelines.

That's why we packaged Scrape.do into a transparent, fully open KNIME Workflow (.knwf). Import it, feed it URLs, watch the data flow. Download the template here and follow along.

Step 1: Import the Engine and Add Your Key

Download the web_scraper.knwf file and open KNIME Analytics Platform. Go to File → Import KNIME Workflow..., select the file, and the node-based scraping engine appears on your canvas.

Before you run anything, authorize the workflow. Double-click the first Table Creator node (the one labeled for your Token) and replace ENTER_YOUR_TOKEN_HERE with your Scrape.do API Token from the dashboard. Click OK. You only have to do this once.

Step 2: Feed the Target URLs

Double-click the second Table Creator node. In the URL column, paste the list of target pages you want to scrape: product pages, real estate listings, directories, whatever you need. Click OK.

Step 3: Fire the Engine

Click the green Play button in the top toolbar (or Shift + F7) to execute all nodes.

The workflow merges your token and URLs and sends them to the GET Request node, which routes each request through Scrape.do's rotating residential proxies. The Row Filter node automatically drops any failed requests, so only clean Status 200 responses pass through.

Step 4: Extract the Data You Actually Need

Raw HTML is useful, but you usually want specific fields. The Regex Split node at the end of the chain handles that. Double-click it and write a regular expression in the Pattern field.

Say you're tracking Amazon product prices. To extract the page title and unit price, use:

(?s).*?<title>(.*?)</title>.*?class="a-price-whole">([^<]+).*

That tells KNIME to scan the HTML body, pull the text inside <title> tags into one column, and extract the number inside the a-price-whole class into another. Right-click the node and open Split Result to see the clean output.