Cloudflare Web Scraper
1 day trial then $20.00/month - No credit card required now
Cloudflare Web Scraper
1 day trial then $20.00/month - No credit card required now
Cloudflare Web Scraper extracts data from Cloudflare-protected websites. You can customize parameters such as proxies, timeouts, and JavaScript execution, making it ideal for reports, spreadsheets, and applications.
What does Cloudflare web Scraper do?
Cloudflare web Scraper allows you to extract data from the web page protected by Cloudflare. It gives you detailed web page data in structured formats like JSON, which you can use in your own reports, spreadsheets, and applications. Cloudflare web Scraper allows you to collect web page protected by Cloudflare, including:
- URL
- HTML
- Result from JS Script
Input & Output
To start collecting Idealo.de product data, simply fill out the input form. Cloudflare web Scraper recognizes the following input parameters:
- URLs - Links to product information pages, seller product listing pages, or product query pages via search, categories, etc. You can paste URLs one at a time or use the Bulk Edit section to add a prepared list.
- js_script - JavaScript code to execute on the web page, You can use it to manipulate web pages or get js command results after being run.
- js_timeout - Maximum time (in seconds) to wait for the JavaScript execution to complete.
- retrieve_result_from_js_script - Determines whether to fetch the result of the executed JavaScript.
- page_is_loaded_before_running_script - Specifies whether to wait for the page to fully load before running the JavaScript.
- execute_js_async" - Indicates if the JavaScript should be executed asynchronously.
- retrieve_html_from_url_after_loaded - Determines whether to retrieve the page's HTML after it has fully loaded.
- Limit the number of retries - Maximum number of retries when collecting data when an unexpected error occurs.
- Timeout on page load - Maximum waiting time when accessing the links you provided.
- Proxy configuration - Add a proxy to ensure that during the data collection process, you are not detected as a bot.
Collect data from web page protected by Cloudflare
Example url: https://gitlab.com
Example Screenshot of product information page:
Input:
1{ 2 "max_retries": 2, // Maximum waiting time when accessing the links you provided. 3 "proxy": { // Add a proxy to ensure that during the data collection process, you are not detected as a bot. 4 "useApifyProxy": true, 5 "apifyProxyGroups": [ 6 "RESIDENTIAL" 7 ], 8 "apifyProxyCountry": "SG" // You should choose an Country that coincides with the Country you want to collect data from 9 }, 10 "timeout": 5, 11 "urls": [ // Links to web pages. 12 "https://gitlab.com" 13 ], 14 "js_script": "return 10 + 10 + 20", // JS script you want to run 15 "js_timeout": 10, 16 "retrieve_result_from_js_script": true, // Retrieve result from JS script 17 "page_is_loaded_before_running_script": true, // Page is loaded before running script 18 "execute_js_async": false, // Execute JS async 19 "retrieve_html_from_url_after_loaded": true, // Retrieve page HTML from url after loaded 20}
Output:
You get the output from the Idealo.de product scraper stored in a tab. The following is an example of the Information Fields collected after running the Actor.
1[ // List of product information 2 { 3 "url": "https://about.gitlab.com/", 4 "result_from_js_script": 40, 5 "html": "<!DOCTYPE html>...</html>" // HTML from web page 6 }, // ... Many other product details 7]
How can I use the data extracted from web page with Cloudflare web Scraper?
1💙 Bypass cloudflare's protection. 2⭐ Execute custom js script
Your feedback
We are always working to improve Actors' performance. So, if you have any technical feedback about Cloudflare web Scraper or simply found a bug, please create an issue on the Actor's Issues tab in Apify Console.
ecomscrapeActor Metrics
2 monthly users
-
0 No stars yet
>99% runs succeeded
Created in Jan 2025
Modified a day ago
